1423 3991 1 SM PDF
1423 3991 1 SM PDF
1423 3991 1 SM PDF
Abstract—Traffic Sign Detection and Recognition (TSDR) plays mainly created for the classification task as each image contains
a crucial role in driver assistance systems, and provides drivers only one TS with a small view of the background. Recently they
with safety and precaution information. It is part of the computer attached 900 new images contains scenes from the roads of
vision which requires a dataset for training and testing the Germany can be used in the detection task. Presented in
detection and recognition techniques. In this paper, a dataset for
“German Traffic Sign Detection Benchmark” competition [5]
Malaysian TS (MTSD) is proposed in order to eliminate the gap in
the previously created datasets. The MTSD includes a variety of but the main problem for this dataset is the lack of video data
TS scenes to be used in TS detection and images contain only TS which can be used to make the detection task more robust.
to assist in the recognition of TS. The KULD includes four videos that can be used in
evaluating the performance of the TSDR system also it has
Index Terms—Traffic Sign Detection and Recognition; Traffic more classes than the GTSDRD but the tradeoff is the limited
Sign Dataset; German TSDR Benchmark; Synthetic Digital number of images. STSD has only seven classes but the images
Images. are actually extracted from the video sequence while LISAD is
the most recent database that is created because of the lack of
I. INTRODUCTION TS databases in the US. It complies with the Manual on
Uniform Traffic Control Devices (MUTCD) And Standard
Traffic Sign Detection & Recognition (TSDR) is part of the on- Highway Signs and Markings (SHSM) [14, 15]. Also, it
road applications in computer vision [1]. It can be used in includes at least a video track for each class in the dataset.
driver-assistance systems, highway maintenance, sign Unlike the previously mentioned datasets, which uses offline
inventories and intelligent autonomous vehicles. images taken from the real world, the UKOD and RTSD use an
Similar to other computer vision applications, TSDR systems online TS database [16, 17] and then applies many distortion
possess many problems, including lighting conditions, motion techniques such as geometric distortion, blurring, and
blur, vehicle speed, signs within the same category that are illumination variations, in trying to imitate the real world
similar to each other, weather affecting the visibility of signs, problems. This technique allows the recognition task to be done
signs that may be bent or that may not be perpendicular to the over all the TS types and avoids the process of labeling the
vision system, and, in the case of the urban or open roads, other database [11].
objects that may appear to be traffic signs. TSDR systems have The previously mentioned datasets are not widely used yet
been an important issue in the area of computer vision since the except the GTSDRB. Recently synthetic images has been tested
first paper was introduced in Japan in 1984 [2]. in [10, 11, 18] for the recognition task but comparing the results
In order to have a rapid progress in TS detection and with the one obtained from the standard dataset is not
recognition techniques eight standard datasets had been satisfactory [12, 19]. The need for a general dataset that can be
introduced: used around the world is enormously increasing.
1. German TSDR Benchmark (GTSDRB) [3-5] Traffic Signs have two main standards in the world and they
2. KUL Belgium Traffic Signs Dataset (KULD) [6] are the Vienna Convention and the MUTCD. Mainly across
3. Swedish Traffic Signs Dataset (STSD) [7] Europe, they use the Vienna Convention and U.S uses the
4. The Netherlands RUG Traffic Signs Database (RUGD) MUTCD. Most other countries use standards that are close to
[8] one of them, or a combination of the two [20, 21].
5. France Stereopolis Database [9] Table 1 illustrates the information regarding the number of
6. United States LISA Dataset (LISAD) [1] classes, number images, and the sizes for the images etc.
7. United Kingdom Online Dataset (UKOD) [10, 11] In this paper, a Malaysian Traffic Sign Dataset (MTSD) is
8. Russian Traffic Sign Dataset (RTSD) [12] created. Malaysian TSs are mostly similar to the TS found in
GTSDRB is the widely used dataset is the GTSDRB, was the USA, the design of TSs in Malaysia must follow
created to the “The German Traffic Sign Recognition “ARAHAN TEKNIK (JALAN) 2A/85” found in [22]. Warning
Benchmark” competition. It contains a large TS dataset suitable signs are diamond shaped and are yellow and black in color.
for detection and recognition of TS found in Germany which Regulatory signs are round with white backgrounds, red
comply with the Vienna Convention [13]. The GTSDRB is
borders, and black pictograms. Mandatory instruction signs are camera and two different mobile cameras resulting in a pixel
round with blue backgrounds and white pictogram. The range from 1920x977 px (FHD) to 4592x3448 px (UHD+)
exceptions are the stop sign and the give way sign. which is considered to be very high. While the recognition
The MTSD is distinct from the other datasets by many unique dataset which is consists of only traffic signs extracted from the
factors that will make it discriminatory used under many scenes is resized to a standard 32x32 px.
conditions:
1. There is no large dataset for Malaysia previously
collected.
2. The dataset is collected with different sources using
Mobile, camera, and google maps.
3. There is no dataset collected while raining or at night
which is a challenge to the TSDR techniques.
The Malaysian Traffic Sign Dataset (MTSD), is publicly
available the following six links:
– http://vcari.net/cairo/downloads/MTSD.part1.rar
– http://vcari.net/cairo/downloads/MTSD.part2.rar
– http://vcari.net/cairo/downloads/MTSD.part3.rar
– http://vcari.net/cairo/downloads/MTSD.part4.rar
– http://vcari.net/cairo/downloads/MTSD.part5.rar
– http://vcari.net/cairo/downloads/MTSD.part6.rar
Table 1
Public Available Traffic Sign Databases
GTSDR Stereo-
KULD STSD RUGD LISAD UKOD RTSD
B (2012 plis
(2009) (2011) (2003) (2012) (2012) (2013)
& 2013) (2010)
Number
of 43 100+ 7 3 10 49 100+ 140
classes
Number
of TS 9000 9006 20000 48 847 6610 43509 N/A
Figure 1: Various captured scenes during different conditions (daylight,
scenes raining and night) captured using a digital camera and google street maps.
39,209
Number training, 1200 80000+
13444 3488 48 251 7855
of TS 12,630 Synthetic Synthetic
testing
100x100
15x15 to 3x5 to 25x25 to 6x6 to
Sign to
250x250 263x248 N/A 204x159 167x168 24x24 px 30x30 px
sizes 1628x12
px px px px
36 px
640x480
Image 1360x80 1628x12 1280x96 360x270 1920x10 to 648x480 1280×720
sizes 0 px 36 px 0 px px 80 px 1024x52 px px
2 px
Yes, for
Include Yes, 4 all
No No No No No No
videos tracks annota-
tions
The
Country Germ- United United
Belgium Sweden Neth- France Russia
of origin any States Kingdom
erlands
Figure 2: Various traffic signs such as speed limit, road work, pedestrian
crossing, traffic sign ahead and no entry.
II. DATASET COLLECTION
III. DATASET STATISTICS
The dataset is composed of two different categories, the
detection dataset consists of one thousand images in which the The detection dataset consist of one thousand scenes is
traffic signs is detected from the captured traffic sign scene. explicated in Figure 3. 797 photos are taken using a digital
Figure 1 shows a various captured scenes and the recognition camera to output 583 images during daylight, 139 images at
dataset which is extracted from the TS scenes to produce more night and 95 images while raining. Another 162 images are
than two thousand traffic signs some of them are illustrated in taken from google street maps during daylight and 41 images
Figure 2. are captured by two different mobile cameras also during
The detection dataset is collected under different daylight. The resolution is recorded in Table 2.
circumstances at morning and night, foggy, raining and clear
weather to offspring a diversified dataset that satisfy
challenging new conditions. The traffic sign scenes are ranging
from having no TS to fifteen TS per scene. The collected
process is done from several sources using google maps, digital
output with the ground truth. In this dataset, the ground truth is
Dataset statistics split into two different categories one for the detection part and
900 the other one for the recognition part.
800 In order to elucidate the information in the ground truth, four
Number of Images 700
95
files are created to clarify all the information needed to describe
139
600 the dataset. The “Statistics_GT.txt” shows the number of TS in
500 each scene in addition to the weather condition and the camera
400 source. Part of this file is shown in Figure 5. The first line shows
300 563 the header name attributes, which are the File Name, Lightning,
200
Image Source, and Number of TS
100 162
0 41
Panasonic Google Mobile File Name;Lightning;Image Source;Number of TS
Camera Images Camera '1.jpg';'Day Light';'Google Street View';2
Raining Images 95 '10.jpg';'Day Light';'Google Street View';2
Night Images 139 '100.jpg';'Day Light';'Google Street View';2
'101.jpg';'Day Light';'Google Street View';1
Day Light Images 563 162 41 '102.jpg';'Day Light';'Google Street View';1
Image Source '103.jpg';'Day Light';'Google Street View';3
'104.jpg';'Day Light';'Google Street View';1
Figure 3: The number of captured scenes taken via different camera’s sources '105.jpg';'Day Light';'Google Street View';1
under different lighting conditions. '106.jpg';'Day Light';'Google Street View';1
'107.jpg';'Day Light';'Google Street View';4
Table 2
The captured scene’s resolution Figure 5: Sample data from the ground truth file “Statistics_GT.txt”.
Image size Image Source The second ground truth file is created to depict the TS scene
1920x977 px (FHD) Google Street Maps which can be used in testing the detection part or detection
3840x2160 px (4K UHD) Mobile Camera techniques. In this file a lot of information such as the TS
4592x3448 px (UHD+) Digital Camera
location, color and shape are required. All the information
needed to accurately examine the detection technique is in the
ground truth file “GT_Detection.txt” which a part of it is shown
While in the recognition dataset, TS is divided into five in Figure 6. It includes the attributes File Name, X (position),
groups, as stated in the Malaysian TS standard, which are Y (position), Width, Height, TS Color, Shape, Class ID,
Warning Danger “WD”, Regulatory Prohibitive “RP”, Lightning, and Image Source.
Regulatory Mandatory “RM”, Guide Information “GI” and
Temporary “T”. The TS group distribution among the one
File Name;X;Y;Width;Height;TS Color;Shape;Class
thousand images is clarified in Figure 4. Each group of TS ID;Lightning;Image Source
consists of a different TS signs showed in the hierarchy in '1.jpg';581;490;19;17;Red;'Red Circle';1;'Day Light';'Google Street
View'
Figure 7. '1.jpg';886;483;14;14;Red;'Red Circle';1;'Day Light';'Google Street
View'
4% 1% TS Type Distribution '10.jpg';801;493;21;21;Red;'Directive No';2;'Day Light';'Google Street
4% View'
'10.jpg';1157;477;23;22;Red;'Directive No';2;'Day Light';'Google
Warning Danger TS
Street View'
"WD",1076
'100.jpg';638;421;38;37;Yellow;'Diamond';3;'Day Light';'Google Street
Regulatory Prohibitive TS View'
"RP",795 '100.jpg';760;415;15;14;Red;'Red Circle';1;'Day Light';'Google Street
Regulatory Mandatory TS View'
52% "RM",88 '101.jpg';356;350;52;53;Red;'Red Circle';1;'Day Light';'Google Street
39%
Guide Information TS "GI",70 View'
'102.jpg';633;390;32;30;Yellow;'Diamond';3;'Day Light';'Google Street
View'
Temporary TS "T",27 '103.jpg';1026;369;42;46;Yellow;'Diamond';3;'Day Light';'Google
Street View'
'103.jpg';1025;420;42;46;Yellow;'Diamond';3;'Day Light';'Google
Figure 4: The TS distribution among the captured scenes. Street View'
Figure 6: Sample data from the ground truth file “GT_Detection.txt”.
'103.jpg';721;439;20;19;Red;'Red Circle';1;'Day Light';'Google Street
IV. DATASET GROUND TRUTH View'
In'104.jpg';660;409;17;24;Blue;'Rectangle';4;'Day
order to understand the attributes in “GT_Detection.txt”,
Light';'Google Street
Table 3 shows all the TS classes found in the Dataset in addition
View'
Ground truth by detention is the actual data (desired output)
to its'105.jpg';272;392;55;64;Blue;'Rectangle';4;'Day
ID, shape and color and these are the attributesLight';'Google
thatStreet
can be
for any classification problem. It used to test the accuracy of the View'
found is this ground truth file.
supervised learning techniques by comparing its classification '106.jpg';787;411;12;16;Blue;'Rectangle';4;'Day Light';'Google Street
View'
MTSR
'RM',2,3
'WD',7,1 'RM',1,7
'WD',7,2 'RP',2,2 'RP',3,1 Compulsory
Caution! Hump, Pass either side,
Hump ahead, 26 No right turn, 6 No U-turn, 20 motor-cycles
74 11
track, 1
'WD',14,1
'WD',13,1 'RP',5,30 'RP',6,2
Pedestrian
Slippery road, 2 crossing opt1, Weight limit Height limit
11 sign 30T, 2 sign 2.-m, 9
'WD',14,2 'WD',15,1
'RP',6,3 'RP',6,4
Pedestrian School children
crossing opt2, crossing opt1, Height limit Height limit
37 37 sign 3.-m, 14 sign 4.-m, 39
'WD',35,1
'WD',36,1
Narrow bridge, 'RP',14,3
Split way, 264 'RP',17,1
6
Widing limit
No overtaking, 2
3.5m, 2
'WD',37,1 'WD',38,1
Two way road, Divided road
1 ending, 1
'WD',38,3 'WD',42,1
Curve on the Crossroads Y-
left, 4 junction, 2
Figure 7: Standard TS hierarchy illustrates the class, class ID, name, and amount of each traffic sign.
Table 3 classes are nine sign classes one for each speed limit sign and
The TS class ID, shape, and color
in “GI,9” is only one sign class.
All the previously mentioned ground truth files was created
using a developed GUI tool using Matlab R2015a to
TS
automatically insert all the required information found in the TS
scene (Figure 9). The developed GUI shows the TS scene to the
right allowing the user to “Select the ROI” by click on the
Class
1 2 3 4 5 button. Then, a drag-able rectangle is visible to select the TS.
ID The position of the rectangle (TS) is known by the left upper
TS
Shape
Red Circle Directive No Diamond Rectangle Blue Circle point (“X” and “Y”) and the “Width” and “Height” of the drag-
TS Yellow or able rectangle. In the field of “Class ID”, the user must enter the
Red Red Blue Blue
Color Orange full ID that includes Sign Type, Sign Group and Sign Class
separated by a semicolon. Finally, the “Time of the Day” and
“Camera Type” must be adjustable to meet the scene
TS conditions. By click “Save ROI” the information is stored in the
main ground truth file “GT.txt” and the “ROI Count”
Class incremented by one. If there is another TS in the same scene the
6 7 8 9 10 same process must be done until all the TS information is
ID
TS
Flip Triangle
Regulatory
No Entry Pentagon Octagon correctly stored in the file. Moving to the next scene is done by
Shape No clicking the “Next Image” button.
TS
Red Red Red Yellow Red
Color
TS
Compulsory
TS Name U-turn Keep right Keep left Pass either side motor-cycles Stop No Left Turn No right turn
track
TS Class 'GI',9,1 'RM',1,5 'RM',1,6 'RM',1,7 'RM',2,3 'RP',1,1 'RP',2,1 'RP',2,2
TS ID 1 2 3 4 5 6 7 8
TS
Weight limit sign Weight limit sign Height limit sign Height limit sign Height limit sign Height limit sign
TS Name No U-turn No entry
5T 30T 2.-m 3.-m 4.-m 5.-m
TS Class 'RP',3,1 'RP',4,1 'RP',5,5 'RP',5,30 'RP',6,2 'RP',6,3 'RP',6,4 'RP',6,5
TS ID 9 10 11 12 13 14 15 16
TS
Height limit sign
TS Name Speed Limit 20 Speed Limit 30 Speed Limit 40 Speed Limit 50 Speed Limit 60 Speed Limit 70 Speed Limit 80
6.-m
TS Class 'RP',6,6 'RP',7,20 'RP',7,30 'RP',7,40 'RP',7,50 'RP',7,60 'RP',7,70 'RP',7,80
TS ID 17 18 19 20 21 22 23 24
TS
No Entry for
Heavy vehicles,
Vehicles
TS Name Speed Limit 90 Speed Limit 110 no driving on No Parking No Stopping Give way Wide limit 3.5m
Exceeding 5T,
right lane
Trucks etc.
TS Class 'RP',7,90 'RP',7,110 'RP',8,1 'RP',8,3 'RP',10,1 'RP',11,1 'RP',13,1 'RP',14,3
TS ID 25 26 27 28 29 30 31 32
TS
Camera
TS Name No overtaking Road Work Crosswind area Caution! Hump Hump ahead Towing zone Left bend
operation zone
TS Class 'RP',17,1 'T',1,1 'WD',5,1 'WD',6,1 'WD',7,1 'WD',7,2 'WD',8,1 'WD',10,2
TS ID 33 34 35 36 37 38 39 40
TS
Pedestrian Pedestrian School children School children Narrow roads on Traffic lights
TS Name Slippery road Caution
crossing opt1 crossing opt2 crossing opt1 crossing opt2 the left ahead
TS Class 'WD',13,1 'WD',14,1 'WD',14,2 'WD',15,1 'WD',15,2 'WD',16,1 'WD',21,1 'WD',22,1
TS ID 41 42 43 44 45 46 47 48
TS
Staggered Crossroads T- Crossroads to the Crossroads to the Minor road on
TS Name Obstacles ahead Exit to the left Crossroads
junctions junction right left right
TS Class 'WD',23,1 'WD',25,1 'WD',27,1 'WD',27,2 'WD',27,3 'WD',27,5 'WD',27,6 'WD',28,1
TS ID 49 50 51 52 53 54 55 56
TS
Minor road on Minor road on Roundabout Divided road
TS Name Cattle crossing Narrow bridge Split way Two way road
left left opt2 ahead ending
TS Class 'WD',28,2 'WD',28,3 'WD',30,1 'WD',31,1 'WD',35,1 'WD',36,1 'WD',37,1 'WD',38,1
TS ID 57 58 59 60 61 62 63 64
TS
Crossroads Y-
TS Name Curve on the left
junction
TS Class 'WD',38,3 'WD',42,1
TS ID 65 66
File Name;X;Y;Width;Height;Sign Type;Sign Group;Sign Class;TS [7] Larsson, F., et al., "Using fourier descriptors and spatial models for traffic
Class;Class ID;TS Color;Shape;Shape ID;Lightning;Image Source sign recognition", in Image Analysis. 2011, Springer. p. 238-249.
'1.jpg';581;490;19;17;'RP';7;60;'RP',7,60;22;Red;'Red Circle';1;'Day [8] Grigorescu, C., et al., "Distance sets for shape filters and shape
Light';'Google Street View' recognition". Image Processing, IEEE Transactions on, 2003. 12(10): p.
'1.jpg';886;483;14;14;'RP';7;60;'RP',7,60;22;Red;'Red Circle';1;'Day 1274-1286.
Light';'Google Street View' [9] Belaroussi, R., et al., "Road sign detection in images: A case study". in
'10.jpg';801;493;21;21;'RP';8;3;'RP',8,3;28;Red;'Directive No';2;'Day Pattern Recognition (ICPR), 2010 20th International Conference on.
Light';'Google Street View' 2010. IEEE.
'10.jpg';1157;477;23;22;'RP';8;3;'RP',8,3;28;Red;'Directive No';2;'Day [10] Greenhalgh, J., et al., "Real-time detection and recognition of road traffic
Light';'Google Street View' signs". Intelligent Transportation Systems, IEEE Transactions on, 2012.
'100.jpg';638;421;38;37;'WD';16;1;'WD',16,1;46;Yellow;'Diamond';3;'D 13(4): p. 1498-1506.
ay Light';'Google Street View' [11] Greenhalgh, J., et al., "Traffic sign recognition using MSER and Random
'100.jpg';760;415;15;14;'RP';7;60;'RP',7,60;22;Red;'Red Circle';1;'Day Forests". in Signal Processing Conference (EUSIPCO), 2012 Proceedings
Light';'Google Street View' of the 20th European. 2012. IEEE.
'101.jpg';356;350;52;53;'RP';7;60;'RP',7,60;22;Red;'Red Circle';1;'Day [12] Moiseev, B., et al., "Evaluation of Traffic Sign Recognition Methods
Light';'Google Street View' Trained on Synthetically Generated Data". in Advanced Concepts for
'102.jpg';633;390;32;30;'T';1;1;'T',1,1;34;Yellow;'Diamond';3;'Day Intelligent Vision Systems. 2013. Springer.
Light';'Google Street View' [13] "Vienna Convention on Road Signs and Signals". 1968; Available from:
'103.jpg';1026;369;42;46;'WD';23;1;'WD',23,1;49;Yellow;'Diamond';3;' http://www.unece.org/fileadmin/DAM/trans/conventn/signalse.pdf.
Day Light';'Google Street View' [Accessed: September 30, 2014]
'103.jpg';1025;420;42;46;'WD';36;1;'WD',36,1;62;Yellow;'Diamond';3;' [14] State of California, Dept. Transp, Sacramento, CA "Manual on Uniform
Day Light';'Google Street View' Traffic Control Devices for Streets and Highways". 2009; Available from:
'103.jpg';721;439;20;19;'RP';7;50;'RP',7,50;21;Red;'Red Circle';1;'Day http://mutcd.fhwa.dot.gov/. [Accessed: September 30, 2014]
Light';'Google Street View' [15] U.S., Department for Transport "Standard Highway Signs and Markings".
'104.jpg';660;409;17;24;'GI';9;1;'GI',9,1;1;Blue;'Rectangle';4;'Day 2004; Available from: http://mutcd.fhwa.dot.gov/ser-
Light';'Google Street View' shs_millennium.htm. [Accessed: September 30, 2014]
'105.jpg';272;392;55;64;'GI';9;1;'GI',9,1;1;Blue;'Rectangle';4;'Day [16] London, U.K., Department for Transport "Traffic Signs Image Database".
2013; Available from: https://www.gov.uk/traffic-sign-images.
Light';'Google Street View'
[Accessed: September 30, 2014]
[17] "Road Signs of Russia". 2015; Available from:
http://commons.wikimedia.org/wiki/Road_signs_of_Russia. [Accessed:
Figure 10: Sample data from the main ground truth file “GT.txt”.
January 8, 2015]
[18] Mogelmose, A., et al., "Learning to detect traffic signs: Comparative
REFERENCES evaluation of synthetic and real-world datasets". in Pattern Recognition
(ICPR), 2012 21st International Conference on. 2012. IEEE.
[1] Mogelmose, A., et al., "Vision-based traffic sign detection and analysis [19] Mogelmose, A., et al., "Traffic Sign Detection and Analysis: Recent
for intelligent driver assistance systems: Perspectives and survey". Studies and Emerging Trends". 2012 15th International Ieee Conference
Intelligent Transportation Systems, IEEE Transactions on, 2012. 13(4): p. on Intelligent Transportation Systems (Itsc), 2012: p. 1310-1314.
1484-1497. [20] Møgelmose, A., "Traffic sign detection using computer vision:
[2] Maldonado-Bascon, S., et al., "Road-sign detection and recognition based Explorations for a driver support system", in Department of Electrical
on support vector machines". Ieee Transactions on Intelligent Engineering. 2012, Aalborg University, Denmark in collaboration with
Transportation Systems, 2007. 8(2): p. 264-278. University of California, San Diego. p. 121.
[3] Stallkamp, J., et al., "The German traffic sign recognition benchmark: a [21] Brkic, K., "An overview of traffic sign detection methods". Department
multi-class classification competition". in Neural Networks (IJCNN), The of Electronics, Microelectronics, Computer and Intelligent Systems
2011 International Joint Conference on. 2011. IEEE. Faculty of Electrical Engineering and Computing Unska, 2010. 3: p.
[4] Stallkamp, J., et al., "Man vs. computer: Benchmarking machine learning 10000.
algorithms for traffic sign recognition". Neural networks, 2012. 32: p. [22] Jabatan Kerja Raya (JKR), Public Work Department, Malaysia "Arahan
323-332. Teknik (Jalan) 2A/85, Manual on Traffic Control Devices : Standard
[5] Houben, S., et al., "Detection of traffic signs in real-world images: The Traffic Signs". 2009; Available from: https://www.jkr.gov.my/.
German Traffic Sign Detection Benchmark". in Neural Networks [Accessed: December 30, 2014]
(IJCNN), The 2013 International Joint Conference on. 2013. IEEE.
[6] Timofte, R., et al., "Multi-view traffic sign detection, recognition, and 3d
localisation". in Applications of Computer Vision (WACV), 2009
Workshop on. 2009. IEEE.