An Efficient Automated Vehicle License Plate Recognition System Under Image Processing
An Efficient Automated Vehicle License Plate Recognition System Under Image Processing
An Efficient Automated Vehicle License Plate Recognition System Under Image Processing
Corresponding Author:
Dilshad Islam
Department of Physical and Mathematical Sciences, Faculty of Food Science and Technology
Chattogram Veterinary and Animal Sciences University (CVASU)
4225 Chattogram, Bangladesh
Email: [email protected]
1. INTRODUCTION
Nowadays vehicles are increasing exponentially because of population growth and their needs. So,
the efficient and effective recognition of the vehicle license plate detection system is very important. License
plate recognition system uses image processing techniques that retrieves the vehicle number from a given
license plate image as a form of digital string of characters. This system is important because it plays an
indispensable role in automatic inspection of effective traffic control, transportation systems and also in
security systems.
Over the past few years, many techniques have been developed by researchers to detect number plates
but still it remains a challenging task. Due to several factors such as poor image resolution, non-uniform
illumination, reflection, varied weather, variety of fonts, camera quality affects the vehicle license plate
detection procedure. And the quality of the vehicle license plate recognition process will directly affect the
speed and the accuracy of the whole vehicle license plate detection system.
Lot of researchers had experimented with different procedures to implement the vehicle license plate
detection system. These approaches applied by researchers to improve the speed and accuracy of the license
plate recognition system progressed day by day. Some of these approaches are reviewed below:
Selmi et al. [1] proposed a framework based on the mask region convolutional neural network which
was applied in three stages of license plate detection, recognition and segmentation. This system was evaluated
under several datasets containing images acquired from several hazardous conditions with multi-languages and
multi-orientation license plates. Kessentini et al. [2] proposed a system architecture for the that used two deep
learning stages pipeline which could be used for multilingual and multi-norm license plate identifications. This
proposed system recognition accuracy was 91.46% over Radar dataset and 97.67% over GAP-LP dataset with
a rational computational time.
A deep learning based vehicle license plate recognition system was introduced that incorporates
optimal K-means (OKM) clustering for license plate segmentation and convolutional neural network (CNN)
for license plate identification [3]. This system used Krill Herd algorithm with OKM clustering technique in
segmenting license plate characters and CNN model was used in recognizing license plate characters. The test
was performed on three datasets namely FZU Cars, Stanford Cars, and HumAIn 2019 challenge dataset and
attained accuracy of 0.981 on the applied datasets. A single neural network for detecting and recognizing a
mixed style license plate where two parallel branches of object classification and detection were presented for
vehicle license plate detection and recognition [4]. This system produced the license plate type and the license
plate string skipping the intermediate and some repetitive steps and classified the branch of license plate to
support mixed style and multiple license plates in an integrated framework.
Chowdhury et al. [5] proposed a model named fractal series expansion method for the enhancement
of license plate image. This model calculates the highest probability for the pixels of the license plate with the
pixels of background and based on this entropy information enhancement was done. An on-line real time
intelligent license plate recognition system was developed using the fusion of connected component analysis
and spectral analysis at the stage of license plate extraction and segmentation [6]. A super-resolution framework
for license plate images was designed which was composed of two components domain priori GAN and
progressive vehicle search that exploits to integrate all the synthetic license plates into an image with highest
resolution [7].
Yousif et al. [8] proposed a methodology that combines the genetic algorithm with an optimized
neutrosophic set and image processing algorithms. The methodology proposed in this paper was used for
increasing the recognition rate of license plates (Arabic-Egyptian) and English. To localize the license plate,
some image processing techniques like edge detection and morphological operation were used and for
extraction the genetic algorithm with a neutrosophic set was used. Farag et al. [9] presented a method using
image processing techniques that regulates the entrance of smart parking. For larger image discrete wavelet
transform was used for feature extraction at the preprocessing step that decreased the time at detection stage.
At segmentation stage correlation method was used and in case of similarity support vector machine [10] was
used for classification in this system.
Marzuki et al. [11] proposed a four-layered CNN architecture which was used for license plate
detection. The learning algorithm used in training the CNN was an improved version of stochastic diagonal
levenberg marquadt algorithm [12] and a smoother error gradient was achieved. Yaacob et al. [13] proposed
an automated system that kept continuous track of entry and exit at the gate of university campus and used
template matching technique at character recognition stage. The accuracy rates of this system for the plate
recognition, character segmentation, and character recognition were 91.58%, 93.11% and 80.25%,
respectively. A comparative analysis of vehicle accidents was done by testing a number of network topologies
and also number matrices were used in the estimation process which identify the best networks [14]. The study
showed that CNNs made classification and detection results more accurate and the network detectors with deep
CNN topologies intensify the accuracy rate of accident classification. A multivehicle detection and vehicle
license plate recognition system under complex traffic backgrounds was proposed based on a hierarchical
region convolutional neural network (RCNN) by Tu and Du [15]. A multiple class classification problem was
solved by this RCNN that used multiple-level networks for smaller blocks of sub tasks that enables the
upgradation of network when complexity arises so that the computational load was reduced.
An automatic layout independent license plate recognition system based upon the state-of-the-art you
only look once (YOLO) object detector which contains an integrated approach for license plate layout
classification and detection and using post-processing rules improves the recognition results [16]-[18]. This
network was trained using images from several datasets such as ChineseLP, OpenALPR-EU, and UFPR-ALPR
with the addition of several data augmentation techniques as a result the system was robust under different
conditions. Ahmed et al. [19] implemented an analysis and design of a simulation model of vehicle license
Indonesian J Elec Eng & Comp Sci, Vol. 29, No. 2, February 2023: 1055-1062
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 1057
plate recognition system using image processing technology. The horizontal and vertical projections and co-
occurrence matrix were applied to extract the editable text from the extracted plate’s information. An algorithm
was designed which integrates an adaptive image segmentation technique named as connected component
analysis and sliding concentric windows with a character recognition neural network for identification of the
vehicle license plate [20]. For the optical character recognition system, a two-layer probabilistic neural network
was used and a trained network was used in identifying characters from the vehicle license plate image based
on the data acquired from algorithmic image processing.
The proposed automatic license plate recognition system consists of four sequential modules which
are: input image preprocessing, license plate region extraction, license plate character segmentation and license
plate character recognition as shown in Figure 1. The system is implemented and simulated on MATLAB
2018b software.
Figure 1. Major steps involved in automated license plate character recognition system
The main contribution of this paper is to provide an efficient way that recognizes license plate
characters with higher accuracies than other existing systems using image processing techniques. The proposed
method can accept all kinds of colored images as input and returns characters present in the license plate as a
form of string more accurately than any other existing system. The rest of the paper is organized as follows. Next
section briefly introduces the research methods involved in converting the input image into a form of string of
characters present in the vehicle license plate image. Experimental works and results are discussed in section 3. And
finally, the paper is concluded in section 4.
2. METHOD
The major steps involved in this system are divided into four major steps which are; 1) input image
preprocessing, 2) license plate region extraction, 3) license plate character segmentation, and 4) license plate
character recognition. In the first phase of preprocessing the input image is processed so that the image quality
is enhanced and it also provides a noise free image. This step is very important for the accuracy of the license
plate character recognition. At the second phase probable plate region is extracted from the larger scene using
some morphological operations. And at the third phase character segmentation is done where each object is
labeled using connected component analysis. Finally, each character is matched with available templates stored
in the database using template matching techniques that will recognize each of the characters present in the
license plate image. The overall flowchart of the processing of license plate detection system is illustrated in
Figure 2.
I=0.299*R+0.587*G+0.114*B (1)
An efficient automated vehicle license plate recognition system under image processing (Dilshad Islam)
1058 ISSN: 2502-4752
The resolution of the image is reduced using MATLAB function as a result the image computing time
will also decrease. Images captured in real time may consist of noises which affect the quality of the image.
So, the reconstruction of the image is necessary in order to get a noise free image and without losing any piece
of information from the image [22]. Bilateral filtering is applied to eliminate these noises and the result of this
filtering preserves edges more effectively than median filtering shown in Figure 3(c).
Figure 3. Effects of preprocessing (a) original image, (b) gray scale image, and (c) applying bilateral filter on
gray scale image
Indonesian J Elec Eng & Comp Sci, Vol. 29, No. 2, February 2023: 1055-1062
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 1059
(a) (b)
(c) (d)
Figure 4. Steps during license plate localization (a) binarized image, (b) results of applying sobel filter,
(c) results of erosion, and (d) extracted candidate region
An efficient automated vehicle license plate recognition system under image processing (Dilshad Islam)
1060 ISSN: 2502-4752
(a) (b)
Figure 5. Segmentation process (a) plate region with connected components and (b) final plate region by
removing small objects
Indonesian J Elec Eng & Comp Sci, Vol. 29, No. 2, February 2023: 1055-1062
Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 1061
4. CONCLUSION
Although a good number of researches are carried out in detection and recognition of vehicle license
plates, still it remains a challenging task. The proposed system is efficient and gives satisfactory results
compared to other existing license plate detection systems and it also works well on low resolution, low contrast
images. But some difficulties may occur while the image plate is broken or similarity between ambiguous
characters like 0 and O; 1 and I; 5 and S and images with serious defects may produce wrong results. The future
research could be carried out in the recognition of high-definition license plate image, multi-style license plate
image and multi-plates license plate image using advanced features of artificial intelligence. The segmentation
procedure can also be improved using advanced tools and techniques of image processing to reduce processing
time and speed up the recognition rate.
REFERENCES
[1] Z. Selmi, M. B. Halima, U. Pal, and M. A. Alimi, "DELP-DAR system for license plate detection and recognition," Pattern
Recognition Letters, vol. 129, pp. 213-223, 2020, doi: 10.1016/j.patrec.2019.11.007.
[2] Y. Kessentini, M. D. Besbes, S. Ammar, and A. Chabbouh, "A two-stage deep neural network for multi-norm license plate detection
and recognition," Expert systems with applications, vol. 136, pp. 159-170, 2019, doi: 10.1016/j.eswa.2019.06.036.
[3] I. V. Pustokhina et al., "Automatic vehicle license plate recognition using optimal K-means with convolutional neural network for
intelligent transportation systems," IEEE Access, vol. 8, pp. 92907-92917, 2020, doi: 10.1109/ACCESS.2020.2993008.
[4] Q. Huang, Z. Cai, and T. Lan, "A single neural network for mixed style license plate detection and recognition," IEEE Access,
vol. 9, pp. 21777-21785, 2021, doi: 10.1109/ACCESS.2021. 3055243.
[5] P. N. Chowdhury, P. Shivakumara, H. A. Jalab, R. W. Ibrahim, U. Pal, and T. Lu, "A new fractal series expansion-based
enhancement model for license plate recognition," Signal Processing: Image Communication, vol. 89, p. 115958, 2020, doi:
10.1016/j.image.2020. 115958.
[6] G. Lekhana and R. Srikantaswamy, "Real time license plate recognition system," International Journal of Advanced Technology &
Engineering Research, vol. 2, pp. 5-9, 2012.
[7] W. Liu, X. Liu, H. Ma, and P. Cheng, "Beyond human-level license plate super-resolution with progressive vehicle search and
domain priori GAN," in Proceedings of the 25th ACM international conference on Multimedia, pp. 1618-1626, 2017, doi:
10.1145/3123266.3123422.
[8] B. B. Yousif, M. M. Ata, N. Fawzy, and M. Obaya, "Toward an optimized neutrosophic K-means with genetic algorithm for automatic
vehicle license plate recognition (ONKM-AVLPR)," IEEE Access, vol. 8, pp. 49285-49312, 2020, doi: 10.1109/ACCESS.2020.2979185.
[9] M. S. Farag, M. El Din, and H. El Shenbary, "Parking entrance control using license plate detection and recognition," Indonesian
Journal of Electrical Engineering and Computer Science, vol. 15, pp. 476-483, 2019, doi: 10.11591/ijeecs.v15.i1.pp476-483.
[10] K. Parasuraman and P. Subin, "SVM based license plate recognition system," in IEEE International Conference on Computational
Intelligence and Computing Research, 2010.
[11] P. Marzuki, A. Syafeeza, Y. Wong, N. Hamid, A. N. Alisa, and M. Ibrahim, "A design of license plate recognition system using
convolutional neural network," International Journal of Electrical and Computer Engineering, vol. 9, p. 2196, 2019, doi:
10.11591/ijece.v9i3.pp2196-2204.
[12] S. S. Liew, M. Khalil-Hani, and R. Bakhteri, "An optimized second order stochastic learning algorithm for neural network training,"
Neurocomputing, vol. 186, pp. 74-89, 2016, doi: 10.1016/j.neucom.2015.12.076.
[13] N. L. Yaacob, A. A. Alkahtani, F. M. Noman, A. M. Zuhdi, and D. Habeeb, "License plate recognition for campus auto-gate system,"
Indonesian Journal of Electrical Engineering and Computer Science, vol. 21, pp. 128-136, 2021, doi: 10.11591/ijeecs.v21.i1.pp128-136.
[14] M. A. Anwer, S. M. Shareef, and A. M. Ali, "Accident vehicle types classification: a comparative study between different deep
learning models," Indonesian Journal of Electrical Engineering and Computer Science, vol. 21, pp. 1474-1484, 2021, doi:
10.11591/ijeecs.v21.i3.pp1474-1484.
[15] C. Tu and S. Du, "A hierarchical RCNN for vehicle and vehicle license plate detection and recognition," International Journal of
Electrical and Computer Engineering, vol. 12, p. 731, 2022, doi: 10.11591/ijece.v12i1.pp731-737.
[16] R. Laroca, L. A. Zanlorensi, G. R. Gonçalves, E. Todt, W. R. Schwartz, and D. Menotti, "An efficient and layout‐independent
automatic license plate recognition system based on the YOLO detector," IET Intelligent Transport Systems, vol. 15, pp. 483-503,
2021, doi: 10.1049/itr2.12030.
[17] R. Laroca et al., "A robust real-time automatic license plate recognition based on the YOLO detector," in 2018 International Joint
Conference on Neural Networks (ijcnn), 2018, pp. 1-10.
[18] P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, "A Review of Yolo algorithm developments," Procedia Computer Science, vol. 199,
pp. 1066-1073, 2022, doi: 10.1016/j.procs.2022.01.135.
[19] A. K. Ahmed, M. Q. Taha, and A. S. Mustafa, "On-road automobile license plate recognition using co-occurrence matrix," Journal
of Advanced Research in Dynamical & Control Systems, vol. 10, 2018.
[20] C. N. E. Anagnostopoulos, I. E. Anagnostopoulos, V. Loumos, and E. Kayafas, "A license plate-recognition algorithm for intelligent
transportation system applications," IEEE Transactions on Intelligent transportation systems, vol. 7, pp. 377-392, 2006, doi:
10.1109/TITS.2006.880641.
[21] X. Zhang, F. Xu, and Y. Su, "Research on the license plate recognition based on MATLAB," Procedia Engineering, vol. 15,
pp. 1330-1334, 2011, doi: 10.1016/j.proeng.2011.08.246.
[22] W. A. Ulbeh, A. Moustafa, and Z. A. Alqadi, "Gray image reconstruction," European Journal of Scientific Research, vol. 27,
pp. 167-173, 2009.
[23] S. L. Bangare, A. Dubal, P. S. Bangare, and S. Patil, "Reviewing Otsu’s method for image thresholding," International Journal of
Applied Engineering Research, vol. 10, pp. 21777-21783, 2015, doi: 10.37622/IJAER/10.9.2015.21777-21783.
[24] S. Israni and S. Jain, "Edge detection of license plate using sobel operator," in 2016 International Conference on Electrical,
Electronics, and Optimization Techniques (ICEEOT), 2016, pp. 3561-3563.
[25] I. Kusumadewi, C. A. Sari, and E. H. Rachmawanto, "License number plate recognition using template matching and bounding box
method," in Journal of Physics: Conference Series, 2019, p. 012067, doi: 10.1088/1742-6596/1201/1/012067.
An efficient automated vehicle license plate recognition system under image processing (Dilshad Islam)
1062 ISSN: 2502-4752
BIOGRAPHIES OF AUTHORS
Indonesian J Elec Eng & Comp Sci, Vol. 29, No. 2, February 2023: 1055-1062