European License Plate Recognition

Based on Convolutional Neural Network Positioning and OCR Recognition

This is my bachelor thesis. Mainly for license plates in Europe. The training set's photos were mainly taken in the parking lot of the car's tail and head. Because the training of neural network is EXTREMELY TARGETED, for example: If there is a big difference between your test photo and my training set (incl. resolution, scene, color), it will lead to an inaccuracy recognition. In this case, you can use the two training functions in the file to train the scenarios that you need to identify. Due to my personal ability, there is no large data set available to me, and my training data is only a few hundred pictures. If you have more training data, you can get better results.

License plate recognition consists of three parts:
1. Image pre-processing
2. license plate positioning
3. Character recognition

For positioning module:
The main idea is firstly to search 8-connectivty areas after appropriate image pre-processing (morphological process). If this step succeeds, the area captured by 8-connectivty is often more accurate than the area locked on by the neural network. License plates and similar signs have distinct regional characteristics that differ from those of the surrounding area: the license plate and its surrounding area are not connected by a common feature. The entire pre-processed image was binarized and stored in a logical matrix. All separate areas that were different from the surrounding area could then be framed by searching for 8-connected objects: the license-plate area had to be within these 8-connected areas. Next, a neural network classifier was used to identify and classify all 8-connectivty areas being searched to determine the license-plate region.

In order to train the classifier, I used MATLAB Image Labeler to label the picture of the training set. I manually labeled the license plates region and some typical non-license plate elements in each picture, and then trimmed the label area. In this way, the resulting pictures are stored in folders in two categories: plate and non-plate.

ResNet-18 is then used for transfer-learning and training, so that a neural network specifically designed to identify whether the target image is a license plate can be deployed.

However, if 8-connectivty areas are not found, two Faster-RCNN object detectors are called for license plate positioning and extraction. I used two different neural networks to build the object detector: ResNet-18 and AlexNet. Among them, the AlexNet-picked license plate area is more accurate, ResNet-18 is more robust: it can locate the license plate in a picture containing a large number of complex non-license plate elements. In fact, two different Faster-RCNN object detectors are just for fun, if you have big data training set and more powerful computing ability, two or more detectors are not necessary, you can train one more powerful detector based on ResNet-50.

Character recognition module:
Attention! All thresholds for character extraction are set for license plates in the training set, and if you use another (i.e. Others1 and Others2 in the folder) license plate image, you must adjust the threshold settings to your situation. Character recognition is achieved by MATLAB OCR function, English letters and numbers can be correctly recognized. But OCR is NOT WORKING for other character such as Cyrillic alphabet, deutsch ÄÜÖß and Chinese.

Besides OCR and template matching, CNN is a better recognition method. However, the disadvantage of using neural networks is that there must be a large data set as a training sample. In fact, in past cases when CNNs were used for character recognition, hundreds of different style patterns per character were needed. This huge workload is difficult for a single person to complete without the support of a data set.

Pay attention to the Required MathWorks Products!!!
Before running, make sure your PC has a separate GPU and compute capability is above 5.0. Check this by "gpuDevice" command in MATLAB.

For more detail, view Readme.mlx

by CNS