Matlab Number Plate

Tar.Aldarion

Hey I am just learning some matlab and I have been processing pictures of number plates, to try isolate and count the numbers/letters on it. i should then highlight the letters/numbers I found in some way.

Question:
A. Isolate and count the main alphanumerical registration characters in the four number plate images
illustrated below. The same programme must work on all images.
B. Each identified character must also be highlighted on the original image.

Original image:





So I have processed the image up to this point:



Or also the inversion of this image with black letters and a white background.

Now I want to know how do I pick the letters out from the rest of the shapes without a template, then count and highlight them? Edge detection or something? I wanted to find the centroid of them but that doesnt really work.

Thanks

Tar.Aldarion

I forgot to say that whole image is surround by a white border, which is stopping me a lot from completing my work, is there a way to remove a border or scan from a a certain percentage into the image?

as you can see I find the centroid of each enclosed white space here and the border causes a 'false' centroid near the O's one. also there is a false one in the bottom left.




although i am not sure if this is even the correct method as I have to 'highlight' the letters/numbers which is very vague.

carveone

I experimented with this stuff years and years ago (on a 386 to give you an idea) and it's a total killer. Frankly, I'm pretty impressed with your first step! You've done the normalisation and moving to isolation.

Edge detection can be used to isolate the letters, given that you know they can be cut into rectangles.

You could try thinning and then rasterisation, which is the stuff I knew about. I get a bit lost after that. Sorry I'm not more help! This stuff is really non trivial!

Joneser

Hi Tar, it's been quite a while since I have done any of this so can't really add much, but your post got me interested myself and I found these 2 pages which seem to try and do something similar to what you are attempting, hopefully they are of some help.

http://www.mathworks.co.uk/help/toolbox/images/f0-8778.html

http://www.instructables.com/id/Image-Processing-and-Counting-using-MATLAB/step5/Find-the-Boundaries-of-the-Objects/ (The next step involves the drawing of boundaries around detected objects.

yuang86

Hi Tar,

it pains me to find out that I am only allowed to post a reply when I register as a forum member. Oh well, such is life plagued with spams.

Back to your question:
You've done well so far. The next thing you will need is to locate the position of the numbers, and this is can be done easily with the function normxcorr2. You can read about this function here: http://www.mathworks.com/help/toolbox/images/ref/normxcorr2.html

Surely there may be other methods other than template matching, but this is by far the most commonly used approach.

Hope this helps.

Tar.Aldarion

Thanks very much guys, I will have a look at each of these. Good to have a topic to look up as was a bit lost! sorry i made you register

Tar.Aldarion

I hit a lto of dead ends, normxcorr wouldnt work with rotation or scaling of the image. and where does one gget a template of their letters? or does it jsut have to be close?

a problem with mine that is the only thing that is holding me back is the stuff in the image that isnt the numbers and letters, how do i get rid of those?

I did sobel edge detection but im not sure if that helps me much, new image:

yuang86

You will have to create your own templates or you can google for "alphabets" and "numbers" on the internet. There are plenty of styles there - pick the most generic one, which has the greatest resemblance to the car plate font style. The higher the resemblance, the better the match.

You will need to do some form of pre-processing on the image before you use normxcorr2. For one, you will need to format the numbers so that they match your template size.

As for the orientation, you will need to rotate the image to correct its angle. As far as I can tell, Hough Transform should do the job nicely. MATLAB has that function.

Tar.Aldarion

Thanks a lot! /work work

I suppose I can convert scale and rotation before doing everything else. damn tutorials just said it ws useless if images need to be scaled or rotated grrr.

yutta

What happens if you fire that image into ocrad?

i.e. `ocrad myimage.png`

Tar.Aldarion

you mean gnu ocrad or something in matlab?
I'm mostly using the vsg toolbox, I downloaded, to do this.

ianhobo

I haven't used matlab in years, but in general the steps would be;
1. Threshold (which you have already achieved)
- try playing with these tolerances, the numbers are very well defined, so you could reduce the contrast to minimise the unwanted areas while leaving the numbers well defined
2. Remove unwanted area's
- Depending on how you want to do step 3, you either remove them or not.
I would say, given that all EU number plates are the same, you could simply crop the image to remove the the stars and IRL on the left, it's perfectly valid - you have known consistent data (location of numbers), so use it (although who ever is marking the question may see it differently)
Other wise you'll have to thin the image or get there outlines and determine their size, if lower than a size of a valid number, removed them, something along those lines.
3. your options for number 3 involve either matrix template matching (used for OCR against known fonts) or feature matching (used for variable font recognition like handwriting,).
matrix template matching would involve making a reference image of each of the potential values (0-9,a-z), then you search the image against each of these template images, record the SAD or other (sum of abs differences), and the location of the result. The better the match, the lower the SAD value. Using the PGM image format is very helpful for this method.

Feature matching involves a further step (probably) of thinning each of the found objects, and analysing their features, again, agianst a set of known features. For example - a 2 has a semi circle on top, connect to horizontal line generally the with of the semi circle.

I'm sure many of these steps can probably be completed with some MatLab library funtions.

Tar.Aldarion

Thanks, this is very helpful!

I would have this all done (need it done by noon ) if I could do the cropping like you suggest. The only problem is i cant use something like imcrop as that is manual interaction and a big part of it is that it has to be automated fully. I just have no idea how to get rid of the irl and borders automatically.

can you crop percentages of pictures? like crop the left side by 10%, the right by 4% etc. that could be automatic and robust to scaling.


I tried a method of getting the areas of each object and removing some of the areas, ie the object with the largest area would clearly be the background, then i would be left with the letters. however i dont think this is good or robust for other images.



I guess i would have to try feature or template matching but it's too late for me to try that further. My other methods would work if i could remove the border and IRL automatically dammit!

thanks for the post, very helpful. I must look up thinning as I dont really seem to understand it. is it similar to sobel edge detection and finding the area of each object like I did?

yutta

This problem is crying out machine learning (SVM, HMM, etc.). It's a well-known problem and there is lots of info out on the web. Template matching is an unelegant hack.

mp3guy

If you want a robust solution;

http://yann.lecun.com/exdb/lenet/