PyConZA 2015: "Numberplate recognition using python, opencv and some other magic." by Kobus Wolvaardt

How to build/construct a numberplate recognition system for you or your neighbourhood. Using Python and OpenCV to detect and recognize numberplates from images, check and log it in a DB. Some challenges in getting usable numberplates at night and how the software was designed will be covered.

Transcript

1. ANPR
2. How I got started • Got involved with neighbourhood watch • Cameras installation came up and I had the best CV: – Under the age of 70 – Own a computer • Got a test camera
3. WIFI • Limits Camera placement • Solve wifi or get fiber • Resources: – Join a local WUG – Online tutorials – Buy a Mikrotik or Ubiquity kit and play with it
4. Cameras • Many good options • Be careful of cheap cameras. – Thermal drift, lightning sensitivity, old codes eat BW • Numberplates at night: – Good sensor – Shutter configurable
5. Plates at night • Car speed and angle • Light • Shutterspeed • Sensor quality
6. Early evening
7. Pitch dark
8. ANPR • I am not a computer vision expert • CDD was used for the development – Wrote a c++/python hybrid many years ago (lost source code) – Want pure python rewrite – Promise you'll give talk at pycon (Conference Driven Development) • ANPR Basic process: – Pre-process image – Localize numberplate(s) in image – De-skew or rotate plate region – Segment characters – Recognize characters – Grammar / post processing
9. ANPR • Pre-process – Equalisation or normalisation – Colour space preparation • Numberplate localization – Edge detection with vertical and horizontal count – Tophat Morphological Operations – Haar (and other trainable) classifiers, usually to find faces – Character detection (what I used)
10. Edge detection localiation
11. Tophat morphology localiation rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 6)) tophat = cv2.morphologyEx(gray, cv2.MORPH_TOPHAT, rectKernel)
12. Character detection localiation • Numberplate characteristics: – High contrast in plate region – Similar sized closely grouped characters – Horizontally oriented • Do adaptive threshold: – thresh = cv2.adaptiveThreshold(image, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, self.config["thesh_window"], self.config["thesh_offset"]) • Detect all blobs (CCA or contour finding) – (cnts, _) = cv2.findContours(self.thresh.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) • Filter blobs based on height and weight • Group filter blobs
13. Adaptive Threshold
14. Contours detected in threshold
15. Size and aspect filtered
16. Grouped and filtered Row 1 Row 2 Row 3 Row 4 0 2 4 6 8 10 12 Column 1 Column 2 Column 3
17. Plate de-skew and rotate • De-skew: – Find edges of numberplate and four corners – Edges sometimes invisible M = cv2.getPerspectiveTransform(rect, dst) warped = cv2.warpPerspective(image, M, (maxWidth, maxHeight)) • Rotate: – Rotate gave me more consistant results – No need for numberplate edge – Use characters to determine plate angle M = cv2.getRotationMatrix2D((cX, cY), degrees, 1.0) rotated = cv2.warpAffine(plateregion, M, (w, h))
18. Character segmentation • Adaptive Threshold • CCA (Contour) • Filter (stricter on height variance) • Copy and resize characters
19. Character Recognition • Compare against averaged samples of all characters – Not accurate • Train ANN – Need lots of data and time to train (900 input layer and 36 output) – Not great at generalisation • Deep Belief Networks – Something to try next, seems to be a good tool import nolearn • Other tools – Tesserac does not like getting 30x30pixel single letter images – Suggestion would be welcome
20. Questions • Code samples and ideas came from http://www.pyimagesearch.com/ check out Adrian's site if you want to learn computer vision • Questions?