Stroke Width Transform

A novel image operator that seeks to find the value of stroke width for each image pixel, and its use on the task of text detection in natural images.

Transcript

1. Detecting Text in Natural Scenes with Stoke Width Transform
2. Text Detection • Image retrieval • Help for visually impaired • Translation • Business geocoding • Robotic navigation
3. Natural OCR ≠ Traditional OCR
4. Natural Images • Large variability of fonts • Varied illumination • Image noise • Blur • No page layout
5. Text Detection important step in OCR
6. Region Based Text Detection • Pixels aggregated based on local features • Connected components extracted, deemed letter candidates • Letter candidates grouped into line text, geometric checks applied
7. The Approach Image SWT (local feature) Connected Component Filter Letters Line Aggregation Filter Lines OCR
8. Feature: Stroke Width WMT
9. Computing Stroke Width
10. Computing Stroke Width
11. Computing Stroke Width 3 2 1
12. Computing Stroke Width 3 3 3
13. Computing Stroke Width 4 4 4 4 3 4 3 4 3 4 4
14. Computing Stroke Width 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 3 4 4 3 3 3 4 4 4 4 4 4 4 4 4 4 4 3 3 3 4 4 4 4 4 4 4 4 4 4 4 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
15. Complex Situations 30
16. Complex Situations 30
17. Complex Situations 7
18. Complex Situations 7 40
19. Complex Situations 7 7
20. Connected Components • Pixels grouped using stroke width • Stoke width ratio < 3.0
21. Geometric Checks • Reject components high stroke width variance • Limit aspect ratio of components • Components too large or too small ignored
22. Group Into Lines W O R D A z C E
23. Issues • Gradient direction will not always point towards opposite stroke edge point • Will travel in opposite direction than intended
25. Solution • SWT must be done twice • Dark text on light background • Light text on dark background • Inverse of gradient direction
26. Demo
27. Future • Detect reading in eye tracking data • Use gaze data to bound text detection to a window • Preform text detection • Preform OCR • Use for different purposes
28. Uses • Finish reading • Google search • Dictation • Save for later