Strokelets: A Learned Multi-Scale Mid-Level Representation for Scene Text Recognition

IEEE Transactions on Image Processing

Volumn 25, Issue 6, 2016, Pages 2789-2802

  Bai, Xiang ^a   Yao, Cong ^a   Liu, Wenyu ^a  

^a HUAZHONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

midlevel representation; multi scale representation; natural images; scene text detection; Scene text recognition

Indexed keywords

IMAGE PROCESSING; MATHEMATICAL MODELS;

CHARACTER IDENTIFICATION; DIFFERENT GRANULARITIES; INTERFERENCE FACTOR; MID-LEVEL REPRESENTATION; MULTISCALE REPRESENTATIONS; NATURAL IMAGES; SCENE TEXT; TEXT RECOGNITION;

CHARACTER RECOGNITION;

EID: 84968644486     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2016.2555080     Document Type: Article

References (86)

1. H. Webster, World History. Boston, MA, USA: Heath, 1921.
2. L. V. Remias, A Real-Time Image Understanding System for an Autonomous Mobile Robot. Washington, DC, USA: Storming Media, 1996.
3. H. Yang and C. Meinel, "Content based lecture video retrieval using speech and video text information," IEEE Trans. Learn. technol., vol. 7, no. 2, pp. 142-154, Apr. 2014.
4. J. Feng, "Mobile product search with bag of hash bits and boundary reranking," in Proc. CVPR, 2012, pp. 3005-3012.
5. D. B. Barber, J. D. Redding, T. W. McLain, R. W. Beard, and C. N. Taylor, "Vision-based target geo-location using a fixed-wing miniature air vehicle," J. Intell. Robot. Syst., vol. 47, no. 4, pp. 361-382, 2006.
6. G. N. DeSouza and A. C. Kak, "Vision for mobile robot navigation: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 2, pp. 237-267, Feb. 2002.
7. Y. K. Ham, M. S. Kang, H. K. Chung, R.-H. Park, and G.-T. Park, "Recognition of raised characters for automatic classification of rubber tires," Opt. Eng., vol. 34, no. 1, pp. 102-109, 1995.
8. A. Mishra, K. Alahari, and C. V. Jawahar, "Scene text recognition using higher order language priors," in Proc. BMVC, 2012, pp. 1-11.
9. B. Epshtein, E. Ofek, and Y. Wexler, "Detecting text in natural scenes with stroke width transform," in Proc. CVPR, 2010, pp. 2963-2970.
10. K. Wang, B. Babenko, and S. Belongie, "End-to-end scene text recognition," in Proc. ICCV, 2011, pp. 1457-1464.
11. C. Yao, X. Bai, W. Liu, Y. Ma, and Z. Tu, "Detecting texts of arbitrary orientations in natural images," in Proc. CVPR, 2012, pp. 1083-1090.
12. L. Neumann and J. Matas, "Real-time scene text localization and recognition," in Proc. CVPR, 2012, pp. 3538-3545.
13. T. Novikova, O. Barinova, P. Kohli, and V. Lempitsky, "Large-lexicon attribute-consistent text recognition in natural images," in Proc. ECCV, 2012, pp. 752-765.
14. J. J. Weinman, Z. Butler, D. Knoll, and J. Feild, "Toward integrated scene text reading," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 2, pp. 375-387, Feb. 2013.
15. C. Yi and Y. Tian, "Scene text recognition in mobile applications by character descriptor and structure configuration," IEEE Trans. Image Process., vol. 23, no. 7, pp. 2972-2982, Jul. 2014.
16. X. C. Yin, W. Y. Pei, J. Zhang, and H. W. Hao, "Multi-orientation scene text detection with adaptive clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1930-1937, Sep. 2015.
17. C. Yao, X. Zhang, X. Bai, W. Liu, Y. Ma, and Z. Tu, "Rotation-invariant features for multi-oriented text detection in natural images," PLoS One, vol. 8, no. 8, p. e70173, 2013.
18. A. Mishra, K. Alahari, and C. V. Jawahar, "An MRF model for binarization of natural scene text," in Proc. ICDAR, 2011, pp. 11-16.
19. C. Shi, C. Wang, B. Xiao, Y. Zhang, S. Gao, and Z. Zhang, "Scene text recognition using part-based tree-structured character detection," in Proc. CVPR, 2013, pp. 2961-2968.
20. C. Y. Lee, A. Bhardwaj, W. Di, V. Jagadeesh, and R. Piramuthu, "Region-based discriminative feature pooling for scene text recognition," in Proc. CVPR, 2014, pp. 4050-4057.
21. J. Gall and V. Lempitsky, "Class-specific Hough forests for object detection," in Proc. CVPR, 2009, pp. 1022-1029.
22. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories," in Proc. CVPR, 2005, pp. 524-531.
23. M. Juneja, A. Vedaldi, C. V. Jawahar, and A. Zisserman, "Blocks that shout: Distinctive parts for scene classification," in Proc. CVPR, 2013, pp. 923-930.
24. C. Yao, X. Bai, B. Shi, and W. Liu, "Strokelets: A learned multiscale representation for scene text recognition," in Proc. CVPR, 2014, pp. 4042-4049.
25. S. M. Lucas, A. Panaretos, L. Sosa, A. Tang, S. Wong, and R. Young, "ICDAR 2003 robust reading competitions," in Proc. ICDAR, 2003, pp. 682-687.
26. K. Wang and S. Belongie, "Word spotting in the wild," in Proc. ECCV, 2010, pp. 591-604.
27. L. Neumann and J. Matas, "A method for text localization and recognition in real-world images," in Proc. ACCV, 2010, pp. 770-783.
28. A. Mishra, K. Alahari, and C. V. Jawahar, "Top-down and bottom-up cues for scene text recognition," in Proc. CVPR, 2012, pp. 2687-2694.
29. G. Yildirim, R. Achanta, and S. Süsstrunk, "Text recognition in natural images using multiclass Hough forests," in Proc. VISAPP, 2013, pp. 737-741.
30. V. Goel, A. Mishra, K. Alahari, and C. V. Jawahar, "Whole is greater than sum of parts: Recognizing scene text words," in Proc. ICDAR, 2013, pp. 398-402.
31. J. A. Rodriguez-Serrano and F. Perronnin, "Label embedding for text recognition," in Proc. BMVC, 2013.
32. R. Wang, N. Sang, and C. Gao, "Text detection approach based on confidence map and context information," Neurocomputing, vol. 157, pp. 153-165, Jun. 2015.
33. Y. Zhu, C. Yao, and X. Bai, "Scene text detection and recognition: Recent advances and future trends," Frontier Comput. Sci., vol. 10, no. 1, pp. 19-36, 2016.
34. H. Chen, S. S. Tsai, G. Schroth, D. M. Chen, R. Grzeszczuk, and B. Girod, "Robust text detection in natural images with edge-enhanced maximally stable extremal regions," in Proc. ICIP, 2011, pp. 2609-2612.
35. H. IlKoo and D. HoonKim, "Scene text detection via connected component clustering and nontext filtering," IEEE Trans. Image Process., vol. 22, no. 6, pp. 2296-2305, Jun. 2013.
36. X.-C. Yin, X. Yin, K. Huang, and H.-E. Hao, "Accurate and robust text detection: A step-in for text retrieval in natural scene images," in Proc. SIGIR, 2013, pp. 1091-1092.
37. X. C. Yin, X. Yin, K. Huang, and H. W. Hao, "Robust text detection in natural scene images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 5, pp. 970-983, May 2014.
38. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. CVPR, 2005, pp. 886-893.
39. L. Neumann and J. Matas, "Scene text localization and recognition with oriented stroke detection," in Proc. ICCV, 2013, pp. 97-104.
40. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
41. J. D. Lafferty, A. McCallum, and F. C. N. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. ICML, 2005, pp. 282-289.
42. S. Singh, A. Gupta, and A. A. Efros, "Unsupervised discovery of midlevel discriminative patches," in Proc. ECCV, 2012, pp. 73-86.
43. J. J. Lim, C. L. Zitnick, and P. Dollár, "Sketch tokens: A learned midlevel representation for contour and object detection," in Proc. CVPR, 2013, pp. 3158-3165.
44. X. Wang, B. Wang, X. Bai, W. Liu, and Z. Tu, "Max-margin multipleinstance dictionary learning," in Proc. ICML, 2013, pp. 846-854.
45. Y. Le Cun et al., "Handwritten digit recognition with a back-propagation network," in Proc. NIPS, 1990, pp. 396-404.
46. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. NIPS, 2012, pp. 1097-1105.
47. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1915-1929, Aug. 2013.
48. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, "DeepFace: Closing the gap to human-level performance in face verification," in Proc. CVPR, 2014, pp. 1701-1708.
49. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. CVPR, 2014, pp. 580-587.
50. A. Coates et al., "Text detection and character recognition in scene images with unsupervised feature learning," in Proc. ICDAR, 2011, pp. 440-445.
51. T. Wang, D. J. Wu, A. Coates, and A. Y. Ng, "End-to-end text recognition with convolutional neural networks," in Proc. ICPR, 2012, pp. 3304-3308.
52. A. Bissacco, M. Cummins, Y. Netzer, and H. Neven, "PhotoOCR: Reading text in uncontrolled conditions," in Proc. ICCV, 2013, pp. 785-792.
53. M. Jaderberg, A. Vedaldi, and A. Zisserman, "Deep features for text spotting," in Proc. ECCV, 2014, pp. 512-528.
54. M. Mohri, F. Pereira, and M. Riley, "Weighted finite-state transducers in speech recognition," Comput. Speech Lang., vol. 16, no. 1, pp. 69-88, 2013.
55. S. McCann and D. G. Lowe, "Spatially local coding for object recognition," in Proc. ACCV, 2012, pp. 204-217.
56. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
57. L. Bourdev and J. Malik, "Poselets: Body part detectors trained using 3D human pose annotations," in Proc. ICCV, 2009, pp. 1365-1372.
58. L. Bourdev, S. Maji, T. Brox, and J. Malik, "Detecting people using mutually consistent poselet activations," in Proc. ECCV, 2010, pp. 168-181.
59. B. Leibe, A. Leonardis, and B. Schiele, "Robust object detection with interleaved categorization and segmentation," Int. J. Comput. Vis., vol. 77, nos. 1-3, pp. 259-289, 2008.
60. Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 8, pp. 790-799, Aug. 1995.
61. C. Yao, X. Bai, W. Liu, and L. J. Latecki, "Human detection using learned part alphabet and pose dictionary," in Proc. ECCV, 2014, pp. 251-266.
62. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. CVPR, 2006, pp. 2169-2178.
63. J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial pyramid matching using sparse coding for image classification," in Proc. CVPR, 2009, pp. 1794-1801.
64. J. Donahue et al., "DeCAF: A deep convolutional activation feature for generic visual recognition," in Proc. ICML, 2014, pp. 647-655.
65. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," in Proc. CVPR, 2009, pp. 248-255.
66. J. Deng, A. Berg, S. Satheesh, H. Su, A. Khosla, and L. Fei-Fei. (2012). ImageNet Large Scale Visual Recognition Competition 2012 (ILSVRC2012). [Online]. Available: http://www.image-net.org/challenges/LSVRC/2012/
67. C. Yao, X. Bai, and W. Liu, "A unified framework for multioriented text detection and recognition," IEEE Trans. Image Process., vol. 23, no. 11, pp. 4737-4749, Nov. 2014.
68. R. Minetto, N. Thome, M. Cord, N. J. Leite, and J. Stolfi, "T-HOG: An effective gradient-based descriptor for single line text regions," Pattern Recognit., vol. 46, no. 3, pp. 1078-1090, 2013.
69. A. Shahab, F. Shafait, and A. Dengel, "ICDAR 2011 robust reading competition challenge 2: Reading text in scene images," in Proc. ICDAR, 2011, pp. 1491-1496.
70. S. M. Lucas, "ICDAR 2005 text locating competition results," in Proc. ICDAR, 2005, pp. 80-84.
71. C. Wolf and J.-M. Jolion, "Object count/area graphs for the evaluation of object detection and segmentation algorithms," Int. J. Document Anal. Recognit., vol. 8, no. 4, pp. 280-296, 2006.
72. M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman. (2014). "Reading text in the wild with convolutional neural networks." [Online]. Available: http://arxiv.org/abs/1412.1842
73. M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep structured output learning for unconstrained text recognition," in Proc. ICLR, 2015, pp. 1-10.
74. ABBYY FineReader 9.0, acessed on Apr. 15, 2016. [Online]. Available: http://www.abbyy.com/
75. M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman, "Synthetic data and artificial neural networks for natural scene text recognition," in Proc. NIPS Deep Learn. Workshop, 2014, pp. 1-10.
76. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The Pascal visual object classes (VOC) challenge," Int. J. Comput. Vis., vol. 88, no. 2, pp. 303-338, Sep. 2009.
77. T. E. de Campos, B. R. Babu, and M. Varma, "Character recognition in natural images," in Proc. VISAPP, 2009, pp. 273-280.
78. W. Huang, Y. Qu, and X. Tang, "Robust scene text detection with convolution neural network induced MSER trees," in Proc. ECCV, 2014, pp. 497-511.
79. L. Neumann and J. Matas, "On combining multiple segmentations in scene text recognition," in Proc. ICDAR, 2013, pp. 523-527.
80. C. Yi and Y. Tian, "Text detection in natural scene images by stroke Gabor words," in Proc. ICDAR, 2011, pp. 171-177.
81. Y.-F. Pan, X. Hou, and C.-L. Liu, "A hybrid approach to detect and localize texts in natural scene images," IEEE Trans. Image Process., vol. 20, no. 3, pp. 800-813, Mar. 2011.
82. S. Lee, M. S. Cho, K. Jung, and J. H. Kim, "Scene text extraction with edge constraint and text collinearity," in Proc. ICPR, 2010, pp. 3983-3986.
83. B. Shi, X. Bai, and C. Yao, "Script identification in the wild via discriminative convolutional neural network," Pattern Recognit., vol. 52, pp. 448-458, Apr. 2016.
84. G.-S. Xia, J. Delon, and Y. Gousseau, "Accurate junction detection and characterization in natural images," Int. J. Comput. Vis., vol. 106, no. 1, pp. 31-56, Jan. 2014.
85. J. Ma, H. Zhou, J. Zhao, Y. Gao, J. Jiang, and J. Tian, "Robust feature matching for remote sensing image registration via locally linear transforming," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 12, pp. 6469-6481, Dec. 2015.
86. X. Bai, S. Bai, Z. Zhu, and L. J. Latecki, "3D shape matching via two layer coding," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 12, pp. 2361-2373, Dec. 2015.