Improving patch-based scene text script identification with ensembles of conjoined networks

Pattern Recognition

Volumn 67, Issue , 2017, Pages 85-96

  Gomez, Lluis ^a   Nicolaou, Anguelos ^a   Karatzas, Dimosthenis ^a  

^a Barcelona Perception Computing Lab   (Spain)

Author keywords

Convolutional neural networks; Ensemble of conjoined networks; Multi language OCR; Scene text understanding; Script identification

Indexed keywords

NEURAL NETWORKS;

CLASSIFICATION FRAMEWORK; CLASSIFICATION SCHEME; CONVOLUTIONAL NEURAL NETWORK; KEY CHARACTERISTICS; LEARNING PROCEDURES; MULTI LANGUAGES; SCENE TEXT; SCRIPT IDENTIFICATION;

ASPECT RATIO;

EID: 85015987251     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2017.01.032     Document Type: Article

References (68)

1. [1] Unnikrishnan, R., Smith, R., Combined script and page orientation estimation using the tesseract ocr engine. Proceedings of the International Workshop on Multilingual OCR, 2009, ACM, 6.
2. [2] Ghosh, D., Dube, T., Shivaprasad, A.P., Script recognitiona review. Pattern Anal Mach Intell, IEEE Trans 32:12 (2010), 2142–2161.
3. [3] Pal, U., Chaudhuri, B., Indian script character recognition: a survey. Pattern Recogn. 37:9 (2004), 1887–1899.
4. [4] Bissacco, A., Cummins, M., Netzer, Y., Neven, H., Photoocr: reading text in uncontrolled conditions. Proceedings of the IEEE International Conference on Computer Vision, 2013, 785–792.
5. [5] Jaderberg, M., Vedaldi, A., Zisserman, A., Deep features for text spotting. Computer Vision–ECCV 2014, 2014, Springer, 512–528.
6. [6] Neumann, L., Matas, J., Real-time lexicon-free scene text localization and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38:9 (2016), 1872–1885.
7. [7] Shi, B., Yao, C., Zhang, C., Guo, X., Huang, F., Bai, X., Automatic script identification in the wild. Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, IEEE, 531–535.
8. [8] Shi, B., Bai, X., Yao, C., Script identification in the wild via discriminative convolutional neural network. Pattern Recogn. 52 (2016), 448–458.
9. [9] Nicolaou, A., Bagdanov, A.D., Gomez-Bigorda, L., Karatzas, D., Visual script and language recognition. DAS, 2016.
10. [10] Gomez-Bigorda, L., Karatzas, D., A fine-grained approach to scene text script identification. DAS, 2016.
11. [11] Tian, S., Bhattacharya, U., Lu, S., Su, B., Wang, Q., Wei, X., Lu, Y., Tan, C.L., Multilingual scene character recognition with co-occurrence of histogram of oriented gradients. Pattern Recogn. 51 (2016), 125–134.
12. [12] Gllavata, J., Freisleben, B., Script recognition in images with complex backgrounds. Signal Processing and Information Technology, 2005. Proceedings of the Fifth IEEE International Symposium on, 2005, IEEE, 589–594.
13. [13] Shivakumara, P., Yuan, Z., Zhao, D., Lu, T., Tan, C.L., New gradient-spatial-structural features for video script identification. Comput. Vision Image Understanding 130 (2015), 35–53.
14. [14] Coates, A., Ng, A.Y., Lee, H., An analysis of single-layer networks in unsupervised feature learning. International Conference on Artificial Intelligence and Statistics, 2011, 215–223.
15. [15] Boiman, O., Shechtman, E., Irani, M., In defense of nearest-neighbor based image classification. Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 2008, IEEE, 1–8.
16. [16] Spitz, A.L., Ozaki, M., Palace: a multilingual document recognition system. Document Analysis Systems, vol. 1, 1995, World Scientific, Singapore, 16–37.
17. [17] Spitz, A.L., Determination of the script and language content of document images. Pattern Anal. Mach. Intell., IEEE Trans. 19:3 (1997), 235–245.
18. [18] Lee, D., Nohl, C.R., Baird, H.S., Language identification in complex, unoriented, and degraded document images. Ser. Mach. Percept. Artif. Intell. 29 (1998), 17–39.
19. [19] Waked, B., Bergler, S., Suen, C., Khoury, S., Skew detection, page segmentation, and script classification of printed document images. Systems, Man, and Cybernetics, 1998. 1998 IEEE International Conference on, vol. 5, 1998, IEEE, 4470–4475.
20. [20] Chaudhury, S., Sheth, R., Trainable script identification strategies for indian languages. Document Analysis and Recognition, 1999. ICDAR’99. Proceedings of the Fifth International Conference on, 1999, IEEE, 657–660.
21. [21] Hochberg, J., Kerns, L., Kelly, P., Thomas, T., Automatic script identification from images using cluster-based templates. Document Analysis and Recognition, 1995., Proceedings of the Third International Conference on, vol. 1, 1995, IEEE, 378–381.
22. [22] Wood, S.L., Yao, X., Krishnamurthi, K., Dang, L., Language identification for printed text independent of segmentation. Image Processing, 1995. Proceedings., International Conference on, vol. 3, 1995, IEEE, 428–431.
23. [23] Tan, T., Rotation invariant texture features and their use in automatic script identification. Pattern Anal. Mach. Intell., IEEE Trans. 20:7 (1998), 751–756.
24. [24] Chan, W., Coghill, G., Text analysis using local energy. Pattern Recogn. 34:12 (2001), 2523–2532.
25. [25] Pan, W., Suen, C.Y., Bui, T.D., Script identification using steerable gabor filters. Document Analysis and Recognition, 2005. Proceedings. Eighth International Conference on, 2005, IEEE, 883–887.
26. [26] Ferrer, M.A., Morales, A., Pal, U., Lbp based line-wise script identification. Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, 2013, IEEE, 369–373.
27. [27] Jain, A.K., Zhong, Y., Page segmentation using texture analysis. Pattern Recogn. 29:5 (1996), 743–770.
28. [28] Chi, Z., Wang, Q., Siu, W.-C., Hierarchical content classification and script determination for automatic document image processing. Pattern Recogn. 36:11 (2003), 2483–2500.
29. [29] Hennig, A., Sherkat, N., Exploiting zoning based on approximating splines in cursive script recognition. Pattern Recogn. 35:2 (2002), 445–454.
30. [30] Schenk, J., Lenz, J., Rigoll, G., Novel script line identification method for script normalization and feature extraction in on-line handwritten whiteboard note recognition. Pattern Recogn. 42:12 (2009), 3383–3393.
31. [31] Zhu, G., Yu, X., Li, Y., Doermann, D., Language identification for handwritten document images using a shape codebook. Pattern Recogn. 42:12 (2009), 3184–3191.
32. [32] Basu, S., Das, N., Sarkar, R., Kundu, M., Nasipuri, M., Basu, D.K., A novel framework for automatic sorting of postal documents with multi-script address blocks. Pattern Recogn. 43:10 (2010), 3507–3521.
33. [33] Zhong, G., Cheriet, M., Tensor representation learning based image patch analysis for text identification and recognition. Pattern Recogn. 48:4 (2015), 1211–1224.
34. [34] Sharma, N., Chanda, S., Pal, U., Blumenstein, M., Word-wise script identification from video frames. Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, 2013, IEEE, 867–871.
35. [35] Phan, T.Q., Shivakumara, P., Ding, Z., Lu, S., Tan, C.L., Video script identification based on text lines. Document Analysis and Recognition (ICDAR), 2011 International Conference on, 2011, IEEE, 1240–1244.
36. [36] Shivakumara, P., Sharma, N., Pal, U., Blumenstein, M., Tan, C.L., Gradient-angular-features for word-wise video script identification. 2014 22nd International Conference on Pattern Recognition (ICPR), 2014, IEEE, 3098–3103.
37. [37] Sharma, N., Mandal, R., Sharma, R., Pal, U., Blumenstein, M., Bag-of-visual words for word-wise video script identification: A study. Neural Networks (IJCNN), 2015 International Joint Conference on, 2015, IEEE, 1–7.
38. [38] Sharma, N., Mandal, R., Sharma, R., Pal, U., Blumenstein, M., Icdar2015 competition on video script identification (cvsi 2015). Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, IEEE, 1196–1200.
39. [39] Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A., Sequential deep learning for human action recognition. International Workshop on Human Behavior Understanding, 2011, Springer, 29–39.
40. [40] Ji, S., Xu, W., Yang, M., Yu, K., 3d convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35:1 (2013), 221–231.
41. [41] Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L., Large-scale video classification with convolutional neural networks. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, 1725–1732.
42. [42] Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 1097–1105.
43. [43] Bromley, J., Bentz, J.W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., Säckinger, E., Shah, R., Signature verification using a ǣsiameseǥ time delay neural network. Int. J. Pattern Recogn. Artif. Intell. 7:04 (1993), 669–688.
44. [44] Pan, Y.-F., Hou, X., Liu, C.-L., Text localization in natural scene images based on conditional random field. Document Analysis and Recognition, 2009. ICDAR’09. 10th International Conference on, 2009, IEEE, 6–10.
45. [45] Yao, C., Bai, X., Liu, W., Ma, Y., Tu, Z., Detecting texts of arbitrary orientations in natural images. Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 2012, IEEE, 1083–1090.
46. [46] de Campos, T.E., Babu, B.R., Varma, M., Character recognition in natural images. VISAPP (2), 2009, 273–280.
47. [47] Kumar, D., Prasad, M., Ramakrishnan, A., Multi-script robust reading competition in icdar 2013. Proceedings of the 4th International Workshop on Multilingual OCR, 2013, ACM, 14.
48. [48] Lee, S., Cho, M.S., Jung, K., Kim, J.H., Scene text extraction with edge constraint and text collinearity. Pattern Recognition (ICPR), 2010 20th International Conference on, 2010, IEEE, 3983–3986.
49. [49] Karatzas, D., Robles, S., Gomez, L., An on-line platform for ground truthing and performance evaluation of text extraction systems. Document Analysis Systems (DAS), 2014 11th IAPR International Workshop on, 2014, IEEE, 242–246.
50. [50] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T., Caffe: Convolutional architecture for fast feature embedding. Proceedings of the ACM International Conference on Multimedia, 2014, ACM, 675–678.
51. [51] Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y., What is the best multi-stage architecture for object recognition?. Computer Vision, 2009 IEEE 12th International Conference on, 2009, IEEE, 2146–2153.
52. [52] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1 (2014), 1929–1958.
53. [53] Lowe, D.G., Object recognition from local scale-invariant features. Computer vision, 1999. The Proceedings of the Seventh IEEE International Conference on, vol. 2, 1999, IEEE, 1150–1157.
54. [54] Vedaldi, A., Fulkerson, B., VLFeat: an open and portable library of computer vision algorithms. 2008. ( http://www.vlfeat.org/).
55. [55] Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J., LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9 (2008), 1871–1874.
56. [56] Nicolaou, A., Bagdanov, A.D., Liwicki, M., Karatzas, D., Sparse radial sampling lbp for writer identification. Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, IEEE, 716–720.
57. [57] McNemar, Q., Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12:2 (1947), 153–157.
58. [58] Gomez, L., Karatzas, D., A fast hierarchical method for multi-script and arbitrary oriented scene text extraction. arXiv preprint arXiv:1407.7504, 2014.
59. [59] Matas, J., Chum, O., Urban, M., Pajdla, T., Robust wide-baseline stereo from maximally stable extremal regions. Image Vision Comput. 22:10 (2004), 761–767.
60. [60] Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A., The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 111:1 (2015), 98–136.
61. [61] Karatzas, D., Gomez-Bigorda, L., Nicolaou, A., Ghosh, S., Bagdanov, A., Iwamura, M., Matas, J., Neumann, L., Chandrasekhar, V.R., Lu, S., et al. Icdar 2015 competition on robust reading. Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, IEEE, 1156–1160.
62. [62] Smith, R., An overview of the tesseract ocr engine. ICDAR, 2007, IEEE, 629–633.
63. [63] Milyaev, S., Barinova, O., Novikova, T., Kohli, P., Lempitsky, V., Image binarization for end-to-end text understanding in natural images. Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, 2013, IEEE, 128–132.
64. [64] Gómez, L., Karatzas, D., Scene text recognition: No country for old men?. Computer Vision-ACCV 2014 Workshops, 2014, Springer, 157–168.
65. [65] Milyaev, S., Barinova, O., Novikova, T., Kohli, P., Lempitsky, V., Fast and accurate scene text understanding with image binarization and off-the-shelf ocr. Int. J. Doc. Anal. Recogn. (IJDAR) 18:2 (2015), 169–182.
66. [66] Wang, K., Babenko, B., Belongie, S., End-to-end scene text recognition. Computer Vision (ICCV), 2011 IEEE International Conference on, 2011, IEEE, 1457–1464.
67. [67] Karatzas, D., Shafait, F., Uchida, S., Iwamura, M., Gomez i Bigorda, L., Robles Mestre, S., Mas, J., Fernandez Mota, D., Almazan Almazan, J., de las Heras, L.-P., Icdar 2013 robust reading competition. Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, 2013, IEEE, 1484–1493.
68. [68] Yousfi, S., Berrani, S.-A., Garcia, C., Alif: A dataset for arabic embedded text recognition in tv broadcast. Document Analysis and Recognition (ICDAR), 2015 13th International Conference on, 2015, IEEE, 1221–1225.