Online and offline handwritten Chinese character recognition: A comprehensive study and new benchmark

Pattern Recognition

Volumn 61, Issue , 2017, Pages 348-360

  Zhang, Xu Yao ^a   Bengio, Yoshua ^b   Liu, Cheng Lin ^a,c  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c Chinese Academy of Sciences   (China)

Author keywords

Adaptation; Chinese characters; Convolutional neural network; Directional feature map; Handwriting recognition; Offline; Online

Indexed keywords

CONVOLUTION; NEURAL NETWORKS;

ADAPTATION; CHINESE CHARACTERS; CONVOLUTIONAL NEURAL NETWORK; DIRECTIONAL FEATURE; HANDWRITING RECOGNITION; OFFLINE; ONLINE;

CHARACTER RECOGNITION;

EID: 84988504068     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.08.005     Document Type: Article

References (83)

1. [1] Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y., Modified quadratic discriminant functions and the application to Chinese character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1 (1987), 149–153.
2. [2] Dai, R.-W., Liu, C.-L., Xiao, B.-H., Chinese character recognition: history, status and prospects. Front. Comput. Sci. China 1:2 (2007), 126–136.
3. [3] Liu, C.-L., Jaeger, S., Nakagawa, M., Online recognition of Chinese characters: the state-of-the-art. IEEE Trans. Pattern Anal. Mach. Intell. 26:2 (2004), 198–213.
4. [4] Liu, C.-L., Koga, M., Fujisawa, H., Lexicon-driven segmentation and recognition of handwritten character strings for Japanese address reading. IEEE Trans. Pattern Anal. Mach. Intell. 24:11 (2002), 1425–1437.
5. [5] Zhou, X.-D., Wang, D.-H., Tian, F., Liu, C.-L., Nakagawa, M., Handwritten Chinese/Japanese text recognition using semi-Markov conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 35:10 (2013), 2484–2497.
6. [6] Wang, Q.-F., Yin, F., Liu, C.-L., Handwritten Chinese text recognition by integrating multiple contexts. IEEE Trans. Pattern Anal. Mach. Intell. 34:8 (2012), 1469–1481.
7. [7] Liu, C.-L., Sako, H., Fujisawa, H., Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings. IEEE Trans. Pattern Anal. Mach. Intell. 26:11 (2004), 1395–1407.
8. [8] C.-L. Liu, F. Yin, D.-H. Wang, Q.-F. Wang, Chinese handwriting recognition contest 2010, in: Proceedings of Chinese Conference on Pattern Recognition (CCPR), 2010.
9. [9] C.-L. Liu, F. Yin, Q.-F. Wang, D.-H. Wang, ICDAR 2011 Chinese handwriting recognition competition, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2011, pp. 1464–1469.
10. [10] F. Yin, Q.-F. Wang, X.-Y. Zhang, C.-L. Liu, ICDAR 2013 Chinese handwriting recognition competition, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2013, pp. 1095–1101.
11. [11] D. Ciresan, U. Meier, J. Masci, L. Gambardella, J. Schmidhuber, Flexible, high performance convolutional neural networks for image classification, in: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), 2011, pp. 1237–1242.
12. [12] B. Graham, Sparse arrays of signatures for online character recognition, 2013, 〈arXiv:1308.0371〉.
13. [13] Bengio, Y., Courville, A., Vincent, P., Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1798–1828.
14. [14] W. Yang, L. Jin, Z. Xie, Z. Feng, Improved deep convolutional neural network for online handwritten Chinese character recognition using domain-specific knowledge, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015.
15. [15] Z. Zhong, L. Jin, Z. Xie, High performance offline handwritten Chinese character recognition using GoogLeNet and directional feature maps, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015.
16. [16] J. Tsukumo and H. Tanaka, Classification of handprinted Chinese characters using non-linear normalization and correlation methods, in: Proceedings of International Conference on Pattern Recognition (ICPR), 1988, pp. 168–171.
17. [17] C.-L. Liu, H. Sako, H. Fujisawa, Handwritten Chinese character recognition: alternatives to nonlinear normalization, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2003, pp. 524–528.
18. [18] Liu, C.-L., Marukawa, K., Pseudo two-dimensional shape normalization methods for handwritten Chinese character recognition. Pattern Recognit. 38:12 (2005), 2242–2255.
19. [19] Liu, C.-L., Yin, F., Wang, D.-H., Wang, Q.-F., Online and offline handwritten Chinese character recognition: benchmarking on new databases. Pattern Recognit. 46:1 (2013), 155–162.
20. [20] M. Okamoto, A. Nakamura, K. Yamamoto, On-line handwriting character recognition method with directional features and directional change features, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 1997, pp. 926–930.
21. [21] Z. Bai, Q. Huo, A study on the use of 8-directional features for online handwritten Chinese character recognition, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2005, pp. 262–266.
22. [22] Liu, C.-L., Normalization-cooperated gradient feature extraction for handwritten character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29:8 (2007), 1465–1469.
23. [23] A. Krizhevsky, I. Sutskever, G. Hinton, ImageNet classification with deep convolutional neural networks, in: Proceedings of Advances in Neural Information Processing Systems (NIPS), 2012, pp. 1097–1105.
24. [24] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proceedings of Computer Vision and Pattern Recognition (CVPR), 2015.
25. [25] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in: Proceedings of International Conference on Learning Representations (ICLR), 2015.
26. [26] Sarkar, P., Nagy, G., Style consistent classification of isogenous patterns. IEEE Trans. Pattern Anal. Mach. Intell. 27:1 (2005), 88–98.
27. [27] Connell, S., Jain, A., Writer adaptation for online handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 24:3 (2002), 329–346.
28. [28] Zhang, X.-Y., Liu, C.-L., Writer adaptation with style transfer mapping. IEEE Trans. Pattern Anal. Mach. Intell. 35:7 (2013), 1773–1787.
29. [29] C. Suen, M. Berthod, S. Mori, Automatic recognition of handprinted characters: the state of the art, in: Proceedings of IEEE, vol. 68, no. 4, 1980, pp. 469–487.
30. [30] Tappert, C., Suen, C., Wakahara, T., The state of the art in online handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 12:8 (1990), 787–808.
31. [31] Hildebrandt, T., Liu, W., Optical recognition of handwritten Chinese characters: advances since 1980. Pattern Recognit. 26:2 (1993), 205–225.
32. [32] Plamondon, R., Srihari, S., Online and offline handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22:1 (2000), 63–84.
33. [33] Fujisawa, H., Forty years of research in character and document recognition: an industrial perspective. Pattern Recognit. 41:8 (2008), 2435–2446.
34. [34] Hinton, G., Salakhutdinov, R., Reducing the dimensionality of data with neural networks. Science 313:5786 (2006), 504–507.
35. [35] LeCun, Y., Bengio, Y., Hinton, G., Deep learning. Nature 521:7553 (2015), 436–444.
36. [36] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, in: Proceedings of IEEE, vol. 86, no. 11, 1998, pp. 2278–2324.
37. [37] D. Ciresan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image classification, in: Proceedings of Computer Vision and Pattern Recognition (CVPR), 2012, pp. 3642–3649.
38. [38] D. Ciresan, J. Schmidhuber, Multi-column deep neural networks for offline handwritten Chinese character classification, 2013, 〈arXiv:1309.0261〉.
39. [39] B. Graham, Spatially-sparse convolutional neural networks, 2014, 〈arXiv:1409.6070〉.
40. [40] C. Wu, W. Fan, Y. He, J. Sun, S. Naoi, Handwritten character recognition by alternately trained relaxation convolutional neural network, in: Proceedings of International Conference on Frontiers in Handwriting Recognition (ICFHR), 2014, pp. 291–296.
41. [41] L. Chen, S. Wang, W. Fan, J. Sun, S. Naoi, Beyond human recognition: A CNN-based framework for handwritten character recognition, in: Proceedings of Asian Conference on Pattern Recognition (ACPR), 2015.
42. [42] Kim, I.-J., Xie, X., Handwritten Hangul recognition using deep convolutional neural networks. Int. J. Doc. Anal. Recognit. 18:1 (2015), 1–13.
43. [43] W. Yang, L. Jin, D. Tao, Z. Xie, Z. Feng, DropSample: a new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition, 2015, 〈arXiv:1505.05354〉.
44. [44] W. Yang, L. Jin, M. Liu, Character-level Chinese writer identification using path signature feature, dropstroke and deep CNN, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015.
45. [45] Y. Wang, X. Li, C. Liu, X. Ding, Y. Chen, An MQDF-CNN hybrid model for offline handwritten Chinese character recognition, in: Proceedings of International Conference on Frontiers in Handwriting Recognition (ICFHR), 2014.
46. [46] M. He, S. Zhang, H. Mao, L. Jin, Recognition confidence analysis of handwritten Chinese character with CNN, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015.
47. [47] M. Wang, Y. Chen, X. Wang, Recognition of handwritten characters in Chinese legal amounts by stacked autoencoders, in: Proceedings of International Conference on Pattern Recognition (ICPR), 2014.
48. [48] M. Jaderberg, A. Vedaldi, A. Zisserman, Deep features for text spotting, in: Proceedings of European Conference on Computer Vision (ECCV), 2014.
49. [49] Bengio, Y., LeCun, Y., Nohl, C., Burges, C., LeRec: a NN/HMM hybrid for on-line handwriting recognition. Neural Comput. 7:6 (1995), 1289–1303.
50. [50] A. Graves, J. Schmidhuber, Offline handwriting recognition with multidimensional recurrent neural networks, in: Proceedings of Advances in Neural Information Processing Systems (NIPS), 2008, pp. 545–552.
51. [51] R. Messina, J. Louradour, Segmentation-free handwritten Chinese text recognition with LSTM-RNN, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015.
52. [52] P. He, W. Huang, Y. Qiao, C. Loy, X. Tang, Reading scene text in deep convolutional sequences, 2015, 〈arXiv:1506.04395〉.
53. [53] B. Shi, X. Bai, C. Yao, An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition, 2015, 〈arXiv:1507.05717〉.
54. [54] Du, J., Huo, Q., A discriminative linear regression approach to adaptation of multi-prototype based classifiers and its applications for Chinese OCR. Pattern Recognit. 46:8 (2013), 2313–2322.
55. [55] B. Li, L. Peng, J. Ji, Historical Chinese character recognition method based on style transfer mapping, in: International Workshop on Document Analysis Systems (DAS), 2014, pp. 96–100.
56. [56] J. Feng, L. Peng, F. Lebourgeois, Gaussian process style transfer mapping for historical Chinese character recognition, in: Proceedings of SPIE, Document Recognition and Retrieval, 2015.
57. [57] Liu, C.-L., Nakagawa, M., Evaluation of prototype learning algorithms for nearest neighbor classifier in application to handwritten character recognition. Pattern Recognit. 34:3 (2001), 601–615.
58. [58] J. Hoffman, E. Tzeng, J. Donahue, Y. Jia, K. Saenko, T. Darrell, One-shot adaptation of supervised deep convolutional models, 2013, 〈arXiv:1312.6204〉.
59. [59] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, T. Darrell, DeCAF: a deep convolutional activation feature for generic visual recognition, in: Proceedings of International Conference on Machine Learning (ICML), 2014.
60. [60] Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H., Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern Recognit. 36:10 (2003), 2271–2285.
61. [61] C.-L. Liu, X.-D. Zhou, Online Japanese character recognition using trajectory-based normalization and direction feature extraction, in: Proceedings of International Workshop on Frontiers in Handwriting Recognition (IWFHR), 2006, pp. 217–222.
62. [62] K. Ding, G. Deng, L. Jin, An investigation of imaginary stroke technique for cursive online handwriting Chinese character recognition, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2009, pp. 531–535.
63. [63] Heiden, R., Gren, F., The box-cox metric for nearest neighbor classification improvement. Pattern Recognit. 30:2 (1997), 273–279.
64. [64] Zhang, X.-Y., Liu, C.-L., Evaluation of weighted Fisher criteria for large category dimensionality reduction in application to Chinese handwriting recognition. Pattern Recognit. 46:9 (2013), 2599–2611.
65. [65] C.-L. Liu, High accuracy handwritten Chinese character recognition using quadratic classifiers with discriminative feature extraction, in: Proceedings of International Conference on Pattern Recognition (ICPR), 2006, pp. 942–945.
66. [66] Liu, C.-L., Sako, H., Fujisawa, H., Discriminative learning quadratic discriminant function for handwriting recognition. IEEE Trans. Neural Netw. 15:2 (2004), 430–444.
67. [67] X.-Y. Zhang, C.-L. Liu, Locally smoothed modified quadratic discriminant function, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2013, pp. 8–12.
68. [68] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1 (2014), 1929–1958.
69. [69] A. Maas, A. Hannun, A. Ng, Rectifier nonlinearities improve neural network acoustic models, in: Proceedings of International Conference on Machine Learning (ICML), 2013.
70. [70] K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, 2015, 〈arXiv:1502.01852〉.
71. [71] S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, 2015, 〈arXiv:1502.03167〉.
72. [72] B. Xu, N. Wang, T. Chen, M. Li, Empirical evaluation of rectified activations in convolution network, 2015, 〈arXiv:1505.00853〉.
73. [73] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2010, pp. 249–256.
74. [74] Rumelhart, D., Hinton, G., Williams, R., Learning representations by back-propagating errors. Nature 323:9 (1986), 533–536.
75. [75] M. Long, Y. Cao, J. Wang, M. Jordan, Learning transferable features with deep adaptation networks, in: Proceedings of International Conference on Machine Learning (ICML), 2015.
76. [76] C.-L. Liu, F. Yin, D.-H. Wang, Q.-F. Wang, CASIA online and offline Chinese handwriting databases, in: Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2011, pp. 37–41.
77. [77] M. Zeiler, AdaDelta: an adaptive learning rate method, 2012, 〈arXiv:1212.5701〉.
78. [78] D. Kingma, J. Ba, Adam: a method for stochastic optimization, in: Proceedings of International Conference on Learning Representations (ICLR), 2015.
79. [79] F. Bastien, P. Lamblin, R. Pascanu, J. Bergstra, I. Goodfellow, A. Bergeron, N. Bouchard, D. Warde-Farley, Y. Bengio, Theano: new features and speed improvements, in: NIPS Deep Learning Workshop, 2012.
80. [80] J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, Y. Bengio, Theano: a CPU and GPU math expression compiler, in: Proceedings of Python for Scientific Computing Conference (SciPy), 2010.
81. [81] C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, Z. Tu, Deeply-supervised nets, in: Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), 2015, pp. 562–570.
82. [82] A. Graves, Supervised sequence labeling with recurrent neural networks, in: Studies in Computational Intelligence, Springer, 2012.
83. [83] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2015, 〈arXiv:1512.03385〉.