Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition

Pattern Recognition

Volumn 72, Issue , 2017, Pages 72-81

  Xiao, Xuefeng ^a   Jin, Lianwen ^a   Yang, Yafeng ^a   Yang, Weixin ^a   Sun, Jun ^b   Chang, Tianhai ^a  

^a SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

^b Fujitsu Research and Development Center Co Ltd   (China)

Author keywords

CNN acceleration; CNN compression; Convolutional neural network; Handwritten Chinese character recognition

Indexed keywords

CONVOLUTION; COSTS; DROPS; NEURAL NETWORKS; PROBLEM SOLVING;

BASELINE MODELS; CHARACTER IMAGES; COMPUTATIONAL COSTS; CONVOLUTIONAL NEURAL NETWORK; HANDWRITTEN CHINESE CHARACTER RECOGNITION; OFFLINE HANDWRITTEN CHINESE CHARACTER RECOGNITION; STATE OF THE ART; STORAGE CAPACITY;

CHARACTER RECOGNITION;

EID: 85027531484     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2017.06.032     Document Type: Article

References (44)

1. Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y., Modified quadratic discriminant functions and the application to Chinese character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 9:1 (1987), 149–153.
2. Jin, L., Huang, J., Yin, J., He, Q., Deformation transformation for handwritten Chinese character shape correction. Proceedings of Advances in Multimodal Interfaces (ICMI), 2000, 450–457.
3. Dai, R., Liu, C., Xiao, B., Chinese character recognition: history, status and prospects. Front. Comput. Sci. China 1:2 (2007), 126–136.
4. Long, T., Jin, L., Building compact MQDF classifier for large character set recognition by subspace distribution sharing. Pattern Recognit. 41:9 (2008), 2916–2925.
5. Liu, C., Yin, F., Wang, D., Wang, Q., Online and offline handwritten Chinese character recognition: benchmarking on new databases. Pattern Recognit. 46:1 (2013), 155–162.
6. Zhang, X., Bengio, Y., Liu, C., Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark. Pattern Recognit. 61 (2017), 348–360.
7. Yin, F., Wang, Q., Zhang, X., Liu, C., ICDAR 2013 Chinese handwriting recognition competition. Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2013, 1464–1470.
8. LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E., Jackel, L.D., Handwritten digit recognition with a back-propagation network. Proceedings of Advances in Neural Information Processing Systems (NIPS), 1990, Morgan-Kaufmann, 396–404.
9. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. Proc. IEEE 86:11 (1998), 2278–2324.
10. Ciresan, D.C., Meier, U., Multi-column deep neural networks for offline handwritten Chinese character classification. Proceedings of International Joint Conference on Neural Networks (IJCNN), 2015, 1–6.
11. Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R., Exploiting linear structure within convolutional networks for efficient evaluation. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2014, 1269–1277.
12. Jaderberg, M., Vedaldi, A., Zisserman, A., Speeding up convolutional neural networks with low rank expansions. Proceedings of British Machine Vision Conference (BMVC), 2014.
13. Zhang, X., Zou, J., Ming, X., He, K., Sun, J., Efficient and accurate approximations of nonlinear convolutional networks. Proceedings of Computer Vision and Pattern Recognition (CVPR), 2015, 1984–1992.
14. Zhang, X., Zou, J., He, K., Sun, J., Accelerating very deep convolutional networks for classification and detection. IEEE Trans. Pattern Anal. Mach. Intell. 38:10 (2016), 1943–1955.
15. Han, S., Pool, J., Tran, J., Dally, W.J., Learning both weights and connections for efficient neural network. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2015, 1135–1143.
16. Guo, Y., Yao, A., Chen, Y., Dynamic network surgery for efficient dnns. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2016, 1379–1387.
17. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., Going deeper with convolutions. Proceedings of Computer Vision and Pattern Recognition (CVPR), 2015, 1–9.
18. He, K., Zhang, X., Ren, S., Sun, J., Deep residual learning for image recognition. Proceedings of Computer Vision and Pattern Recognition (CVPR), 2016, 770–778.
19. Wu, C., Fan, W., He, Y., Sun, J., Naoi, S., Handwritten character recognition by alternately trained relaxation convolutional neural network. Proceedings of International Conference on Frontiers in Handwriting Recognition (ICFHR), 2014, 291–296.
20. Zhong, Z., Jin, L., Xie, Z., High performance offline handwritten Chinese character recognition using GoogLeNet and directional feature maps. Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2015, 846–850.
21. Zhou, S., Wu, J., Wu, Y., Zhou, X., Exploiting local structures with the Kronecker layer in convolutional networks. CoRR, abs/1512.09194, 2015.
22. Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition. Proceedings of International Conference on Learning Representations (ICLR), 2014.
23. Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Proceedings of Advances in Neural Information Processing Systems (NIPS), 2012, 1106–1114.
24. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R.B., Guadarrama, S., Darrell, T., Caffe: convolutional architecture for fast feature embedding. Proceedings of International Conference on Multimedia (ICM), 2014, 675–678.
25. Cong, J., Xiao, B., Minimizing computation in convolutional neural networks. Proceedings of International Conference on Artificial Neural Networks (ICANN), 2014, 281–290.
26. Mathieu, M., Henaff, M., LeCun, Y., Fast training of convolutional networks through ffts. CoRR, abs/1312.5851, 2013.
27. Lavin, A., Gray, S., Fast algorithms for convolutional neural networks. Proceedings of Computer Vision and Pattern Recognition (CVPR), 2016, 4013–4021.
28. Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J., Quantized convolutional neural networks for mobile devices. Proceedings of Computer Vision and Pattern Recognition (CVPR), 2016, 4820–4828.
29. Yanai, K., Tanno, R., Okamoto, K., Efficient mobile implementation of A CNN-based object recognition system. Proceedings of International Conference on Multimedia(ACM MM), 2016, 362–366.
30. Chen, W., Wilson, J.T., Tyree, S., Weinberger, K.Q., Chen, Y., Compressing neural networks with the hashing trick. Proceedings of International Conference on Machine Learning (ICML), 2015, 2285–2294.
31. Vanhoucke, V., Senior, A., Mao, M.Z., Improving the speed of neural networks on cpus. NIPS Deep Learning and Unsupervised Feature Learning Workshop, 2011, Citeseer.
32. Courbariaux, M., Bengio, Y., Binarynet: training deep neural networks with weights and activations constrained to +1 or −1. CoRR, abs/1602.02830, 2016.
33. Lin, S., Ji, R., Guo, X., Li, X., Towards convolutional neural networks compression via global error reconstruction. Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), 2016, 1753–1759.
34. Han, S., Mao, H., Dally, W.J., Deep compression: compressing deep neural network with pruning, trained quantization and Huffman coding. Proceedings of International Conference on Learning Representations (ICLR), 2016.
35. He, K., Zhang, X., Ren, S., Sun, J., Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of International Conference on Computer Vision (ICCV), 2015, 1026–1034.
36. Nair, V., Hinton, G.E., Rectified linear units improve restricted Boltzmann machines. Proceedings of International Conference on Machine Learning (ICML), 2010, 807–814.
37. Ioffe, S., Szegedy, C., Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of International Conference on Machine Learning (ICML), 2015, 448–456.
38. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1 (2014), 1929–1958.
39. Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K., SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and < 0.5MB model size. CoRR, abs/1602.07360, 2016.
40. Liu, C., Yin, F., Wang, D., Wang, Q., CASIA online and offline Chinese handwriting databases. Proceedings of International Conference on Document Analysis and Recognition (ICDAR), 2011, 37–41.
41. Chen, L., Wang, S., Fan, W., Sun, J., Naoi, S., Beyond human recognition: A CNN-based framework for handwritten character recognition. Proceedings of Asian Conference on Pattern Recognition (ACPR), 2015, 695–699.
42. Graves, A., Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, 385, 2012, Springer.
43. Zhang, X., Yin, F., Zhang, Y., Liu, C., Bengio, Y., Drawing and recognizing Chinese characters with recurrent neural network. CoRR, abs/1606.06539, 2016.
44. Xie, Z., Sun, Z., Jin, L., Ni, H., Lyons, T., Learning spatial-semantic context with fully convolutional recurrent network for online handwritten Chinese text recognition. CoRR, abs/1610.02616, 2016.