A New design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition

Procedia Computer Science

Volumn 80, Issue , 2016, Pages 1712-1723

  Elleuch, Mohamed ^a   Maalej, Rania ^b   Kherallah, Monji ^c  

^a UNIVERSITY OF MANOUBA   (Tunisia)

^b UNIVERSITY OF SFAX   (Tunisia)

^c UNIVERSITY OF SFAX   (Tunisia)

Author keywords

Arabic handwritten recognition; Based SVM; CNN; Dropout; Features; HACDB; Over fitting

Indexed keywords

CHARACTER RECOGNITION; CONVOLUTION; NETWORK ARCHITECTURE; NEURAL NETWORKS; OPTICAL CHARACTER RECOGNITION; SUPPORT VECTOR MACHINES;

ARABIC HANDWRITTEN RECOGNITION; BASED-SVM; DROPOUT; FEATURES; HACDB; OVERFITTING;

IMAGE CLASSIFICATION;

EID: 84978472842     PISSN: None     EISSN: 18770509     Source Type: Conference Proceeding    
DOI: 10.1016/j.procs.2016.05.512     Document Type: Conference Paper

References (41)

1. H. Lee, R. Grosse, R. Ranganath, A. Y. Ng, "Unsupervised learning of hierarchical representations with convolutional deep belief networks," Communications of the ACM, 54(10) (2011) 95-103.
2. G.-B. Huang, H. Zhou, X. Ding, R. Zhang, "Extreme Learning Machine for Regression and Multiclass Classification," IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, vol. 42(2), pp. 513-529, 2012.
3. H. Lee, P.T. Pham, Y. Largman, A.Y. Ng, "Unsupervised feature learning for audio classification using convolutional deep belief networks," Advances in Neural Information Processing Systems (NIPS), pp. 1096-1104, 2009.
4. Y. Ren, Y. Wu, "Convolutional Deep Belief Networks for Feature Extraction of EEG Signal," International Joint Conference on Neural Networks (IJCNN), pp. 2850-2853, 2014.
5. D.C. Ciresan, U. Meier, J. Schmidhuber, "Transfer Learning for Latin and Chinese Characters with Deep Neural Networks," in Proceedings of International Joint Conference on Neural Networks, 2012.
6. D.C. Ciresan, J. Schmidhuber, "Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification," In Proceedings of CoRR. 2013.
7. X.-X. Niu, Ching Y. Suen, "Novel hybrid CNN-SVM classifier for recognizing handwritten digits," Pattern Recognition, vol. 45, pp. 1318-1325, 2012.
8. M. Elleuch, N. Tagougui and M. Kherallah, "A Novel Architecture of CNN based on SVM Classifier for Recognizing Arabic Handwritten," International Journal of Intelligent Systems Technologies and Applications, vol. 15, 2016, in press.
9. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," arXiv preprint arXiv:1207.0580, 2012.
10. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86(11), pp. 2278-2324, 1998.
11. D. C. Ciresan, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber, "High performance neural networks for visual object classification," Technical Report IDSIA-01-11, Dalle Molle Institute for Artificial Intelligence, 2011.
12. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, vol. 1, pp. 541-551, 1989.
13. V. Vapnik, "Statistical Learn Theory," John Wiley, New York, 1998.
14. H. Byun, S.-W. Lee, "A survey on pattern recognition applications of Support Vector Machines", International Journal of Pattern Recognition and Artificial Intelligence, vol. 17, pp. 459-486, 2003.
15. D. Gorgevik, D. Cakmakov, V. Radevski, "Handwritten digit recognition by combining support vector machines using rule-based reasoning," Proc. 23rd Int. Conf. Information Technology Interfaces (ITI), pp. 139-144, 2001.
16. G. Guo, S. Z. Li, K. Chan, "Face Recognition by Support Vector Machines," Proc. 4th IEEE Intl. Conf. on Face and Gesture Recognition, pp. 196-201, 2000.
17. MNIST: http://yann.lecun.com/exdb/mnist/
18. P. J. Grother, "Nist special database 19 - handprinted forms and characters database," National Institute of Standards and Thechnology (NIST), Tech. Rep., 1995.
19. D. C. Ciresan, U. Meier, L. M. Gambardella, J. Schmidhuber, "Convolutional Neural Network Committees For Handwritten Character Classification," 11th International Conference on Document Analysis and Recognition (ICDAR), 2011.
20. B. Théodore, N. Hermann, K. Christopher, "Tandem HMM with convolutional neural network for handwritten word recognition," In: 38th International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 2390-2394, 2013.
21. U. V. Marti and H. Bunke, "The IAM-database: an English sentence database for offline handwriting recognition," International Journal on Document Analysis and Recognition, vol. 5, no. 1, pp. 39-46, 2002.
22. E. Augustin, M. Carré, E. Grosicki, J.-M. Brodin, E. Geoffrois, F. Preteux, "RIMES evaluation campaign for handwritten mail processing," In Workshop on Frontiers in Handwriting Recognition, 2006(1).
23. H. Byun, S.-W. Lee, "Applications of Support Vector Machines for Pattern Recognition: A Survey," in Proceedings of the First International Workshop, SVM 2002, pp. 213-236, 2002.
24. J. Chen, H. Cao, R. Prasad, A. Bhardwaj, P. Natarajan, "Gabor features for offline arabic handwriting recognition," in Proceedings of the 9th IAPR International Workshop on Document Analysis Systems (DAS), pp. 53-58, 2010.
25. M. Elleuch and M. Kherallah, "An Improved Arabic Handwritten Recognition System using Deep Support Vector Machines," International Journal of Multimedia Data Engineering and Management, vol. 7(2), pp. 1-14, 2016.
26. N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting," The Journal of Machine Learning Research, 2014, vol. 15, No 1, pp. 1929-1958.
27. R. Maalej, N. Tagougui and M. Kherallah, "Online Arabic Handwriting Recognition with Dropout Applied in Deep Recurrent Neural Networks," in 12th IAPR International Workshop on Document Analysis Systems (DAS), 2016, in press.
28. V. Pham, T. Bluche, C. Kermorvant, and J. Louradour, "Dropout improves recurrent neural networks for handwriting recognition," arXiv preprint arXiv:1312.4569, 2013.
29. Y. LeCun, F.J. Huang, L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting," Proc. Computer Vision and Pattern Recognition Conference (CVPR), IEEE Press, 2004.
30. M. Ranzato, F. Huang, Y. Boureau, Y. LeCun, "Unsupervised learning of invariant feature hierarchies with applications to object recognition," Proc. Computer Vision and Pattern Recognition Conference (CVPR), IEEE Press, 2007.
31. C. Cortes, V. Vapnik, "Support vector networks," Machine Learning, vol. 20, pp. 273-297, 1995.
32. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowledge Discovery, vol. 2(2), pp. 121-167, 1998.
33. A. Lawgali, M. Angelova, A. Bouridane, "HACDB: Handwritten Arabic characters database for automatic character recognition," EUropean Workshop on Visual Information Processing (EUVIP), pp. 255-259, 2013.
34. M. Pechwitz, S.S. Maddouri, V. Märgner, N. Ellouze and H. Amiri, "IFN/ENIT database of handwritten Arabic words," In: Colloque International Francophone sur l'Ecrit et le Document (CIFED), pp. 127-136, 2002.
35. P. Simard, D. Steinkraus, J. C. Platt, "Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis," International Conference on Document Analysis and Recognition (ICDAR), pp. 958-962, 2003.
36. C.C. Chang, C.J. Lin, "LIBSVM: A Library for Support Vector Machines," Software Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
37. M. Elleuch, N. Tagougui, and M. Kherallah, "Towards Unsupervised Learning for Arabic Handwritten Recognition Using Deep Architectures," Neural Information Processing - 22nd International Conference, ICONIP 2015, Istanbul, Turkey, part. (1), pp. 363-372, 2015.
38. A. Lawgali, A. Bouridane, M. Angelova and Z. Ghassemlooy, "Handwritten Arabic character recognition: Which feature extraction method?," International Journal of Advanced Science and Technology, vol. 34, pp. 1-8, 2011.
39. S. A. Azeem and H. Ahmed, "Effective technique for the recognition of offline Arabic handwritten words using hidden Markov models," International Journal on Document Analysis and Recognition (IJDAR), vol. 16(4), pp. 399-412, 2013.
40. P. J. Grother, "Nist special database 19 - handprinted forms and characters database," National Institute of Standards and Thechnology (NIST), Tech. Rep., 1995.
41. Applied Media Analysis, Arabic-Handwritten-1.0, http://appliedmediaanalysis.com/Datasets.htm. 2007.