A snapshot of image Pre-Processing for convolutional neural networks: Case study of MNIST

International Journal of Computational Intelligence Systems

Volumn 10, Issue 1, 2017, Pages 555-568

  Tabik, Siham ^a   Peralta, Daniel ^b,c   Herrera Poyatos, Andrés ^a   Herrera, Francisco ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b DEPARTMENT OF PLANT SYSTEMS BIOLOGY   (Belgium)

^c GHENT UNIVERSITY   (Belgium)

Author keywords

Classification; Convolutional Neural Networks (CNNs); Data augmentation; Deep learning; Handwritten digits; Preprocessing

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVOLUTION; DEEP LEARNING; ELASTIC DEFORMATION; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; DATA AUGMENTATION; DATA PREPROCESSING TECHNIQUE; HANDWRITTEN DIGIT; IMAGE PREPROCESSING; PATTERN CLASSIFICATION PROBLEMS; PREPROCESSING; PREPROCESSING TECHNIQUES;

CHARACTER RECOGNITION;

EID: 85017024771     PISSN: 18756891     EISSN: 18756883     Source Type: Journal    
DOI: 10.2991/ijcis.2017.10.1.38     Document Type: Article

References (33)

1. Richard O Duda, Peter E Hart, and David G Stork. Pattern classification. John Wiley & Sons, 2012.
2. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012.
3. Quoc V. Le. Building high-level features using large scale unsupervised learning. In 2013 IEEE Int. Conf. Acoust. Speech Signal Process., pages 8595-8598, 2013.
4. Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86(11):2278-2324, 1998.
5. Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups. IEEE Signal Process. Mag., 29(6):82-97, 2012.
6. Tara N. Sainath, Abdel-rahman Mohamed, Brian Kingsbury, and Bhuvana Ramabhadran. Deep convolutional neural networks for LVCSR. In 2013 IEEE Int. Conf. Acoust. Speech Signal Process., pages 8614-8618, 2013.
7. Salvador García, Julián Luengo, and Francisco Herrera. Data preprocessing in data mining. Springer, 2015.
8. Salvador García, Julián Luengo, and Francisco Herrera. Tutorial on practical tips of the most influential data preprocessing algorithms in data mining. Knowledge-Based Systems, 98:1-29, 2016.
9. Dan Ciresan, Ueli Meier, Jonathan Masci, and J¨urgen Schmidhuber. Multi-column deep neural network for traffic sign classification. Neural Networks, 32:333-338, 2012.
10. P.Y. Simard, D. Steinkraus, and J.C. Platt. In Seventh Int. Conf. Doc. Anal. Recognition, 2003. Proceedings.
11. Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/, 2013. Online; accessed 09 octuber 2014.
12. Rodrigo Benenson. Classification datasets results. http://rodrigob.github.io/are_we_ there_yet/build/classification_datasets_ results.html, 2013. Online; accessed 09 november 2016.
13. Michael A Nielsen. Neural networks and deep learning. URL: http://neuralnetworksanddeeplearning.com, 2015.
14. Li Wan, Matthew Zeiler, Sixin Zhang, Yann L Cun, and Rob Fergus. Regularization of neural networks using dropconnect. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), pages 1058-1066, 2013.
15. Michael A. Nielsen. Neural Networks and Deep Learning. Determination Press, 2015.
16. Daniel Strigl, Klaus Kofler, and Stefan Podlipnig. Performance and scalability of gpu-based convolutional neural networks. In PDP, pages 317-324, 2010.
17. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis., 115(3):211-252, 2015.
18. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323(6088):533-536, 1986.
19. Ning Qian. On the momentum term in gradient descent learning algorithms. Neural Networks, 12(1):145-151, 1999.
20. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
21. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. J. Mach. Learn. Res.
22. George E Dahl, Tara N Sainath, and Geoffrey E Hinton. Improving deep neural networks for lvcsr using rectified linear units and dropout. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 8609-8613. IEEE, 2013.
23. Michał Wózniak, Manuel Graña, and Emilio Corchado. A survey of multiple classifier systems as hybrid systems. Information Fusion, 16:3-17, 2014.
24. Esther Levin, Naftali Tishby, and SARA A Solla. A statistical approach to learning and generalization in layered neural networks. Proceedings of the IEEE, 78(10):1568-1574, 1990.
25. Deep Learning Libraries by Language. http://www.teglor.com/b/ deep-learning-libraries-language-cm569/, 2016. Online; accessed 14 octuber 2016.
26. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pages 675-678. ACM, 2014.
27. Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.02688.
28. Fast convolutional neural networks in C++/CUDA. https://code.google.com/archive/p/ cuda-convnet2/, 2016. Online; accessed 31 octuber 2016.
29. Shaohuai Shi, Qiang Wang, Pengfei Xu, and Xiaowen Chu. Benchmarking state-of-the-art deep learning software tools. arXiv preprint arXiv:1608.07249, 2016.
30. Chen-Yu Lee, Patrick W Gallagher, and Zhuowen Tu. Generalizing pooling functions in convolutional neural networks: Mixed, gated, and tree. In International Conference on Artificial Intelligence and Statistics, 2016.
31. Jia-Ren Chang and Yong-Sheng Chen. Batchnormalized maxout network in network. arXiv preprint arXiv:1511.02583, 2015.
32. Ikuro Sato, Hiroki Nishimura, and Kensuke Yokoi. Apac: Augmented pattern classification with neural networks. arXiv preprint arXiv:1505.03229, 2015.
33. Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber. Multi-column deep neural networks for image classification. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 3642-3649. IEEE, 2012.