Understanding convolutional neural networks with a mathematical model

Journal of Visual Communication and Image Representation

Volumn 41, Issue , 2016, Pages 406-413

  Kuo, C C Jay ^a  

^a University of Southern California   (United States)

Author keywords

Convolutional neural network (CNN); MNIST dataset; Nonlinear activation; RECOS model; Rectified linear unit (ReLU)

Indexed keywords

CHEMICAL ACTIVATION; CONVOLUTION;

CONVOLUTIONAL NEURAL NETWORK; INTERMEDIATE LAYERS; LINEAR UNITS; MNIST DATASET; MULTI-LAYER SYSTEM; NON-LINEAR ACTIVATION; NONLINEAR ACTIVATION FUNCTIONS; SPECTRAL COMPONENTS;

NEURAL NETWORKS;

EID: 84996534050     PISSN: 10473203     EISSN: 10959076     Source Type: Journal    
DOI: 10.1016/j.jvcir.2016.11.003     Document Type: Article

References (20)

1. [1] Juang, B.H., Deep neural networks – a developmental perspective. APSIPA Trans. Sign. Inform. Process., 5, 2016, e7.
2. [2] LeCun, Y., Bengio, Y., Hinton, G.E., Deep learning. Nature 521 (2015), 436–444.
3. [3] Mallat, S., Group invariant scattering. Commun. Pure Appl. Math. 65:10 (2012), 1331–1398.
4. [4] Bruna, J., Mallat, S., Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35:8 (2013), 1872–1886.
5. [5] T. Wiatowski, H. Bölcskei, A mathematical theory of deep convolutional neural networks for feature extraction. Available from: < arxiv:1512.06293>.
6. [6] N. Cohen, O. Sharir, A. Shashua, On the expressive power of deep learning: a tensor analysis. Available from: and .
7. [7] J. Dai, Y. Lu, Y.-N. Wu, Generative modeling of convolutional neural networks. Available from: < arxiv:1412.6296>.
8. [8] Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., Samek, W., On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS One, 10(7), 2015, e0130140.
9. [9] G. Montavon, S. Bach, A. Binder, W. Samek, K.-R. Müller, Explaining nonlinear classification decisions with deep Taylor decomposition. Available from: < arxiv:1512.02479>.
10. [10] K. Simonyan, A. Vedaldi, A. Zisserman, Deep inside convolutional networks: visualising image classification models and saliency maps. Available from: < arxiv:1312.6034>.
11. [11] Zeiler, M.D., Fergus, R., Visualizing and understanding convolutional networks. European Conference on Computer Vision, 2014, Springer, 818–833.
12. [12] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, A. Torralba, Object detectors emerge in deep scene CNNs. Available from: < arxiv:1412.6856>.
13. [13] Hornik, K., Stinchcombe, M., White, H., Multilayer feedforward networks are universal approximators. Neural Netw. 2:5 (1989), 359–366.
14. [14] LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. Proc. IEEE 86:11 (1998), 2278–2324.
15. [15] Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q., (eds.) Advances in Neural Information Processing Systems, vol. 25, 2012, Curran Associates, Inc., 1097–1105.
16. [16] McCulloch, W.S., Pitts, W., A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5:4 (1943), 115–133.
17. [17] Zhang, L., Zhang, B., A geometrical representation of McCulloch-Pitts neural model and its applications. IEEE Trans. Neural Netw. 10:4 (1999), 925–929.
18. [18] A. Coates, H. Lee, A.Y. Ng, An analysis of single-layer networks in unsupervised feature learning, in: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 15, JMLR:W&CP 15, 2011, pp. 215–223.
19. [19] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I.J. Goodfellow, R. Fergus, Intriguing properties of neural networks. Available from: < arxiv:1312.6199>.
20. [20] Audhkhasi, K., Osoba, O., Kosko, B., Noise-enhanced convolutional neural networks. Neural Netw. 78 (2016), 15–23.