Understanding deep convolutional networks

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 374, Issue 2065, 2016, Pages

  Mallat, Stéphane ^a  

^a ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Deep convolutional neural networks; Learning; Wavelets

Indexed keywords

NEURAL NETWORKS;

CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; HIGH-DIMENSIONAL PROBLEMS; LEARNING; LINEAR FILTERS; MATHEMATICAL FRAMEWORKS; STATE OF THE ART; WAVELETS;

CONVOLUTION;

EID: 84960344531     PISSN: 1364503X     EISSN: None     Source Type: Journal    
DOI: 10.1098/rsta.2015.0203     Document Type: Review

References (32)

1. Le Cun Y., Bengio Y, Hinton G. 2015 Deep learning. Nature 521, 436-444. (doi:10.1038/nature14539)
2. Krizhevsky A, Sutskever I, Hinton G. 2012 ImageNet classification with deep convolutional neural networks. In Proc. 26th Annual Conf. on Neural Information Processing Systems, Lake Tahoe, NV, 3-6 December 2012, pp. 1090-1098.
3. Hinton G et al. 2012 Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29, 82-97. (doi:10.1109/MSp.2012.2205597)
4. Leung MK, Xiong HY, Lee L.J., Frey BJ. 2014 Deep learning of the tissue regulated splicing code. Bioinformatics 30, i121-i129. (doi:10.1093/bioinformatics/btu277)
5. Sutskever I, Vinyals O, Le QV 2014 Sequence to sequence learning with neural networks. In Proc. 28th Annual Conf. on Neural Information Processing Systems, Montreal, Canada, 8-13 December 2014.
6. Anselmi F, Leibo J, Rosasco L., Mutch J, Tacchetti A, Poggio T. 2013 Unsupervised learning of invariant representations in hierarchical architectures. (http://arxiv.org/abs/1311.4158)
7. Candès E., Donoho D. 1999 Ridglets: a key to high-dimensional intermittency? Phil. Trans. R. Soc. A 357, 2495-2509. (doi:10.1098/rsta.1999.0444)
8. Le Cun Y., Boser B, Denker J., Henderson D, Howard R, Hubbard W., Jackelt L. 1990 Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems 2(ed. DS Touretzky), pp. 396-404. San Francisco, CA: Morgan Kaufmann.
9. Mallat S. 2012 Group invariant scattering. Commun. Pure Appl. Math. 65, 1331-1398. (doi:10.1002/cpa.21413)
10. Agrawal A, Girshick R, Malik J. 2014 Analyzing the performance of multilayer neural networks for object recognition. In Computer vision - ECCV 2014(eds D Fleet, T Pajdla, B Schiele, T Tuytelaars). Lecture Notes in Computer Science, Vol. 8695, pp. 329-344. Cham, Switzerland: Springer. (doi:10.1007/978-3-319-10584-0-22)
11. Hastie T, Tibshirani R, Friedman J. 2009 The elements of statistical learning. Springer Series in Statistics. New York, NY: Springer.
12. Radford A, Metz L, Chintala S. 2016 Unsupervised representation learning with deep convolutional generative adversarial networks. In 4th Int. Conf. on Learning Representations (ICLR 2016), San Juan, Puerto Rico, 2-4 May 2016.
13. Mesgarani M, Slaney M, Shamma S. 2006 Discrimination of speech from nonspeech based on multiscale spectro-temporal modulations. IEEE Trans. Audio Speech Lang. Process. 14, 920-930. (doi:10.1109/TSA.2005.858055)
14. Lowe DG. 2004 SIFT: scale invariant feature transform. J. Comput. Vision 60, 91-110.
15. Carrier J, Greengard L, Rokhlin V. 1988 A fast adaptive multipole algorithm for particle simulations. SIAM J. Sci. Stat. Comput. 9, 669-686.
16. Choromanska A, Henaff M, Mathieu M., Ben Arous G, Le Cun Y. 2014 The loss surfaces of multilayer networks. (http://arxiv.org/abs/1412.0233)
17. Szegedy C, Erhan D, Zaremba W., Sutskever I, Goodfellow I, Bruna J., Fergus R. 2014 Intriguing properties of neural networks. In Int. Conf. on Learning Representations (ICLR 2014), Banff, Canada, 14-16 April 2014.
18. Waldspurger I. 2015 Wavelet transform modulus: phase retrieval and scattering. PhD thesis, Ecole Normale Supèrieure, Paris, France.
19. Bruna J, Mallat S. 2013 Invariant scattering convolution networks. IEEE Trans. PAMI 35, 1872-1886. (doi:10.1109/TPAMI.2012.230)
20. Andèn J., Mallat S. 2014 Deep scattering spectrum. IEEE Trans. Signal Process. 62, 4114-4128. (doi:10.1109/TSp.2014.2326991)
21. Sifre L, Mallat S. 2013 Rotation, scaling and deformation invariant scattering for texture discrimination. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1233-1240. (doi:10.1109/CVPR.2013.163)
22. Bruna J, Mallat S. Submitted. Stationary scattering models.
23. Bruna J, Mallat S, Bacry E., Muzy JF. 2015 Intermittent process analysis with scattering moments. Ann. Stat. 43, 323-351. (doi:10.1214/14-AOS1276)
24. Gatys LA, Ecker AS, Bethge M. 2015 Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. (http://arxiv.org/abs/1505.07376)
25. Denton E, Chintala S, Szlam A., Fergus R. 2015 Deep generative image models using a Laplacian pyramid of adversarial networks. In Proc. 29th Annual Conf. on Neural Information Processing Systems, Montreal, Canada, 7-12 December 2015.
26. Petitot J. 2008 Neurogémétrie de la vision. Paris, France: Éditions de l'École Polytechnique.
27. Andèn J., Lostanlen V, Mallat S. 2015 Joint time-frequency scattering for audio classification. In Proc. IEEE Int. Workshop on Machine Learning for Signal Processing, Boston, MA, 17-20 September 2015. (doi:10.1109/MLSp.2015.7324385)
28. Bruna J, Szlam A, Le Cun Y. 2014 Learning stable group invariant representations with convolutional networks. In Int. Conf. on Learning Representations (ICLR 2014), Banff, Canada, 14-16 April 2014.
29. Oyallon E, Mallat S. 2015 Deep roto-translation scattering for object classification. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2865-2873. (doi:10.1109/CVPR.2015.7298904)
30. Hirn M, Poilvert N, Mallat S. 2015 Quantum energy regression using scattering transforms. (http://arxiv.org/abs/1502.02077)
31. Aubry M, Russell B. 2015 Understanding deep features with computer-generated imagery. (http://arxiv.org/abs/1506.01151)
32. Glinsky M. 2011 A new perspective on renormalization: the scattering transformation. (http://arxiv.org/abs/1106.4369)