Deep Haar scattering networks

Information and Inference

Volumn 5, Issue 2, 2016, Pages 105-133

  Cheng, Xiuyuan ^a   Chen, Xu ^b   Mallat, Stéphane ^c  

^a YALE UNIVERSITY   (United States)

^b Princeton University   (United States)

^c ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Classification; Deep learning; Graphs; Haar wavelet; Images; Neural network; Scattering transform

Indexed keywords

EID: 85032765704     PISSN: None     EISSN: 20498772     Source Type: Journal    
DOI: 10.1093/imaiai/iaw007     Document Type: Article

References (45)

1. Ankenman, J. I. (2014) Geometry and analysis of dual networks on questionnaires. PhD thesis, Yale University.
2. Arora, S., Bhaskara, A., Ge, R. & Ma, T. (2013) Provable bounds for learning some deep representations. Preprint arXiv:1310.6343.
3. Bengio, Y., Courville, A. & Vincent, P. (2013a) Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35, 1798-1828.
4. Bengio, Y., Courville, A. & Vincent, P. (2013b) Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35, 1798-1828.
5. Bengio, Y., Thibodeau-Laufer, E. & Yosinski, J. (2013) Deep generative stochastic networks trainable by backprop. Preprint arXiv:1306.1091.
6. Bruna, J. & Mallat, S. (2013) Invariant scattering convolution networks. IEEE Trans. PAMI, 35, 1872-1886.
7. Bruna, J., Zaremba, W., Szlam, A. & LeCun, Y. (2014) Spectral networks and deep locally connected networks on graphs. Conference Proceedings of International Conference on Learning Representations 2014. (ICLR'14).
8. Chen, X., Cheng, X. & Mallat, S. (2014) Unsupervised deep Haar scattering on graphs. Advances in Neural Information Processing Systems 27 (NIPS'14). Curran Associates, Inc., pp. 1709-1717.
9. Chen, S., Cowan, C. F. & Grant, P. M. (1991) Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans. Neural Netw., 2, 302-309.
10. Coifman, R. & Gavish, M. (2011) Harmonic analysis of digital data bases. Wavelets and Multiscale analysis. Berlin, Heidelberg, New York: Springer, pp. 161-197.
11. Coifman, R. & Maggioni, M. (2006) Diffusion wavelets. Appl. Comput. Harmonic Anal., 21, 53-94.
12. Coifman, C., Meyer, Y. & Wickerhauser, M. (1992) Wavelet analysis and signal processing. Wavelets and Their Applications (Ruskai ed.). Boston: Jones and Bartlett, pp. 153-178.
13. Edmonds, J. (1965) Paths, trees, and flowers. Can. J. Math., 17, 449-467.
14. Erhan, D., Manzagol, P., Bengio, Y., Bengio, S. & Vincent, P. (2009) The difficulty of training deep architectures and the effect of unsupervised pre-training. Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS '09), pp. 153-160.
15. Gabow, H. N. (1976) An efficient implementation of Edmonds' algorithm for maximum matching on graphs. J. ACM, 23, 221-234.
16. Gavish, M., Nadler, B. & Coifman, R. R. (2010) Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning. Proceedings of the 27th International Conference on Machine Learning (ICML '10), pp. 367-374.
17. Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A. & Benjio, Y. (2013) Maxout networks. Preprint arXiv:1302.4389.
18. He, K., Zhang, X., Ren, S. & Sun, J. (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Preprint arXiv:1502.01852.
19. Hinton, G., Deng, L., Yu, D., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Dahl, G. & Kingsbury, B. (2012) Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag., 29, 82-97.
20. Hinton, G., Osindero, S. & Teh, Y.-W. (2006) A fast learning algorithm for deep belief nets. Neural Comput., 18, 1527-1554.
21. Hinton, G. & Salakhutdinov, R. R. (2006) Reducing the dimensionality of data with neural networks. Science, 313, 504-507.
22. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. (2012) Improving neural networks by preventing co-adaptation of feature detectors. Preprint arXiv:1207. 0580.
23. Jia, Y., Huang, C. & Darrell, T. (2012) Beyond spatial pyramids: Receptive field learning for pooled image features. CVPR, 2012 IEEE Conference on, Providence, RI, pp. 3370-3377.
24. Jones, P. W., Osipov, A. & Rokhlin, V. (2011) Randomized approximate nearest neighbors algorithm. Proc. Natl. Acad. Sci., 108, 15679-15686.
25. Labusch, K., Barth, E. & Martinetz, T. (2008) Simple method for highperformance digit recognition based on sparse coding. IEEE TNN, 19, 1985-1989.
26. Le, Q., Sarlos, T. & Smola, A. (2013) Fastfood-approximating kernel expansions in loglinear time. ICML. Proceedings of the 30th International Conference on Machine Learning (ICML'13), JMLR: W&CP, pp. 224-252.
27. LeCun, Y., Kavukvuoglu, K. & Farabet, C. (2010) Convolutional networks and applications in vision. Proceedings of 2010 IEEE International Symposium on, Circuits and Systems (ISCAS), pp. 253-256.
28. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z. & Tu, Z. (2014) Deeply supervised nets. Preprint arXiv:1409.5185.
29. Lin, T.-H. & Kung, H.-T. (2014) Stable and efficient representation learning with nonnegativity constraints. Proceedings of the 31st International Conference on Machine Learning (ICML'14), pp. 1323-1331. ICML.
30. Mallat, S. (2012) Group invariant scattering. Commun. Pure Appl. Math., 65, 1331-1398.
31. Mallat, S. (2016) Understanding deep convolutional networks. Philosphical Trans. A, 374, 2065.
32. Mallat, S. & Waldspurger, I. (2013) Deep learning by scattering. Preprint arXiv:1306.5532.
33. Maurey, B. (1991) Some deviation inequalities. Geom. Funct. Anal., 1, 188-197.
34. Ngiam, J., Chen, Z., Chia, D., Koh, P. W., Le, Q. V. & Ng, A. Y. (2010) Tiled convolutional neural networks. Adv. Neural Inf. Process. Syst., 23, 1279-1287.
35. Oyallon, E. & Mallat, S. (2014) Deep roto-translation scattering for object classification. Preprint arXiv:1412.8659.
36. Pisier, G. (1985) Probabilistic Methods in the Geometry of Banach Spaces. Springer Lecture Notes in Math., vol. 1206. Springer: Berlin, Heidelberg. pp. 167-241.
37. Preis, R. (1999) Linear Time 12 -Approximation Algorithm for Maximum Weighted Matching in General Graphs, STACS99: 16th Annual Symposium on Theoretical Aspects of Computer Science, Trier, Germany, Springer: Berlin Heidelberg. pp. 259-269.
38. Rifai, S., Vincent, P., Muller, X., Glorot, X. & Bengio, Y. (2011) Contractive auto-encoders: Explicit invariance during feature extraction. Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 833-840.
39. Roux, N. L., Bengio, Y., Lamblin, P., Joliveau, M. & Kégl, B. (2008) Learning the 2-D topology of images. Adv. Neural Inf. Process. Syst., 20, 841-848.
40. Rustamov, R. & Guibas, L. (2013) Wavelets on Graphs via Deep Learning, Advances in Neural Information Processing Systems 26. ed. by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Q. Weinberger, Curran Associates, Inc. pp. 998-1006.
41. Salakhutdinov, R. (2015) Learning deep generative models. Annu. Rev. Stat. Appl., 2, 361-385.
42. Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A. & Vandergheynst, P. (2013) The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag., 30, 83-98.
43. Szlam, A. D., Maggioni, M., Coifman, R. & Bremer Jr, J. C. (2005) Diffusion-driven multiscale analysis on manifolds and graphs: Top-down and bottom-up constructions. Optics & Photonics 2005. International Society for Optics and Photonics, pp. 59141D.
44. Taigman, Y., Yang, M., Ranzato, M. & Wolf, L. (2014) DeepFace: Closing the gap to human-level performance in face verification. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701-1708.
45. Yu, D. & Deng, L. (2011) Deep convex net: A scalable architecture for speech pattern classification. Proceedings of INTERSPEECH, pp. 2285-2288.