On invariance and selectivity in representation learning

Information and Inference

Volumn 5, Issue 2, 2016, Pages 134-158

  Anselmi, Fabio ^a   Rosasco, Lorenzo ^a,b   Poggio, Tomaso ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

Invariance; Machine learning

Indexed keywords

EID: 85071659312     PISSN: None     EISSN: 20498772     Source Type: Journal    
DOI: 10.1093/imaiai/iaw009     Document Type: Article

References (42)

1. Akhiezer, N. (1965) The Classical Moment Problem: And Some Related Questions in Analysis. University mathematical monographs. Edinburgh: Oliver & Boyd.
2. Andoni, A., Ba, K. D., Indyk, P. & Woodruff, D. P. (2009) Efficient sketches for earth-mover distance, with applications. FOCS. IEEE Computer Society, pp. 324-330.
3. Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A. & Poggio, T. (2013) Unsupervised learning of invariant representations in hierarchical architectures. Theor. Comput. Sci. (preprint) arXiv:1311.4158.
4. Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A. & Poggio, T. (2013) Magic materials: A theory of deep hierarchical architectures for learning sensory representations. CBCL Paper.
5. Anselmi, F. & Poggio, T. (2010) Representation learning in sensory cortex: A theory. CBMM Memo No. 26.
6. Bengio, Y. (2009) Learning deep architectures for AI. Foundations and Trends in Machine Learning 2. Boston, Delft: Now publishers inc.
7. Berger, J. (1993) Statistical Decision Theory and Bayesian Analysis. Springer Series in Statistics. New York: Springer-Verlag.
8. Berlinet, A. & Thomas-Agnan, C. (2004) Reproducing Kernel Hilbert Spaces in Probability and Statistics. Boston: Kluwer Academic.
9. Cadieu, C., Kouh, M., Pasupathy, A., Connor, C., Riesenhuber, M. & Poggio, T. (2004) A model of V4 shape selectivity and invariance. J. Neurophysiol., 98, 1733-1750. Published 1 September 2007.
10. Chen, X., Chen, X. & Mallat, S. (2015) Deep Haar scattering networks. Inf. Inference (this issue).
11. Cohen, T. & Welling, M. (2015) Transformation properties of learned visual representations. International Conference on Learning Representations.
12. Cramer, H. & Wold, H. (1936) Some theorems on distribution functions. J. Lond. Math. Soc., 4, 290-294.
13. Cucker, F. & Smale, S. (2002) On the mathematical foundations of learning. Bull. Am. Math. Soc., 39, 1-49.
14. Edelsbrunner, H. & Harer, J. L. (2010) Computational Topology, An Introduction. New York: American Mathematical Society.
15. Fukushima, K. (1980) Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern., 36, 193-202.
16. Galanti, T., Wolf, L. & Hazan, T. (2015) A PAC theory of the transferable. Inf. Inference (Current Issue).
17. Gens, R. & Domingos, P. (2014) Deep symmetry networks, Neural Information Processing Systems (NIPS).
18. Haasdonk, B. & Burkhardt, H. (2007) Invariant Kernel functions for pattern analysis and machine learning. Mach. Learn., 68, 35-61.
19. Hein, M. & Bousquet, O. (2005) Hilbertian metrics and positive definite kernels on probability measures. AISTATS 2005. Max-Planck-Gesellschaft, pp. 136-143.
20. Hubel, D. & Wiesel, T. (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol., 160, 106.
21. Hubel, D. & Wiesel, T. (1965) Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol., 28, 229.
22. Hubel, D. & Wiesel, T. (1968) Receptive fields and functional architecture of monkey striate cortex. J. Physiol., 195, 215.
23. Jan Boman, F. L. (2009) Support theorems for the radon transform and Cramer-Wold theorems. J. Theor. Probab., 22, 683-710.
24. Johnson, W. B. & Lindenstrauss, J. (1984) Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math., 54, 129-138.
25. Kazhdan, M., Funkhouser, T. & Rusinkiewicz, S. (2003) Rotation invariant spherical harmonic representation of 3D shape descriptors. Proceedings of the 2003 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, SGP'03, pp. 156-164.
26. LeCun, Y. & Bengio, Y. (1995) Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks. MA: MIT Press Cambridge, 255-258.
27. Leibo, J. Z., Liao, Q., Anselmi, F. & Poggio, T. (2014) The invariance hypothesis implies domain-specific regions in visual cortex. http://dx.doi.org/10.1101/004473.
28. Mallat, S. (2012) Group invariant scattering. Commun. Pure Appl. Math., 65, 1331-1398.
29. Olver, P. (2009) Equivalence, Invariants and Symmetry. Cambridge: Cambridge University Press.
30. Poggio, T., Mutch, J., Anselmi, F., Tacchetti, A., Rosasco, L. & Leibo, J. Z. (2013) Does invariant recognition predict tuning of neurons in sensory cortex? MIT-CSAIL-TR-2013-019, CBCL-313. CBCL Technical Report.
31. Ramm, A. G. (1998) On the theory of reproducing kernel hilbert spaces. J. Inverse Ill-Posed Probl, 6, 515-520.
32. Reed, M. & Simon, B. (1978) Methods of Modern Mathematical Physics II Fourier Analysis, Self-Adjointness. London: Academic Press.
33. Riesenhuber, M. & Poggio, T. (1999) Hierarchical models of object recognition in cortex. Nat. Neurosci., 2, 1019-1025.
34. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R. & LeCun, Y. (2014) OverFeat: Integrated recognition, localization and detection using convolutional networks. International Conference on Learning Representations (ICLR2014), Calgary.
35. Soatto, S. (2009) Actionable information in vision. Computer Vision, 2009 IEEE 12th International Conference on. IEEE, pp. 2138-2145.
36. Soatto, S. & Chiuso, A. Visual scene representations: Sufficiency, minimality, invariance and deep approximation. Proceedings of the ICLR Workshop, 2015 (also ArXiv: 1411.7676, 2014).
37. Sriperumbudur, B. K., Gretton, A., Fukumizu, K., Schölkopf, B. & Lanckriet, G. R. G. (2010) Hilbert space embeddings and metrics on probability measures. J. Mach. Learn. Res., 11, 1517-1561.
38. Tosic, I. & Frossard, P. (2011) Dictionary learning: what is the right representation for my signal? IEEE Signal Process. Mag., 28, 27-38.
39. Vapnik, V. (1982) Estimation of Dependencies Based on Empirical Data. Berlin, Heidelberg, New York: Springer.
40. Vedaldi, A. & Zisserman, A. (2011) Efficient additive kernels via explicit feature maps. Pattern Anal.Mach. Intell., 34, 480-492.
41. Xing, E. P., Ng, A. Y., Jordan, M. I. & Russell, S. (2003) Distance metric learning, with application to clustering with side-information. Advances in Neural Information Processing Systems. MA: MIT Press, pp. 505-512.
42. Zacks, S. (1975) The Theory of Statistical Inference. New York: Wiley.