Unsupervised learning of invariant representations

Theoretical Computer Science

Volumn 633, Issue , 2016, Pages 112-121

  Anselmi, Fabio ^a,b   Leibo, Joel Z ^a   Rosasco, Lorenzo ^a,b,c   Mutch, Jim ^a   Tacchetti, Andrea ^a   Poggio, Tomaso ^a,b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^c UNIVERSITY OF GENOA   (Italy)

Author keywords

Convolutional networks; Cortex; Hierarchy; Invariance

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPLEX NETWORKS; INVARIANCE; LEARNING SYSTEMS; OBJECT RECOGNITION; SPEECH RECOGNITION; SUPERVISED LEARNING; UNSUPERVISED LEARNING;

CONVOLUTIONAL NETWORKS; CORTEX; GROUP TRANSFORMATIONS; HIERARCHICAL REPRESENTATION; HIERARCHY; INVARIANT REPRESENTATION; SIMPLE AND COMPLEX CELLS; VISUAL OBJECT RECOGNITION;

LEARNING ALGORITHMS;

EID: 84939500710     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2015.06.048     Document Type: Article

References (37)

1. Hubel D., Wiesel T. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 1962, 160(1):106. http://jp.physoc.org/content/160/1/106.full.pdf.
2. Riesenhuber M., Poggio T. Models of object recognition. Nat. Neurosci. 2000, 3(11):1199-1204. http://www.nature.com/neuro/journal/v3/n11s/full/nn1100_1199.html.
3. Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybernet. 1980, 36(4):193-202. http://www.springerlink.com/content/r6g5w3tt54528137.
4. LeCun Y., Boser B., Denker J., Henderson D., Howard R., Hubbard W., Jackel L. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1989, 1(4):541-551. http://www.mitpressjournals.org/doi/abs/10.1162/neco.1989.1.4.541.
5. Lee T., Soatto S. Video-based descriptors for object recognition. Image Vis. Comput. 2012, 29:639-652.
6. Anselmi F., Leibo J., Rosasco L., Mutch J., Tacchetti A., Poggio T. Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning? Center for Brains, Minds and Machines (CBMM) Memo No. 1. arxiv:1311.4158v5.
7. Schulz-Mirbach H. Constructing invariant features by averaging techniques. Proceedings of the 12th IAPR International Conference on Pattern Recognition, 1994, Conference B: Computer Vision and Image Processing, vol. 2 1994, 387-390.
8. Cramer H., Wold H. Some theorems on distribution functions. J. London Math. Soc. 1936, 4:290-294.
9. Koloydenko A. Symmetric measures via moments. Bernoulli 2008, 14(2):362-390.
10. Adelson E., Bergen J. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Amer. A 1985, 2(2):284-299. http://www.opticsinfobase.org/abstract.cfm?URI=josaa-2-2-284.
11. Anselmi F., Leibo J., Rosasco J., Mutch L., Tacchetti A., Poggio T. Magic materials: a theory of deep hierarchical architectures for learning sensory representations CBCL paper. http://cbcl.mit.edu/publications/ai-publications/2013/Magic_working_paper_May6_2013.pdf.
12. Jarrett K., Kavukcuoglu K., Ranzato M., LeCun Y. What is the best multi-stage architecture for object recognition?. IEEE International Conference on Computer Vision 2009, 2146-2153. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5459469.
13. Voinea S., Zhang C., Evangelopoulos G., Rosasco L., Poggio T. Word-level invariant representations from acoustic waveforms. 15th Annual Conf. of the International Speech Communication Association 2014.
14. Zhang C., Voinea S., Evangelopoulos G., Rosasco L., Poggio T. Phone classification by a hierarchy of invariant representation layers. 15th Annual Conf. of the International Speech Communication Association 2014.
15. Liao Q., Leibo J.Z., Poggio T. Learning invariant representations and applications to face verification. Advances in Neural Information Processing Systems 2013.
16. Leibo J.Z., Mutch J., Poggio T. Why the brain separates face recognition from object recognition. Advances in Neural Information Processing Systems 2011, http://cbcl.mit.edu/publications/ps/Leibo_Mutch_Poggio_face:invar_v08_cam_rdy_letter_Dec2011.pdf.
17. Leibo J.Z., Anselmi F., Mutch J., Ebihara A.F., Freiwald W., Poggio T. View-invariance and mirror-symmetric tuning in a model of the macaque face-processing system. Computational and Systems Neuroscience 2013, I-54.
18. Pinto N., Doukhan D., DiCarlo J., Cox D. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS Comput. Biol. 2009, 5(11):e1000579. http://dx.plos.org/10.1371/journal.pcbi.1000579.
19. Olshausen B., et al. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996, 381(6583):607-609.
20. Mallat S. Group invariant scattering. Comm. Pure Appl. Math. 2012, 65(10):1331-1398. 10.1002/cpa.21413.
21. Candes E., Romberg J. Sparsity and incoherence in compressive sampling. Invers. Probl. 2007, 23:969-985.
22. J.Z. Leibo, Q. Liao, F. Anselmi, T. Poggio, The invariance hypothesis implies domain-specific regions in visual cortex, bioRxiv. doi:10.1101/004473.
23. Soatto S. Steps towards a theory of visual information: active perception, signal-to-symbol conversion and the interplay between sensing and control 2011, 1-151. arxiv:1110.2053.
24. Smale S., Rosasco L., Bouvrie J., Caponnetto A., Poggio T. Mathematics of the neural response. Found. Comput. Math. 2010, 10(1):67-91. http://www.springerlink.com/index/k650503137550j23.pdf.
25. Serre T., Kouh M., Cadieu C., Knoblich U., Kreiman G., Poggio T. A theory of object recognition: computations and circuits in the feedforward path of the ventral stream in primate visual cortex CBCL paper #259/AI memo #2005-036. http://dspace.mit.edu/handle/1721.1/36407.
26. Chikkerur S.S., Serre T., Tan C., Poggio T. What and where: a Bayesian inference theory of attention. Vis. Res. 2010, 50(22):2233-2247. 10.1016/j.visres.2010.05.013.
27. George D., Hawkins J. A hierarchical bayesian model of invariant pattern recognition in the visual cortex. Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 3 2005, 1812-1817.
28. Geman S. Invariance and selectivity in the ventral visual pathway. J. Physiol. 2006, 100(4):212-224. http://linkinghub.elsevier.com/retrieve/pii/S0928425707000034.
29. McCulloch W.S., Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5(4):115-133. http://link.springer.com/article/10.1007/BF02478259.
30. Földiák P. Learning invariance from transformation sequences. Neural Comput. 1991, 3(2):194-200. http://www.mitpressjournals.org/doi/abs/10.1162/neco.1991.3.2.194.
31. Kanwisher N. Functional specificity in the human brain: a window into the functional architecture of the mind. Proc. Natl. Acad. Sci. USA 2010, 25(107):11163.
32. Tsao D., Freiwald W. Faces and objects in macaque cerebral cortex. Nature 2003, 6(9):989-995. http://www.nature.com/neuro/journal/v6/n9/abs/nn1111.html.
33. Marr D., Poggio T. From understanding computation to understanding neural circuitry AIM-357. http://scholar.google.com/scholar?q=marr+poggio+levels&hl=en&btnG=Search&as_sdt=40000001&as_sdtp=on.
34. Mutch J., Lowe D. Multiclass object recognition with sparse, localized features. Comput. Vis. Pattern Recognit. 2006, vol. 1 2006, 11-18. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1640736.
35. Serre T., Oliva A., Poggio T. A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. USA 2007, 104(15):6424-6429. http://cat.inist.fr/?aModele=afficheN&cpsidt=18713198.
36. LeCun Y., Bengio Y. Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks 1995, 255-258. URL. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.32.9297&rep=rep1&type=pdf.
37. LeCun Y., Huang F., Bottou L. Learning methods for generic object recognition with invariance to pose and lighting. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 2004, II-97. IEEE.