Slow, decorrelated features for pretraining complex cell-like networks

Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

Volumn , Issue , 2009, Pages 99-107

  Bergstra, James ^a   Bengio, Yoshua ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; ITERATIVE METHODS; NEURAL NETWORKS; PHYSIOLOGICAL MODELS;

ACTIVATION FUNCTIONS; AREA V1; COMPLEX CELLS; IMAGE MODELING; NETWORK ACTIVATIONS; NEURAL-NETWORKS; PRE-TRAINING; RATE MODELS; SINGLE-HIDDEN-LAYER NEURAL NETWORKS; VISUAL AREAS;

NETWORK LAYERS;

EID: 84858720675     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, 2, 284-99.
2. Becker, S., & Hinton, G. E. (1993). Learning mixture models of spatial coherence. Neural Computation, 5, 267-277.
3. Bengio, Y. (2009). Learning deep architectures for AI. Foundations and Trends in Machine Learning, to appear.
4. Berkes, P., & Wiskott, L. (2005). Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5, 579-602.
5. Cadieu, C., & Olshausen, B. (2009). Learning transformational invariants from natural movies. In Advances in neural information processing systems 21 (nips'08), 209-216. MIT Press.
6. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience. The MIT Press.
7. Decoste, D., & Schölkopf, B. (2002). Training invariant support vector machines. Machine Learning, 46, 161-190.
8. Erhan, D., Manzagol, P.-A., Bengio, Y., Bengio, S., & Vincent, P. (2009). The difficulty of training deep architectures and the effect of unsupervised pre-training. AISTATS'2009 (pp. 153-160). Clearwater (Florida), USA.
9. Finn, I., & Ferster, D. (2007). Computational diversity in complex cells of cat primary visual cortex. Journal of Neuroscience, 27, 9638-48.
10. Hinton, G. E., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554.
11. Hurri, J., & Hyvärinen, A. (2003). Temporal coherence, natural image sequences, and the visual cortex. Advances in Neural Information Processing Systems 15 (NIPS'02) (pp. 141-148).
12. Körding, K. P., Kayser, C., Einhäuser, W., & König, P. (2004). How are complex cell properties adapted to the statistics of natural stimuli? Journal of Neurophysiology, 91, 206-212.
13. Kouh, M. M., & Poggio, T. T. (2008). A canonical neural circuit for cortical nonlinear operations. Neural Computation, 20, 1427-1451.
14. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007). An empirical evaluation of deep architectures on problems with many factors of variation. ICML 2007 (pp. 473-480). Corvallis, OR: ACM.
15. Lee, H., Ekanadham, C., & Ng, A. (2008). Sparse deep belief net model for visual area V2. In Advances in neural information processing systems 20 (nips'07). Cambridge, MA: MIT Press.
16. Loosli, G., Canu, S., & Bottou, L. (2007). Training invariant support vector machines using selective sampling. In L. Bottou, O. Chapelle, D. DeCoste and J. Weston (Eds.), Large scale kernel machines, 301-320. Cambridge, MA.: MIT Press.
17. Mobahi, H., Collobert, R., & Weston, J. (2009). Deep learning from temporal coherence in video. ICML 2009. ACM. To appear.
18. Ranzato, M., Boureau, Y., & LeCun, Y. (2008). Sparse feature learning for deep belief networks. NIPS 20.
19. Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007). Efficient learning of sparse representations with an energy-based model. NIPS 19.
20. Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019-1025.
21. Rust, N., Schwartz, O., Movshon, J. A., & Simoncelli, E. (2005). Spatiotemporal elements of macaque V1 receptive fields. Neuron, 46, 945-956.
22. Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008). Extracting and composing robust features with denoising autoencoders. ICML 2008 (pp. 1096-1103). ACM.
23. Weston, J., Ratle, F., & Collobert, R. (2008). Deep learning via semi-supervised embedding. ICML 2008 (pp. 1168-1175). New York, NY, USA: ACM.
24. Wiskott, L., & Sejnowski, T. (2002). Slow feature analysis: Unsupervised learning of invariances. Neural Computation, 14, 715-770.