Topographic product models applied to natural scene statistics

Neural Computation

Volumn 18, Issue 2, 2006, Pages 381-414

  Osindero, Simon ^a   Welling, Max ^b   Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b Irvine   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; METHODOLOGY; PHYSIOLOGY; VISUAL SYSTEM;

ALGORITHMS; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; VISUAL PATHWAYS;

EID: 33644900506     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/089976606775093936     Document Type: Article

References (36)

1. Andrews, D., & Mallows, C. (1974). Scale mixtures of normal distributions. Journal of the Royal Statistical Society, 36, 99-102.
2. Atick, J.J., & Redlich, A. N. (1992). What does the retina know about natural scenes. Neural Computation, 4(2), 196-210.
3. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
4. Bell, A. J., & Sejnowski, T. J. (1997). The "independent components" of natural scenes are edge filters. Vision Research, 37(23), 3327-3338.
5. Carreira-Perpinan, M., & Hinton, G. (2005). On contrastive divergence learning. In R. Cowell & Z. Ghahrami (Eds.), Proceedings of the Society for Artificial Intelligence and Statistics, Barbados. Available online at http://www.gatsby.ucl.ac.uk/aistats.
6. Freund, Y., & Haussler, D. (1992). Unsupervised learning of distributions of binary vectors using 2-layer networks. In C. L. Giles, S. J. Hanson, & J. D. Cawan (Eds.), Advances in neural information processing systems, 4 (pp. 912-919). San Francisco: Morgan Kaufmann.
7. Heskes, T. (1998). Selecting weighting factors in logarithmic opinion pools. In M. I. Jordan, M. J. Kearns, & S. A. Solla (Eds.), Advances in neural information processing systems, 10. Cambridge, MA: MIT Press.
8. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771-1800.
9. Hinton, G., & Teh, Y. (2001). Discovering multiple constraints that are frequently approximately satisfied. In Proceedings of the Conference on Uncertainty in Artificial intelligence (pp. 227-234). San Francisco: William Kaufmann.
10. Hoyer, P. O., & Hyvarinen, A. (2000). Independent component analysis applied to feature extraction from colour and stereo images. Network Computation in Neural Systems, 11(3), 191-210.
11. Hyvarinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413-2423.
12. Hyvarinen, A., Hoyer, P. O., & Inki, M. (2001). Topographic independent component analysis. Neural Computation, 13(7), 1527-1558.
13. Karklin, Y., & Lewicki, M. S. (2003). Learning higher-order structures in natural images. Network Computation in Neural Systems, 14, 483-499.
14. Karklin, Y., & Lewicki, M. S. (2005). A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals. Neural Computation, 17, 397-423.
15. Lewicki, M., & Sejnowski, T. (2000). Learning overcomplete representations. Neural Computation, 12, 337-365.
16. Marks, T. K., & Movellan, J. R. (2001). Diffusion networks, products of experts, and factor analysis (Tech. Rep. UCSD MPLab TR 2001.02). San Diego: University of California, San Diego.
17. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607-610.
18. Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23), 3311-3325.
19. Portilla, J., Strela, V., Wainwright, M., & Simoncelli, E. P. (2003). Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans. Image Processing, 12(11), 1338-1351.
20. Ringach, D. L. (2002). Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J. Neurophysiol., 88(1), 455-463.
21. Simoncelli, E. (1997). Statistical models for images: Compression, restoration and synthesis. In Proceedings of the 31st Asilomar Conference on Signals, Systems and Computers. Pacific Grove, CA: IEEE Computer Society.
22. Smolensky, P. (1986). Information processing in dynamical systems: Foundations of harmony theory. In D. Rumelhart & J. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Vol. 1: Foundations. New York: McGraw-Hill.
23. Teh, Y., Welling, M., Osindero, S., & Hinton, G. (2003). Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research [Special Issue] 4, 1235-1260.
24. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B Biol. Sci., 265(1394), 359-366.
25. Wainwright, M. J., & Simoncelli, E. P. (2000). Scale mixtures of gaussians and the statistics of natural images. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 855-861). Cambridge, MA: MIT Press.
26. Wainwright, M. J., Simoncelli, E. P., & Willsky, A. S. (2000). Random cascades of gaussian scale mixtures and their use in modeling natural images with application to denoising. In Proceedings of the 7th International Conference on Image Processing. Vancouver, BC: IEEE Computer Society.
27. Welling, M., Agakov, F., and Williams, C. (2003). Extreme components analysis. In S. Thrün, L. K. Savl, & B. Schölkopf (Eds.), Advances in neural information processing systems, 16. Cambridge, MA: MIT Press.
28. Welling, M., Hinton, G., & Osindero, S. (2002). Learning sparse topographic representations with products of student-t distributions. In S. Becker, S. Thrün, & K. Obermayer (Eds.), Advances in neural information processing systems, 15. Cambridge, MA: MIT Press.
29. Welling, M., Zemel, R., & Hinton, G. (2002). Self-supervised boosting. In S. Becker, S. Thrün, & K. Obermayer (Eds.), Advances in neural information processing systems, 15. Cambridge, MA: MIT Press.
30. Welling, M., Zemel, R., & Hinton, G. (2003). A tractable probabilistic model for projection pursuit. In Proceedings of the Conference on Uncertainty in Artificial Intelligence. San Francisco: Morgan Kaufmann.
31. Welling, M., Zemel, R., & Hinton, G. (2004). Probabilistic sequential independent components analysis. IEEE Transactions in Neural Networks [Special Issue], 15, 838-849.
32. Williams, C. K. I., & Agakov, F. (2002). An analysis of contrastive divergence learning in gaussian Boltzmann machines (Tech. Rep. EDI-INF-RR-0120). Edinburgh: School of Informatics, University of Edinburgh.
33. Williams, C., Agakov, F., & Felderhof, S. (2001). Products of gaussians. In T. G. Dietterich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural information processing systems, 14. Cambridge, MA: MIT Press.
34. Willmore, B., & Tolhurst, D. J. (2001). Characterizing the sparseness of neural codes. Network Computation in Neural Systems, 12(3), 255-270.
35. Yuille, A. (2004). A comment on contrastive divergence (Tech. Rep.). Los Angeles: Department Statistics and Psychology, UCLA.
36. Zhu, S. C., Wu, Y. N., & Mumford, D. (1998). Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2), 107-126.