The 'independent components' of natural scenes are edge filters

Vision Research

Volumn 37, Issue 23, 1997, Pages 3327-3338

  Bell, Anthony J ^a   Sejnowski, Terrence J ^a  

^a HOWARD HUGHES MEDICAL INSTITUTE   (United States)

Author keywords

Independent components; Information theory; Neural network learning

Indexed keywords

ALGORITHM; ARTICLE; CAT; HUMAN; LEARNING; MONKEY; NERVE CELL; NERVE CELL NETWORK; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; RETINA IMAGE; VISUAL CORTEX; VISUAL DISCRIMINATION;

ALGORITHMS; ANIMALS; CONTRAST SENSITIVITY; HUMANS; MODELS, PSYCHOLOGICAL; VISUAL PERCEPTION;

EID: 0030832881     PISSN: 00426989     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0042-6989(97)00121-1     Document Type: Article

References (54)

1. Amari, S., Cichocki, A. and Yang, H. H. (1996). A new learning algorithm for blind signal separation, Advances in neural information processing systems (Vol. 8). Cambridge, MA: MIT Press.
2. Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network, 3, 213-251.
3. Atick, J. J. & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2, 308-320.
4. Atick, J. J. & Redlich, A. N. (1993). Convergent algorithm for sensory receptive field development. Neural Computation, 5, 45-60.
5. Baddeley, R. (1996). Searching for filters with "interesting" output distributions: an uninteresting direction to explore? Network, in press.
6. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295-311.
7. Barlow, H. B. (1994). What is the computational goal of the neocortex? In Koch, C. (Ed.) Large-scale neuronal theories of the brain. Cambridge, MA: MIT Press.
8. Barlow, H. B. & Tolhurst, D. J. (1992). Why do you have edge detectors? Optical Society of America: Technical Digest, 23, 172
9. Bell, A. J. (1992). Self-organisation in real neurons: anti-Hebb in channel space? In Moody, J. et al. (Eds) Advances in neural information processing systems (Vol. 4, pp. 59-66). Morgan-Kaufmann.
10. Bell, A. J. & Sejnowski, T. J. (1995a). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129-1159.
11. Bell, A. J. & Sejnowski, T. J. (1995b). Fast blind separation based on information theory, in Proc. Intern. Symp. on Nonlinear Theory and Applications, Las Vegas, Dec. 1995.
12. Bell, A. J. & Sejnowski, T. J. (1996). Learning the higher-order structure of a natural sound. Network: Computation in Neural Systems, 7, 2.
13. Bialek, W., Ruderman, D. L. & Zee, A. (1991). Optimal sampling of natural images: a design principle for the visual system? In Touretzky, D. (Ed.), Advances in Neural Information Processing Systems (Vol. 1). Morgan-Kaufmann.
14. Bienenstock, E. L., Cooper, L. N. & Munro, P. W. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. Journal of Neuroscience, 21, 32-48.
15. Burr, D. C. & Morrone, M. C. (1990). Feature detection in biological and artificial vision systems. In Blakemore, C. (Ed.), Vision: coding and efficiency. Cambridge, U.K.: Cambridge University Press.
16. Cardoso, J.-F. & Laheld, B. (1996) Equivariant adaptive source separation, IEEE Trans. on Signal Proc., to appear.
17. Cichocki, A., Unbehauen, R. & Rummert, E. (1994). Robust learning algorithm for blind separation of signals. Electronics Letters, 3017, 1386-1387.
18. Comon, P. (1994). Independent component analysis, a new concept? Signal Processing, 36, 287-314.
19. Cover, T. M. & Thomas, J. A. (1991). Elements of information theory. New York: John Wiley.
20. Daugman, J. G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America A, 27, 1160-1169.
21. Ferster, D., Chung, S. & Wheat, H. (1996). Orientation selectivity of thalamic input to simple cells of cat visual cortex. Nature, 380, 249-252.
22. Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A, 412, 2370-2393.
23. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559-601.
24. Földiák, P. (1990). Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, 64, 165-170.
25. Fyfe, C. & Baddeley, R. (1995). Finding compact and sparse-distributed representations of visual images. Network, 6, 333-344.
26. Goodall, M. C. (1960). Performance of stochastic net. Nature, 185, 557-558.
27. Hancock, P. J. B., Baddeley, R. J. & Smith, L. S. (1992). The principal components of natural images. Network, 3, 61-72.
28. Harpur, G. F. & Prager, R. W. (1996). Development of low entropy coding in a recurrent network. Network, in press.
29. Haykin, S. (Ed.) (1994). Blind deconvolution. New Jersey: Prentice-Hall.
30. Hubel, D. H. & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195, 215-244.
31. Hubel, D. H. & Wiesel, T. N. (1974). Uniformity of monkey striate cortex: a parallel relationship between field size, scatter, and magnification factor. Journal of Comparative Neurology, 158, 295-306.
32. Huber, P. J. (1985). Projection pursuit. The Annals of Statistics, 13, 435-475.
33. Intrator, N. (1992). Feature extraction using an unsupervised neural network. Neural Computation, 4, 98-107.
34. Jutten, C. & Hérault, J. (1991). Blind separation of sources, Part I: an adaptive algorithm based on neuromimetic architecture. Signal Processing, 24, 1-10.
35. Karhunen, J., Wang, L. & Jousensalo, J. (1995). Neural estimation of basis vectors in Independent Component Analysis, Proc. ICANN, Paris, 1995.
36. Karhunen, J., Oja, E., Wang, L., Vigario, R. & Joutsenalo, J. (1996). A class of neural networks for independent component analysis. submitted to IEEE Trans. on Neural Networks.
37. Laughlin, S. (1981). A simple coding procedure enhances a neuron's information capacity. Z. Naturforsch., 36, 910-912.
38. Law, C. C. & Cooper, L. N. (1994). Formation of receptive fields in realistic visual environments according to the Bienestock, Cooper and Munro (BCM) theory. Proceedings of the National Academy of Sciences USA, 91, 7797-7801.
39. Lee, T.-W., Bell, A. J. & Lambert, R. (1997). Blind separation of delayed and convolved sources, in Advances in neural information processing systems (Vol. 9). Cambridge, MA: MIT Press.
40. Linsker, R. (1992). Local synaptic learning rules suffice to maximise mutual information in a linear network. Neural Computation, 4, 691-702.
41. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105-117.
42. Marr, D. & Hildreth, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London B, 207, 187-217.
43. Miller, K. D. (1988). Correlation-based models of neural development. In Gluck, M. & Rumelhart, D. (Eds), Neuroscience and connectionist theory (pp. 267-353). Hillsdale, NJ: Lawrence Erlbaum.
44. Nadal, J-P. & Parga, N. (1994). Non-linear neurons in the low noise limit: a factorial code maximises information transfer. Network, 5, 565-581.
45. Oja, E. (1989). Neural networks, principal components and linear neural networks. Neural Networks, 5, 927-935.
46. Olshausen, B. A. (1996) Learning linear, sparse, factorial codes, MIT AI-memo No. 1580, AI-lab, MIT.
47. Olshausen, B. A. & Field, D. J. (1996). Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7, 2.
48. Pham, D. T., Garrat, P. & Jutten, C. (1992). Separation of a mixture of independent sources through a maximum likelihood approach, in Proc. EU-SIPCO, 771-774.
49. Ruderman, D. L. (1994). The statistics of natural images. Network: Computation in Neural Systems, 5, 517-548.
50. Sanger, T. D. (1989). Optimal unsupervised learning in a single-layer network. Neural Networks, 2, 459-473.
51. Schmidhuber, J., Eldracher, M. & Foltin, B. (1996). Semi-linear predictability minimization produces well-known feature detectors, Neural Computation, in press.
52. Shouval, H. (1995). Formation and organisation of receptive fields, with an input environment composed of natural scenes, Ph.D. thesis, Dept. of Physics, Brown University.
53. Torkkola, K. (1996). Blind separation of convolved sources based on information maximisation. Proc. IEEE Workshop on Neural Networks and Signal Processing, Kyota, Japan, Sept. 1996.
54. van Hateren, J. H. (1992). A theory of maximising sensory information. Biol. Cybern., 68, 23-29.