Sparse coding with an overcomplete basis set: A strategy employed by V1?

Vision Research

Volumn 37, Issue 23, 1997, Pages 3311-3325

  Olshausen, Bruno A ^b   Field, David J ^a  

^a Department of Obstetrics Gynecology   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Coding V1; Gabor wavelet; Natural images

Indexed keywords

ARTICLE; CODING; IMAGING; PHOTORECEPTOR; PRIORITY JOURNAL; STRIATE CORTEX; VISUAL CORTEX; VISUAL STIMULATION;

ALGORITHMS; ANIMALS; MAMMALS; MODELS, PSYCHOLOGICAL; VISUAL CORTEX; VISUAL PERCEPTION;

EID: 0030779611     PISSN: 00426989     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0042-6989(97)00169-7     Document Type: Article

References (44)

1. Amari, S., Cichocki, A. & Yang, H. H. (1996). A new learning algorithm for blind signal separation. In Advances in neural information processing systems (Vol. 8). Cambridge, MA: MIT Press.
2. Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network, 3, 213-251.
3. Atick, J. J. & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2, 308-320.
4. Atick, J. J. & Redlich, A. N. (1992). What does the retina know about natural scenes? Neural Computation, 4, 196-210.
5. Baddeley, R. (1996). An efficient code in V1 Nature, 381, 560-561.
6. Barlow, H. B. (1961). Possible principles underlying the transformations of sensory messages. In Rosenblith, W. A. (Ed.), Sensory communication (pp. 217-234). Cambridge, MA: MIT Press.
7. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295-311.
8. Baum, E. B., Moody, J. & Wilczek, F. (1988). Internal representations for associative memory. Biological Cybernetics, 59, 217-228.
9. Bell, A. J. & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129-1159.
10. Comon, P. (1994). Independent component analysis, a new concept? Signal Processing, 36, 287-314.
11. Dan, Y., Atick, J. J. & Reid, R. C. (1996). Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. Journal of Neuroscience, 16, 3351-3362.
12. Daugman, J. G. (1989). Entropy reduction and decorrelation in visual coding by oriented neural receptive fields. IEEE Transactions on Biomedical Engineering, 36, 107-114.
13. Dayan, P., Hinton, G. E., Neal. R. M. & Zemel, R. S. (1995). The Helmholtz machine. Neural Computation, 7, 889-904.
14. De Valois, R. L., Albrecht, D. G. & Thorell, L. G. (1982). Spatial frequency selectivity of cells in macaque visual cortex. Vision Research, 22, 545-559.
15. Dong, D. W. & Atick, J. J. (1995). Temporal decorrelation: a theory of lagged and non-lagged responses in the lateral geniculate nucleus. Network: Computation in Neural Systems, 6, 159-178.
16. Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A, 4, 2379-2394.
17. Field, D. J. (1993). Scale-invariance and self-similar "wavelet" transforms: an analysis of natural scenes and mammalian visual systems. In Farge, M., Hunt, J. & Vascillicos, C. (Eds), Wavelets, fractals, and Fourier transforms (pp. 151-193). Oxford: Oxford University Press.
18. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559-601.
19. Foldiak, P. (1990). Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, 64, 165-170.
20. Foldiak, P. (1995). Sparse coding in the primate cortex. In Arbib, M. A. (Ed.), The handbook of brain theory and neural networks (pp. 895-989). Cambridge, MA: MIT Press.
21. Friedman, J. H. (1987). Exploratory projection pursuit. Journal of the American Statistical Association, 82, 249-266.
22. Fyfe, C. & Baddeley, R. (1995). Finding compact and sparse-distributed representations of visual images. Network, 6, 333-344.
23. Harpur, G. F. (1997). Low entropy coding with unsupervised neural networks. Ph.D. Thesis, Dept. of Electrical Engineering, Cambridge University.
24. Harpur, G. F. & Prager, R. W. (1996). Development of low entropy coding in a recurrent network. Network, 7, 277-284.
25. Hinton, G. E. & Ghahramani, Z. (1997) Generative models for discovering sparse distributed representations. Philosophical Transactions of the Royal Society B (in press).
26. Intrator, N. (1992). Feature extraction using an unsupervised neural network. Neural Computation, 4, 98-107.
27. Kullback, S. (1959). Information theory and statistics. New York: John Wiley & Sons.
28. Law, C. C. & Cooper, L. N. (1994). Formation of receptive fields in realistic visual environments according to the Bienenstock, Cooper, and Munro (BCM) theory. Proceedings of the National Academy of Sciences USA, 91, 7797-7801.
29. Lewicki, M. & Sejnowski, T. J. (1996). Bayesian unsupervised learning of higher order structure. In: Advances in neural information processing systems (Vol. 9). Cambridge, MA: MIT Press.
30. Lu, Z. L., Chubb, C. & Sperling, G. (1996). Independence rejection: an unsupervised learning algorithm for extracting latent source structures from arbitrary image populations. Technical Report MBS 96-15. Institute for Mathematical Behavioral Sciences, UC Irvine.
31. Mallat, S. G. (1989). A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 117, 674-693.
32. Mumford, D. (1994). Neuronal architectures for pattern-theoretic problems. In Koch, C. & Davis, J. L. (Eds), Large scale neuronal theories of the brain (pp. 125-152). Cambridge, MA: MIT Press.
33. Olshausen, B. A. (1996). Learning linear, sparse, factorial codes. AI Memo 1580, Massachusetts Institute of Technology.
34. Olshausen, B. A. & Anderson, C. H. (1994). A model of the spatial-frequency organization in primate striate cortex. In Bower, J. M. (Ed.), The neurobiology of computation (pp. 275-280). Kluwer.
35. Olshausen, B. A. & Field, D. J. (1996a) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607-609.
36. Olshausen, B. A. & Field, D. J. (1996b) Natural image statistics and efficient coding. Network, 7, 333-339.
37. Parker, A. J. & Hawken, M. J. (1988). Two-dimensional spatial structure of receptive fields in monkey striate cortex. Journal of the Optical Society of America A, 5, 598-605.
38. Pearlmutter, B. A. & Parra, L. C. (1996). A context-sensitive generalization of ICA. International Conf. on Neural Information Processing. September 1996, Hong Kong.
39. Press, W. A. & Lee, C. W. (1996). Projection pursuit analysis of the statistical structure in natural scenes. Paper presented at CNS96, Cambridge, MA.
40. Rao, R. P. N. & Ballard, D. H. (1997). Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Computation, 9, 721-763.
41. Saund, E. (1995). A multiple cause mixture model for unsupervised learning. Neural Computation, 7, 51-71.
42. Simoncelli, E. P., Freeman, W. T., Adelson, E. H. & Heeger, D. J. (1992). Shiftable multiscale transforms. IEEE Transactions on Information Theory, 382, 587-607.
43. Tadmor, Y. & Tolhurst, D. J. (1989). The effect of threshold on the relationship between the receptive field profile and the spatial-frequency tuning curve in simple cells of the cat's striate cortex. Visual Neuroscience, 3, 445-454.
44. Zemel, R. S. (1993). A minimum description length framework for unsupervised learning. Ph.D. Thesis, University of Toronto, Department of Computer Science.