Probabilistic framework for the adaptation and comparison of image codes

Journal of the Optical Society of America A: Optics and Image Science, and Vision

Volumn 16, Issue 7, 1999, Pages 1587-1601

  Lewicki, Michael S ^a   Olshausen, Bruno A ^b  

^a HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FUNCTIONS; MATHEMATICAL MODELS; PROBABILITY; SIGNAL TO NOISE RATIO; SPURIOUS SIGNAL NOISE; VECTORS;

BAYESIAN SOLUTION; GABOR FUNCTIONS;

IMAGE CODING;

EID: 0032606945     PISSN: 10847529     EISSN: 15208532     Source Type: Journal    
DOI: 10.1364/JOSAA.16.001587     Document Type: Article

References (46)

1. J. G. Daugman, “Uncertainty relation for resolution in space, spatial-frequency, and orientation optimized by two-dimensional visual cortical filters,” J. Opt. Soc. Am. A 2, 1160–1169 (1985).
2. J. G. Daugman, “Complete discrete 2-D Gabor transforms by neural networks for image-analysis and compression,” IEEE Trans. Acoust., Speech, Signal Process. 36, 1169–1179 (1988).
3. J. G. Daugman, “Entropy reduction and decorrelation in visual coding by oriented neural receptive-fields,” IEEE Trans. Biomed. Eng. 36, 107–114 (1989).
4. D. J. Field, “What is the goal of sensory coding,” Neural Comput. 6, 559–601 (1994).
5. T. S. Lee, “Image representation using 2D Gabor wavelets,” IEEE Trans. Pattern. Anal. Mach. Intell. 18, 959–971 (1996).
6. M. S. Lewicki and T. J. Sejnowski, “Learning overcomplete representations,” Neural Comput. (to be published).
7. C. Jutten and J. Herault, “Blind separation of sources. 1. An adaptive algorithm based on neuromimetic architecture,” Signal Process. 24, 1–10 (1991).
8. P. Comon, “Independent component analysis, a new concept,” Signal Process. 36, 287–314 (1994).
9. A. J. Bell and T. J. Sejnowski, “An information maximization approach to blind separation and blind deconvolution,” Neural Comput. 7, 1129–1159 (1995).
10. B. A. Olshausen and D. J. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V1?” Vision Res. 37, 3311–3325 (1997).
11. E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, “Shiftable multiscale transforms,” IEEE Trans. Inf. Theory 38, 587–607 (1992).
12. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” tech. rep. (Stanford University, Stanford, Calif., 1996).
13. R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. Inf. Theory 38, 713–718 (1992).
14. S. G. Mallat and Z. F. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Trans. Signal Process. 41, 3397–3415 (1993).
15. S. C. Zhu, Y. N. Wu, and D. Mumford, “Minimax entropy principle and its application to texture modeling,” Neural Comput. 9, 1627–1660 (1997).
16. B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive-field properties by learning a sparse code for natural images,” Nature (London) 381, 607–609 (1996).
17. P. J. B. Hancock, R. J. Baddeley, and L. S. Smith, “The principal components of natural images,” Network Comput. Neural Syst. 3, 61–70 (1992).
18. C. Fyfe and R. Baddeley, “Finding compact and sparse-distributed representations of visual images,” Network Comput. Neural Syst. 6, 333–344 (1995).
19. R. P. N. Rao and D. H. Ballard, “Dynamic-model of visual recognition predicts neural response properties in the visual-cortex,” Neural Comput. 9, 721–763 (1997).
20. R. P. N. Rao and D. H. Ballard, “Development of localized oriented receptive-fields by learning a translation-invariant code for natural images,” Network Comput. Neural Syst. 9, 219–234 (1998).
21. A. J. Bell and T. J. Sejnowski, “The ‘independent components’ of natural scenes are edge filters,” Vision Res. 37, 3327–3338 (1997).
22. J. H. van Hateren and A. van der Schaaf, “Independent component filters of natural images compared with simple cells in primary visual cortex,” Proc. R. Soc. London, Ser. B 265, 359–366 (1998).
23. C. Zetzsche, E. Barth, and B. Wegmann, “The importance of intrinsically two-dimensional image features in biological vision and picture coding,” in Digital Images and Human Vision, A. B. Watson, ed. (MIT Press, Cambridge, Mass., 1993), pp. 109–138.
24. D. L. Ruderman, “The statistics of natural images,” Network Comput. Neural Syst. 5, 517–548 (1994).
25. H. B. Barlow, “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, W. A. Rosenbluth, ed. (MIT Press, Cambridge, Mass., 1961), pp. 217–234.
26. H. B. Barlow, “Unsupervised learning,” Neural Comput. 1, 295–311 (1989).
27. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipies in C: The Art of Scientific Programming, 2nd ed. (Cambridge U. Press, Cambridge, England, 1992).
28. S. Marcelja, “Mathematical description of the responses of simple cortical cells,” J. Opt. Soc. Am. 70, 1297–1300 (1980).
29. R. L. De Valois, D. G. Albrecht, and L. G. Thorell, “Spatial frequency selectivity of cells in macaque visual cortex,” Vision Res. 22, 545–559 (1982).
30. A. J. Parker and M. J. Hawken, “Two-dimensional spatial structure of receptive fields in monkey striate cortex,” J. Opt. Soc. Am. A 5, 598–605 (1988).
31. J. H. van Hateren and D. L. Ruderman, “Independent component analysis of natural images sequences yield spatiotemporal filters similar to simple cells in primary visual cortex,” Proc. R. Soc. London Ser. B 265, 2315–2320 (1998).
32. I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math. XLI, 909–996 (1988).
33. R. W. Buccigrossi and E. P. Simoncelli, “Image compression via joint statistical characterization in the wavelet domain,” Tech. Rep. 414 (University of Pennsylvania, Philadelphia, Penn., May 1997).
34. E. P. Simoncelli and E. H. Adelson, “Noise removal via Bayesian wavelet coring,” in Proceedings of International Conference IEEE on Image Processing, III Lausanne, Switzerland (Institute of Electrical and Electronics Engineers, New York, 1996), pp. 379–382.
35. S. Chen, “Basis pursuit,” Ph.D. dissertation (Stanford University, Stanford, Calif., 1995). Available at http://www-stat.stanford.edu/reports/chen.s.
36. R. Everson and L. Sirovich, “Karhunen–Loève procedure for gappy data,” J. Opt. Soc. Am. A 12, 1657–1664 (1995).
37. B. A. Pearlmutter and L. C. Parra, “Maximum likelihood blind source separation: a context-sensitive generalization of ICA,” in Advances in Neural and Information Processing Systems M. C. Mozer, M. I. Jordan, and T. Petsche, eds. (Morgan Kaufmann, Los Altos, Calif., 1997), Vol. 9, pp. 613–619.
38. H. Attias, “Independent factor analysis,” Neural Comput. 11, 803–851 (1998).
39. B. D. Rao and K. Kreutz-Delgado, “An affine scaling methodology for best basis selection,” tech. rep. (Center for Information Engineering, University of California, San Diego, San Diego, Calif., 1997).
40. R. M. Neal, Bayesian Learning for Neural Networks (Springer-Verlag, New York, 1996).
41. J.-P. Nadal and N. Parga, “Nonlinear neurons in the lownoise limit: a factorial code maximizes information transfer,” Network 5, 565–581 (1994).
42. J.-P. Nadal and N. Parga, “Redundancy reduction and independent component analysis: conditions on cumulants and adaptive approaches,” Network 5, 565–581 (1994).
43. J-F. Cardoso, “Infomax and maximum likelihood for blind source separation,” IEEE Signal Process. Lett. 4, 109–111 (1997).
44. G. E. Hinton and T. J. Sejnowski, “Learning and relearning in Boltzmann machines,” in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, Chap. 7, pp. 282–317.
45. R. Linsker, “Self-organization in a perceptual network,” Computer 21, 105–117 (1988).
46. J. J. Atick, “Could information-theory provide an ecological theory of sensory processing,” Network Comput. Neural Syst. 3, 213–251 (1992).