The problem of sparse image coding

Journal of Mathematical Imaging and Vision

Volumn 17, Issue 2, 2002, Pages 89-108

  Pece, Arthur E C ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Adaptive representation; Atomic decomposition; ICA; Matching pursuit; Sparse coding; Wavelet

Indexed keywords

COMPUTER VISION; ESTIMATION; IMAGE ANALYSIS; LINEAR EQUATIONS; MATHEMATICAL MODELS; PATTERN MATCHING; STATISTICAL METHODS;

ADAPTIVE REPRESENTATION; GENERATIVE-MODEL FRAMEWORK; IMAGE-GENERATION PROCESS; LINEAR EXPANSIONS; NON-ZERO COEFFICIENTS; SPARSE IMAGE CODING;

IMAGE CODING;

EID: 0036761902     PISSN: 09249907     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1020677318841     Document Type: Article

References (78)

1. H. Attias, "Independent factor analysis," Neural Comp., Vol. 11, No. 4, pp. 803-851, 1999.
2. F. Attneave, "Informational aspects of visual perception," Psychol. Review, Vol. 61, pp. 183-193, 1954.
3. H.B. Barlow, "Possible principles underlying the transformation of sensory messages," in Sensory Communication, W. Rosenblith (Ed.), MIT Press: Cambridge, 1961, pp. 217-234.
4. H.B. Barlow, "Single units and sensation: A neuron doctrine for perceptual psychology?" Perception, Vol. 1, pp. 371-394, 1972.
5. H.B. Barlow, "Unsupervised learning," Neural Comp., Vol. 1, pp. 295-311, 1989.
6. H.B. Barlow, "What is the computational goal of the neocortex?" in Large Scale Neuronal Theories of the Brain, C. Koch (Ed.), MIT Press: Cambridge, MA, 1994, pp. 1-22.
7. E.B. Baum, J. Moody, and F. Wilczeck, "Internal representations for associative memory," Biol. Cybern., Vol. 59, pp. 217-228, 1988.
8. A.J. Bell and T.J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Computation, Vol. 7, pp. 1129-1159, 1995.
9. A.J. Bell and T.J. Sejnowski, "The 'independent components' of natural scenes are edge filters," Vision Res., Vol. 37, pp. 3327-3338, 1997.
10. C.E. Bredfeldt and D.L. Ringach, "Dynamics of spatial frequency tuning in macaque VI," J. Neurosci., Vol. 22, No. 5, pp. 1976-1984, 2002.
11. E.J. Candes and D.L. Donoho, "Curvelets-A surprisingly effective nonadaptive representation for objects with edges," in Curves and Surfaces, L.L. Schumaker et al. (Eds.), Vanderbilt University Press, 1999.
12. S. Celebrini, S. Thorpe, Y. Trotter, and M. Imbert, "Dynamics of orientation coding in area V1 of the awake monkey," Visual Neurosci., Vol. 10, pp. 811-825, 1993.
13. S. Chen, S.A. Billings, and W. Luo, "Orthogonal least squares methods and their application to non-linear system identification," Int. J. Control, Vol. 50, No. 5, pp. 1873-1896, 1989.
14. S. Chen and D.L. Donoho, "Examples of basis pursuit," in Proc. of the SPIE, Vol. 2569, No. 2, pp. 564-574, 1995.
15. R. Coifman and V. Wickerhauser, "Entropy-based algorithms for best basis selection," IEEE Trans. Info. Theory, Vol. 38, No. 2, pp. 713-718, 1992.
16. I. Daubechies, "The wavelet transform, time-frequency localization and signal analysis," IEEE Trans. Info. Theory, Vol. 36, pp. 961-1005, 1990.
17. J.G. Daugman, "Uncertainty relations for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters," J. Opt. Soc. of Am. A, Vol. 2, pp. 1160-1169, 1985.
18. G. Davis, S. Mallat, and M. Avellaneda, "Greedy adaptive approximation," J. Constructive Approximation, Vol. 13, pp. 57-98, 1997.
19. D.L. Donoho, "De-noising by soft thresholding," IEEE Trans. Info. Theory, Vol. 41, pp. 613-627, 1995.
20. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, 2nd edn., Wiley: New York, 2001.
21. D.J. Field, "Relations between the statistics of natural images and the response properties of cortical cells," J. Opt. Soc. Am. A, Vol. 4, pp. 2379-2394, 1987.
22. D.J. Field, "What is the goal of sensory coding?" Neural Comp., Vol. 6, pp. 559-601, 1994.
23. P. Foldiak, "Adaptive network for optimal linear feature extraction," in Proc. Int. Joint Conf. Neural Net., Washington, DC, June 18-22, 1989, pp. 401-405.
24. P. Foldiak and M. Young, "Sparse coding in the primate cortex," in The Handbook of Brain Theory and Neural Networks, Michael A. Arbib (Ed.), 1995, pp. 895-898.
25. J.H. Friedman and W. Stuetzle, "Projection pursuit regression," J. Am. Stat. Assoc., Vol. 76, No. 376, pp. 817-823, 1981.
26. E.I. George and R.E. McCulloch, "Approaches for Bayesian variable selection," Statistica Sinica, Vol. 7, pp. 339-373, 1997.
27. A. Gersho and R.M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers: Boston, MA, 1992.
28. G. Golub and C. van Loan, Matrix Computations, 3rd edn., Johns Hopkins University Press: Baltimore, 1996.
29. G.F. Harpur and R.W. Prager, "Development of low entropy coding in a recurrent network," Network, Vol. 7, No. 2, pp. 277-284, 1996.
30. P.A.d.F.R. Højen-Sørensen, O. Winther, and L.K. Hansen, "Mean field approaches to independent component analysis," Neural Computation, Vol. 14, pp. 889-918, 2002.
31. J. Huang and D. Mumford, "Statistics of natural images and models," in Proc. IEEE Conf. Computer Vis. Pattern Rec., Fort Collins, CO, 2000, Vol. 1, pp. 541-547.
32. A. Hyvärinen, "Gaussian moments for noisy independent component analysis," IEEE Signal Proc. Letters, Vol. 6, No. 6, pp. 145-147, 1999.
33. A. Hyvärinen, P.O. Hoyer, and M. Inki, "Topographic independent component analysis," Neural Computation, Vol. 13, No. 7, pp. 1525-1558, 2001.
34. A. Hyvärinen and M. Inki, "Estimating overcomplete independent component bases for image windows," Vol. 17, No. 2, pp. 139-152, 2002.
35. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis, Wiley: New York, 2001.
36. S. Jaggi, W. Karl, S. Mallat, and A. Willsky, "Silhouette recognition using high-resolution pursuit," Pattern Recognition, Vol. 32, No. 5, pp. 753-771, 1999.
37. J. Jones and L. Palmer, "An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex," J. Neurophysiol., Vol. 58, No. 6, pp. 1233-1258, 1987.
38. R.E. Kalman and R.S. Bucy, "New results in linear filtering and prediction theory," Trans. Am. Soc. Mech. Eng. D and J. Basic Eng., Vol. 83, pp. 95-108, 1961.
39. S. Kaski, "Dimensionality reduction by random mapping: Fast similarity computation for clustering," in Proc. of IEEE Int. Joint Conf. Neural Net., IJCNN'98, Anchorage, Alaska, 1998, Vol. 1, pp. 413-418.
40. F.P. Kourouniotis, R.F. Kubichek, N. Boyd, III, and A.K. Majumdar, "Application of the wavelet transform and matching pursuit algorithm in seismic data processing for the development of new noise reduction techniques," in Int. Symposium on Optical Sci. Eng. and Instrumentation, Denver, CO, Aug. 1996.
41. D.D. Lee and H.S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, Vol. 401, pp. 788-791, 1999.
42. M. Lewicki and B. Olshausen, "A probabilistic framework for the adaptation and comparison of image codes," J. Opt. Soc. Am. A, Vol. 16, No. 7, pp. 1587-1601, 1998.
43. S. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," IEEE Trans. Pat. Anal. Mach. Intell., Vol. 11, pp. 674-693, 1989.
44. S. Mallat, A Wavelet Tour of Image Processing, Academic Press: San Diego, 1999.
45. S. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries," IEEE Trans. Signal Processing, Vol. 41, No. 12, pp. 3397-3415, 1993.
46. A.J. Miller, Subset Selection in Regression, Chapman and Hall: London, 1990.
47. J. Miskin and D. MacKay, "Ensemble learning for blind source separation," in Independent Component Analysis: Principle and Practice, S. Roberts and R. Everson (Eds.), Cambridge University Press, 2001, pp. 209-233.
48. T.J. Mitchell and J.J. Beuchamp, "Bayesian variable selection in linear regression," J. Am. Stat. Assoc., Vol. 83, pp. 1023-1032, 1988.
49. D. Mumford and B. Gidas, "Stochastic models for generic images," Quarterly of Applied Mathematics, Vol. 59, No. 11, pp. 85-111, 2001.
50. P.C. Murphy and A.M. Sillito, "Corticofugal feedback influences the generation of length tuning in the visual pathway," Nature, Vol. 329, pp. 727-729, 1987.
51. M. Nafie, M. Ali, and A.H. Tewfik, "Optimal subset selection for adaptive signal representation," in Proc. of the IEEE Int. Conf. on Acoust. Speech and Signal Proc., 1996, pp. 2511-2514.
52. B.K. Natarajan, "Sparse approximate solutions to linear systems," SIAM J. Computing, Vol. 24, No. 2, pp. 227-234, 1995.
53. R. Neff and A. Zakhor, "Very low bit-rate video coding based on matching pursuits," IEEE Trans. Circuits and Syst. for Video Technology, Vol. 7, No. 1, pp. 158-171, 1997.
54. A.N. Netravali and B.G. Haskell, Digital Pictures: Representation and Compression, Plenum Press: New York, 1988.
55. B.A. Olshausen and D.J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, Vol. 381, pp. 607-609, 1996.
56. B.A. Olshausen and D.J. Field, "Sparse coding with an over-complete basis set: A strategy employed by V1?" Vision Res., Vol. 37, pp. 3311-3325, 1997.
57. B.A. Olshausen and K.J. Millman, "Learning sparse codes with a mixture-of-Gaussians prior," in Advances in Neural Information Processing Systems 12, S.A. Solla, T.K. Leen, and K.R. Muller, (Eds.), MIT Press: Cambridge, MA, 2000, pp. 841-847.
58. B.A. Olshausen, P. Sallee, and M.S. Lewicki, "Learning sparse image codes using a wavelet pyramid architecture," in Advances in Neural Information Processing Systems 13, T.K. Leen, T.G. Dietterich, V. Tresp (Eds.), MIT Press: Cambridge, MA, 2000, pp. 887-893.
59. G. Palm, "On associative memory," Biol. Cybern., Vol. 36, pp. 19-31, 1980.
60. A.E.C. Pece, "Redundancy reduction of a Gabor representation: A possible computational role for feedback from primary visual cortex to lateral geniculate nucleus," in Artificial Neural Networks 2, I. Aleksander and J. Taylor (Eds.), Elsevier Science Publishers: Amsterdam, 1992, pp. 865-868.
61. A.E.C. Pece and N. Petkov, "Fast atomic decomposition by the inhibition method," in Proceedings of the 15th International Conference on Pattern Recognition (ICPR 2000), Barcelona, Spain, Sept. 2-8, 2000, pp. 215-218.
62. P.J. Phillips, "Matching pursuit filters applied to face identification," IEEE Trans. Image Proc., Vol. 7, No. 8, pp. 1150-1164, 1998.
63. M.D. Plumbley, "Information processing in negative feedback neural networks," Network, Vol. 7, No. 2, pp. 301-305, 1996.
64. W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C, 2nd edn., Cambridge University Press: Cambridge, 1992.
65. J.G. Robson, "Frequency domain visual processing," in Physical and Biological Processing of Images, O.J. Braddick and A.C. Sleigh (Eds.), Springer, 1983, pp. 73-87.
66. S. Roweis and Z. Ghahramani, "A unifying review of linear Gaussian models," Neural Computation, Vol. 11, No. 2, pp. 305-345, 1999.
67. D.L. Ruderman, "The statistics of natural images," Network, Vol. 5, pp. 517-548, 1994.
68. A. Said and W.A. Pearlman, "A new, fast, and efficient image codec based on set partitioning in hierarchical trees," IEEE Trans. Circuits and Syst. for Video Technol., Vol. 6, No. 3, pp. 243-250, 1996.
69. A. Shmilovici and O. Maimon, "Application of adaptive matching pursuit to adaptive control of nonlinear dynamic systems," IEE Proceedings-Control Theory and Application, Vol. 145, No. 6, pp. 575-582, 1998.
70. E.P. Simoncelli and E.H. Adelson, "Noise removal via bayesian wavelet coring," in Proc. Int. Conf. Im. Proc., Lausanne (CH), Sept. 1996, pp. 379-383.
71. E.P. Simoncelli, W.T. Freeman, E.H. Adelson, and D.J. Heeger, "Shiftable multiscale transforms," IEEE Trans. Info. Theory, Vol. 38, No. 2, pp. 587-607, 1992.
72. M.J. Wainwright, E.P. Simoncelli and A.S. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural images," Applied and Computational Harmonic Analysis, Vol. 11, No. 1, pp. 89-123, 2001.
73. S. Watanabe, Pattern Recognition: Human and Mechanical, Wiley: New York, 1985.
74. D.A. Winfeld, K.C. Gatter, and T.P.S. Powell, "An electron microscopic study of the types and proportions of neurons in the cortex of the motor and visual areas of the cat and rat," Brain, Vol. 103, pp. 245-258, 1990.
75. Y.N. Wu, S.C. Zhu, and C. Guo, "Statistical modeling of texture sketch," in Proceedings of the European Conference on Computer Vision (ECCV 2002), Springer, 2002.
76. H. Yoshida, "Matching pursuit with optimally weighted wavelet packets for extraction of microcalcifications in mammograms," Applied Signal Processing, Vol. 5, No. 3, pp. 127-141, 1999.
77. C. Zetzsche, "Sparse coding: The link between low level vision and associative memory," in Parallel Processing in Neural Systems and Computers, R. Eckmiller, G. Hartmann, and G. Hauske (EDs.), North-Holland: Amsterdam, 1990, pp. 273-276.
78. M. Zibulevsky and B.A. Pearlmutter, "Blind source separation by sparse decomposition in a signal dictionary," Neural Computation, Vol. 13, No. 4, pp. 863-882, 2001.