Learning sparse overcomplete codes for images

Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology

Volumn 45, Issue 1-2, 2006, Pages 97-110

  Murray, Joseph F ^a   Kreutz Delgado, Kenneth ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

FOCUSS CNDL; Image compression; Independent components analysis (ICA); Matching pursuit; Non negative; Sparse Bayesian learning; Sparse overcomplete coding

Indexed keywords

ALGORITHMS; FEATURE EXTRACTION; IMAGE COMPRESSION; INDEPENDENT COMPONENT ANALYSIS; LEARNING SYSTEMS; PATTERN RECOGNITION;

FEATURE SELECTION; OVERCOMPLETE CODES; SPARSE CODING;

IMAGE CODING;

EID: 33846364251     PISSN: 13875485     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11265-006-9774-5     Document Type: Conference Paper

References (39)

1. R. C. Gonzales and R. E. Woods, "Digital Image Processing," Addison-Wesley, Reading, Massachusetts, 1993.
2. E. Oja, "A Simplified Neuron Model as a Principal Component Analyzer," J. Math. Biol., vol. 15, 1982, pp. 267-273.
3. p Deconvolution," IEEE Trans. Acoust. Speech Signal Process, vol. 37, no. 1. January 1989, pp. 73-82.
4. C. Jutten and J. Hérault, "Blind Separation of Sources. Part I. An Adaptive Algorithm Based on Neuromimetic Architecture," Signal Process, vol. 24, 1991, pp. 1-10.
5. K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski, "Dictionary Learning Algorithms for Sparse Representation," Neural Comput., vol. 15, no. 2, February 2003, pp. 349-396.
6. M. S. Lewicki and T. J. Sejnowski, "Learning Overcomplete Representations," Neural Comput., vol. 12, no. 2, February 2000, pp. 337-365.
7. K. Engan, J. H. Husoy, and S. O. Aase, "Frame Based Representation and Compression of Still Images," in Proc. ICIP, 2001, pp. 1-4.
8. B. A. Olshausen and D. J. Field, "Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1?," Vis. Res., vol. 37, 1997, pp. 3311-3325.
9. D. M. Weber and D. Casasent, "Quadratic Gabor Filters for Object Detection," IEEE Trans. Image Process, vol. 10, no. 2, February 2001, pp. 218-230.
10. B. D. Rao and K. Kreutz-Delgado, "An Affine Scaling Methodology for Best Basis Selection," IEEE Trans. Signal Process, vol. 47, 1999, pp. 187-200.
11. M. E. Tipping, "Sparse Bayesian Learning and the Relevance Vector Machine," J. Mach. Learn. Res., vol. 1, 2001, pp. 211-244.
12. S. F. Cotter, J. Adler, B. D. Rao, and K. Kreutz-Delgado, "Forward Sequential Algorithms for Best Basis Selection," IEE Proc., Vis. Image Signal Process, vol. 146, no. 5, October 1999, pp. 235-244.
13. J. F. Murray and K. Kreutz-Delgado, "An Improved FOCUSS-based Learning Algorithm for Solving Sparse Linear Inverse Problems," in Conference Record of the 35th Asilomar Conference on Signals, Systems and Computers (IEEE), vol. 1, Pacific Grove, California, November 2001, pp. 347-351.
14. M. S. Lewicki and B. A. Olshausen, "A probabilistic Framework for the Adaptation and Comparison of Image Codes," J. Opt. Soc. Am. A, vol. 16, no. 7, July 1999, pp. 1587-1601.
15. K. Engan, "Frame Based Signal Representation and Compression." PhD thesis, Stavanger Universty College, Norway, 2000.
16. P. Paatero and U. Tapper, "Positive Matrix Factorization: A Nonnegative Factor Model with Optimal Utilization of Error Estimates of Data Values," Environmetrics, vol. 5, 1994, pp. 111-126.
17. D. D. Lee and H. S. Seung, "Learning the Parts of Objects by Non-negative Matrix Factorization," Nature, vol. 401, October 1999, pp. 788-791.
18. P. O. Hoyer, "Non-negative Sparse Coding," in Proc. of the 12th IEEE Workshop on Neural Networks for Sig. Proc., 2002, pp. 557-565.
19. M. D. Plumbley, "Algorithms for Nonnegative Independent Component Analysis," IEEE Trans. Neural Netw., vol. 14, no. 3, May 2003, pp. 534-543.
20. S. Chen and D. Donoho, "Atomic Decomposition by Basis Pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, 1998, pp. 33-61.
21. 1 Minimization," Proc. Natl. Acad. Sci. U. S. A., vol. 100, no. 5, March 2003, pp. 2197-2202.
22. p Diversity Measures," in Proc. of the Intl. Conf. on Acoustics, Speech, and Signal Processing, 2004 (ICASSP '04), vol. 2, May 2004.
23. D. P. Wipf and B. D. Rao, "Sparse Bayesian Learning for Basis Selection," IEEE Trans. Signal Process, vol. 52, no. 8, 2004, pp. 2153-2164.
24. I. F. Gorodnitsky, J. S. George, and B. D. Rao, "Neuromagnetic Source Imaging with FOCUSS: A Recursive Weighted Minimum Norm Algorithm," Electroencephalogr. Clin. Neurophysiol., vol. 95, no. 4, 1995, pp. 231-251.
25. R. Vigário and E. Oja, "Independence: A New Criterion for the Analysis of the Electromagnetic Fields in the Global Brain?," Neural Netw., vol. 13, 2000, pp. 891-907.
26. C. M. Bishop and M. E. Tipping, "Bayesian Regression and Classification," in Advances in Learning Theory: Methods, Models and Applications, volume 190 of NATO Science Series III: Computer & Systems Sciences, J. Suykens, G. Horvath, S. Basu, C. Micchelli, and J. Vandewalle (Eds.), IOS, Amsterdam, 2003, pp. 267-285.
27. R. J. Muirhead, "Aspects of Multivariate Statistical Theory," Wiley, New York, 1982.
28. S. G. Mallat and Z. Zhang, "Matching Pursuits with Time-frequency Dictionaries," IEEE Trans. Signal Process, vol. 41, no. 12, 1993, pp. 3397-3415.
29. S. F. Cotter, "Subset Selection Algorithms with Applications." PhD thesis, University of California at San Diego, 2001.
30. K. Engan, K. Skretting, and J. H. Husoy, "A Family of Iterative LS-based Dictionary Learning Algorithms, ILS-DLA, for Sparse Signal Representation," to appear, IEEE Trans. on Signal Processing.
31. M. Girolami, "A Variational Method for Learning Sparse and Overcomplete Representations," Neural Comput., vol. 13, 2001, pp. 2517-2532.
32. J. A. Palmer and K. Kreutz-Delgado, "A General Framework for Component Estimation," in Proc. 4th Intl. Symp. on ICA, 2003.
33. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation," to appear, IEEE Trans. on Signal Processing.
34. S. A. Nene, S. K. Nayar, and H. Murase, "Columbia Object Image Library (COIL-100)," Technical Report CUCS-006-96, Columbia University, 1996.
35. G. H. Golub and C. F. V. Loan, "Matrix Computations," John Hopkins University Press, Baltimore, Maryland, 1983.
36. S. Peng and S. Sedukhin, "Parallel Algorithm and Architectures for Two-step Division-free Gaussian Elimination," in Algorithms and Architectures for Parallel Processing, 3rd International Conference on (ICAPP 97), 1997, pp. 489-502.
37. J.-C. Tsay and P.-Y. Chang, "Design of Efficient Regular Arrays for Matrix Multiplication by Two-step Regularization," IEEE Trans. Parallel Distrib. Syst., vol. 6, no. 2, February 1995, pp. 215-222.
38. K. Muhammad and K. Roy, "Reduced Computational Redundancy Implementation of DSP Algorithms Using Computation Sharing Vector Scaling," IEEE Transactions on VLSI Systems, vol. 10, no. 3, 2002, pp. 292-300.
39. D. Zhang, "Parallel VLSI Neural System Design," Springer, Berlin Heidelberg New York, 1998.