Soft-competitive learning of sparse codes and its application to image reconstruction

Neurocomputing

Volumn 74, Issue 9, 2011, Pages 1418-1428

  Labusch, Kai ^a   Barth, Erhardt ^a   Martinetz, Thomas ^a  

^a UNIVERSITY OF LÜBECK   (Germany)

Author keywords

Dictionary learning; Image reconstruction; Sparse approximation; Vector quantization

Indexed keywords

COMPETITIVE LEARNING; COMPUTATION TIME; DICTIONARY LEARNING; DISCRETE COSINE TRANSFORMATION; GRADIENT APPROACH; HAAR-WAVELETS; IMAGE ENCODING; LIMITED TRAINING DATA; NEURAL GAS; OPTIMAL DIRECTION; OVER-COMPLETE; OVERCOMPLETE DICTIONARIES; SPARSE APPROXIMATIONS; SPARSE CODES; SPARSE CODING; STATE-OF-THE-ART METHODS;

IMAGE CODING; IMAGE RECONSTRUCTION; VECTOR QUANTIZATION;

LEARNING ALGORITHMS;

ARTICLE; CONTROLLED STUDY; IMAGE ANALYSIS; IMAGE DISPLAY; IMAGE RECONSTRUCTION; INTERMETHOD COMPARISON; LEARNING ALGORITHM; MATHEMATICAL ANALYSIS; NEURAL GAS OF DICTIONARY ALGORITHM; PRIORITY JOURNAL;

EID: 79953050592     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2011.02.002     Document Type: Article

References (41)

1. Antonini M., Barlaud M., Mathieu P., Daubechies I. Image coding using wavelet transform. IEEE Transactions on Image Processing 1992, 1(2):205-220.
2. Starck J.L., Bijaoui A., Lopez B., Perrier C. Image reconstruction by the wavelet transform applied to aperture synthesis. Astronomy and Astrophysics 1994, 283(1):349-360.
3. Chang S.G., Bin Y., Vetterli M. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing 2000, 9(9):1532-1546.
4. A.M. Bruckstein, D.L. Donoho, M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, 2007.
5. S. Mallat, A wavelet tour of signal processing, The Sparse Way, third ed., Academic Press, December 2008.
6. Aharon M., Elad M., Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing 2006, 54(11):4311-4322. (see also IEEE Transactions on Acoustics, Speech, and Signal Processing).
7. Mairal J., Elad M., Sapiro G. Sparse representation for color image restoration. IEEE Transactions on Image Processing 2008, 17(1).
8. Bruckstein A.M., Donoho D.L., Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review 2009, 51(1):34-81.
9. Donoho D.L., Elad M., Temlyakov V.N. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory 2006, 52(1):6-18.
10. Gribonval R., Figueras R., Vandergheynst P. A simple test to check the optimality of a sparse signal approximation. Signal Processing 2006, 86:496-510.
11. Davis G., Mallat S., Avellaneda M. Greedy adaptive approximation. Journal of Construction Approximation 1997, 13:57-89.
12. Candés E.J., Romberg J., Tao T. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics 2006, 59:1207-1233.
13. Donoho D.L. For most large underdetermined systems of linear equations, the minimal l1-norm near-solution approximates the sparsest near solution. Communications on Pure and Applied Mathematics 2006, 59:907-934.
14. Donoho D.L. For most large underdetermined systems of linear equations, the minimal l1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics 2006, 59:797-829.
15. Chen S.S., Donoho D.L., Saunders M.A. Atomic decomposition by basis pursuit. SIAM Review 2001, 43(1):129-159.
16. Rao B.D., Engan K., Cotter S.F., Palmer J., Kreutz-Delgado K. Subset selection in noise based on diversity measure minimization. IEEE Transactions on Signal Processing 2003, 51:760-770.
17. Osborne M.R., Presnell B., Turlach B.A. A new approach to variable selection in least squares problems. Journal of Numerical Analysis 2000, 20:389-403.
18. Daubechies I., Defrise M., De-Mol C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics 2004, 57:1413-1457.
19. Rozell C.J., Johnson D.H., Baraniuk R.G. Sparse coding via thresholding and local competition in neural circuits. Neural Computation 2008, 20:2526-2563.
20. Y. Pati, R. Rezaiifar, P. Krishnaprasad, Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decomposition. Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, November 1993.
21. Rebollo-Neira L., Lowe D. Optimized orthogonal matching pursuit approach. IEEE Signal Processing Letters 2002, 9(4):137-140.
22. Tropp J.A. Greed is good: algorithmic results for sparse approximation. IEEE Transactions on Information Theory 2004, 50(10):2231-2242.
23. Tropp J.A., Gilbert A.C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory 2007, 53(12):4655-4666.
24. Rehn M., Sommer F.T. A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. Journal of Computational Neuroscience 2007, 22(2):135-146.
25. Donoho D.L. Compressed sensing. IEEE Transactions on Information Theory 2006, 52(4):1289-1306.
26. B. Olshausen, D. Field, Sparse coding of natural images produces localized, oriented, bandpass receptive fields, Technical Report CCN-110-95, Department of Psychology, Cornell University, Ithaca, New York 14853, 1995.
27. Olshausen B.A., Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 1996, 381:607-609.
28. Lewicki M.S., Sejnowski T.J. Learning nonlinear overcomplete representations for efficient coding. NIPS '97: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 1998, vol. 10:556-562. MIT Press, Cambridge, MA, USA.
29. Lewicki M.S., Sejnowski T.J. Learning overcomplete representations. Neural Computation 2000, 12(2):337-365.
30. Kreutz-Delgado K., Murray J.F., Rao B.D., Engan K., Lee T.-W., Sejnowski T.J. Dictionary learning algorithms for sparse representation. Neural Computation 2003, 15(2):349-396.
31. Lee H., Battle A., Raina R., Ng A.Y. Efficient sparse coding algorithms. Advances in Neural Information Processing Systems 2007, vol. 19:801-808. MIT Press, Cambridge, MA.
32. Engan K., Aase S.O., Husoy J.H. Method of optimal directions for frame design. ICASSP '99: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing 1999, 2443-2446. IEEE Computer Society, Washington, DC, USA.
33. K. Labusch, E. Barth, T. Martinetz, Learning data representations with sparse coding neural gas, in: M. Verleysen (Ed.), Proceedings of the 16th European Symposium on Artificial Neural Networks, D-Side Publishers, 2008, pp. 233-238.
34. Labusch K., Barth E., Martinetz T. Sparse coding neural gas: learning of overcomplete data representations. Neurocomputing 2009, 72(7-9):1547-1555.
35. Hartigan J.A., Wong M.A. A K-means clustering algorithm. Applied Statistics 1979, 28:100-108.
36. Linde Y., Buzo A., Gray R. An algorithm for vector quantizer design. IEEE Transactions on Communications 1980, 28(1):84-95.
37. Martinetz T., Schulten K. A "neural-gas network" learns topologies. Artificial Neural Networks I 1991, 397-402.
38. Martinetz T., Berkovich S., Schulten K. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks 1993, 4(4):558-569.
39. Cottrell M., Hammer B., Hasenfuß A., Villmann T. Batch and median neural gas. Neural Networks 2006, 19(6):762-771.
40. Mallat S., Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 1993, 41:3397-3415.
41. Monaci G., Vandergheynst P., Sommer F.T. Learning bimodal structure in data. IEEE Transactions on Neural Networks 2009, 20:1898-1910.