Sparse Coding Neural Gas: Learning of overcomplete data representations

Neurocomputing

Volumn 72, Issue 7-9, 2009, Pages 1547-1555

  Labusch, Kai ^a   Barth, Erhardt ^a   Martinetz, Thomas ^a  

^a UNIVERSITY OF LÜBECK   (Germany)

Author keywords

Matching Pursuit; Sparse coding; Unsupervised learning; Vector quantization

Indexed keywords

ARTIFICIAL DATUM; BASIS FUNCTIONS; BASIS VECTORS; DATA REPRESENTATIONS; LINEAR COMBINATIONS; MATCHING PURSUIT; NATURAL IMAGES; NEURAL GAS; NEURAL GAS ALGORITHMS; ORTHOGONAL MATCHING PURSUITS; OVER-COMPLETE; OVERCOMPLETE REPRESENTATIONS; PRIMARY VISUAL CORTICES; RECEPTIVE FIELDS; REFERENCE IMPLEMENTATIONS; SPARSE CODES; SPARSE CODING; STATE-OF-THE-ART METHODS; TRAINING SAMPLES;

EDUCATION; UNSUPERVISED LEARNING; VECTOR QUANTIZATION;

LEARNING ALGORITHMS;

ANALYTIC METHOD; ARTICLE; DATA ANALYSIS; IMAGE ANALYSIS; INTERMETHOD COMPARISON; LEARNING ALGORITHM; LEARNING STYLE; LINEAR SYSTEM; MATHEMATICAL COMPUTING; NERVE CELL; PRIORITY JOURNAL; RECEPTIVE FIELD; VISUAL CORTEX;

EID: 61849103486     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2008.11.027     Document Type: Article

References (38)

1. Aharon M., Elad M., and Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing 54 11 (2006) 4311-4322
2. Bell A.J., and Sejnowski T.J. The "independent components" of natural scenes are edge filters. Vision Research 37 23 (1997) 3327-3338
3. Davies M., and Mitianoudis N. Simple mixture model for sparse overcomplete ICA. IEEE Proceedings on Vision, Image and Signal Processing 151 1 (2004) 35-43
4. Davis G., Mallat S., and Avellaneda M. Greedy adaptive approximation. Journal of Constructive Approximations 13 (1997) 57-89
5. Donoho D.L., Elad M., and Temlyakov V.N. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory 52 1 (2006) 6-18
6. Engan K., Aase S.O., and Husoy J.H. Multi-frame compression: theory and design. Signal Processing 80 10 (2000) 2121-2140
7. Field D.J. What is the goal of sensory coding?. Neural Computation 6 4 (1994) 559-601
8. Hartigan J.A., and Wong M.A. A K-means clustering algorithm. Applied Statistics 28 (1979) 100-108
9. Hoyer P., and Oja E. Image denoising by sparse code shrinkage. Intelligent Signal Processing (2000), IEEE Press, New York
10. Hyvärinen A. Gaussian moments for noisy independent component analysis. IEEE Signal Processing Letters 6 6 (1999) 145-147
11. A. Hyvarinen, R. Cristescu, E. Oja, A fast algorithm for estimating overcomplete ICA bases for image windows, in: Proceedings of the International Joint Conference on Neural Networks, IJCNN'99, vol. 2, 1999, pp. 894-899.
12. Hyvärinen A., and Hoyer P. Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation 12 7 (2000) 1705-1720
13. Hyvarinen A., Karhunen J., and Oja E. Independent Component Analysis (May 2001), Wiley, New York
14. Kreutz-Delgado K., Murray J.F., Rao B.D., Engan K., Lee T.-W., and Sejnowski T.J. Dictionary learning algorithms for sparse representation. Neural Computation 15 2 (2003) 349-396
15. Kushner H.J., and Clark D.S. Stochastic Approximation Methods for Constrained and Unconstrained Systems (1978), Springer, Berlin
16. Labusch K., Barth E., and Martinetz T. Learning data representations with Sparse Coding Neural Gas. In: Verleysen M. (Ed). Proceedings of the 16th European Symposium on Artificial Neural Networks (2008), D-Side Publishers 233-238
17. Labusch K., Barth E., and Martinetz T. Simple method for high-performance digit recognition based on sparse coding. IEEE Transactions on Neural Networks 19 11 (2008) 1985-1989
18. K. Labusch, E. Barth, T. Martinetz, Sparse coding neural gas for the separation of noisy overcomplete sources, in: V. Kurková, R. Neruda, J. Koutník (Eds.), Artificial Neural Networks-ICANN 2008, 18th International Conference, Prague, Czech Republic, September 3-6, 2008, Proceedings, Part II, Lecture Notes in Computer Science, vol. 5163, Springer, 2008, pp. 788-797.
19. Lee H., Battle A., Raina R., and Ng A.Y. Efficient sparse coding algorithms. In: Schölkopf B., Platt J., and Hoffman T. (Eds). Advances in Neural Information Processing Systems vol. 19 (2007), MIT Press, Cambridge, MA 801-808
20. Lee T.-W., Lewicki M.S., Girolami M., and Sejnowski T.J. Blind source separation of more sources than mixtures using overcomplete representations. IEEE Signal Processing Letters 6 4 (1999) 87-90
21. Lewicki M.S., and Olshausen B.A. Probabilistic framework for the adaptation and comparison of image codes. Journal of Optical Society of America A 16 7 (1999) 1587-1601
22. Lewicki M.S., and Sejnowski T.J. Learning overcomplete representations. Neural Computation 12 2 (2000) 337-365
23. Martinetz T., Berkovich S., and Schulten K. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE-Transactions on Neural Networks 4 4 (1993) 558-569
24. Martinetz T., and Schulten K. A "neural-gas network" learns topologies. Artificial Neural Networks I (1991) 397-402
25. Mutch J., and Lowe D.G. Multiclass object recognition with sparse, localized features. CVPR '06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006), IEEE Computer Society, Washington, DC, USA 11-18
26. Oja E. A simplified neuron model as a principal component analyzer. Journal of Mathematical Biology 15 (1982) 267-273
27. B. Olshausen, D. Field, Sparse coding of natural images produces localized, oriented, bandpass receptive fields, Technical Report CCN-110-95, Department of Psychology, Cornell University, Ithaca, New York 14853, 1995.
28. Olshausen B.A., and Field D.J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (1996) 607-609
29. Olshausen B.A., and Field D.J. Sparse coding with an overcomplete basis set: a strategy employed by V1?. Vision Research 37 23 (1997) 3311-3325
30. Y. Pati, R. Rezaiifar, P. Krishnaprasad, Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition, in: Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, November 1993.
31. Rebollo-Neira L., and Lowe D. Optimized orthogonal matching pursuit approach. IEEE Signal Processing Letters 9 4 (2002) 137-140
32. Schleif F.-M., Ongyerth M., and Villmann T. Sparse coding neural gas for analysis of nuclear magnetic resonance spectroscopy. CBMS (2008), IEEE Computer Society, Silver Spring, MD 620-625
33. Serre T., Wolf L., Bileschi S., Riesenhuber M., and Poggio T. Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence 29 3 (2007) 411-426
34. Simoncelli E.P., and Olshausen B.A. Natural image statistics and neural representation. Annual Review of Neuroscience 24 (2001) 1193-1216
35. Mallat S., and Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41 (1993) 3397-3415
36. Theis F., Lang E., and Puntonet C. A geometric algorithm for overcomplete linear ICA. Neurocomputing 56 (2004) 381-398
37. Tropp J.A. Greed is good: algorithmic results for sparse approximation. IEEE Transactions on Information Theory 50 10 (2004) 2231-2242
38. Tropp J.A., and Gilbert A.C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory 53 12 (2007) 4655-4666