Learning overcomplete representations

Neural Computation

Volumn 12, Issue 2, 2000, Pages 337-365

  Lewicki, Michael S ^a   Sejnowski, Terrence J ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b HOWARD HUGHES MEDICAL INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ANIMAL; ARTICLE; BIOLOGICAL MODEL; HUMAN; LEARNING; PROBABILITY; SPEECH; STATISTICAL MODEL;

ALGORITHMS; ANIMALS; HUMANS; LEARNING; LIKELIHOOD FUNCTIONS; MODELS, NEUROLOGICAL; MODELS, STATISTICAL; PROBABILITY; SPEECH;

EID: 0034133184     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015826     Document Type: Article

References (34)

1. Amari, S., Cichocki, A., & Yang, H. H. (1996). A new learning algorithm for blind signal separation. In D. Touretzy, M. Mozer, & M. Hasselmo (Eds.), Advances in neural and information processing systems, 8 (pp. 757-763). San Mateo, CA: Morgan Kaufmann.
2. Atick, J. J. (1992). Could information-theory provide an ecological theory of sensory processing? Network-Computation in Neural Systems, 3(2), 213-251.
3. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenbluth (Ed.), Senosory communication (pp. 217-234). Cambridge, MA: MIT Press.
4. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295-311.
5. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
6. Buccigrossi, R. W., & Simoncelli, E. P. (1997). Image compression via joint statistical characterization in the wavelet domain (Tech. Rep. No. 414). Philadelphia: University of Pennsylvania.
7. Cardoso, J.-F. (1997). Infomax and maximum likelihood for blind source separation. IEEE Signal Processing Letters, 4, 109-111.
8. Chen, S., Donoho, D. L., & Saunders, M. A. (1996). Atomic decomposition by basis pursuit (Technical Rep.). Stanford, CA: Department of Statistics, Stanford University.
9. Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713-718.
10. Comon, P. (1994). Independent component analysis, a new concept. Signal Processing, 36(3), 287-314.
11. Daubechies, I. (1990). The wavelet transform, time-frequency localization, and signal analysis. IEEE Transactions on Information Theory, 36(5), 961-1004.
12. Daugman, J. G. (1988). Complete discrete 2-D Gabor transforms by neural networks for image-analysis and compression. IEEE Transactions on Acoustics Speech and Signal Processing, 36(7), 1169-1179.
13. Daugman, J. G. (1989). Entropy reduction and decorrelation in visual coding by oriented neural receptive-fields. IEEE Trans. Bio. Eng., 36(1), 107-114.
14. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6(4), 559-601.
15. Hinton, G. E., & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann machines. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing (Vol. 1, pp. 282-317). Cambridge, MA: MIT Press.
16. Jutten, C., & Hérault, J. (1991). Blind separation of sources. 1. An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24(1), 1-10.
17. Lee, T-W., Lewicki, M., Girolami, M., & Sejnowski, T. (1999). Blind source separation of more sources than mixtures using overcomplete representations. IEEE Sig. Proc. Lett., 6, 87-90.
18. Lewicki, M. S., & Olshausen, B. A. (1998). Inferring sparse, overcomplete image codes using an efficient coding framework. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural and information processing systems, 10. San Mateo, CA: Morgan Kaufmann.
19. Lewicki, M. S., & Olshausen, B. A. (1999). A probabilistic framework for the adaptation and comparison of image codes. J. Opt. Soc. Am. A: Optics, Image Science, and Vision, 16(7), 1587-1601.
20. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21(3), 105-117.
21. MacKay, D. J. C. (1996). Maximum likelihood and covariant algorithms for independent component analysis. Unpublished manuscript. Cambridge: University of Cambridge, Cavendish Laboratory. Available online at: ftp://wol.ra.phy.cam.ac.uk/pub/mackay/ica.ps.gz.
22. Makeig, S., Jung, T. P., Bell, A. J., Ghahremani, D., & Sejnowski, T. J. (1996). Blind separation of event-related brain response components. Psychophysiology, 33, S58-S58.
23. Mallat, S. G., & Zhang, Z. F. (1993). Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12), 3397-3415.
24. McKeown, M. J., Jung, T.-P., Makeig, S., Brown, G. G., Kindermann, S. S., Lee, T., & Sejnowski, T. (1998). Spatially independent activity patterns in functional magnetic resonance imaging data during the stroop color-naming task. Proc. Natl. Acad. Sci. USA, 95, 803-810.
25. Meszaros, C. (1997). BPMPD: An interior point linear programming solver. Code available online at: ftp://ftp.netlib.org/opt/bpmpd.tar.gz.
26. Nadal, J.-P. & Parga, N. (1994a). Nonlinear neurons in the low-noise limit: A factorial code maximizes information transfer. Network, 5, 565-581.
27. Nadal, J.-P., & Parga, N. (1994b). Redundancy reduction and independent component analysis: Conditions on cumulants and adaptive approaches. Network, 5, 565-581.
28. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature, 381, 607-609.
29. Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res., 37(23).
30. Osuna, E., Freund, R., & Girosi, F. (1997). An improved training algorithm for support vector machines. Proc. of IEEE NNSP'97 (pp. 24-26).
31. Pearlmutter, B. A., & Parra, L. C. (1997). Maximum likelihood blind source separation: A context-sensitive generalization of ICA. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural and information processing systems, 9. San Mateo, CA: Morgan Kaufmann.
32. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in C: The art of scientific programming (2nd ed.). Cambridge: Cambridge University Press.
33. Silverman, B. W. (1986). Density estimation for statistics and data analysis. New York: Chapman and Hall.
34. Simoncelli, E. P., Freeman, W. T., Adelson, E. H., & Heeger, D. J. (1992). Shiftable multiscale transforms. IEEE Trans. Info. Theory, 38, 587-607.