Maximal causes for non-linear component extraction

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 1227-1267

  Lücke, Jörg ^a   Sahani, Maneesh ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Approximate EM; Competitive learning; Component extraction; Maximum likelihood; Neural networks

Indexed keywords

ACOUSTICS; ARTIFICIAL INTELLIGENCE; BLIND SOURCE SEPARATION; COMPETITION; EDUCATION; EIGENVALUES AND EIGENFUNCTIONS; EXTRACTION; FACTORIZATION; FUNCTION EVALUATION; INDEPENDENT COMPONENT ANALYSIS; INDUSTRIAL ECONOMICS; LEARNING SYSTEMS; MATRIX ALGEBRA; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; PHOTOACOUSTIC EFFECT; VEGETATION;

ACTIVATION FUNCTION (AF); ANALYTICAL RESULTS; BENCH MARK TEST; EXPECTATION MAXIMIZATION (EM); GENERATIVE MODELING; HEBBIAN LEARNING; IMAGE DATA; INDEPENDENT COMPONENTS; LEARNING MODELS; MAXIMUM LIKELIHOOD (MLET); NEURAL NETWORK (NN) MODELS; NON LINEAR INTERACTIONS; NON-LINEAR; NON-NEGATIVE MATRIX FACTORIZATION (DNMF); REALISTIC MODELING; SPARSE CODING; VISUAL DATA;

NEURAL NETWORKS;

EID: 46749096794     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (44)

1. A. J. Bell and T. J. Sejnowski. The "independent components" of natural scenes are edge filters. Vision Research, 37(23):3327-3338, 1997.
2. N. J. Butko and J. Triesch. Learning sensory representations with intrinsic plasticity. Neurocomputing, 70(7-9): 1130-1138, 2007.
3. D. Charles and C. Fyfe. Modelling multiple-cause structure using rectification constraints. Network: Computation in Neural Systems, 9:167-182, 1998.
4. D. Charles, C. Fyfe, D. MacDonald, and J. Koetsier. Unsupervised neural networks for the identification of minimum overcomplete basis in visual data. Neurocomputing, 47(1-4):119-143, 2002.
5. P. Comon. Independent component analysis, a new concept? Signal Processing, 36(3):287-314, 1994.
6. P. Dayan and L. F. Abbott. Theoretical Neuroscience. MIT Press, Cambridge, 2001.
7. P. Dayan and R. S. Zemel. Competition and multiple cause models. Neural Computation, 7:565-579, 1995.
8. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B, 39:1-38, 1977.
9. B. S. Everitt. An Introduction to Latent Variable Models. Chapman and Hall, 1984.
10. P. Fòldiák. Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, 64:165-170, 1990.
11. C. Fyfe. A neural network for PCA and beyond. Neural Processing Letters, 6:33-41, 1997.
12. N. M. Grzywacz and A. L. Yuille. A model for the estimate of local image velocity by cells in the visual cortex. Proceedings of the Royal Society of London B, 239:129-161, 1990.
13. G. F. Harpur and R. W. Prager. Development of low entropy coding in a recurrent network. Network-Computation in Neural Systems, 7:277-284, 1996.
14. G. E. Hinton and Z. Ghahramani. Generative models for discovering sparse distributed representations. Philosophical Transactions of the Royal Society B, 352:1177-1190, 1997.
15. G. E. Hinton, P Dayan, B. J. Frey, and R. M. Neal. The 'wake-sleep' algorithm for unsupervised neural networks. Science, 268:1158-1161, 1995.
16. S. Hochreiter and J. Schmidhuber. Feature extraction through LOCOCODE. Neural Computation, 11:679-714, 1999.
17. P. O. Hoyer. Non-negative sparse coding. In Neural Networks for Signal Processing XII: Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, pages 557-565. 2002.
18. P. O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457-1469, 2004.
19. M. I. Jordan, Z. Ghahramani, T. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine Learning, 37(2): 183-233, 1999.
20. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788-791, 1999.
21. D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In Advances in Neural Information Processing Systems 13, pages 556-562, 2001.
22. J. Lücke. Hierarchical self-organization of minicolumnar receptive fields. Neural Networks, 17/8-9: 1377-1389, 2004.
23. J. Lücke. A dynamical model for receptive field self-organization in V1 cortical columns. In Proceedings of the International Conference on Artificial Neural Networks, LNCS 4669, pages 389-398. Springer, 2007.
24. J. Lücke and J. D. Bouecke. Dynamics of cortical columns - self-organization of receptive fields. In Proceedings of the International Conference on Artificial Neural Networks, LNCS 3696, pages 31-37. Springer, 2005.
25. J. Lücke and M. Sahani. Generalized softmax networks for non-linear component extraction. In Proceedings of the International Conference on Artificial Neural Networks, LNCS 4668, pages 657-667. Springer, 2007.
26. J. Lücke and C. von der Malsburg. Rapid processing and unsupervised learning in a model of the cortical macrocolumn. Neural Computation, 16:501-533, 2004.
27. G. McLachlan and D. Peel. Finite Mixture Models. Wiley, 2000.
28. R. Neal and G. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan, editor, Learning in Graphical Models. Kluwer, 1998.
29. S. J. Nowlan. Maximum likelihood competitive learning. In Advances in Neural Information Processing Systems 2, pages 574-582, 1990.
30. B. A. Olshausen and D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:607-609, 1996.
31. R. C. O'Reilly. Generalization in interactive networks: The benefits of inhibitory competition and Hebbian learning. Neural Computation, 13:1199-1241, 2001.
32. R. P. N. Rao, B. A. Olshausen, and M. S. Lewicki, editors. Probabilistic Models of the Brain: Perception and Neural Function. Neural Information Processing. The MIT Press, Cambridge, MA, 2002.
33. M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11): 1019-1025, 1999.
34. S. T. Roweis. Factorial models and refiltering for speech separation and denoising. In Proceedings Eurospeech, volume 7, pages 1009-1012, 2003.
35. M. Sahani. Latent Variable Models for Neural Data Analysis. PhD thesis, California Institute of Technology, Pasadena, California, 1999. URL http://www.gatsby.ucl.ac.uk/~maneesh/thesis/.
36. E. Saund. A multiple cause mixture model for unsupervised learning. Neural Computation, 7: 51-71, 1995.
37. M. W. Spratling. Learning image components for object recognition. Journal of Machine Learning Research, 7:793-815, 2006.
38. M. W. Spratling and M. H. Johnson. Preintegration lateral inhibition enhances unsupervised learning. Neural Computation, 14:2157-2179, 2002.
39. Sunsite. Sun-sounds: Phonemes. Data retrieved in 2007 through ibiblio.org from ftp://sunsite.unc.edu/pub/multimedia/suri-sounds/phonemes/, 1997.
40. N. Ueda and R. Nakano. Deterministic annealing EM algorithm. Neural Networks, 11(2):271-282, 1998.
41. J. H. van Hateren and A. van der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London B, 265: 359-366, 1998.
42. Y. Weiss. Phase transitions and the perceptual organization of video sequences. In Advances in Neural Information Processing Systems 10, pages 850-856, 1998.
43. H. Wersing and E. Körner. Learning optimized features for hierarchical models of invariant object recognition. Neural Computation, 15(7): 1559-1588, 2003.
44. A. L. Yuille and D. Geiger. Winner-take-all networks. In M.A. Arbib, editor, The handbook of brain theory and neural networks, pages 1228-1231. MIT Press, 2003.