A Hebbian/Anti-Hebbian network for online sparse dictionary learning derived from symmetric matrix factorization

Conference Record - Asilomar Conference on Signals, Systems and Computers

Volumn 2015-April, Issue , 2015, Pages 613-619

  Hu, Tao ^a   Pehlevan, Cengiz ^b,c   Chklovskii, Dmitri B ^c  

^a TEXAS A AND M UNIVERSITY   (United States)

^b HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^c Simons Center for Data Analysis   (United States)

Author keywords

matrix factorization; neuromorphic computing; neuron; online algorithm; sparse dictionary learning

Indexed keywords

COMPUTATION THEORY; COST FUNCTIONS; E-LEARNING; FACTORIZATION; GABOR FILTERS; LEARNING ALGORITHMS; MULTILAYER NEURAL NETWORKS; NETWORK LAYERS; NEURONS;

MATRIX FACTORIZATIONS; NEURAL COMPUTATIONS; NEUROMORPHIC COMPUTING; ON-LINE ALGORITHMS; PRIMARY VISUAL CORTEX; SPARSE DICTIONARIES; SYMMETRIC FACTORIZATIONS; SYNAPTIC PLASTICITY;

MATRIX ALGEBRA;

EID: 84940550977     PISSN: 10586393     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACSSC.2014.7094519     Document Type: Conference Paper

References (28)

1. D. H. Hubel, Eye, Brain and Vision, W. H. Freeman; 2nd edition, 1995
2. D. H. Hubel, &T. N. Wiesel. Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, London, 195, 215-243
3. B. Olshausen and D. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, pp. 607-609, 1996
4. B. Olshausen and D. Field, "Sparse coding with an overcomplete basis set: a strategy employed by V1?," Vision Res., vol. 37, no. 23, pp. 33113325, 1997
5. B. A. Olshausen and D. J. Field, "Sparse coding of sensory inputs.," Curr Opin Neurobiol, vol. 14, no. 4, pp. 481-7, Aug. 2004
6. C. J. Rozell, D. H. Johnson, R. G. Baraniuk, and B.A. Olshausen, "Sparse coding via thresholding and local competition in neural circuits," Neural Computation, vol. 20, pp. 2526-2563, 2008
7. S. Boyd, &L. Vandenberghe. Convex optimization. Cambridge university press (2009)
8. T. Hastie, R. Tibshirani, J. Friedman., The elements of statistical learning (Vol. 2, No. 1). New York: Springer (2009)
9. P. Foldiak. Forming Sparse representations by local anti-Hebbian learning, Biol. Cybern. 64, 165-170 (1990)
10. J. Zylberberg, J. T. Murphy, &M. R. DeWeese, A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields. PLoS computational biology,7(10), e1002250 (2011)
11. A. A. Koulakov and D. Rinberg. "Sparse incomplete representations: a potential role of olfactory granule cells." Neuron 72.1 (2011): 124-136
12. S. Druckmann, T. Hu, &D. B. Chklovskii. A mechanistic model of early sensory processing based on subtracting sparse representations. InAdvances in Neural Information Processing Systems (pp. 1979-1987). 2012
13. J. Carroll and J. Chang. Idioscal (individual differences in orientation scaling)A generalization of indscal allowing idiosyncratic reference systems as well as an analytic approximation to indscal. In Psychometric-meeting, Princeton, NJ (1972)
14. C. Ding, X. He, and H. Simon, "On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering.," SDM, no. 4, 2005
15. E. Oja, "A simplified neuron model as a principal component analyzer," Journal of Mathematical Biology, vol. 15, pp. 267-273, 1982
16. T. Hu; Z.J. Towfic, C. Pehlevan, A. Genkin, D.B. Chklovskii. "A neuron as a signal processing device," Asilomar Conference on Signals, Systems and Computers, 2013, vol., no., pp.362,366, 3-6 Nov. 2013
17. C. Pehlevan, T. Hu, &D.B. Chklovskii. "A Hebbian/Anti-Hebbian Network for Linear Subspace Tracking: A Derivation form Multidimensional Scaling of Streaming Data," Neural Computation, submitted
18. C. Pehlevan D.B. Chklovskii. "A Hebbian/Anti-Hebbian Network for Dervied from Online Nonnegative Matrix Factorization Can Cluster and Discover Sparse Features," Asilomar Conference on Signals, Systems and Computers, 2-5 Nov. 2014
19. A. Hyvarinen, J. Hurri and P.P. Hoyer. Natural Image Statistics-A probabilistic approach to early computational vision (Springer-Verlag, London), 2009
20. D.L. Ringach. Spatial structure and symmetry of simple-cell receptive fields in macaque primary visual cortex. J Neurophysiol. 88(1):455-63 (2002)
21. A. Hyvarinen and E. Oja, "Independent component analysis: algorithms and applications," Neural networks, vol. 13, no. 4-5, pp. 411-430, Jun. 2000
22. A. Bell and T. J. Sejnowski, "The 'independent components' of natural scenes are edge filters.," Vision Res., vol. 37, no. 23, pp. 3327-38, Dec. 1997
23. M. Crair and R. Malenka, "A critical period for long-term potentiation at thalamocortical synapses," Nature, vol. 375, pp. 325-327, 1995
24. A. Kirkwood, H. K. Lee, and M. F. Bear, "Co-regulation of long-term potentiation and experience-dependent synaptic plasticity in visual cortex by age and experience," Nature, vol. 375, pp. 328-331, 1995
25. C. Poo and J. S. Isaacson, "An early critical period for long-term plasticity and structural modification of sensory synapses in olfactory cortex," J. Neurosci, vol. 27, pp. 7553-7558, 2007
26. T. Hromadka, MR Deweese, AM Zador. Sparse representation of sounds in the unanesthetized auditory cortex. PLoS Biol 6:e16 (2008)
27. R.H. Masland, "The fundamental plan of the retina," Nature Neuroscience, 4(9), 877-886, 2001
28. B Li, MR Peterson, RD Freeman, Journal of Neurophysiology, 90(1) 204-217, Jul 2003