A normative theory of adaptive dimensionality reduction in neural networks

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 2269-2277

  Pehlevan, Cengiz ^a   Chklovskii, Dmitri B ^a  

^a Simons Center for Data Analysis   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE ALGORITHMS; ALGORITHMS; COVARIANCE MATRIX; EIGENVALUES AND EIGENFUNCTIONS; IMAGE MATCHING; INFORMATION SCIENCE; MATRIX ALGEBRA; NETWORKS (CIRCUITS); NEURAL NETWORKS; SENSORY ANALYSIS;

DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION ALGORITHMS; HIGH DIMENSIONAL DATASETS; MULTI-DIMENSIONAL SCALING; NEURONAL ACTIVITIES; OBJECTIVE FUNCTIONS; SENSORY PROCESSING; SIMILARITY-MATCHING;

NEURONS;

EID: 84965117085     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. David H Hubel. Eye, brain, and vision. Scientific American Library/Scientific American Books, 1995.
2. E Oja. Simplified neuron model as a principal component analyzer. J Math Biol, 15(3):267-273, 1982.
3. K I Diamantaras and S Y Kung. Principal component neural networks: theory and applications. John Wiley & Sons, Inc., 1996.
4. B Yang. Projection approximation subspace tracking. IEEE Trans. Signal Process., 43(1):95-107, 1995.
5. T Hu, Z J Towfic, C Pehlevan, A Genkin, and D B Chklovskii. A neuron as a signal processing device. In Asilomar Conference on Signals, Systems and Computers, pages 362-366. IEEE, 2013.
6. E Oja. Principal components, minor components, and linear neural networks. Neural Networks, 5(6):927-935, 1992.
7. R Arora, A Cotter, K Livescu, and N Srebro. Stochastic optimization for pca and pls. In Allerton Conf. on Communication, Control, and Computing, pages 861-868. IEEE, 2012.
8. J Goes, T Zhang, R Arora, and G Lerman. Robust stochastic principal component analysis. In Proc. 17th Int. Conf. on Artificial Intelligence and Statistics, pages 266-274, 2014.
9. Todd K Leen. Dynamics of learning in recurrent feature-discovery networks. NIPS, 3, 1990.
10. P Földiak. Adaptive network for optimal linear feature extraction. In Int. Joint Conf. on Neural Networks, pages 401-405. IEEE, 1989.
11. T D Sanger. Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural networks, 2(6):459-473, 1989.
12. J Rubner and P Tavan. A self-organizing network for principal-component analysis. EPL, 10:693, 1989.
13. M D Plumbley. A hebbian/anti-hebbian network which optimizes information capacity by orthonormalizing the principal subspace. In Proc. 3rd Int. Conf. on Artificial Neural Networks, pages 86-90, 1993.
14. M D Plumbley. A subspace network that determines its own output dimension. Tech. Rep., 1994.
15. M D Plumbley. Information processing in negative feedback neural networks. Network-Comp Neural, 7(2):301-305, 1996.
16. P Vertechi, W Brendel, and C K Machens. Unsupervised learning of an efficient short-term memory network. In NIPS, pages 3653-3661, 2014.
17. B A Olshausen and D J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Res, 37(23):3311-3325, 1997.
18. A A Koulakov and D Rinberg. Sparse incomplete representations: a potential role of olfactory granule cells. Neuron, 72(1):124-136, 2011.
19. S Druckmann, T Hu, and D B Chklovskii. A mechanistic model of early sensory processing based on subtracting sparse representations. In NIPS, pages 1979-1987, 2012.
20. A L Fairhall, G D Lewen, W Bialek, and R R R van Steveninck. Efficiency and ambiguity in an adaptive neural code. Nature, 412(6849):787-792, 2001.
21. S E Palmer, O Marre, M J Berry, and W Bialek. Predictive information in a sensory population. PNAS, 112(22):6908-6913, 2015.
22. E Doi, J L Gauthier, G D Field, J Shlens, et al. Efficient coding of spatial information in the primate retina. J Neurosci, 32(46):16256-16264, 2012.
23. R Linsker. Self-organization in a perceptual network. Computer, 21(3):105-117, 1988.
24. C Pehlevan, T Hu, and D B Chklovskii. A hebbian/anti-hebbian neural network for linear subspace learning: A derivation from multidimensional scaling of streaming data. Neural Comput, 27:1461-1495, 2015.
25. G Young and A S Householder. Discussion of a set of points in terms of their mutual distances. Psychometrika, 3(1):19-22, 1938.
26. W S Torgerson. Multidimensional scaling: I. theory and method. Psychometrika, 17(4):401-419, 1952.
27. H G Barrow and J M L Budd. Automatic gain control by a basic neural circuit. Artificial Neural Networks, 2:433-436, 1992.
28. E J Candès and B Recht. Exact matrix completion via convex optimization. Found Comput Math, 9(6):717-772, 2009.
29. J Mairal, F Bach, J Ponce, and G Sapiro. Online learning for matrix factorization and sparse coding. JMLR, 11:19-60, 2010.
30. S. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.
31. P Gao and S Ganguli. On simplicity and complexity in the brave new world of large-scale neuroscience. Curr Opin Neurobiol, 32:148-155, 2015.
32. M Zhu and C J Rozell. Modeling inhibitory interneurons in efficient sensory coding models. PLoS Comput Biol, 11(7):e1004353, 2015.
33. P D King, J Zylberberg, and M R De Weese. Inhibitory interneurons decorrelate excitatory cells to drive sparse code formation in a spiking model of v1. J Neurosci, 33(13):5475-5485, 2013.
34. N Kriegeskorte, M Mur, D A Ruff, R Kiani, et al. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, 60(6):1126-1141, 2008.
35. R Kiani, H Esteky, K Mirpour, and K Tanaka. Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. J Neurophysiol, 97(6):4296-4309, 2007.
36. G Tkačik, E Granot-Atedgi, R Segev, and E Schneidman. Retinal metric: a stimulus distance measure derived from population neural responses. PRL, 110(5):058104, 2013.
37. H S Seung, T J Richardson, J C Lagarias, and J J Hopfield. Minimax and hamiltonian dynamics of excitatory-inhibitory networks. NIPS, 10:329-335, 1998.
38. C Pehlevan and D B Chklovskii. Optimization theory of hebbian/anti-hebbian networks for pca and whitening. In Allerton Conf. on Communication, Control, and Computing, 2015.