On simplicity and complexity in the brave new world of large-scale neuroscience

Current Opinion in Neurobiology

Volumn 32, Issue , 2015, Pages 148-155

  Gao, Peiran ^a   Ganguli, Surya ^b  

^a Stanford University   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL NEURAL NETWORK; BEHAVIOR; CONCEPTUAL FRAMEWORK; CONNECTOME; DATA ANALYSIS; HUMAN; INFORMATION; MULTIFACTOR DIMENSIONALITY REDUCTION; NERVE CELL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVOUS SYSTEM FUNCTION; NEURONAL TASK COMPLEXITY; NEUROSCIENCE; NONHUMAN; PRIORITY JOURNAL; PROBLEM SOLVING; REVIEW; ANIMAL; INFORMATION PROCESSING; PHYSIOLOGY; PROCEDURES; THEORETICAL MODEL;

ANIMALS; DATASETS AS TOPIC; MODELS, THEORETICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); NEUROSCIENCES;

EID: 84928527851     PISSN: 09594388     EISSN: 18736882     Source Type: Journal    
DOI: 10.1016/j.conb.2015.04.003     Document Type: Review

References (90)

1. Stevenson I.H., Kording K.P. How advances in neural recording affect data analysis. Nat Neurosci 2011, 14:139-142.
2. Robinson J.T., Jorgolli M., Shalek A.K., Yoon M.H., Gertner R.S., Park H. Vertical nanowire electrode arrays as a scalable platform for intracellular interfacing to neuronal circuits. Nat Nano 2012, 7:180-184.
3. Ahrens M.B., Li J.M., Orger M.B., Robson D.N., Schier A.F., Engert F., Portugues R. Brain-wide neuronal dynamics during motor adaptation in zebrafish. Nature 2012, 485:471-477.
4. Schrodel T., Prevedel R., Aumayr K., Zimmer M., Vaziri A. Brain-wide 3D imaging of neuronal activity in Caenorhabditis elegans with sculpted light. Nat Methods 2013, 10:1013-1020.
5. Ziv Y., Burns L.D., Cocker E.D., Hamel E.O., Ghosh K.K., Kitch L.J., El Gamal A., Schnitzer M.J. Long-term dynamics of CA1 hippocampal place codes. Nat Neurosci 2013, 16:264-266.
6. Prevedel R., Yoon Y., Hoffmann M., Pak N. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat Methods 2014, 11:727-730.
7. Micheva K., Smith S. Array tomography: a new tool for imaging the molecular architecture and ultrastructure of neural circuits. Neuron 2007, 55:25-36.
8. Wickersham I.R., Lyon D.C., Barnard R.J., Mori T., Finke S., Conzelmann K.K., Young J.A., Callaway E.M. Monosynaptic restriction of transsynaptic tracing from single, genetically targeted neurons. Neuron 2007, 53:639-647.
9. Li A., Gong H., Zhang B., Wang Q., Yan C., Wu J., Liu Q., Zeng S., Luo Q. Micro-optical sectioning tomography to obtain a high-resolution atlas of the mouse brain. Science 2010, 330:1404-1408.
10. Ragan T., Kadiri L.R., Venkataraju K.U., Bahlmann K., Sutin J., Taranda J., Arganda-Carreras I., Kim Y., Seung H.S., Osten P. Serial two-photon tomography for automated ex vivo mouse brain imaging. Nat Methods 2012, 9:255-258.
11. Chung K., Deisseroth K. Clarity for mapping the nervous system. Nat Methods 2013, 10:508-513.
12. Takemura S., Bharioke y., Lu A., Nern Z., Vitaladevuni A., Rivlin S. A visual motion detection circuit suggested by Drosophila connectomics. Nature 2013, 500:175-181.
13. Pestilli F., Yeatman J.D., Rokem A., Kay K.N., Wandell B.A. Evaluation and statistical inference for human connectomes. Nat Methods 2014, 11:1058-1063.
14. Oh S.W., Harris J.A., Ng L., Winslow B., Cain N., Mihalas S., et al. A mesoscale connectome of the mouse brain. Nature 2014, 508:207-214.
15. Bock D.D., Lee W.C.A., Kerlin A.M., Andermann M.L., Hood G., Wetzel A.W., Yurgenson S., Soucy E.R., Kim H.S., Reid R.C. Network anatomy and in vivo physiology of visual cortical neurons. Nature 2011, 471:177-182.
16. Rancz E.A., Franks K.M., Schwarz M.K., Pichler B., Schaefer A.T., Margrie T.W. Transfection via whole-cell recording in vivo: bridging single-cell physiology, genetics and connectomics. Nat Neurosci 2011, 14:527-532.
17. Cunningham J.P., Byron M.Y. Dimensionality reduction for large-scale neural recordings. Nat Neurosci 2014, 1500-1509.
18. Stevenson I.H., Kording K.P. How advances in neural recording affect data analysis. Nat Neurosci 2011, 14:139-142.
19. Shepherd G.M. The Synaptic Organization of the Brain 2004, vol 3. Oxford University Press, New York.
20. Mazor O., Laurent G. Transient dynamics versus fixed points in odor representations by locust antennal lobe projection neurons. Neuron 2005, 48:661-673.
21. Machens C.K., Romo R., Brody C.D. Functional, but not anatomical, separation of "what" and "when" in prefrontal cortex. J Neurosci 2010, 30:350-360.
22. Mante V., Sussillo D., Shenoy K.V., Newsome W.T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 2013, 503:78-84.
23. Stopfer M., Jayaraman V., Laurent G. Intensity versus identity coding in an olfactory system. Neuron 2003, 39:991-991004.
24. Assisi C., Stopfer M., Laurent G., Bazhenov M. Adaptive regulation of sparseness by feedforward inhibition. Nat Neurosci 2007, 10:1176-1184.
25. Raman B., Joseph J., Tang J., Stopfer M. Temporally diverse firing patterns in olfactory receptor neurons underlie spatiotemporal neural codes for odors. J Neurosci 2010, 30:1994-2006.
26. Haddad R., Weiss T., Khan R., Nadler B., Mandairon N., Bensafi M., Schneidman E., Sobel N. Global features of neural activity in the olfactory system form a parallel code that predicts olfactory behavior and perception. J Neurosci 2010, 30:9017-9026.
27. Bathellier B., Buhl D.L., Accolla R., Carleton A. Dynamic ensemble odor coding in the mammalian olfactory bulb: sensory information at different timescales. Neuron 2008, 57:586-598.
28. Narayanan N.S., Laubach M. Delay activity in rodent frontal cortex during a simple reaction time task. J Neurophysiol 2009, 101:2859-2871.
29. Peyrache A., Khamassi M., Benchenane K., Wiener S.I., Battaglia F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci 2009, 12:919-926.
30. Warden M.R., Miller E.K. Task-dependent changes in short-term memory in the prefrontal cortex. J Neurosci 2010, 30:15801-15810.
31. Paz R., Natan C., Boraud T., Bergman H., Vaadia E. Emerging patterns of neuronal responses in supplementary and primary motor areas during sensorimotor adaptation. J Neurosci 2005, 25:10941-10951.
32. Churchland M.M., Cunningham J.P., Kaufman M.T., Foster J.D., Nuyujukian P., Ryu S.I., Shenoy K.V. Neural population dynamics during reaching. Nature 2012, 487:51-56.
33. Chapin J., Nicolelis M. Principal component analysis of neuronal ensemble activity reveals multidimensional somatosensory representations. J Neurosci 1999, 94:121-140.
34. Hegdé J., Van Essen D.C. Temporal dynamics of shape analysis in macaque visual area V2. J Neurophysiol 2004, 92:3030-3042.
35. Matsumoto N., Okada M., Yasuko S., Yamane S., Kawano K. Population dynamics of face-responsive neurons in the inferior temporal cortex. Cereb Cortex 2005, 15:1103-1112.
36. Sasaki T., Matsuki N., Ikegaya Y. Metastability of active CA3 networks. J Neurosci 2007, 27:517-528.
37. Bromberg-Martin E.S., Hikosaka O., Nakamura K. Coding of task reward value in the dorsal raphe nucleus. J Neurosci 2010, 30:6262-6272.
38. Johnson W., Lindenstrauss J. Extensions of Lipschitz mappings into a Hilbert space. Contemp Math 1984, 26. 1-1.
39. Dasgupta S., Gupta A. An elementary proof of a theorem of johnson and Lindenstrauss. Random Struct Algorithms 2003, 22:60-65.
40. Baraniuk R., Wakin M. Random projections of smooth manifolds. Found Comput Math 2009, 9:51-77.
41. Gao P., Trautmann E., Yu B., Santhanam G., Ryu S., Shenoy K.V., Ganguli S. A theory of neural dimensionality and measurement. Computational and Systems Neuroscience Conference (COSYNE) 2014.
42. Gao P., Ganguli S. Dimensionality, Coding and Dynamics of Single-Trial Neural Data. Computational and Systems Neuroscience Conference (COSYNE) 2015.
43. Byron M.Y., Kemere C., Santhanam G., Afshar A., Ryu S.I., Meng T.H., Sahani M., Shenoy K.V. Mixture of trajectory models for neural decoding of goal-directed movements. J Neurophysiol 2007, 97:3763-3780.
44. Gavish M., Donoho D. Optimal Shrinkage of Singular Values 2014, arXiv preprint arXiv:14057511.
45. Lennart L. System Identification: Theory for the User 1999.
46. Yao J. A note on a marcenko-pasteur type theorem for time-series. Stat Probab Lett 2012, 10.1016/j.spl.2011.08.011.
47. Benaych-Georges F., Nadakuditi R. The singular values and vectors of low rank perturbations of large rectangular random matrices. J Multivar Anal 2012, 111:120135. 10.1016/j.jmva.2012.04.019.
48. Nica A., Speicher R. On the multiplication of free N-tuples of noncommutative random variables. Am J Math 1996, 799-837. http://www.jstor.org/stable/25098492, 10.2307/25098492.
49. Amelunxen D., Lotz M., McCoy M.B., Tropp J.A. Living on the edge: phase transitions in convex programs with random data. Inf Inference 2014, iau005.
50. Donoho D., Maleki A., Montanari A. Message-passing algorithms for compressed sensing. Proc Natl Acad Sci U S A 2009, 106:18914.
51. Ganguli S., Sompolinsky H. Statistical mechanics of compressed sensing. Phys Rev Lett 2010, 104:188701.
52. Donoho D.L., Gavish M., Montanari A. The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising. Proc Natl Acad Sci U S A 2013, 110:8405-8410.
53. Ganguli S., Sompolinsky H. Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. Annu Rev Neurosci 2012, 35:485-508.
54. Advani M., Lahiri S., Ganguli S. Statistical mechanics of complex neural systems and high dimensional data. J Stat Mech Theory Exp 2013, 2013:P03014.
55. Bengio Y., Courville A., Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 2013, 35:1798-1828.
56. Bengio Y. Learning deep architectures for AI. Found Trends® Mach Learn 2009, 2:1-127.
57. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015, 61:85-117.
58. Krizhevsky A., Sutskever I., Hinton G.E. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 2012, 1097-1105.
59. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhouchke V., Rabinovich A. Going Deeper with Convolutions 2014, arXiv preprint arXiv:14094842.
60. Sun Y., Chen Y., Wang X., Tang X. Deep learning face representation by joint identification-verification. Advances in Neural Information Processing Systems 2014, vol 27:1988-1996. Curran Associates, Inc. Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, K. Weinberger (Eds.).
61. Taigman Y., Yang M., Ranzato M., Wolf L. Deepface: closing the gap to human-level performance in face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2014, 1701-1708. IEEE.
62. Hannun A.Y., Case C., Casper J., Catanzaro B.C., Diamos G., Elsen E., et al. Deep Speech: Scaling up End-to-End Speech Recognition 2014, CoRR abs/1412.5567.
63. Sutskever I., Vinyals O., Le Q.V.V. Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 2014, vol 27:3104-3112. Curran Associates, Inc. Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, K. Weinberger (Eds.).
64. Xiong H.Y., Alipanahi B., Lee L.J., Bretschneider H., Merico D., Yuen R.K., et al. The human splicing code reveals new insights into the genetic determinants of disease. Science 2015, 347:1254806.
65. Ciresan D., Giusti A., Gambardella L.M., Schmidhuber J. Deep neural networks segment neuronal membranes in electron microscopy images. Advances in Neural Information Processing Systems 2012, 2843-2851.
66. Serre T., Oliva A., Poggio T. A feedforward architecture accounts for rapid categorization. Proc Natl Acad Sci U S A 2007, 104:6424-6429.
67. Yamins D.L., Hong H., Cadieu C.F., Solomon E.A., Seibert D., DiCarlo J.J. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci U S A 2014, 201403112.
68. Cadieu C.F., Hong H., Yamins D.L., Pinto N., Ardila D., Solomon E.A., Majaj N.J., Dicarlo J.J. Deep neural networks rival the representation of primate IT cortex for core visual object recognition. PLoS comput Biol 2014, 10:e1003963.
69. Agrawal P., Stansbury D., Malik J., Gallant J.L. Pixels to Voxels: Modeling Visual Representation in the Human Brain 2014, arXiv preprint arXiv:14075104.
70. Bianchini M., Scarselli F. On the complexity of neural network classifiers: a comparison between shallow and deep architectures. IEEE Trans Neural Netw 2014, 25:1553-1565.
71. Pascanu R., Montúfar G., Bengio Y. On the number of inference regions of deep feed forward networks with piece-wise linear activations. Internal Conference on Learning Representations (ICLR) 2014.
72. Saxe A., McClelland J., Ganguli S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. International Conference on Learning Representations (ICLR) 2014.
73. Saxe A.M., McClelland J.L., Ganguli S. Learning hierarchical category structure in deep neural networks. Proc. of 35th Annual Cog. Sci. Society 2013.
74. Dauphin Y.N., Pascanu R., Gulcehre C., Cho K., Ganguli S., Bengio Y. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. Advances in Neural Information Processing Systems 2014, 2933-2941.
75. Li F., Long T., Lu Y., Ouyang Q., Tang C. The yeast cell-cycle network is robustly designed. Proc Natl Acad Sci U S A 2004, 101:4781-4786.
76. Lau K., Ganguli S., Tang C. Function constrains network architecture and dynamics: a case study on the yeast cell cycle Boolean network. Phys Rev E 2007, 75:051907.
77. Li H., Helling R., Tang C., Wingreen N. Emergence of preferred structures in a simple model of protein folding. Science 1996, 273:666-669.
78. Wingreen N.S., Li H., Tang C. Designability and thermal stability of protein structures. Polymer 2004, 45:699-705.
79. Li H., Tang C., Wingreen N.S. Are protein folds atypical?. Proc Natl Acad Sci U S A 1998, 95:4987-4990.
80. Maass W., Natschlager T., Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 2002, 14:2531-2560.
81. Jaeger H. Short term memory in echo state networks. GMD Report 152 German National Research Center for Information Technology 2001.
82. White O., Lee D., Sompolinsky H. Short-term memory in orthogonal neural networks. Phys Rev Lett 2004, 92:148102.
83. Ganguli S., Huh D., Sompolinsky H. Memory traces in dynamical systems. Proc Natl Acad Sci U S A 2008, 105:18970.
84. Ganguli S., Sompolinsky H. Short-term memory in neuronal networks through dynamical compressed sensing. Neural Information Processing Systems (NIPS) 2010.
85. Fusi S., Drew P.J., Abbott L.F. Cascade models of synaptically stored memories. Neuron 2005, 45:599-611. 10.1016/j.neuron.2005.02.001.
86. Lahiri S., Ganguli S. A memory frontier for complex synapses. Neural Information Processing Systems (NIPS) 2014.
87. Prinz A.A., Bucher D., Marder E. Similar network activity from disparate circuit parameters. Nat Neurosci 2004, 7:1345-1352.
88. Schulz D.J., Goaillard J.M., Marder E.E. Quantitative expression profiling of identified neurons reveals cell-specific constraints on highly variable levels of gene expression. Proc Natl Acad Sci U S A 2007, 104:13187-13191.
89. O'Leary T., Williams A.H., Caplan J.S., Marder E. Correlations in ion channel expression emerge from homeostatic tuning rules. Proc Natl Acad Sci U S A 2013, 110:E2645-E2654.
90. Shakespeare W. The Tempest 2001, vol 9. Classic Books Company.