Biologically plausible learning in neural networks with modulatory feedback

Neural Networks

Volumn 88, Issue , 2017, Pages 32-48

  Grant, W Shane ^a   Tanner, James ^a   Itti, Laurent ^a,b  

^a University of Southern California ^*  (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Border ownership; Computational modeling; Feedback; Modulatory; Plasticity; Self organization

Indexed keywords

COGNITIVE SYSTEMS; FEEDBACK; PLASTICITY;

BORDER OWNERSHIP; COMPUTATIONAL MODEL; FEEDBACK CONNECTION; MODULATORY; NEURAL PLASTICITY; NOCV1; ORIENTATION SELECTIVITY; SELF ORGANIZATIONS; STRONG DEPENDENCES;

ARTIFICIAL INTELLIGENCE;

ARTICLE; CONFLICT LEARNING; CONNECTOME; FEEDBACK SYSTEM; HEBBIAN LEARNING; LEARNING; LEARNING ALGORITHM; LONG TERM DEPRESSION; LONG TERM POTENTIATION; MATHEMATICAL COMPUTING; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NEUROMODULATION; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; BRAIN; HUMAN; MACHINE LEARNING; PHYSIOLOGY; PROCEDURES;

BRAIN; FEEDBACK; HUMANS; LEARNING; MACHINE LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONAL PLASTICITY; PATTERN RECOGNITION, AUTOMATED;

EID: 85011807602     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2017.01.007     Document Type: Article

References (68)

1. [1] Abbott, L.F., Nelson, S.B., Synaptic plasticity: taming the beast. Nature Neuroscience 3 (2000), 1178–1183.
2. [2] Baluch, F., Itti, L., Mechanisms of top-down attention. Trends in Neurosciences 34:4 (2011), 210–224.
3. [3] Bar, M., Kassam, K.S., Ghuman, A.S., Boshyan, J., Schmid, A.M., Dale, A.M., et al. Top-down facilitation of visual recognition. Proceedings of the National Academy of Sciences of the United States of America 103:2 (2006), 449–454.
4. [4] Bayerl, P., Neumann, H., Disambiguating visual motion through contextual feedback modulation. Neural Computation 16:10 (2004), 2041–2066.
5. [5] Bednar, J.A., Building a mechanistic model of the development and function of the primary visual cortex. Journal of Physiology-Paris 106:5 (2012), 194–211.
6. [6] Bednar, J.A., Topographica: building and analyzing map-level simulations from Python, C/C++, MATLAB, NEST, or NEURON components. Python in Neuroscience, 2015, 104.
7. [7] Beuth, F., Hamker, F.H., A mechanistic cortical microcircuit of attention for amplification, normalization and suppression. Vision Research 116 (2015), 241–257.
8. [8] Bienenstock, E.L., Cooper, L.N., Munro, P.W., Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. The Journal of Neuroscience 2:1 (1982), 32–48.
9. [9] Bridson, R., Fast Poisson disk sampling in arbitrary dimensions. SIGGRAPH sketches, 2007, 22.
10. [10] Brosch, T., Neumann, H., Computing with a canonical neural circuits model with pool normalization and modulating feedback. Neural Computation, 2014.
11. [11] Brosch, T., Neumann, H., Interaction of feedforward and feedback streams in visual cortex in a firing-rate model of columnar computations. Neural Networks 54 (2014), 11–16.
12. [12] Buschman, T.J., Kastner, S., From behavior to neural dynamics: An integrated theory of attention. Neuron 88:1 (2015), 127–144.
13. [13] Callaway, E.M., Feedforward, feedback and inhibitory connections in primate visual cortex. Neural Networks 17:5 (2004), 625–632.
14. [14] Carandini, M., Heeger, D.J., Normalization as a canonical neural computation. Nature Reviews Neuroscience 13:1 (2012), 51–62.
15. [15] Chapman, B., Stryker, M.P., Bonhoeffer, T., Development of orientation preference maps in ferret primary visual cortex. The Journal of Neuroscience 16:20 (1996), 6443–6453.
16. [16] Clopath, C., Büsing, L., Vasilaki, E., Gerstner, W., Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nature Neuroscience 13:3 (2010), 344–352.
17. [17] Craft, E., Schütze, H., Niebur, E., Von Der Heydt, R., A neural model of figure–ground organization. Journal of Neurophysiology 97:6 (2007), 4310–4326.
18. [18] Cudeiro, J., Sillito, A.M., Looking back: corticothalamic feedback and early visual processing. Trends in Neurosciences 29:6 (2006), 298–306.
19. [19] Engert, F., Bonhoeffer, T., Synapse specificity of long-term potentiation breaks down at short distances. Nature 388:6639 (1997), 279–284.
20. [20] Faisal, A.A., Selen, L.P., Wolpert, D.M., Noise in the nervous system. Nature Reviews Neuroscience 9:4 (2008), 292–303.
21. [21] Field, D.J., Relations between the statistics of natural images and the response properties of cortical cells. JOSA A 4:12 (1987), 2379–2394.
22. [22] Fino, E., Paille, V., Cui, Y., Morera-Herreras, T., Deniau, J.-M., Venance, L., Distinct coincidence detectors govern the corticostriatal spike timing-dependent plasticity. The Journal of Physiology 588:16 (2010), 3045–3062.
23. [23] Friston, K., Büchel, C., Attentional modulation of effective connectivity from V2 to V5/MT in humans. Proceedings of the National Academy of Sciences 97:13 (2000), 7591–7596.
24. [24] Froemke, R.C., Plasticity of cortical excitatory-inhibitory balance. Annual Review of Neuroscience, 38, 2015, 195.
25. [25] Fukushima, K., Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36:4 (1980), 193–202.
26. [26] Grant, W.S., Tanner, J. and Itti, L. (2016). Con ict learning source code. ilab.usc.edu/conflictlearning/ Accessed 03.07.16.
27. [27] Grossberg, S., Adaptive resonance theory: How a brain learns to consciously attend, learn, and recognize a changing world. Neural Networks 37 (2013), 1–47.
28. [28] Hebb, D.O., The organization of behavior: A neuropsychological approach. 1949, John Wiley & Sons.
29. [29] Hupe, J.-M., James, A.C., Girard, P., Lomber, S.G., Payne, B.R., Bullier, J., Feedback connections act on the early part of the responses in monkey visual cortex. Journal of Neurophysiology 85:1 (2001), 134–145.
30. [30] Jones, H.E., Andolina, I.M., Ahmed, B., Shipp, S.D., Clements, J.T., Grieve, K.L., et al. Differential feedback modulation of center and surround mechanisms in parvocellular cells in the visual thalamus. The Journal of Neuroscience 32:45 (2012), 15946–15951.
31. [31] Jones, H.E., Andolina, I.M., Shipp, S.D., Adams, D.L., Cudeiro, J., Salt, T.E., et al. Figure-ground modulation in awake primate thalamus. Proceedings of the National Academy of Sciences 112:22 (2015), 7085–7090.
32. [32] Kaschube, M., Schnabel, M., Löwel, S., Coppola, D.M., White, L.E., Wolf, F., Universality in the evolution of orientation columns in the visual cortex. Science 330:6007 (2010), 1113–1116.
33. [33] Keil, W., Kaschube, M., Schnabel, M., Kisvarday, Z.F., Löwel, S., Coppola, D.M., et al. Response to comment on universality in the evolution of orientation columns in the visual cortex. Science, 336(6080), 2012, 413.
34. [34] Kogo, N., van Ee, R., Neural mechanisms of figure-ground organization: Border-ownership, competition and perceptual switching. Handbook of perceptual organization, 2014.
35. [35] Larkum, M., A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex. Trends in Neurosciences 36:3 (2013), 141–151.
36. [36] Levy, W., Steward, O., Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience 8:4 (1983), 791–797.
37. [37] Lim, S., McKee, J.L., Woloszyn, L., Amit, Y., Freedman, D.J., Sheinberg, D.L., et al. Inferring learning rules from distributions of firing rates in cortical neurons. Nature Neuroscience, 2015.
38. [38] Linden, D.J., Connor, J.A., Long-term synaptic depression. Annual Review of Neuroscience 18:1 (1995), 319–357.
39. [39] Major, G., Larkum, M.E., Schiller, J., Active properties of neocortical pyramidal neuron dendrites. Annual Review of Neuroscience 36 (2013), 1–24.
40. [40] Markov, N.T., Vezoli, J., Chameau, P., Falchier, A., Quilodran, R., Huissoud, C., et al. Anatomy of hierarchy: feedforward and feedback pathways in macaque visual cortex. Journal of Comparative Neurology 522:1 (2014), 225–259.
41. [41] Markram, H., Toledo-Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., Wu, C., Interneurons of the neocortical inhibitory system. Nature Reviews Neuroscience 5:10 (2004), 793–807.
42. [42] Martin, A.B., von der Heydt, R., Spike synchrony reveals emergence of proto-objects in visual cortex. The Journal of Neuroscience 35:17 (2015), 6860–6870.
43. [43] Miconi, T., VanRullen, R., A feedback model of attention explains the diverse effects of attention on neural firing rates and receptive field structure. PLoS Computational Biology, 12(2), 2016, e1004770.
44. [44] Mihalas, S., Dong, Y., von der Heydt, R., Niebur, E., Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects. Proceedings of the National Academy of Sciences 108:18 (2011), 7583–7588.
45. [45] Oja, E., Simplified neuron model as a principal component analyzer. Journal of Mathematical Biology 15:3 (1982), 267–273.
46. [46] Olshausen, B.A., Field, D.J., Sparse coding with an overcomplete basis set: A strategy employed by V1?. Vision Research 37:23 (1997), 3311–3325.
47. [47] Qiu, F.T., Sugihara, T., von der Heydt, R., Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience 10:11 (2007), 1492–1499.
48. [48] Roelfsema, P.R., Lamme, V.A., Spekreijse, H., Bosch, H., Figureground segregation in a recurrent network architecture. Journal of Cognitive Neuroscience 14:4 (2002), 525–537.
49. [49] Sakai, K., Nishimura, H., Surrounding suppression and facilitation in the determination of border ownership. Journal of Cognitive Neuroscience 18:4 (2006), 562–579.
50. [50] Sanger, T.D., Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2:6 (1989), 459–473.
51. [51] Serre, T., Wolf, L., Poggio, T., Object recognition with features inspired by visual cortex. Computer vision and pattern recognition, 2005. CVPR 2005. IEEE computer society conference on. Vol. 2, 2005, IEEE, 994–1000.
52. [52] Shouval, H.Z., Samuel, S., Wittenberg, G.M., Spike timing dependent plasticity: a consequence of more fundamental learning rules. Spike-Timing Dependent Plasticity, 2010, 60.
53. [53] Song, S., Miller, K.D., Abbott, L.F., Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3:9 (2000), 919–926.
54. [54] Stevens, J.-L.R., Law, J.S., Antolík, J., Bednar, J.A., Mechanisms for stable, robust, and adaptive development of orientation maps in the primary visual cortex. The Journal of Neuroscience 33:40 (2013), 15747–15766.
55. [55] Supèr, H., Romeo, A., Keil, M., Feed-forward segmentation of figure-ground and assignment of border-ownership. PLoS One, 5(5), 2010, e10705.
56. [56] Turrigiano, G.G., Nelson, S.B., Homeostatic plasticity in the developing nervous system. Nature Reviews Neuroscience 5:2 (2004), 97–107.
57. [57] Varela, C., Thalamic neuromodulation and its implications for executive networks. Frontiers in Neural Circuits, 8, 2014, 69.
58. [58] Vogels, T.P., Froemke, R.C., Doyon, N., Gilson, M., Haas, J.S., Liu, R., et al. Inhibitory synaptic plasticity: spike timing-dependence and putative network function. Frontiers in Neural Circuits, 7(EPFL-REVIEW-189448), 2013.
59. [59] von der Heydt, R., Figure–ground organization and the emergence of proto-objects in the visual cortex. Frontiers in Psychology, 6, 2015.
60. [60] Wagatsuma, N., von der Heydt, R., Niebur, E., Spike Synchrony Generated by Modulatory Common Input through NMDA-type Synapses. Journal of Neurophysiology, 2016, jn–01142.
61. [61] Wang, L., Maffei, A., Inhibitory plasticity dictates the sign of plasticity at excitatory synapses. The Journal of Neuroscience 34:4 (2014), 1083–1093.
62. [62] Widloski, J., Fiete, I.R., A model of grid cell development through spatial exploration and spike time-dependent plasticity. Neuron 83:2 (2014), 481–495.
63. [63] Wiesel, T.N., Hubel, D.H., et al. Single-cell responses in striate cortex of kittens deprived of vision in one eye. Journal of Neurophysiology 26:6 (1963), 1003–1017.
64. [64] Yantis, S., The neural basis of selective attention cortical sources and targets of attentional modulation. Current Directions in Psychological Science 17:2 (2008), 86–90.
65. [65] Zenke, F., Agnes, E.J., Gerstner, W., Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nature Communications, 6, 2015.
66. [66] Zhaoping, L., Border ownership from intracortical interactions in visual area V2. Neuron 47:1 (2005), 143–153.
67. [67] Zhou, H., Friedman, H.S., Von Der Heydt, R., Coding of border ownership in monkey visual cortex. The Journal of Neuroscience 20:17 (2000), 6594–6611.
68. [68] Zucker, R.S., Regehr, W.G., Short-term synaptic plasticity. Annual Review of Physiology 64:1 (2002), 355–405.