Self-organizing neural systems based on predictive learning

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 361, Issue 1807, 2003, Pages 1149-1175

  Rao, Rajesh P N ^a   Sejnowski, Terrence J ^b,c  

^a University of Washington   (United States)

^b HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Cerebral cortex; Conditioning; Neuroscience; Prediction; Synaptic plasticity; Visual perception

Indexed keywords

ACTION POTENTIAL; ADAPTATION; ANIMAL; ARTICLE; BEE; BIOLOGICAL MODEL; BRAIN CORTEX; FEEDING BEHAVIOR; HAPLORHINI; INSTRUMENTAL CONDITIONING; LEARNING; NERVE CELL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; PHYSIOLOGY;

ACTION POTENTIALS; ADAPTATION, PHYSIOLOGICAL; ANIMALS; BEES; CEREBRAL CORTEX; CONDITIONING, OPERANT; FEEDING BEHAVIOR; HAPLORHINI; LEARNING; MODELS, NEUROLOGICAL; NERVE NET; NEURONAL PLASTICITY; NEURONS;

EID: 0042233985     PISSN: 1364503X     EISSN: None     Source Type: Journal    
DOI: 10.1098/rsta.2003.1190     Document Type: Conference Paper

References (76)

1. Abbott, L. F. & Blum, K. I. 1996 Functional significance of long-term potentiation for sequence learning and prediction. Cereb. Cortex 6, 406-416.
2. Abbott, L. F. & Song, S. 1999 Temporally asymmetric Hebbian learning, spike timing and neural response variability. In Advances in neural information processing systems vol. 11, pp. 69-75. Cambridge, MA: MIT Press.
3. Atick, J. J. & Redlich, A. N. 1992 What does the retina know about natural scenes? Neural Comput. 4, 196-210.
4. Barlow, H. B. 1961 Possible principles underlying the transformation of sensory messages. In Sensory communication (ed. W. A. Rosenblith), pp. 217-234. Cambridge, MA: MIT Press.
5. Barlow, H. B. 1998 Cerebral predictions. Perception 27, 885-888.
6. Bell, A. J. &: Sejnowski, T. J. 1997 The 'independent components' of natural scenes are edge filters. Vision Res. 37, 3327-3338.
7. Bell, C., Bodznick, D., Montgomery, J. & Bastian, J. 1997 The generation and subtraction of sensory expectations within cerebellum-like structures. Brain Behav. Evol. 50, 17-31.
8. Berry, D. A. & Fristedt, B. 1985 Bandit problems: sequential allocation of experiments. London: Chapman and Hall.
9. Bi, G. Q. & Poo, M. M. 1998 Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464- 10472.
10. Bliss, T. V. & Collingridge, G. L. 1993 A synaptic model of memory: long-term potentiation in the hippocampus. Nature 361, 31-39.
11. Chance, F. S., Nelson, S. B. & Abbott, L. F. 1999 Complex cells as cortically amplified simple cells. Nature Neurosci. 2, 277-282.
12. Cole, B. J. & Robbins, T. W. 1992 Forebrain norepinephrine: role in controlled information processing in the rat. Neuropsychopharmacology 7, 129-142.
13. Daugman, J. G. & Downing, C. J. 1995 Demodulation, predictive coding, and spatial vision. J. Opt. Soc. Am. A12, 641-660.
14. Dayan, P. 2002 Matters temporal. Trends Cogn. Sci. 6, 105-106.
15. Dayan, P. & Hinton, G. 1996 Varieties of Helmholtz machine. Neural Netw. 9, 1385-1403.
16. Destexhe, A., Mainen, Z. & Sejnowski, T. 1998 Kinetic models of synaptic transmission. In Methods in neuronal modeling (ed. C. Koch & I. Segev). Cambridge, MA: MIT Press.
17. Dong, D. W. & Atick, J. J. 1995 Statistics of natural time-varying images. Network Computat. Neural Syst. 6, 345-358.
18. Douglas, R. J., Martin, K. A. C. & Whitteridge, D. 1991 An intracellular analysis of the visual responses of neurons in cat visual cortex. J. Physiol. 440, 659-696.
19. Doya, K. 2000 Reinforcement learning in continuous time and space. Neural Comput. 12, 219- 245.
20. Eckert, M. P. & Buchsbaum, G. 1993 Efficient encoding of natural time varying images in the early visual system. Phil. Trans. R. Soc. Land. B 339, 385-395.
21. Emptage, N. J. 1999 Calcium on the up: supralinear calcium signaling in central neurons. Neuron 24, 495-497.
22. Feldman, D. 2000 Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron 27, 45-56.
23. Franks, K. M., Bartol, T. M., Egelman, D. M., Poo, M. M. & Sejnowski, T. J. 1999 Simulated dendritic influx of calcium ions through voltage- and ligand-gated channels using MCELL. Abstr. Soc. Neurosci. 25, 1989.
24. Gerstner, W., Kempter, R., Hemmen, J. L. & van Wagner, H. 1996 A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76-81.
25. Ghahramani, Z. 2001 An introduction to hidden Markov models and Bayesian networks. Int. J. Patt. Recog. Art. Intel. 15, 9-42.
26. Gould, J. L. 1987 In Foraging behavior (ed. A. C. Kamil, J. R. Krebs & H. R. Pulliam). New York: Plenum.
27. Harder, L. D. & Real, L. A. 1987 Why are bumble-bees risk averse? Ecology 68, 1104-1108.
28. Hebb, D. O. 1949 The organization of behaviour: a neuropsychological theory. Wiley.
29. Koch, C. 1999 Biophysics of computation: information processing in single neurons. Oxford University Press.
30. Koester, H. J. & Sakmann, B. 1998 Calcium dynamics in single spines during coincident pre- and postsynaptic activity depend on relative timing of back-propagating action potentials and subthreshold excitatory postsynaptic potentials. Proc. Natl Acad. Sci. USA 95, 9596-9601.
31. Krebs, J. R., Kacelnik, A. & Taylor, P. 1978 Nature 275, 27.
32. Levy, W. & Steward, O. 1983 Temporal contiguity requirements for long-term associative poten- tiation/depression in the hippocampus. Neuroscience 8, 791-797.
33. Linden, D. J. 1999 The return of the spike: postsynaptic action potentials and the induction of LTP and LTD. Neuron 22, 661-666.
34. Livingstone, M. 1998 Mechanisms of direction selectivity in macaque VI. Neuron 20, 509-526.
35. MacKay, D. M. 1956 The epistemological problem for automata. In Automata studies pp. 235- 251. Princeton, NJ: Princeton University Press.
36. Magee, J. C. & Johnston, D. 1997 A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science 275, 209-213.
37. Mainen, Z. & Sejnowski, T. 1996 Influence of dendritic structure on firing pattern in model neocortical neurons. Nature 382, 363-366.
38. Markram, H., Lubke, J., Frotscher, M. & Sakmann, B. 1997 Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213-215.
39. Mehta, M. R. & Wilson, M. 2000 From hippocampus to VI: effect of LTP on spatiotemporal dynamics of receptive fields. In Computational neuroscience, trends in research 1999 (ed. J. Bower). Amsterdam: Elsevier Press.
40. Mehta, M. R., Barnes, C. A. & McNaughton, B. L. 1997 Experience-dependent, asymmetric expansion of hippocampal place fields. Proc. Natl Acad. Sci. USA 94, 8918-8921.
41. Menzel, R. & Erber, J. 1978 Learning and memory in bees. Scient. Am. 239, 80-87.
42. Menzel, R., Erber, J. & Masuhr, J. 1974 Experimental analysis of insect behavior, vol. 195. Springer.
43. Minai, A. A. & Levy, W. 1993 Sequence learning in a single trial. In Proc. 1993 INNS World Congr. Neural Networks II, pp. 505-508. New Jersey: Erlbaum.
44. Montague, P. R. & Sejnowski, T. J. 1994 The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learn. Mem. 1, 1-33.
45. Montague, P. R., Dayan, P. & Sejnowski, T. J. 1994 Foraging in an uncertain environment using predictive Hebbian learning. In Advances in neural information processing systems, vol. 6, pp. 598-605. San Mateo, CA: Morgan Kaufmann.
46. Montague, P. R., Dayan, P., Person, C. & Sejnowski, T. J. 1995 Bee foraging in uncertain environments using predictive Hebbian learning. Nature 377, 725-728.
47. Montague, P. R., Dayan, P. & Sejnowski, T. J. 1996 A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936-1947.
48. Morrison, J. H. & Magistretti, P. J. 1983 Monoamines and peptides in cerebral cortexcontrasting principles of cortex organization. Trends Neurosci. 6, 146-151.
49. Movshon, J. A. & Sluyters, R. C. V. 1981 Visual neural development. A. Rev. Psychol. 32, 477-522.
50. Olshausen, B. A. & Field, D. J. 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607-609.
51. Paulsen, O. & Sejnowski, T. J. 2000 Natural patterns of activity and long-term synaptic plasticity. Curr. Opin. Neurobiol. 10, 172-179.
52. Rao, R. P. N. 1999 An optimal estimation approach to visual perception and learning. Vision Res. 39, 1963-1989.
53. Rao, R. P. N. & Ballard, D. H. 1997 Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 9, 721-763.
54. Rao, R. P. N. & Ballard, D. H. 1999 Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive field effects. Nature Neurosci. 2, 79-87.
55. Rao, R. P. N. & Sejnowski, T. J. 2000 Predictive sequence learning in recurrent neocortical circuits. In Advances in neural information processing systems, vol. 12, pp. 164-170. Cambridge, MA: MIT Press.
56. Rao, R. P. N. & Sejnowski, T. J. 2001 Spike-timing dependent Hebbian plasticity as temporal difference learning. Neural Comput. 13, 2221-2237.
57. Real, L. A. 1991 Animal choice behavior and the evolution of cognitive architecture. Science 253, 980-986.
58. Real, L. A., Ellner, S. & Harder, L. D. 1990 Short-term energy maximization and risk-aversion in bumblebees: a reply to Possingham et al. Ecology 71, 1625-1628.
59. Rescorla, R. A. 1988 Behavioral studies of Pavlovian conditioning. A. Rev. Neurosci. 11, 329-352.
60. Roberts, P. D. 1999 Computational consequences of temporally asymmetric learning rules. I. Differential Hebbian learning. J. Comput. Neurosci. 7, 235-246.
61. Schiller, J., Schiller, Y. & Clapham, D. E. 1998 NMDA receptors amplify calcium influx into dendritic spines during associative pre- and postsynaptic activation. Nature Neurosci, 1, 114-118.
62. Schultz, W., Romo, R., Ljungberg, T., Mirenowicz, J., Hollerman, J. R. & Dickinson, A. 1995 Reward-related signals carried by dopamine neurons. In Models of information processing in the basal ganglia (ed. J. C. Houk, J. L. Davis &: D. G. Beiser), pp. 233-248. Cambridge, MA: MIT Press.
63. Schultz, W., Dayan, P. & Montague, P. R. 1997 A neural substrate of prediction and reward. Science 275, 1593-1598.
64. Schwartz, O. & Simoncelli, E. P. 2001 Natural signal statistics and sensory gain control. Nature Neurosci. 4, 819-825.
65. Sejnowski, T. J. 1999 The book of Hebb. Neuron 24, 773-776.
66. Song, S., Miller, K. D. & Abbott, L. F. 2000 Competitive Hebbian learning through spike-timing dependent synaptic plasticity. Nature Neurosci. 3, 919-926.
67. Stiles, J. S., Bartol, T. M., Salpeter, M. M., Salpeter, E. E. & Sejnowski, T. J. 2000 Synaptic variability: new insights from reconstructions and Monte Carlo simulations with MCELL. In Synapses (ed. M. Cowan & K. Davies). Baltimore: Johns Hopkins University Press.
68. Button, R. S. 1988 Learning to predict by the method of temporal differences. Mach. Learn. 3, 9-44.
69. Sutton, R. S. & Barto, A. 1990 Time-derivative models of Pavlovian reinforcement. In Learning and computational neuroscience: foundations of adaptive networks (ed. M. Gabriel & J. W. Moore). Cambridge, MA: MIT Press.
70. Sutton, R. S. & Barto, A. G. 1998 Reinforcement learning: an introduction. Cambridge, MA: MIT Press.
71. Tesauro, G. 1989 Neurogammon wins Computer Olympiad. Neural Comput. 1, 321-323.
72. Tootell, R. B., Switkes, E., Silverman, M. S. & Hamilton, S. L. 1988 Functional anatomy of macaque striate cortex. II. Retinotopic organization. J. Neurosci. 8, 1531-1568.
73. Van Essen, D. 1985 Functional organization of primate visual cortex. In Cerebral cortex (ed. A. Peters & E. Jones), vol. 3, pp. 259-329. New York: Plenum.
74. Westerman, W. C., Northmore, D. P. M. & Elias, J. G. 1999 Antidromic spikes drive Hebbian learning in an artificial dendritic tree. Analog Integr. Circuits Signal Process. 18, 141-152.
75. Wise, R. A. 1982 Neuroleptics and operant behavior: the anhedonia hypothesis. Behav. Brain Sci. 5, 39-53.
76. Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A. & Poo, M. M. 1998 A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37-44.