Goal-directed control and its antipodes

Neural Networks

Volumn 22, Issue 3, 2009, Pages 213-219

  Dayan, Peter ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Basal ganglia; Declarative control; Goal directed control; Habitual control; Model based reinforcement learning; Model free reinforcement learning; Prefrontal cortex; Procedural control

Indexed keywords

BASAL GANGLIA; DECLARATIVE CONTROL; GOAL-DIRECTED CONTROL; HABITUAL CONTROL; MODEL-BASED REINFORCEMENT LEARNING; MODEL-FREE REINFORCEMENT LEARNING; PREFRONTAL CORTEX; PROCEDURAL CONTROL;

EDUCATION; LIGHT MEASUREMENT; REINFORCEMENT LEARNING;

REINFORCEMENT;

ACCURACY; ARTICLE; BASAL GANGLION; BAYES THEOREM; COGNITION; CONTROL; ENVIRONMENT; EVOLUTION; HUMAN; LEARNING; MEMORY; NERVE CELL NETWORK; NONHUMAN; PREFRONTAL CORTEX; PRIORITY JOURNAL; RECOGNITION; REINFORCEMENT;

ANIMALS; BASAL GANGLIA; BRAIN; COGNITION; GOALS; HABITUATION, PSYCHOPHYSIOLOGIC; HUMANS; LEARNING; MODELS, NEUROLOGICAL; PREFRONTAL CORTEX; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY);

EID: 67349170462     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.03.004     Document Type: Article

References (114)

1. Anderson J. Acquisition of cognitive skill. Psychological Review 89 4 (1982) 369-406
2. Anderson J., and Lebiere C. The atomic components of thought (1998), Erlbaum, Mahwah, NJ
3. Anderson J.R., Bothell D., Byrne M.D., Douglass S., Lebiere C., and Qin Y. An integrated theory of the mind. Psychological Review 111 4 (2004) 1036-1060. http://dx.doi.org/10.1037/0033-295X.111.4.1036 URL: http://dx.doi.org/10.1037/0033-295X.111.4.1036
4. Anderson J.R., Fincham J.M., Qin Y., and Stocco A. A central circuit of the mind. Trends in Cognitive Sciences 12 4 (2008) 136-143. http://dx.doi.org/10.1016/j.tics.2008.01.006 URL: http://dx.doi.org/10.1016/j.tics.2008.01.006
5. Badre D., and D'esposito M. Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex. Journal of Cognitive Neuroscience 19 12 (2007) 2082-2099. http://dx.doi.org/10.1162/jocn.2007.19.12.2082 URL: http://dx.doi.org/10.1162/jocn.2007.19.12.2082
6. Balleine B.W. Neural bases of food-seeking: Affect arousal and reward in corticostriatolimbic circuits. Physiological Behaviour 86 5 (2005) 717-730. http://dx.doi.org/10.1016/j.physbeh.2005.08.061 URL: http://dx.doi.org/10.1016/j.physbeh.2005.08.061
7. Balleine B.W., and Killcross S. Parallel incentive processing: An integrated view of amygdala function. Trends in Neuroscience 29 5 (2006) 272-279. http://dx.doi.org/10.1016/j.tins.2006.03.002 URL: http://dx.doi.org/10.1016/j.tins.2006.03.002
8. Barto A. Adaptive critics and the basal ganglia. In: Houk J., Davis J., and Beiser D. (Eds). Models of information processing in the basal ganglia (1995), MIT Press, Cambridge, MA 215-232
9. Barto A.G., Sutton R.S., and Watkins C.J.C.H. Learning and sequential decision making. Learning and computational neuroscience (1990), MIT Press, Cambridge, MA 539-602
10. Belin D., and Everitt B.J. Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 57 3 (2008) 432-441. http://dx.doi.org/10.1016/j.neuron.2007.12.019 URL: http://dx.doi.org/10.1016/j.neuron.2007.12.019
11. Bertsekas D.P. Dynamic programming and optimal control (2 vol set) (2007), Athena Scientific. http://amazon.com/o/ASIN/1886529086/ URL: http://amazon.com/o/ASIN/1886529086/
12. Bolles R.C. Species-specific defense reactions and avoidance learning. Psychological Review 77 (1970) 32-48
13. Braver T., and Cohen J. On the control of control: The role of dopamine in regulating prefrontal function and working memory. In: Monsell S., and Driver J. (Eds). Attention and performance XVIII: Control of cognitive processes (2000), MIT Press, Cambridge, MA 713-737
14. Bray S., Rangel A., Shimojo S., Balleine B., and O'Doherty J.P. The neural mechanisms underlying the influence of Pavlovian cues on human decision making. Journal of Neuroscience 28 22 (2008) 5861-5866. http://dx.doi.org/10.1523/JNEUROSCI.0897-08.2008 URL: http://dx.doi.org/10.1523/JNEUROSCI.0897-08.2008
15. Breland K., and Breland M. The misbehavior of organisms. American Psychologist 16 (1961) 681-684
16. Chaiken S., and Trope Y. Dual-process theories in social psychology (1999), Guilford Press, New York, NY
17. Crossman E. A theory of the acquisition of speed-skill. Ergonomics 2 (1959) 153-166
18. Daw N.D., Kakade S., and Dayan P. Opponent interactions between serotonin and dopamine. Neural Networks 15 4-6 (2002) 603-616
19. Daw N.D., Niv Y., and Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience 8 12 (2005) 1704-1711. http://dx.doi.org/10.1038/nn1560 URL: http://dx.doi.org/10.1038/nn1560
20. Dayan P. Improving generalization for temporal difference learning: The successor representation. Neural Computation 5 4 (1993) 613-624
21. Dayan P. Recurrent sampling models for the helmholtz machine. Neural Computation 11 3 (1999) 653-678
22. Dayan P. Bilinearity rules and prefrontal cortex. Frontiers in Computational Neuroscience 1 (2007) 1
23. Dayan P., Hinton G.E., Neal R.M., and Zemel R.S. The Helmholtz machine. Neural Computation 7 5 (1995) 889-904
24. Dayan P., and Huys Q. Serotonin in affective control. Annual Review of Neuroscience 32 (2009)
25. Dayan P., Niv Y., Seymour B., and Daw N.D. The misbehavior of value and the discipline of the will. Neural Networks 19 8 (2006) 1153-1160. http://dx.doi.org/10.1016/j.neunet.2006.03.002 URL: http://dx.doi.org/10.1016/j.neunet.2006.03.002
26. Dayan P., and Seymour B. Values and actions in aversion. In: Glimcher P., Camerer C., Poldrack R., and Fehr E. (Eds). Neuroeconomics: Decision making and the brain (2008), Academic Press, New York, NY 175-191
27. Deakin J.F.W. Roles of brain serotonergic neurons in escape avoidance and other behaviors. Journal of Psychopharmacology 43 (1983) 563-577
28. Deakin J.F.W., and Graeff F.G. 5-HT and mechanisms of defence. Journal of Psychopharmacology 5 (1991) 305-316
29. Diba K., and Buzsáki G. Forward and reverse hippocampal place-cell sequences during ripples. Nature Neuroscience 10 10 (2007) 1241
30. Dickinson A. Actions and habits: The development of behavioural autonomy. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 308 1135 (1985) 67-78
31. Dickinson A., and Balleine B. The role of learning in motivation. In: Gallistel C. (Ed). Stevens' handbook of experimental psychology Vol. 3 (2002), Wiley, New York, NY 497-533
32. Epstein S. Integration of the cognitive and the psychodynamic unconscious. American Psychologist 49 8 (1994) 709-724
33. Evans J.S.B.T. In two minds: Dual-process accounts of reasoning. Trends in Cognitive Sciences 7 10 (2003) 454-459
34. Felleman D.J., and Essen D.C.V. Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex 1 1 (1991) 1-47
35. Fitts P. Perceptual-motor skill learning. In: Melton A. (Ed). Categories of human learning (1964), Academic Press, New York, NY 243-285
36. Foster D., and Wilson M. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440 7084 (2006) 680-683
37. Frank M.J., Loughry B., and O'Reilly R.C. Interactions between frontal cortex and basal ganglia in working memory: A computational model. Cognitive, Affective, & Behavioral Neuroscience 1 2 (2001) 137-160
38. Fu W.-T., and Anderson J.R. From recurrent choice to skill learning: A reinforcement-learning model. Journal of Experimental Psychology Generol 135 2 (2006) 184-206. http://dx.doi.org/10.1037/0096-3445.135.2.184 URL: http://dx.doi.org/10.1037/0096-3445.135.2.184
39. Fu W.-T., and Anderson J.R. Solving the credit assignment problem: Explicit and implicit learning of action sequences with probabilistic outcomes. Psychological Research 72 3 (2008) 321-330. http://dx.doi.org/10.1007/s00426-007-0113-7 URL: http://dx.doi.org/10.1007/s00426-007-0113-7
40. Gelly, S., & Silver, D. (2008). Achieving master level play in 9×9 computer Go. In: Proceedings of the 23rd AAAI conference on artificial intelligence-Nectar track
41. Haber S.N., Fudge J.L., and Mcfarland N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. Journal of Neuroscience 20 6 (2000) 2369-2382
42. Hammond K. Human judgment and social policy (1996), Oxford University Press, Oxford, England
43. Hertwig R., Barron G., Weber E.U., and Erev I. Decisions from experience and the effect of rare events in risky choice. Psychological Science 15 8 (2004) 534-539. http://dx.doi.org/10.1111/j.0956-7976.2004.00715.x URL: http://dx.doi.org/10.1111/j.0956-7976.2004.00715.x
44. Hinton G.E., Dayan P., Frey B.J., and Neal R.M. The "wake-sleep" algorithm for unsupervised neural networks. Science 268 5214 (1995) 1158-1161
45. Huys, Q. J. M. (2007). Reinforcers and control. towards a computational ætiology of depression. Ph.D. thesis, Gatsby Computational Neuroscience Unit, UCL, University of London
46. Huys Q.J.M., and Dayan P. A Bayesian formulation of behavioral control. Advances in Neural Information Processing Systems (2008)
47. Joel D., and Weiner I. The connections of the dopaminergic system with the striatum in rats and primates: An analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96 3 (2000) 451-474
48. Just M., and Carpenter P. A capacity theory of comprehension: Individual differences in working memory. Psychological Review 99 (1992) 122-149
49. Just M., Carpenter P., and Varma S. Computational modeling of high-level cognition and brain function. Human Brain Mapping 8 (1999) 128-136
50. Kable J.W., and Glimcher P.W. The neural correlates of subjective value during intertemporal choice. Nature Neuroscience 10 12 (2007) 1625-1633. http://dx.doi.org/10.1038/nn2007 URL: http://dx.doi.org/10.1038/nn2007
51. Kahneman D., and Frederick S. Representativeness revisited: Attribute substitution in intuitive judgment. In: Gilovich T., Griffin D., and Kahneman D. (Eds). Heuristics and biases: The psychology of intuitive judgment (2002), Cambridge University Press 49-81
52. Kahneman D., and Frederick S. Frames and brains: Elicitation and control of response tendencies. Trends in Cognitive Sciences 11 2 (2007) 45-46. http://dx.doi.org/10.1016/j.tics.2006.11.007 URL: http://dx.doi.org/10.1016/j.tics.2006.11.007
53. Kawato M., Hayakawa H., and Inui T. A forward-inverse optics model of reciprocal connections between visual cortical areas. Network: Computation in Neural Systems 4 4 (1993) 415-422
54. Kawato, M., & Wolpert, D. (1998). Internal models for motor control. In: Novartis Found Symp, 218 (pp. 291-304); discussion 304-7
55. Killcross S., and Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cerebral Cortex 13 4 (2003) 400-408
56. Koechlin E., and Hyafil A. Anterior prefrontal function and the limits of human decision-making. Science 318 5850 (2007) 594-598. http://dx.doi.org/10.1126/science.1142995 URL: http://dx.doi.org/10.1126/science.1142995
57. Koechlin E., Ody C., and Kouneiher F. The architecture of cognitive control in the human prefrontal cortex. Science 302 5648 (2003) 1181-1185. http://www.sciencemag.org/cgi/content/abstract/302/5648/1181 URL: http://www.sciencemag.org/cgi/content/abstract/302/5648/1181
58. Koechlin E., and Summerfield C. An information theoretical approach to prefrontal executive function. Trends in Cognitive Sciences 11 6 (2007) 229-235. http://dx.doi.org/10.1016/j.tics.2007.04.005 URL: http://dx.doi.org/10.1016/j.tics.2007.04.005
59. Krueger K.A., and Dayan P. Flexible shaping: How learning in small steps helps. Cognition 110 3 (2009) 380-394
60. Lammel S., Hetzel A., Hckel O., Jones I., Liss B., and Roeper J. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57 5 (2008) 760-773. http://dx.doi.org/10.1016/j.neuron.2008.01.022 URL: http://dx.doi.org/10.1016/j.neuron.2008.01.022
61. Lee A., and Wilson M. Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep. Neuron 36 6 (2002) 1183-1194
62. Logan G. Toward an instance theory of automatization. Psychological Review 95 4 (1988) 492-527
63. Louie K., and Wilson M. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29 1 (2001) 145-156
64. Maier S., and Seligman M. Learned helplessness: Theory and evidence. Journal of Experimental Psychology: General 105 1 (1976) 3-46
65. March J.G. Learning to be risk averse. Psychological Review 103 2 (1996) 309-319
66. McCarthy J., and Hayes P. Some philosophical problems from the standpoint of artificial intelligence. In: Meltzer B., and Michie D. (Eds). Machine intelligence Vol. 4 (1969) 463-502
67. McClure S.M., Laibson D.I., Loewenstein G., and Cohen J.D. Separate neural systems value immediate and delayed monetary rewards. Science 306 5695 (2004) 503-507. http://dx.doi.org/10.1126/science.1100907 URL: http://dx.doi.org/10.1126/science.1100907
68. Meyer D.E., and Kieras D.E. A computational theory of executive cognitive processes and multiple-task performance: Part 1. basic mechanisms. Psychological Review 104 1 (1997) 3-65
69. Montague P.R., Dayan P., and Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive hebbian learning. Journal of Neuroscience 16 5 (1996) 1936-1947
70. Morris G., Nevet A., Arkadir D., Vaadia E., and Bergman H. Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience 9 8 (2006) 1057-1063. http://dx.doi.org/10.1038/nn1743 URL: http://dx.doi.org/10.1038/nn1743
71. Newell A. Unified theories of cognition (1990), Harvard University Press, Cambridge, MA
72. Newell A. Unified theories of cognition (The William James lectures) (1994), Harvard University Press. http://amazon.com/o/ASIN/0674921011/ URL: http://amazon.com/o/ASIN/0674921011/
73. Newell A., and Rosenbloom P. Mechanisms of skill acquisition and the law of practice. In: Anderson J. (Ed). Cognitive skills and their acquisition (1981), Lawrence Erlbaum Associates, Hillsdale, NJ 1-56
74. Niv Y., Daw N.D., Joel D., and Dayan P. Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berl) 191 3 (2007) 507-520. http://dx.doi.org/10.1007/s00213-006-0502-4 URL: http://dx.doi.org/10.1007/s00213-006-0502-4
75. Niv Y., Joel D., Meilijson I., and Ruppin E. Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adaptive Behavior 10 1 (2002) 5-24
76. O'Doherty J.P. Lights camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Annals of the New York Academy of Sciences 1121 December (2007) 254-272. http://dx.doi.org/10.1196/annals.1401.036 URL: http://dx.doi.org/10.1196/annals.1401.036
77. O'Reilly R., Braver T., and Cohen J. A biologically based computational model of working memory. In: Miyake A., and Shah P. (Eds). Models of working memory: Mechanisms of active maintenance and executive control (1999), Cambridge University Press, New York, NY 375-411 [Chapter 11]
78. O'Reilly R.C., and Frank M.J. Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation 18 2 (2006) 283-328. http://neco.mitpress.org/cgi/content/abstract/18/2/283 URL: http://neco.mitpress.org/cgi/content/abstract/18/2/283
79. Puterman M.L. Markov decision processes: Discrete stochastic dynamic programming (Wiley series in probability and statistics) (2005), Wiley-Interscience. http://amazon.com/o/ASIN/0471727822/ URL: http://amazon.com/o/ASIN/0471727822/
80. Rao R., Olshausen B., and Lewicki M. Probabilistic models of the brain: Perception and neural function (2002), MIT Press, Bradford Books
81. Reynolds S.M., and Berridge K.C. Fear and feeding in the nucleus accumbens shell: Rostrocaudal segregation of gaba-elicited defensive behavior versus eating behavior. Journal of Neuroscience 21 9 (2001) 3261-3270
82. Reynolds S.M., and Berridge K.C. Positive and negative motivation in nucleus accumbens shell: Bivalent rostrocaudal gradients for gaba-elicited eating taste "liking"/"disliking" reactions, place preference/avoidance, and fear. Journal of Neuroscience 22 16 (2002) 7308-7320. http://dx.doi.org/20026734 URL: http://dx.doi.org/20026734
83. Reynolds S.M., and Berridge K.C. Emotional environments retune the valence of appetitive versus fearful functions in nucleus accumbens. Nature Neuroscience 11 4 (2008) 423-425. http://dx.doi.org/10.1038/nn2061 URL: http://dx.doi.org/10.1038/nn2061
84. Roesch M.R., Calu D.J., and Schoenbaum G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience 10 12 (2007) 1615-1624. http://dx.doi.org/10.1038/nn2013 URL: http://dx.doi.org/10.1038/nn2013
85. Rolls E.T., and Grabenhorst F. The orbitofrontal cortex and beyond: From affect to decision-making. Progress in Neurobiology Sep (2008). http://dx.doi.org/10.1016/j.pneurobio.2008.09.001 URL: http://dx.doi.org/10.1016/j.pneurobio.2008.09.001
86. Rougier N.P., Noelle N.P., Braver T.S., Cohen J.D., and O'Reilly R.C. Prefrontal cortex and flexible cognitive control: Rules without symbols. Proceedings of the National Academy of Sciences 102 20 (2005) 7339-7343
87. Russell S., and Norvig P. Artificial intelligence: A modern approach (1995), Prentice Hall, Englewood Cliffs, NJ
88. Schneider W., and Shiffrin R. Controlled and automatic human information processing: 1 detection, search, and attention. Psychological Review 84 1 (1977) 1-66
89. Schultz W., Dayan P., and Montague P.R. A neural substrate of prediction and reward. Science 275 5306 (1997) 1593-1599
90. Shiffrin R.M., and Schneider W. Automatic and controlled processing revisited. Psychological Review 91 2 (1984) 269-276
91. Skaggs W., and Mcnaughton B. Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271 5257 (1996) 1870
92. Skinner B.F. The behavior of organisms: An experimental analysis (1938), Appleton-Century-Crofts
93. Sloman S. The empirical case for two systems of reasoning. Psychological Bulletin 119 (1996) 3-22
94. Squire L.R. Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory 82 3 (2004) 171-177. http://dx.doi.org/10.1016/j.nlm.2004.06.005 URL: http://dx.doi.org/10.1016/j.nlm.2004.06.005
95. Stanovich K., and West R. Individual differences in reasoning: Implications for the rationality debate?. In: Gilovich T., Griffin D., and Kahneman D. (Eds). Heuristics and biases: The psychology of intuitive judgment (2002), Cambridge University Press 421-440
96. Suri R.E., and Schultz W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91 3 (1999) 871-890
97. Sutton R. Dyna an integrated architecture for learning, planning, and reacting. ACM SIGART Bulletin 2 4 (1991) 160-163
98. Sutton, R. (1995). TD models: Modeling the world at a mixture of time scales. In: Proceedings of the twelfth international conference on machine learning (pp. 531-539)
99. Sutton R., Precup D., and Singh S. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112 (1999) 181-211
100. Sutton R.S., and Barto A.G. Reinforcement learning: An introduction (Adaptive computation and machine learning) (1998), The MIT Press. http://amazon.com/o/ASIN/0262193981/ URL: http://amazon.com/o/ASIN/0262193981/
101. Taatgen N.A., and Anderson J.R. Why do children learn to say "broke"? A model of learning the past tense without feedback. Cognition 86 2 (2002) 123-155
102. Taatgen N.A., Huss D., Dickison D., and Anderson J.R. The acquisition of robust and flexible cognitive skills. Journal of Experimental Psychology Generol 137 3 (2008) 548-565. http://dx.doi.org/10.1037/0096-3445.137.3.548 URL: http://dx.doi.org/10.1037/0096-3445.137.3.548
103. Talmi D., Seymour B., Dayan P., and Dolan R.J. Human Pavlovian-instrumental transfer. Journal of Neuroscience 28 2 (2008) 360-368. http://dx.doi.org/10.1523/JNEUROSCI.4028-07.2008 URL: http://dx.doi.org/10.1523/JNEUROSCI.4028-07.2008
104. Tanaka S.C., Balleine B.W., and O'Doherty J.P. Calculating consequences: Brain systems that encode the causal effects of actions. Journal of Neuroscience 28 26 (2008) 6750-6755. http://dx.doi.org/10.1523/JNEUROSCI.1808-08.2008 URL: http://dx.doi.org/10.1523/JNEUROSCI.1808-08.2008
105. Todorov E. Optimal control theory. In: Doya K., Ishii S., Pouget A., and Rao R. (Eds). Bayesian brain (2007), MIT Press 269-298
106. Tversky A., and Kahneman D. Belief in the law of small numbers. Psychological Bulletin 76 2 (1971) 105-110
107. Valentin V.V., Dickinson A., and O'Doherty J.P. Determining the neural substrates of goal-directed learning in the human brain. Journal of Neuroscience 27 15 (2007) 4019-4026. http://dx.doi.org/10.1523/JNEUROSCI.0564-07.2007 URL: http://dx.doi.org/10.1523/JNEUROSCI.0564-07.2007
108. van der Velde F., and de Kamps M. Neural blackboard architectures of combinatorial structures in cognition. Behavioral and Brain Sciences 29 1 (2006) 37-70. http://dx.doi.org/10.1017/S0140525X06009022 Discussion 70-108. URL: http://dx.doi.org/10.1017/S0140525X06009022
109. Watkins, C. (1989). Learning from delayed rewards. Ph.D. thesis. University of Cambridge
110. Williams D.R., and Williams H. Auto-maintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behaviour 12 4 (1969) 511-520
111. Williams G.V., and Goldman-Rakic P.S. Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376 6541 (1995) 572-575. http://dx.doi.org/10.1038/376572a0 URL: http://dx.doi.org/10.1038/376572a0
112. Williams R. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement Learning 8 (1992) 229-256
113. Wolpert D., and Kawato M. Multiple paired forward and inverse models for motor control. Neural Networks 11 7-8 (1998) 1317-1329
114. Wolpert D.M., Ghahramani Z., and Jordan M.I. An internal model for sensorimotor integration. Science 269 5232 (1995) 1880-1882