Active Inference, homeostatic regulation and adaptive behavioural control

Progress in Neurobiology

Volumn 134, Issue , 2015, Pages 17-35

  Pezzulo, Giovanni ^a   Rigoli, Francesco ^b   Friston, Karl ^b  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Active Inference; Adaptive control; Homeostatic regulation; Model based control; Model free control; Pavlovian control

Indexed keywords

ADAPTIVE BEHAVIOR; ANIMAL BEHAVIOR; CONDITIONED REFLEX; CORPUS STRIATUM; FUNCTIONAL ANATOMY; HIPPOCAMPUS; HOMEOSTASIS; INFERIOR TEMPORAL CORTEX; INSULA; MOTIVATION; MOTOR CORTEX; NONHUMAN; PREFRONTAL CORTEX; PREMOTOR CORTEX; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; STATE DEPENDENT LEARNING; ANIMAL; BEHAVIOR; BIOLOGICAL MODEL; BRAIN; HUMAN; LEARNING; PHYSIOLOGY; PSYCHOLOGICAL MODEL;

ANIMALS; ASSOCIATION LEARNING; BEHAVIOR; BRAIN; HOMEOSTASIS; HUMANS; MODELS, NEUROLOGICAL; MODELS, PSYCHOLOGICAL;

EID: 84946080033     PISSN: 03010082     EISSN: 18735118     Source Type: Journal    
DOI: 10.1016/j.pneurobio.2015.09.001     Document Type: Review

References (134)

1. Adams R.A., Shipp S., Friston K.J. Predictions not commands: active inference in the motor system. Brain Struct. Funct. 2013, 218:611-643. 10.1007/s00429-012-0475-5.
2. Arbib M.A. Schema theory. Encyclopedia of Artificial Intelligence 1992, 1427-1443. Wiley, New York. 2nd ed. S. Shapiro (Ed.).
3. Ashby R. Principles of the self-organizing dynamic system. J. Gen. Psychol. 1947, 37:125-128.
4. Balleine B.W., Daw N.D., O'Doherty J.P., Balleine B. Multiple forms of value learning and the function of dopamine. Neuroeconomics: Decision Making and the Brain 2009, 367-387. Academic Press. P. Glimcher, C. Camerer, E. Fehr, R. Poldrack (Eds.).
5. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407-419.
6. Barrett L.F., Simmons W.K. Interoceptive predictions in the brain. Nat. Rev. Neurosci. 2015, 16:419-429. 10.1038/nrn3950.
7. Bastos A.M., Usrey W.M., Adams R.A., Mangun G.R., Fries P., Friston K.J. Canonical microcircuits for predictive coding. Neuron 2012, 76:695-711.
8. Bates E., Elman J.L. Connectionism and the study of change. Brain Dev. Cogn. Read. 1993, 623-642.
9. Baum E.B. What is Thought? 2004, The MIT Press.
10. Belin D., Jonkman S., Dickinson A., Robbins T.W., Everitt B.J. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav. Brain Res. 2009, 199:89-102. (special issue on the role of the basal ganglia in learning and memory). 10.1016/j.bbr.2008.09.027.
11. Berridge K.C. Motivation concepts in behavioral neuroscience. Physiol. Behav. 2004, 81:179-209.
12. Botvinick M., Braver T.S., Barch D.M., Carter C.S., Cohen J.D. Conflict monitoring and cognitive control. Psychol. Rev. 2001, 108:624-652.
13. Botvinick M.M. Hierarchical models of behavior and prefrontal function. Trends Cogn. Sci. 2008, 12:201-208. 10.1016/j.tics.2008.02.009.
14. Botvinick M., Toussaint M. Planning as inference. Trends Cogn. Sci. 2012, 16:485-488. 10.1016/j.tics.2012.08.006.
15. Botvinick M., Weinstein A. Model-based hierarchical reinforcement learning and human action control. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2014, 369:20130480. 10.1098/rstb.2013.0480.
16. Brooks R.A. Intelligence without representation. Artif. Intell. 1991, 47:139-159.
17. Buzsáki G. The hippocampo-neocortical dialogue. Cereb. Cortex 1996, 6:81-92.
18. Buzsáki G., Moser E.I. Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nat. Neurosci. 2013, 16:130-138. 10.1038/nn.3304.
19. Buzsáki G., Peyrache A., Kubie J. Emergence of cognition from action. Cold Spring Harb. Symp. Quant. Biol. 2014, 79:41-50. 10.1101/sqb.2014.79.024679.
20. Caligiore D., Pezzulo G., Miall R.C., Baldassarre G. The contribution of brain sub-cortical loops in the expression and acquisition of action understanding abilities. Neurosci. Biobehav. Rev. 2013, 10.1016/j.neubiorev.2013.07.016.
21. Cannon W.B. Organization for physiological homeostasis. Physiol. Rev. 1929, 9:399-431.
22. Cisek P., Kalaska J.F. Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 2010, 33:269-298. 10.1146/annurev.neuro.051508.135409.
23. Clark D., White F.J. Review: D1 dopamine receptor-the search for a function: a critical evaluation of the D1/D2 dopamine receptor classification and its functional implications. Synapse 1987, 1:347-388.
24. Conant R.C., Ashby W.R. Every good regulator of a system must be a model of that system. Int. J. Syst. Sci. 1970, 89-97.
25. Cools R., Esposito M.D'. Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry 2011, 69:e113-e125.
26. Corbit L.H., Balleine B.W. The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J. Neurosci. 2011, 31:11786-11794. 10.1523/JNEUROSCI.2711-11.2011.
27. Courville A.C., Daw N.D., Touretzky D.S. Bayesian theories of conditioning in a changing world. Trends Cogn. Sci. 2006, 10:294-300.
28. Craig A.D.B. The sentient self. Brain Struct. Funct. 2010, 214:563-577. 10.1007/s00429-010-0248-y.
29. Damasio A., Carvalho G.B. The nature of feelings: evolutionary and neurobiological origins. Nat. Rev. Neurosci. 2013, 14:143-152. 10.1038/nrn3403.
30. Daw N.D., Dayan P. The algorithmic anatomy of model-based evaluation. Philos. Trans. R. Soc. B: Biol. Sci. 2014, 369:20130478.
31. Daw N.D., Gershman S.J., Seymour B., Dayan P., Dolan R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 2011, 69:1204-1215.
32. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8:1704-1711. 10.1038/nn1560.
33. Dayan P. Goal-directed control and its antipodes. Neural Netw. 2009, 22:213-219.
34. Dayan P., Hinton G.E., Neal R.M., Zemel R.S. The Helmholtz machine. Neural Comput. 1995, 7:889-904.
35. Dezfouli A., Balleine B.W. Habits, action sequences and reinforcement learning. Eur. J. Neurosci. 2012, 35:1036-1051. 10.1111/j.1460-9568.2012.08050.x.
36. Dezfouli A., Lingawi N.W., Balleine B.W. Habits as action sequences: hierarchical action control and changes in outcome value. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2014, 369:20130482. 10.1098/rstb.2013.0482.
37. Dickinson A. Associative learning and animal cognition. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2012, 367:2733-2742. 10.1098/rstb.2012.0220.
38. Dolan R.J., Dayan P. Goals and habits in the brain. Neuron 2013, 80:312-325. 10.1016/j.neuron.2013.09.007.
39. Drazen D.L., Vahl T.P., D'Alessio D.A., Seeley R.J., Woods S.C. Effects of a fixed meal pattern on ghrelin secretion: evidence for a learned response independent of nutrient status. Endocrinology 2006, 147:23-30. 10.1210/en.2005-0973.
40. Edwards M.J., Adams R.A., Brown H., Pareés I., Friston K.J. A Bayesian account of "hysteria". Brain J. Neurol. 2012, 135:3495-3512. 10.1093/brain/aws129.
41. Feldman H., Friston K.J. Attention, uncertainty, and free-energy. Front. Hum. Neurosci. 2010, 4:215. 10.3389/fnhum.2010.00215.
42. FitzGerald T.H.B., Dolan R.J., Friston K. Model averaging, optimal inference, and habit formation. Front. Hum. Neurosci. 2014, 8:457. 10.3389/fnhum.2014.00457.
43. Friston K. What is optimal about motor control?. Neuron 2011, 72:488-498.
44. Friston K. The free-energy principle: a unified brain theory?. Nat. Rev. Neurosci. 2010, 11:127-138. 10.1038/nrn2787.
45. Friston K. Hierarchical models in the brain. PLoS Comput. Biol. 2008, 4:e1000211.
46. Friston K., Daunizeau J., Kiebel S.J. Reinforcement learning or active inference?. PLoS ONE 2009, 4:e6421. 10.1371/journal.pone.0006421.
47. Friston K., Daunizeau J., Kilner J., Kiebel S.J. Action and behavior: a free-energy formulation. Biol. Cybern. 2010, 102:227-260. 10.1007/s00422-010-0364-z.
48. Friston K., Rigoli F., Ognibene D., Mathys C., Fitzgerald T., Pezzulo G. Active inference and epistemic value. Cogn. Neurosci. 2015, 13:1-28. 10.1080/17588928.2015.1020053.
49. Friston K., Schwartenbeck P., FitzGerald T., Moutoussis M., Behrens T., Dolan R.J. The anatomy of choice: dopamine and decision-making. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2014, 369:20130481. 10.1098/rstb.2013.0481.
50. Friston K., Schwartenbeck P., FitzGerald T., Moutoussis M., Behrens T., Dolan R.J. The anatomy of choice: active inference and agency. Front. Hum. Neurosci. 2013, 7:598. 10.3389/fnhum.2013.00598.
51. Friston K., Shiner T., FitzGerald T., Galea J.M., Adams R., Brown H., Dolan R.J., Moran R., Stephan K.E., Bestmann S. Dopamine, affordance and active inference. PLoS Comput. Biol. 2012, 8:e1002327. 10.1371/journal.pcbi.1002327.
52. Friston K., Thornton C., Clark A. Free-energy minimization and the dark-room problem. Front. Percept. Sci. 2012, 3:130. 10.3389/fpsyg.2012.00130.
53. Fuster J.M. The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe 1997, Lippincott-Raven, Philadelphia, PA.
54. Gallistel C.R. Extinction from a rationalist perspective. Behav. Process. 2012, 90:66-80.
55. Gershman S.J., Daw N. Perception, action and utility: the tangled skein. Principles of Brain Dynamics: Global State Interactions 2012, MIT Press. M. Rabinovich, M.K. Friston, P. Varona (Eds.).
56. Gershman S.J., Markman A.B., Otto A.R. Retrospective revaluation in sequential decision making: a tale of two systems. J. Exp. Psychol. Gen. 2014, 143:182.
57. Glascher J., Daw N.D., Dayan P., O'Doherty J.P. States vs rewards: dissociable neural prediction error signals underlying mdoel-based and model-free reinforcement learning. Neuron 2010, 66:585-595.
58. Gregory R.L. Perceptions as hypotheses. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 1980, 290:181-197. 10.1098/rstb.1980.0090.
59. Gremel C.M., Costa R.M. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 2013, 4. 10.1038/ncomms3264.
60. Grush R. The emulation theory of representation: motor control, imagery, and perception. Behav. Brain Sci. 2004, 27:377-396.
61. Gu X., Hof P.R., Friston K.J., Fan J. Anterior insular cortex and emotional awareness. J. Comp. Neurol. 2013, 521:3371-3388. 10.1002/cne.23368.
62. Hassabis D., Maguire E.A. The construction system of the brain. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2009, 364:1263-1271. 10.1098/rstb.2008.0296.
63. Helmholtz H.von Concerning the perceptions in general. Treatise on Physiological Optics 1866, Dover, New York. J.P.C. Southall (Ed.).
64. Hesslow G. Conscious thought as simulation of behaviour and perception. Trends Cogn. Sci. 2002, 6:242-247.
65. Hobson J.A., Friston K.J. Waking and dreaming consciousness: neurobiological and functional considerations. Prog. Neurobiol. 2012, 98:82-98. 10.1016/j.pneurobio.2012.05.003.
66. Hoffmann J. Anticipatory behavioral control. Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems 2003, 44-65. Springer-Verlag, Berlin/Heidelberg. M.V. Butz, O. Sigaud, P. Gerard (Eds.).
67. Hommel B., Musseler J., Aschersleben G., Prinz W. The Theory of Event Coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 2001, 24(5):849-878.
68. Imamizu H., Kuroda T., Miyauchi S., Yoshioka T., Kawato M. Modular organization of internal models of tools in the human cerebellum. Proc. Natl. Acad. Sci. U. S. A. 2003, 100:5461-5466. 10.1073/pnas.0835746100.
69. Ito M., Doya K. Distinct neural representation in the dorsolateral, dorsomedial, and ventral parts of the striatum during fixed- and free-choice tasks. J. Neurosci. 2015, 35:3499-3514. 10.1523/JNEUROSCI.1962-14.2015.
70. Jeannerod M. Neural simulation of action: a unifying mechanism for motor cognition. Neuroimage 2001, 14:S103-S109.
71. Johnson A., Redish A.D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 2007, 27:12176-12189. 10.1523/JNEUROSCI.3761-07.2007.
72. Keramati M., Dezfouli A., Piray P. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput. Biol. 2011, 7:e1002055. 10.1371/journal.pcbi.1002055.
73. Kiebel S.J., Friston K.J. Free energy and dendritic self-organization. Front. Syst. Neurosci. 2011, 5:80. 10.3389/fnsys.2011.00080.
74. Koechlin E., Summerfield C. An information theoretical approach to prefrontal executive function. Trends Cogn. Sci. 2007, 11:229-235. 10.1016/j.tics.2007.04.005.
75. Kunde W. Response-effect compatibility in manual choice reaction tasks. J. Exp. Psychol. Hum. Percept. Perform. 2001, 27:387-394.
76. Kunde W. Temporal response-effect compatibility. Psychol. Res. 2003, 67:153-159.
77. Lee S.W., Shimojo S., O'Doherty J.P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 2014, 81:687-699. 10.1016/j.neuron.2013.11.028.
78. Lepora N., Pezzulo G. Embodied choice: how action influences perceptual decision making. PLOS Comput. Biol. 2015, 11:e1004110.
79. Lengyel M., Dayan P. Hippocampal contributions to control: the third way. Advances in Neural Information Processing Systems 20 2008, 889-896. MIT Press, Cambridge, MA. J.C. Platt, D. Koller, Y. Singer, S. Roweis (Eds.).
80. Liljeholm M., O'Doherty J.P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 2012, 16:467-475. 10.1016/j.tics.2012.07.007.
81. Mackintosh N.J. Conditioning and Associative Learning 1983, Oxford University Press, USA.
82. Maisto D., Donnarumma F., Pezzulo G. Divide et Impera: subgoaling reduces the complexity of probabilistic inference and problem solving. J. R. Soc. Interface 2015, 12.
83. Mannella F., Gurney K., Baldassarre G. The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 2013, 7:135. 10.3389/fnbeh.2013.00135.
84. Mathys C., Daunizeau J., Friston K.J., Stephan K.E. A bayesian foundation for individual learning under uncertainty. Front. Hum. Neurosci. 2011, 5:39. 10.3389/fnhum.2011.00039.
85. Mayer E.A. Gut feelings: the emerging biology of gut-brain communication. Nat. Rev. Neurosci. 2011, 12:453-466. 10.1038/nrn3071.
86. Miall R.C., Wolpert D.M. Forward models for physiological motor control. Neural Netw. 1996, 9:1265-1279. 10.1016/S0893-6080(96)00035-4.
87. Miller E.K., Cohen J.D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 2001, 24:167-202. 10.1146/annurev.neuro.24.1.167.
88. Miller G.A., Galanter E., Pribram K.H. Plans and the Structure of Behavior 1960, Holt, Rinehart and Winston, New York.
89. Montague P.R. Why Choose this Book?: How We Make Decisions 2006, Dutton.
90. Montague P.R., Dolan R.J., Friston K.J., Dayan P. Computational psychiatry. Trends Cogn. Sci. 2012, 16:72-80. 10.1016/j.tics.2011.11.018.
91. Niv Y., Daw N.D., Joel D., Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 2007, 191:507-520.
92. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454. 10.1126/science.1094285.
93. O'Doherty J.P., Lee S.W., McNamee D. The structure of reinforcement-learning mechanisms in the human brain. Curr. Opin. Behav. Sci. 2015, 1:94-100. 10.1016/j.cobeha.2014.10.004.
94. O'Regan J.K., Noe A. A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 2001, 24:883-917.
95. Otto A.R., Gershman S.J., Markman A.B., Daw N.D. The curse of planning dissecting multiple reinforcement-learning systems by taxing the central executive. Psychol. Sci. 2013, 10.1177/0956797612463080.
96. Passingham R.E., Wise S.P. The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight 2012, Oxford University Press, USA. 1st ed.
97. Pavlov (1927) I. Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Ann. Neurosci. 2010, 17(3):136-141. (Lecture One).
98. Pennartz C.M.A., Ito R., Verschure P.F.M.J., Battaglia F.P., Robbins T.W. The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci. 2011, 34:548-559. 10.1016/j.tins.2011.08.001.
99. Penny W.D. Comparing dynamic causal models using AIC, BIC and free energy. Neuroimage 2012, 59:319-330. 10.1016/j.neuroimage.2011.07.039.
100. Penny W.D., Zeidman P., Burgess N. Forward and backward inference in spatial cognition. PLoS Comput. Biol. 2013, 9:e1003383. 10.1371/journal.pcbi.1003383.
101. Pezzulo G. Why do you fear the Bogeyman? An embodied predictive coding model of perceptual inference. Cogn. Affect. Behav. Neurosci. 2013.
102. Pezzulo G. An Active Inference view of cognitive control. Front. Theor. Philos. Psychol. 2012, 10.3389/fpsyg.2012.00478.
103. Pezzulo G. Grounding procedural and declarative knowledge in sensorimotor anticipation. Mind Lang. 2011, 26:78-114.
104. Pezzulo G., Barca L., Bocconi A.L., Borghi A.M. When affordances climb into your mind: advantages of motor simulation in a memory task performed by novice and expert rock climbers. Brain Cogn. 2010, 73:68-73.
105. Pezzulo G., Castelfranchi C. Thinking as the control of imagination: a conceptual framework for goal-directed systems. Psychol. Res. 2009, 73:559-577.
106. Pezzulo G., Rigoli F., Chersi F. The mixed instrumental controller: using value of information to combine habitual choice and mental simulation. Front. Cogn. 2013, 4:92. 10.3389/fpsyg.2013.00092.
107. Pezzulo G., van der Meer M.A.A., Lansink C.S., Pennartz C.M.A. Internally generated sequences in learning and executing goal-directed behavior. Trends Cogn. Sci. 2014, 18:647-657. 10.1016/j.tics.2014.06.011.
108. Pezzulo G., Verschure P.F., Balkenius C., Pennartz C.M. The principles of goal-directed decision-making: from neural mechanisms to computation and robotics. Philos. Trans. R. Soc. B: Biol. Sci. 2014, 369:20130470.
109. Pfeiffer B.E., Foster D.J. Hippocampal place-cell sequences depict future paths to remembered goals. Nature 2013, 497:74-79. 10.1038/nature12112.
110. Prinz W. Perception and action planning. Eur. J. Cogn. Psychol. 1997, 9:129-154.
111. Rescorla R.A. Pavlovian conditioning: it's not what you think it is. Am. Psychol. 1988, 151-160.
112. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century Company, New York. A.H. Black, W.F. Prokasy (Eds.).
113. Robbins T.W., Everitt B.J. Drug addiction: bad habits add up. Nature 1999, 398:567-570. 10.1038/19208.
114. Salamone J.D., Correa M. The mysterious motivational functions of mesolimbic dopamine. Neuron 2012, 76:470-485. 10.1016/j.neuron.2012.10.021.
115. Schuetz-Bosbach S., Prinz W. Perceptual resonance: action-induced modulation of perception. Trends Cogn. Sci. 2007, 11:349-355. 10.1016/j.tics.2007.06.005.
116. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
117. Sella G., Hirsh A.E. The application of statistical physics to evolutionary biology. Proc. Natl. Acad. Sci. U. S. A. 2005, 102:9541-9546. 10.1073/pnas.0501865102.
118. Seth A.K. The cybernetic Bayesian brain. Open MIND 2014, MIND Group, Frankfurt am Main. T. Metzinger, J.M. Windt (Eds.).
119. Seth A.K. Interoceptive inference, emotion, and the embodied self. Trends Cogn. Sci. 2013, 17:565-573. 10.1016/j.tics.2013.09.007.
120. Seth A.K., Suzuki K., Critchley H.D. An interoceptive predictive coding model of conscious presence. Front. Psychol. 2012, 2. 10.3389/fpsyg.2011.00395.
121. Shipp S., Adams R.A., Friston K.J. Reflections on agranular architecture: predictive coding in the motor cortex. Trends Neurosci. 2013, 36:706-716. 10.1016/j.tins.2013.09.004.
122. Solway A., Botvinick M.M. Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol. Rev. 2012, 119:120-154. 10.1037/a0026435.
123. Sterling P., Eyer J. Allostasis: a new paradigm to explain arousal pathology. Handbook of Life Stress, Cognition and Health 1988, 629-649. John Wiley & Sons, Oxford, England. S. Fisher, J. Reason (Eds.).
124. Stoianov I., Genovesio A., Pezzulo G. Prefrontal goal-codes emerge as latent states in probabilistic value learning. J. Cognit. Neurosci. 2015, (in press).
125. Strange B.A., Duggins A., Penny W., Dolan R.J., Friston K.J. Information theory, novelty and hippocampal responses: unpredicted or unpredictable?. Neural Netw. 2005, 18:225-230. 10.1016/j.neunet.2004.12.004.
126. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
127. Tani J., Nolfi S. Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems. Neural Netw. 1999, 12:1131-1141.
128. Verschure P.F.M.J., Voegtlin T., Douglas R.J. Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 2003, 425:620-624. 10.1038/nature02024.
129. Verschure P., Pennartz C.M.A., Pezzulo G. The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2014, 369:20130483.
130. Watkins C.J.C.H., Dayan P. Q-learning. Mach. Learn. 1992, 8:279-292.
131. Wolpert D., Miall C., Kawato M. Internal models in the cerebellum. Trends Cogn. Sci. 1998, 2:338-347.
132. Woods S.C., Hutton R.A., Makous W. Conditioned insulin secretion in the albino rat. Exp. Biol. Med. 1970, 133:964-968. 10.3181/00379727-133-34605.
133. Yin H.H., Knowlton B.J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 2006, 7:464-476.
134. Zeki S., Shipp S. The functional logic of cortical connections. Nature 1988.