The principles of goal-directed decision-making: From neural mechanisms to computation and robotics

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 369, Issue 1655, 2014, Pages

  Pezzulo, Giovanni ^a   Verschure, Paul F M J ^b   Balkenius, Christian ^c   Pennartz, Cyriel ^d,e  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

^b INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

^c LUND UNIVERSITY   (Sweden)

^d UNIVERSITY OF AMSTERDAM   (Netherlands)

^e UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Computational model; Decision making; Goal directed; Neural mechanism; Prediction

Indexed keywords

COMPUTER SIMULATION; DECISION MAKING; MATHEMATICAL ANALYSIS; NUMERICAL MODEL; PREDICTION; ROBOTICS;

DECISION MAKING; HUMAN; MOTIVATION; NEUROSCIENCE; ROBOTICS;

DECISION MAKING; GOALS; HUMANS; NEUROSCIENCES; ROBOTICS;

EID: 84907494705     PISSN: 09628436     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rstb.2013.0470     Document Type: Article

References (39)

1. Passingham RE, Wise ESP. 2012 The neurobiology of the prefrontal cortex: anatomy, evolution, and the origin of insight, 1st edn. New York, NY: Oxford University Press.
2. Pennartz CMA, Ito R, Verschure PFMJ, Battaglia FP, Robbins ETW. 2011 The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci. 34, 548–559. (doi:10.1016/j.tins. 2011.08.001)
3. Pezzulo G, Castelfranchi EC. 2009 Thinking as the control of imagination: a conceptual framework for goal-directed systems. Psychol. Res. 73, 559–577. (doi:10.1007/s00426-009-0237-z)
4. Verschure PFM, Althaus EP. 2003 A real-world rational agent: unifying old and new AI. Cogn. Sci. 27, 561–590. (doi:10.1207/s15516709cog2704_1)
5. Pezzulo G. 2011 Grounding procedural and declarative knowledge in sensorimotor anticipation. Mind Lang. 26, 78–114. (doi:10.1111/j.1468-0017. 2010.01411.x)
6. O’Reilly RC, Hazy TE, Mollick J, Mackie P, Herd ES. 2014 Goal-driven cognition in the brain: a computational framework. (http://arxiv.org/abs/14047591Q-Bio).
7. Pezzulo G. 2014 Goals reconfigure cognition by modulating predictive processes in the brain. Behav. Brain Sci. 37, 154–155. (doi:10.1017/S014052 5X13002148)
8. Adams GK, Watson KK, Pearson J, Platt EML. 2012 Neuroethology of decision-making. Curr. Opin. Neurobiol. 22, 982–989. (doi:10.1016/j.conb.2012. 07.009)
9. Cisek P, Kalaska EJF. 2010 Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 33, 269–298. (doi:10.1146/annurev. neuro.051508.135409)
10. Pezzulo G, Barsalou LW, Cangelosi A, Fischer MH, McRae K, Spivey EMJ. 2013 Computational grounded cognition: a new alliance between grounded cognition and computational modeling. Front. Psychol. 3, 612. (doi:10.3389/fpsyg. 2012.00612)
11. Pezzulo G, Cangelosi BLWA, Fischer MH, McRae K, Spivey EM. 2011 The mechanics of embodiment: a dialogue on embodiment and computational modeling. Front. Cogn. 2, 1–21.
12. Balkenius C. 1995 Natural intelligence in artificial creatures, vol. 37. Lund, Sweden: Lund University.
13. Buschman TJ, Miller EK. 2014 Goal-direction and top-down control. Phil. Trans. R. Soc. B 369, 20130471. (doi:10.1098/rstb.2013.0471)
14. Koechlin E. 2014 An evolutionary computational theory of prefrontal executive function in decision-making. Phil. Trans. R. Soc. B 369, 20130474. (doi:10.1098/rstb.2013.0474)
15. Genovesio A, Ferraina S. 2014 The influence of recent decisions on future goal selection. Phil. Trans. R. Soc. B 369, 20130477. (doi:10.1098/rstb.2013.0477)
16. Lisman J. 2014 Two-phase model of the basal ganglia: implications for discontinuous control of the motor system. Phil. Trans. R. Soc. B 369, 20130489. (doi:10.1098/rstb.2013.0489)
17. Stott JJ, Redish AD. 2014 A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour. Phil. Trans. R. Soc. B 369, 20130472. (doi:10.1098/rstb.2013.0472)
18. Larsen T, O’Doherty JP. 2014 Uncovering the spatio-temporal dynamics of value-based decision-making in the human brain: a combined fMRI–EEG study. Phil. Trans. R. Soc. B 369, 20130473. (doi:10.1098/ rstb.2013.0473)
19. Botvinick M, Niv Y, Barto EA. 2008 Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113, 262–280. (doi:10.1016/j.cognition.2008.08.011)
20. Daw ND, Niv Y, Dayan EP. 2005 Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711. (doi:10.1038/nn1560)
21. Friston K. 2010 The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127–138. (doi:10.1038/nrn2787)
22. Pezzulo G, Rigoli F, Chersi EF. 2013 The mixed instrumental controller: using value of information to combine habitual choice and mental simulation. Front. Cogn. 4, 92.
23. Sutton RS, Barto AG. 1998 Reinforcement learning: an introduction. Cambridge, MA: MIT Press.
24. Pezzulo G, van der Meer MAA, Lansink CS, Pennartz ECMA. In press. Internally generated sequences in learning and executing goal-directed behavior. Trends Cogn. Sci.
25. Schultz W, Dayan P, Montague EPR. 1997 A neural substrate of prediction and reward. Science 275, 1593–1599. (doi:10.1126/science.275.5306.1593)
26. Daw ND, Dayan P. 2014 The algorithmic anatomy of model-based evaluation. Phil. Trans. R. Soc. B 369, 20130478. (doi:10.1098/rstb.2013.0478)
27. Friston K, Schwartenbeck P, FitzGerald T, Moutoussis M, Behrens T, Dolan RJ. 2014 The anatomy of choice: dopamine and decision-making. Phil. Trans. R. Soc. B 369, 20130481. (doi:10.1098/rstb. 2013.0481)
28. Botvinick M, Weinstein A. 2014 Model-based hierarchical reinforcement learning and human action control. Phil. Trans. R. Soc. B 369, 20130480. (doi:10.1098/rstb.2013.0480)
29. Kachergis G, Wyatte D, O’Reilly RC, de Kleijn R, Hommel B. 2014 A continuous-time neural model for sequential action. Phil. Trans. R. Soc. B 369, 20130623. (doi:10.1098/rstb.2013.0623)
30. O’Reilly RC, Munakata EY. 2000 Computational explorations in cognitive neuroscience: understanding the mind by simulating the brain. Cambridge, MA: MIT Press.
31. Hommel B, Musseler J, Aschersleben G, Prinz EW. 2001 The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–878. (doi:10.1017/S0140525X01000103)
32. Dezfouli A, Lingawi NW, Balleine BW. 2014 Habits as action sequences: hierarchical action control and changes in outcome value. Phil. Trans. R. Soc. B 369, 20130482. (doi:10.1098/rstb.2013.0482)
33. Misyak JB, Chater N. 2014 Virtual bargaining: a theory of social decision-making. Phil. Trans. R. Soc. B 369, 20130487. (doi:10.1098/rstb.2013. 0487)
34. Cisek P, Pastor-Bernier A. 2014 On the challenges and mechanisms of embodied decisions. Phil. Trans. R. Soc. B 369, 20130479. (doi:10.1098/rstb. 2013.0479)
35. Verschure PFMJ, Pennartz CMA, Pezzulo G. 2014 The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Phil. Trans. R. Soc. B 369, 20130483. (doi:10.1098/rstb. 2013.0483)
36. Verschure PFMJ, Voegtlin T, Douglas ERJ. 2003 Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425, 620–624. (doi:10.1038/nature02024)
37. Milford M, Schulz R. 2014 Principles of goal-directed spatial robot navigation in biomimetic models. Phil. Trans. R. Soc. B 369, 20130484. (doi:10.1098/rstb.2013.0484)
38. Gillan CM, Robbins TW. 2014 Goal-directed learning and obsessive–compulsive disorder. Phil.Trans. R. Soc. B 369, 20130475. (doi:10.1098/rstb. 2013.0475)
39. Osvath M, Martin-Ordas G. 2014 The future of future-oriented cognition in non-humans: theory and the empirical case of the great apes. Phil. Trans. R. Soc. B 369, 20130486. (doi:10.1098/rstb. 2013.0486)