Model-based analyses: Promises, pitfalls, and example applications to the study of cognitive control

Quarterly Journal of Experimental Psychology

Volumn 65, Issue 2, 2012, Pages 252-267

  Mars, Rogier B ^a   Shea, Nicholas J ^b   Kolling, Nils ^a   Rushworth, Matthew F S ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Action regulation; Cognitive control; Model; Model based approach; Reinforcement learning

Indexed keywords

EID: 84861117877     PISSN: 17470218     EISSN: 17470226     Source Type: Journal    
DOI: 10.1080/17470211003668272     Document Type: Article

References (61)

1. Akaike, H. 1973. "Information theory as an extension of the maximum likelihood principle". In Second international symposium on information theory, Edited by: Petrov, B. N. and Csaki, F. 267-281. Budapest, Hungary: Akademiai Kiado.
2. Behrens, T. E. J., Hunt, L. T., Woolrich, M. W. and Rushworth, M. F. S. 2008. Associative learning of social value. Nature, 456: 245-250.
3. Behrens, T. E. J., Woolrich, M. W., Walton, M. E. and Rushworth, M. F. S. 2007. Learning the value of information in an uncertain world. Nature Neuroscience, 10: 1214-1221.
4. Bestmann, S., Harrison, L. M., Blankenburg, F., Mars, R. B., Haggard, P.Friston, K. J. 2008. Influences of contextual uncertainty and surprise on human corticospinal excitability during preparation for action. Current Biology, 18: 775-780.
5. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S. and Cohen, J. D. 2001. Conflict monitoring and cognitive control. Psychological Review, 108: 624-652.
6. Brown, S. D. and Heathcote, A. J. 2008. The simplest complete model of choice reaction time: Linear ballistic accumulation. Cognitive Psychology, 57: 153-178.
7. Brown, J. W., Reynolds, J. R. and Braver, T. S. 2007. A computational model of fractionated conflict-control mechanisms in task-switching. Cognitive Psychology, 55: 37-85.
8. Burnham, K. P. and Anderson, D. R. 2002. Model selection and multi-model inference. A practical information-theoretic approach, 2, New York: Springer.
9. Burnham, K. P. and Anderson, D. R. 2004. Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods Research, 33: 261-304.
10. Cohen, J. D., Dunbar, K. and McClelland, J. L. 1990. On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97: 332-361.
11. Cohen, M. X. 2007. Individual differences and the neural representations of reward expectation and reward prediction errors. Social Cognitive and Affective Neuroscience, 2: 20-30.
12. Cohen, M. X. 2008. Neurocomputational mechanisms of reinforcement-guided learning in humans: A review. Cognitive, Affective, and Behavioral Neuroscience, 8: 113-125.
13. Cohen, M. X. and Ranganath, C. 2007. Reinforcement learning signals predict future decisions. Journal of Neuroscience, 27: 371-378.
14. Corrado, G. and Doya, K. 2007. Understanding neural coding through the model-based analysis of decision making. Journal of Neuroscience, 27: 8178-8180.
15. Corrado, G. S., Sugrue, L. P., Brown, J. R. and Newsome, W. T. 2009. "The trouble with choice: Studying decision variables in the brain". In Neuroeconomics: Decision making and the brain, Edited by: Glimcher, P. W., Camerer, C. F., Fehr, E. and Poldrack, R. A. 463-480. Amsterdam: Elsevier.
16. Critchley, H. D., Mathias, C. J., Josephs, O., O'Doherty, J., Zanini, S.Dewar, B. K. 2003. Human cingulate cortex and autonomic control: Converging neuroimaging and clinical evidence. Brain, 126: 2139-2152.
17. D'Ardenne, K., McClure, S. M., Nystrom, L. E. and Cohen, J. D. 2008. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science, 319: 1264-1267.
18. Dehaene, S., Posner, M. I. and Tucker, D. M. 1994. Localization of a neural system for error detection and compensation. Psychological Science, 5: 303-305.
19. Den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R. and Stephan, K. E. 2009. A dual role for prediction error in associative learning. Cerebral Cortex, 19: 1175-1185.
20. Dolan, R. J. 2008. Neuroimaging of cognition: Past, present, and future. Neuron, 60: 496-502.
21. Donchin, E. and Coles, M. G. H. 1988. Is the P300 component a manifestation of context updating?. Behavioral and Brain Science, 11: 357-374.
22. Duncan, J. and Owen, A. M. 2000. Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neurosciences, 23: 475-483.
23. Duncan-Johnson, C. and Donchin, E. 1977. On quantifying surprise: The variation of event-related brain potentials with subjective probability. Psychophysiology, 14: 456-467.
24. Forstmann, B. U., Dutilh, G., Brown, S., Neumann, J., Von Cramon, D. Y.Ridderinkhof, K. R. 2008. Striatum and pre-SMA facilitate decision-making under time pressure. Proceedings of the National Academy of Sciences USA, 105: 17538-17542.
25. Friston, K. 2003. Learning and inference in the brain. Neural Networks, 16: 1325-1352.
26. Gilbert, S. J. and Shallice, T. 2002. Task switching: A PDP model. Cognitive Psychology, 44: 297-337.
27. Godfrey-Smith, P. 2006. The strategy of model-based science. Biology and Philosophy, 21: 725-740.
28. Gold, J. I. and Shadlen, M. N. 2007. The neural basis of decision making. Annual Reviews of Neuroscience, 30: 535-574.
29. Hampton, A. N., Bossaerts, P. and O'Doherty, J. P. 2006. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. Journal of Neuroscience, 26: 8360-8367.
30. Haynes, J. D., Sakai, K., Rees, G., Gilbert, S., Frith, C. and Passingham, R. E. 2007. Reading hidden intentions in the human brain. Current Biology, 17: 323-328.
31. Hester, R., Barre, N., Murphy, K., Silk, T. J. and Mattingley, J. B. 2008. Human medial frontal cortex activity predicts learning from errors. Cerebral Cortex, 18: 1933-1940.
32. Holroyd, C. B. and Coles, M. G. H. 2002. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109: 679-709.
33. Holroyd, C. B., Nieuwenhuis, S., Mars, R. B. and Coles, M. G. H. 2004. "Anterior cingulate cortex, selection for action, and error processing". In Cognitive neuroscience of attention, Edited by: Posner, M. I. 219-231. New York: Guilford Press.
34. Kass, R. E. and Raftery, A. E. 1993. Bayes factors and model uncertainty, University of Washington, Seattle, WA. (Tech. Rep. No. 254)
35. Kiebel, S. J., Garrido, M. I., Moran, R., Chen, C. C. and Friston, K. J. 2009. Dynamic causal modelling for EEG and MEG. Human Brain Mapping, 20: 1866-1876.
36. Lau, B. and Glimcher, P. W. 2005. Dynamic response-by-response models of matching behaviour in rhesus monkeys. Journal of the Experimental Analysis of Behavior, 84: 555-579.
37. Logothetis, N. K., Pauls, J., Auguth, M., Trinath, T. and Oeltermann, A. 2001. Neurophysiological investigation of the basis of the fMRI signal. Nature, 412: 150-152.
38. MacKay, D. J. C. 1992. Bayesian interpolation. Neural Computation, 4: 415-447.
39. Marr, D. 1982. Vision. A computational investigation into the human representation and processing of visual information, New York: W.H. Freeman and Company.
40. Mars, R. B., Coles, M. G. H., Grol, M. J., Holroyd, C. B., Nieuwenhuis, S.Hulstijn, W. 2005. Neural dynamics of error processing in human medial frontal cortex. NeuroImage, 28: 1007-1013.
41. Mars, R. B., Debener, S., Gladwin, T. E., Harrison, L. M., Haggard, P.Rothwell, J. C. 2008. Trial-by-trial fluctuations in the event-related electroencephalogram reflect dynamic changes in the degree of surprise. Journal of Neuroscience, 28: 12539-12545.
42. Miller, E. K. 2000. The prefrontal cortex and cognitive control. Nature Reviews Neuroscience, 1: 59-65.
43. Monsell, S. 2003. Task switching. Trends in Cognitive Sciences, 7: 134-140.
44. Nieuwenhuis, S., Aston-Jones, G. and Cohen, J. D. 2005. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychological Bulletin, 131: 510-532.
45. Nieuwenhuis, S., Ridderinkhof, K. R., Talsma, D., Coles, M. G. H., Holroyd, C. B.Kok, A. 2002. A computational account of altered error processing in older age: Dopamine and the error-related negativity. Cognitive, Affective, and Behavioral Neuroscience, 2: 19-36.
46. O'Doherty, J. P., Hampton, A. and Kim, H. 2007. Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104: 35-53.
47. Rescorla, R. A. and Wagner, A. R. 1972. "A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement". In Classical conditioning. II: Current research and theory, Edited by: Black, A. H. and Prokasy, W. F. 64-99. New York: Appleton-Century-Crofts.
48. Ridderinkhof, K. R., Ullsperger, M., Crone, E. A. and Nieuwenhuis, S. 2004. The role of the medial frontal cortex in cognitive control. Science, 306: 443-447.
49. Roesch, M. R., Calu, D. J. and Schoenbaum, G. 2007. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience, 10: 1615-1624.
50. Rosa, M. J., Bestmann, S., Harrison, L. and Penny, W. 2010. Bayesian model selection maps for group studies. NeuroImage, 49: 217-224.
51. Rushworth, M. F. S. and Behrens, T. E. J. 2008. Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11: 389-397.
52. Rushworth, M. F. S., Mars, R. B. and Summerfield, C. 2009. General mechanisms for making decisions?. Current Opinion in Neurobiology, 19: 75-83.
53. Schultz, W., Dayan, P. and Montague, P. R. 1997. A neural substrate of prediction and reward. Science, 275: 1593-1599.
54. Shannon, C. E. 1948. A mathematical theory of communication. Bell Systems Technical Journal, 27: 379-423.
55. Sterret, S. G. 2002. Physical models and fundamental laws: Using one piece of the world to tell about another. Mind & Society, 5: 51-66.
56. Sutton, R. S. and Barto, A. G. 1981. Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88: 135-170.
57. Sutton, R. S. and Barto, A. G. 1998. Reinforcement learning: An introduction, Cambridge, MA: MIT Press.
58. Van Veen, V. and Carter, C. S. 2002. The timing of action-related processes in the anterior cingulate cortex. Journal of Cognitive Neuroscience, 14: 593-602.
59. Walton, M. E. and Mars, R. B. 2007. Probing human and monkey anterior cingulate cortex in variable environments. Cognitive, Affective, and Behavioral Neuroscience, 7: 413-422.
60. Weisberg, M. 2007. Who is a modeler?. British Journal for Philosophy of Science, 58: 207-233.
61. Yu, A. and Dayan, P. 2005. Uncertainty, neuromodulation, and attention. Neuron, 46: 681-692.