Action selection in multi-effector decision making

NeuroImage

Volumn 70, Issue , 2013, Pages 66-79

  Madlon Kay, Seth ^a   Pesaran, Bijan ^b   Daw, Nathaniel D ^b  

^a DUKE UNIVERSITY   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

Bimanual; Decision making; Effector specificity; Value maps; VmPFC

Indexed keywords

ACTION SELECTION; ADULT; ANTERIOR MID CINGULATE CORTEX; ARTICLE; BOLD SIGNAL; BRAIN FUNCTION; BRAIN REGION; CINGULATE GYRUS; CONTROLLED STUDY; DECISION MAKING; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HAND; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; MEDIAL MOTOR CORTEX; MULTIEFFECTOR DECISION MAKING; NERVE CELL NETWORK; NORMAL HUMAN; PREFRONTAL CORTEX; PRIORITY JOURNAL; REWARD; VENTROMEDIAL PREFRONTAL CORTEX;

EID: 84872382730     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2012.12.001     Document Type: Article

References (110)

1. Alexander G.E., DeLong M.R., Strick P.L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 1986, 9:357-381.
2. Andersen R.A., Buneo C.A. Intentional maps in posterior parietal cortex. Annu. Rev. Neurosci. 2002, 25:189-220.
3. Batista A.P., Buneo C.A., Snyder L.H., Andersen R.A. Reach plans in eye-centered coordinates. Science 1999, 285(5425):257-260.
4. Beckmann M., Johansen-Berg H., Rushworth M.F.S. Connectivity-based parcellation of human cingulate cortex and its relation to functional specialization. J. Neurosci. 2009, 29(4):1175-1190.
5. Berns G.S., McClure S.M., Pagnoni G., Montague P.R. Predictability modulates human brain response to reward. J. Neurosci. 2001, 21(8):2793-2798.
6. Blanke O., Spinelli L., Thut G., Michel C.M., Perrig S., Landis T., Seeck M. Location of the human frontal eye field as defined by electrical cortical stimulation: anatomical, functional and electrophysiological characteristics. Neuroreport 2000, 11(9):1907-1913.
7. Breiter H.C., Gollub R.L., Weisskoff R.M., Kennedy D.N., Makris N., Berke J.D., Goodman J.M., Kantor H.L., Gastfriend D.R., Riorden J.P., Mathew R.T., Rosen B.R., Hyman S.E. Acute effects of cocaine on human brain activity and emotion. Neuron 1997, 19(3):591-611. (Sep).
8. Brinkman C. Supplementary motor area of the monkey's cerebral cortex: short-and long-term deficits after unilateral ablation and the effects of subsequent callosal section. J. Neurosci. 1984, 4(4):918-929.
9. Büchel C., Bornhovd K., Quante M., Glauche V., Bromm B., Weiller C. Dissociable neural responses related to pain intensity, stimulus intensity, and stimulus awareness within the anterior cingulate cortex: a parametric single-trial laser functional magnetic resonance imaging study. J. Neurosci. 2002, 22(3):970-976.
10. Calton J.L., Dickinson A.R., Snyder L.H. Non-spatial, motor-specific activation in posterior parietal cortex. Nat. Neurosci. 2002, 5(6):580-588.
11. Camille N., Griffiths C.A., Vo K., Fellows L.K., Kable J.W. Ventromedial frontal lobe damage disrupts value maximization in humans. J. Neurosci. 2011, 31(20):7527-7532.
12. Daw N.D. Trial-by-trial data analysis using computational models. Affect, Learning and Decision Making, Attention and Performance XXIII 2011, Oxford University Press, Oxford. M.R. Delgado, E.A. Phelps, T.W. Robbins (Eds.).
13. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8(12):1704-1711.
14. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441(7095):876-879.
15. Deichmann R., Gottfried J., Hutton C., Turner R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage 2003, 19(2):430-441.
16. Dickinson A. Actions and habits: The development of behavioural autonomy. Philos. Trans. R. Soc. Lond. B Biol. Sci. 1985, 308(1135):67-78.
17. Dickinson A., Balleine B. The role of learning in the operation of motivational systems. Stevens Handbook of Experimental Psychology: Learning, Motivation and Emotion 2002, 497-534. John Wiley & Sons, New York. 3rd edition. C.R. Gallistel (Ed.).
18. Diedrichsen J., Grafton S., Albert N., Hazeltine E., Ivry R.B. Goal-selection and movement-related conflict during bimanual reaching movements. Cereb. Cortex 2006, 16(12):1729-1738.
19. Donchin O., Gribova A., Steinberg O., Mitz A.R., Bergman H., Vaadia E. Single-unit activity related to bimanual arm movements in the primary and supplementary motor cortices. J. Neurophysiol. 2002, 88(6):3498-3517.
20. Frank M.J., Scheres A., Sherman S.J. Understanding decision-making deficits in neurological conditions: insights from models of natural action selection. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362(1485):1641-1654.
21. Franz E.A., Waldie K.E., Smith M.J. The effect of callosotomy on novel versus familiar bimanual actions: a neural dissociation between controlled and automatic processes?. Psychol. Sci. 2000, 11(1):82-86.
22. Friston K.J., Worsley K.J., Frackowiak R.S.J., Mazziotta J.C., Evans A.C. Assessing the significance of focal activations using their spatial extent. Hum. Brain Mapp. 1993, 1(3):210-220.
23. Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 1997, 6(3):218-229.
24. Gershman S.J., Pesaran B., Daw N.D. Human reinforcement learning subdivides structured action spaces by learning effector-specific values. J. Neurosci. 2009, 29(43):13524-13531.
25. Gläscher J., Hampton A.N., O'Doherty J.P. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb. Cortex 2009, 19(2):483-495.
26. Glimcher P.W. Choice: towards a standard back-pocket model. Neuroeconomics: Decision Making and the Brain 2008, 503-521. Academic Press, San Diego, CA, (Ch. 31). P.W. Glimcher, C.F. Camerer, E. Fehr, R.A. Poldrack (Eds.).
27. Grootoonk S., Hutton C., Ashburner J., Howseman A.M., Josephs O., Rees G., Friston K.J., Turner R. Characterization and correction of interpolation effects in the realignment of fMRI time series. Neuroimage 2000, 11(1):49-57.
28. Hare T.A., Camerer C.F., Rangel A. Self-control in decision-making involves modulation of the vmPFC valuation system. Science 2009, 324(5927):646-648.
29. Hare T.A., Schultz W., Camerer C.F., O'Doherty J.P., Rangel A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. U.S.A. 2011, 108(44):18120-18125.
30. Harvey A.H., Kirk U., Denfield G.H., Montague P.R. Monetary favors and their influence on neural responses and revealed preference. J. Neurosci. 2010, 30(28):9597-9602. (Jul).
31. Hayden B.Y., Platt M.L. Neurons in anterior cingulate cortex multiplex information about reward and action. J. Neurosci. 2010, 30(9):3339-3346.
32. Hayden B.Y., Pearson J.M., Platt M.L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 2011, 14(7):933-939.
33. Henson R. What can functional neuroimaging tell the experimental psychologist?. Q. J. Exp. Psychol. 2005, 58A(2):193-233.
34. Inati S., Velasco P., Vessel E. A new method for robust functional to structural alignment using multi-echo GRE B0 and R2* mapping. Poster at Hum. Brain Mapp. 2011, (Quebec City (Canada)).
35. Jäncke L., Peters M., Himmelbach M., Nösselt T., Shah J., Steinmetz H. fMRI study of bimanual coordination. Neuropsychologia 2000, 38(2):164-174.
36. Jenkins I., Brooks D., Nixon P., Frackowiak R., Passingham R. Motor sequence learning: a study with positron emission tomography. J. Neurosci. 1994, 14(6):3775-3790.
37. Kable J.W., Glimcher P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 2007, 10(12):1625-1633.
38. Karni A., Meyer G., Jezzard P., Adams M.M., Turner R., Ungerleider L.G. Functional MRI evidence for adult motor cortex plasticity during motor skill learning. Nature 1995, 377(6545):155-158.
39. Kelso J.A., Southard D.L., Goodman D. On the coordination of two-handed movements. J. Exp. Psychol. Hum. Percept. Perform. 1979, 5(2):229-238.
40. Kennerley S.W., Wallis J.D. Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J. Neurophysiol. 2009, 102(6):3352-3364.
41. Kennerley S.W., Diedrichsen J., Hazeltine E., Semjen A., Ivry R.B. Callosotomy patients exhibit temporal uncoupling during continuous bimanual movements. Nat. Neurosci. 2002, 5(4):376-381.
42. Kennerley S.W., Walton M.E., Behrens T.E.J., Buckley M.J., Rushworth M.F.S. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 2006, 9(7):940-947.
43. Knutson B., Westdorp A., Kaiser E., Hommer D. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 2000, 12(1):20-27.
44. Kolling N., Behrens T.E.J., Mars R.B., Rushworth M.F.S. Neural mechanisms of foraging. Science 2012, 336(6077):95-98.
45. Laplane D., Talairach J., Meininger V., Bancaud J., Orgogozo J. Clinical consequences of corticectomies involving the supplementary motor area in man. J. Neurol. Sci. 1977, 34(3):301-314.
46. Lau B., Glimcher P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 2005, 84(3):555-579.
47. Levy D.J., Glimcher P.W. Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. J. Neurosci. 2011, 31(41):14693-14707.
48. Levy I., Schluppeck D., Heeger D.J., Glimcher P.W. Specificity of human cortical areas for reaches and saccades. J. Neurosci. 2007, 27(17):4687-4696.
49. Leyton A.S.F., Sherrington C.S. Observations on the excitable cortex of the chimpanzee, orangutan, and gorilla. Q. J. Exp. Physiol. 1917, 11(2):135-222.
50. Luk C.-H., Wallis J.D. Dynamic encoding of responses and outcomes by neurons in medial prefrontal cortex. J. Neurosci. 2009, 29(23):7526-7539.
51. Maldjian J.A., Laurienti P.J., Kraft R.A., Burdette J.H. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. Neuroimage 2003, 19(3):1233-1239.
52. Martin L.N., Delgado M.R. The influence of emotion regulation on decision-making under risk. J. Cogn. Neurosci. 2011, 23(9):2569-2581.
53. Matsumoto K., Suzuki W., Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 2003, 301(5630):229-232.
54. Mazaika P., Hoeft F., Glover G.H., Reiss A.L. Methods and software for fMRI analysis for clinical subjects. Poster at Hum. Brain Mapp. 2009.
55. McClure S.M., Laibson D.I., Loewenstein G., Cohen J.D. Separate neural systems value immediate and delayed monetary rewards. Science 2004, 306(5695):503-507.
56. McDannald M.a, Lucantonio F., Burke K.a, Niv Y., Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 2011, 31(7):2700-2705.
57. Mechsner F., Kerzel D., Knoblich G., Prinz W. Perceptual basis of bimanual coordination. Nature 2001, 414(6859):69-73.
58. Nichols T., Brett M., Andersson J., Wager T., Poline J.-B. Valid conjunction inference with the minimum statistic. Neuroimage 2005, 25(3):653-660.
59. Nieuwenhuis S., Forstmann B.U., Wagenmakers E.-J. Erroneous analyses of interactions in neuroscience: a problem of significance. Nat. Neurosci. 2011, 14(9):1105-1107.
60. Noonan M.P., Walton M.E., Behrens T.E.J., Sallet J., Buckley M.J., Rushworth M.F.S. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl. Acad. Sci. U.S.A. 2010, 107(47):20547-20552.
61. O'Doherty J.P., Deichmann R., Critchley H.D., Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron 2002, 33(5):815-826.
62. O'Reilly R.C., Frank M.J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 2006, 18(2):283-328.
63. O'Reilly J.X., Woolrich M.W., Behrens T.E.J., Smith S.M., Johansen-Berg H. Tools of the trade: psychophysiological interactions and functional connectivity. Soc. Cogn. Affect. Neurosci. 2012, 7(5):604-609. (Jun).
64. Oliveira F.T.P., Ivry R.B. The representation of action: insights from bimanual coordination. Curr. Dir. Psychol. Sci. 2008, 17(2):130-135.
65. Padoa-Schioppa C., Assad J.a. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441(7090):223-226.
66. Palminteri S., Boraud T., Lafargue G., Dubois B., Pessiglione M. Brain hemispheres selectively track the expected value of contralateral options. J. Neurosci. 2009, 29(43):13465-13472.
67. Penfield W., Jasper H. Epilepsy and the Functional Anatomy of the Human Brain 1954, Little Brown, Boston.
68. Penny W.D., Henson R. Analysis of Variance. Statistical Parametric Mapping: The Analysis of Functional Brain Images 2006, 166-177. Academic Press, London, (Ch. 13). K.J. Friston, J.T. Ashburner, S.J. Kiebel, T.E. Nichols, W.D. Penny (Eds.).
69. Pesaran B., Nelson M.J., Andersen R.A. Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning. Neuron 2006, 51(1):125-134.
70. Peters J., Büchel C. Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. J. Neurosci. 2009, 29(50):15727-15734.
71. Phelps E.A., Delgado M.R., Nearing K.I., LeDoux J.E. Extinction learning in humans: role of the amygdala and vmPFC. Neuron 2004, 43(6):897-905.
72. Picard N., Strick P.L. Imaging the premotor areas. Curr. Opin. Neurobiol. 2001, 11(6):663-672.
73. Plassmann H., O'Doherty J., Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 2007, 27(37):9984-9988.
74. Plassmann H., O'Doherty J., Shiv B., Rangel A. Marketing actions can modulate neural representations of experienced pleasantness. Proc. Natl. Acad. Sci. U.S.A. 2008, 105(3):1050-1054. (Jan).
75. Platt M.L., Glimcher P.W. Neural correlates of decision variables in parietal cortex. Nature 1999, 400(6741):233-238.
76. Platt M., Padoa-Schioppa C. Neuronal representations of value. Neuroeconomics: Decision Making and the Brain 2008, 441-462. Academic Press, San Diego, CA, (Ch. 29). P.W. Glimcher, C.F. Camerer, E. Fehr, R.A. Poldrack (Eds.).
77. Poldrack R.A., Clark J., Paré-Blagoev E.J., Shohamy D., Creso Moyano J., Myers C., Gluck M.A. Interactive memory systems in the human brain. Nature 2001, 414(6863):546-550.
78. Poldrack R.A., Fletcher P.C., Henson R.N., Worsley K.J., Brett M., Nichols T.E. Guidelines for reporting an fMRI study. Neuroimage 2008, 40(2):409-414.
79. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century-Crofts. A.H. Black, W.F. Prokasy (Eds.).
80. Rosenbaum D.A., Dawson A.M., Challis J.H. Haptic tracking permits bimanual independence. J. Exp. Psychol. Hum. Percept. Perform. 2006, 32(5):1266-1275.
81. Rudebeck P.H., Murray E.A. Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 2011, 31(29):10569-10578.
82. Rudebeck P.H., Behrens T.E., Kennerley S.W., Baxter M.G., Buckley M.J., Walton M.E., Rushworth M.F.S. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 2008, 28(51):13775-13785.
83. Sadato N., Yonekura Y., Waki A., Yamada H., Ishii Y. Role of the supplementary motor area and the right premotor cortex in the coordination of bimanual finger movements. J. Neurosci. 1997, 17(24):9667-9674.
84. Scherberger H., Andersen R.a. Target selection signals for arm reaching in the posterior parietal cortex. J. Neurosci. 2007, 27(8):2001-2012.
85. Schlag J., Schlag-Rey M. Evidence for a supplementary eye field. J. Neurophysiol. 1987, 57(1):179-200.
86. Schönberg T., Daw N.D., Joel D., O'Doherty J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 2007, 27(47):12860-12867.
87. Schwarz G. Estimating the dimension of a model. Ann. Stat. 1978, 6(2):461-464.
88. Seo H., Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 2007, 27(31):8366-8377.
89. Shackman A.J., Salomons T.V., Slagter H.a., Fox A.S., Winter J.J., Davidson R.J. The integration of negative affect, pain and cognitive control in the cingulate cortex. Nat. Rev. Neurosci. 2011, 12(3):154-167.
90. Shadlen M.N., Newsome W.T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 2001, 86(4):1916-1936.
91. Shadmehr R. Neural correlates of motor memory consolidation. Science 1997, 277(5327):821-825.
92. Simon D.A., Daw N.D. Environmental statistics and the trade-off between model-based and TD learning in humans. Advances in Neural Information Processing Systems 2011, 24:127-135. J. Shawe-Taylor, R.S. Zemel, P. Bartlett, F.C.N. Pereira, K.Q. Weinberger (Eds.).
93. Simon D.A., Daw N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 2011, 31(14):5526-5539.
94. Snyder L., Batista A., Andersen R. Coding of intention in the posterior parietal cortex. Nature 1997, 386(6621):167-170.
95. Sperry R.W. Cerebral organization and behavior: the split brain behaves in many respects like two separate brains, providing new research possibilities. Science 1961, 133(3466):1749-1757.
96. Stephan K.E., Penny W.D., Daunizeau J., Moran R.J., Friston K.J. Bayesian model selection for group studies. Neuroimage 2009, 46(4):1004-1017.
97. Sugrue L., Corrado G. Matching behavior and the representation of value in the parietal cortex. Science 2004, 304(June):1782-1787.
98. Swinnen S., Wenderoth N. Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn. Sci. 2004, 8(1):18-25.
99. Tanji J., Okano K., Sato K.C. Neuronal activity in cortical motor areas related to ipsilateral, contralateral, and bilateral digit movements of the monkey. J. Neurophysiol. 1988, 60(1):325-343.
100. Tom S.M., Fox C.R., Trepel C., Poldrack R.A. The neural basis of loss aversion in decision-making under risk. Science 2007, 315(5811):515-518.
101. Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Mazoyer B., Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 2002, 15(1):273-289.
102. Venkatraman V., Rosati A.G., Taren A.A., Huettel S.A. Resolving response, decision, and strategic control: evidence for a functional topography in dorsomedial prefrontal cortex. J. Neurosci. 2009, 29(42):13158-13164.
103. Wallis J.D. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 2012, 15(1):13-19.
104. Wallis J.D., Miller E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 2003, 18(7):2069-2081.
105. Walton M.E., Behrens T.E.J., Buckley M.J., Rudebeck P.H., Rushworth M.F.S. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 2010, 65(6):927-939.
106. Wimmer G.E., Daw N.D., Shohamy D. Generalization of value in reinforcement learning by humans. Eur. J. Neurosci. 2012, 35(7):1092-1104.
107. Wunderlich K., Rangel A., O'Doherty J.P. Neural computations underlying action-based decision making in the human brain. Proc. Natl. Acad. Sci. U.S.A. 2009, 106(40):17199-17204.
108. Wunderlich K., Rangel A., O'Doherty J.P. Economic choices can be made using only stimulus values. Proc. Natl. Acad. Sci. U.S.A. 2010, 107(34):15005-15010.
109. Xue G., Ghahremani D.G., Poldrack R.A. Neural substrates for reversing stimulus-outcome and stimulus-response associations. J. Neurosci. 2008, 28(44):11196-11204.
110. Yu C., Zhou Y., Liu Y., Jiang T., Dong H., Zhang Y., Walter M. Functional segregation of the human cingulate cortex is confirmed by functional connectivity based neuroanatomical parcellation. Neuroimage 2011, 54(4):2571-2581.