Trend detection via temporal difference model predicts inferior prefrontal cortex activation during acquisition of advantageous action selection

NeuroImage

Volumn 21, Issue 2, 2004, Pages 733-743

  Paulus, Martin P ^a,b,c   Feinstein, Justin S ^a   Tapert, Susan F ^b,c   Liu, Thomas T ^c,d  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c VA SAN DIEGO HEALTHCARE SYSTEM   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Functional magnetic resonance imaging; Inferior prefrontal cortex; Temporal difference model

Indexed keywords

ADULT; ARTICLE; BRAIN DEPTH STIMULATION; BRAIN FUNCTION; CONTROLLED STUDY; FEMALE; FRONTAL LOBE; GLOBUS PALLIDUS; HEALTH STATUS; HEMODYNAMICS; HUMAN; HUMAN EXPERIMENT; HYPOTHESIS; MALE; MODEL; NORMAL HUMAN; NUCLEAR MAGNETIC RESONANCE IMAGING; PREDICTION; PREFRONTAL CORTEX; PRIORITY JOURNAL; RIGHT HANDEDNESS; STATISTICAL SIGNIFICANCE; TASK PERFORMANCE; TEMPORAL LOBE; VOLUNTEER;

EID: 1242338920     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2003.09.060     Document Type: Article

References (73)

1. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders (4th Edition): DSM-IV. 1994;The American Psychiatric Association, Washington.
2. Baxter M.G., Murray E.A. The amygdala and reward. Nat. Rev. Neurosci. 3:2002;563-573.
3. Bechara A., Damasio H. Decision-making and addiction (Part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia. 40:2002;1675-1689.
4. Bechara A., Damasio H., Tranel D., Damasio A.R. Deciding advantageously before knowing the advantageous strategy \[see comments]. Science. 275:1997;1293-1295.
5. Bechara A., Dolan S., Denburg N., Hindes A., Anderson S.W., Nathan P.E. Decision-making deficits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia. 39:2001;376-389.
6. Boynton G.M., Engel S.A., Glover G.H., Heeger D.J. Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16:1996;4207-4221.
7. Breiter H.C., Gollub R.L., Weisskoff R.M., Kennedy D.N., Makris N., Berke J.D., Goodman J.M., Kantor H.L., Gastfriend D.R., Riorden J.P., Mathew R.T., Rosen B.R., Hyman S.E. Acute effects of cocaine on human brain activity and emotion. Neuron. 19:1997;591-611.
8. Bush G., Vogt B.A., Holmes J., Dale A.M., Greve D., Jenike M.A., Rosen B.R. Dorsal anterior cingulate cortex: a role in reward-based decision making. Proc. Natl. Acad. Sci. U. S. A. 99:2002;523-528.
9. Byrd R.H.L.P. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16:1995;1190-1208.
10. Childress A.R., Mozley P.D., McElgin W., Fitzgerald J., Reivich M., O'Brien C.P. Limbic activation during cue-induced cocaine craving. Am. J. Psychiatry. 156:1999;11-18.
11. Cools R., Clark L., Owen A.M., Robbins T.W. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci. 22:2002;4563-4567.
12. Cox R.W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29:1996;162-173.
13. Craig A.D. A new view of pain as a homeostatic emotion. Trends Neurosci. 26:2003;303-307.
14. Critchley H.D., Mathias C.J., Dolan R.J. Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron. 29:2001;537-545.
15. Damasio A.R. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos. Trans. R. Soc. Lond., B Biol. Sci. 351:1996;1413-1420.
16. Damasio A.R., Damasio H., Christen Y. Neurobiology of Decision-Making. 1996;Springer, Berlin.
17. Daw N.D., Kakade S., Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw. 15:2002;603-616.
18. Dayan P., Hinton G.E., Neal R.M., Zemel R.S. The Helmholtz machine. Neural Comput. 7:1995;889-904.
19. Egelman D.M., Person C., Montague P.R. A computational role for dopamine delivery in human decision-making. J. Cogn. Neurosci. 10:1998;623-630.
20. Elliott R., Dolan R.J. Activation of different anterior cingulate foci in association with hypothesis testing and response selection. NeuroImage. 8:1998;17-29.
21. Elliott R., Rees G., Dolan R.J. Ventromedial prefrontal cortex mediates guessing. Neuropsychologia. 37:1999;403-411.
22. Elliott R., Dolan R.J., Frith C.D. Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. Cereb. Cortex. 10:2000;308-317.
23. Elliott R., Friston K.J., Dolan R.J. Dissociable neural responses in human reward systems. J. Neurosci. 20:2000;6159-6165.
24. Forman S.D., Cohen J.D., Fitzgerald M., Eddy W.F., Mintun M.A., Noll D.C. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 33:1995;636-647.
25. Frith C.D., Friston K., Liddle P.F., Frackowiak R.S. Willed action and the prefrontal cortex in man: a study with PET. Proc. R. Soc. Lond., B. Biol. Sci. 244:1991;241-246.
26. Gallagher H.L., Jack A.I., Roepstorff A., Frith C.D. Imaging the intentional stance in a competitive game. NeuroImage. 16:2002;814-821.
27. Gold J.I., Shadlen M.N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5:2001;10-16.
28. Gold J.I., Shadlen M.N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron. 36:2002;299-308.
29. Grant S., London E.D., Newlin D.B., Villemagne V.L., Liu X., Contoreggi C., Phillips R.L., Kimes A.S., Margolin A. Activation of memory circuits during cue-elicited cocaine craving. Proc. Natl. Acad. Sci. U. S. A. 93:1996;12040-12045.
30. Grant S., Contoreggi C., London E.D. Drug abusers show impaired performance in a laboratory test of decision making. Neuropsychologia. 38:2000;1180-1187.
31. Huettel S.A., Mack P.B., McCarthy G. Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex. Nat. Neurosci. 5:2002;485-490.
32. Ihaka R., Gentleman R.R. A language for data analysis and graphics. J. Comput. Graph. Stat. 5(3):1996;299-314. (GENERIC) Ref Type: Journal (Full).
33. Jueptner M., Stephan K.M., Frith C.D., Brooks D.J., Frackowiak R.S., Passingham R.E. Anatomy of motor learning: I. Frontal cortex and attention to action. J. Neurophysiol. 77:1997;1313-1324.
34. Kahn I., Yeshurun Y., Rotshtein P., Fried I., Ben Bashat D., Hendler T. The role of the amygdala in signaling prospective outcome of choice. Neuron. 33:2002;983-994.
35. Knutson B., Adams C.M., Fong G.W., Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21:2001;159-164.
36. Knutson B., Fong G.W., Bennett S.M., Adams C.M., Hommer D. A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. NeuroImage. 18:2003;263-272.
37. London E.D., Ernst M., Grant S., Bonson K., Weinstein A. Orbitofrontal cortex and human drug abuse: functional imaging. Cereb. Cortex. 10:2000;334-342.
38. McClure S.M., Berns G.S., Montague P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 38:2003;339-346.
39. Mogg K., Mathews A., Eysenck M., May J. Biased cognitive operations in anxiety: artefact, processing priorities or attentional search? Behav. Res. Ther. 29:1991;459-467.
40. Montague P., Berns G. Neural economics and the biological substrates of valuation. Neuron. 36:2002;265.
41. O'Doherty J., Kringelbach M.L., Hornak J., Andrews C., Rolls E.T. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4:2001;95-102.
42. O'Doherty J.P., Critchley H.D., Deichmann R., Dolan R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23:2003;7931-7939.
43. O'Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron. 38:2003;329-337.
44. Pagnoni G., Zink C.F., Montague P.R., Berns G.S. Activity in human ventral striatum locked to errors of reward prediction. Nat. Neurosci. 5:2002;97-98.
45. Passingham R.E., Toni I., Rushworth M.F. Specialisation within the prefrontal cortex: the ventral prefrontal cortex and associative learning. Exp. Brain Res. 133:2000;103-113.
46. Paulus M.P., Hozack N., Zauscher B., McDowell J.E., Frank L., Brown G.G., Braff D.L. Prefrontal, parietal, and temporal cortex networks underlie decision-making in the presence of uncertainty. NeuroImage. 13:2001;91-100.
47. Paulus M.P., Hozack N.E., Zauscher B.E., Frank L., Brown G.G., Braff D.L., Schuckit M.A. Behavioral and functional neuroimaging evidence for prefrontal dysfunction in methamphetamine-dependent subjects. Neuropsychopharmacology. 26:2002;53-63.
48. Pearce J.M., Bouton M.E. Theories of associative learning in animals. Annu. Rev. Psychol. 52:2001;111-139.
49. Platt M.L., Glimcher P.W. Neural correlates of decision variables in parietal cortex. Nature. 400:1999;233-238.
50. Pochon J.B., Levy R., Fossati P., Lehericy S., Poline J.B., Pillon B., Le Bihan D., Dubois B. The neural system that bridges reward and cognition in humans: an fMRI study. Proc. Natl. Acad. Sci. U. S. A. 99:2002;5669-5674.
51. Pouget A., Dayan P., Zemel R.S. Inference and computation with population codes. Annu. Rev. Neurosci. 26:2003;381-410.
52. Rahman S., Sahakian B.J., Hodges J.R., Rogers R.D., Robbins T.W. Specific cognitive deficits in mild frontal variant frontotemporal dementia. Brain. 122:1999;1469-1493.
53. Rahman S., Sahakia J., Cardinal N., Rogers R., Robbins T. Decision making and neuropsychiatry. Trends Cogn. Sci. 5:2001;271-277.
54. Real L.A. Animal choice behavior and the evolution of cognitive architecture. Science. 253:1991;980-986.
55. Robbins T.W., Everitt B.J., Ryan C.N., Marston H.M., Jones G.H., Page K.J. Comparative effects of quisqualic and ibotenic acid-induced lesions of the substantia innominata and globus pallidus on the acquisition of a conditional visual discrimination: differential effects on cholinergic mechanisms. Neuroscience. 28:1989;337-352.
56. Rode C., Cosmides L., Hell W., Tooby J. When and why do people avoid unknown probabilities in decisions under uncertainty? Testing some predictions from optimal foraging theory. Cognition. 72:1999;269-304.
57. Rogers R.D., Everitt B.J., Baldacchino A., Blackshaw A.J., Swainson R., Wynne K., Baker N.B., Hunter J., Carthy T., Booker E., London M., Deakin J.F., Sahakian B.J., Robbins T.W. Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: evidence for monoaminergic mechanisms. Neuropsychopharmacology. 20:1999;322-339.
58. Salinas E., Abbott L.F. A model of multiplicative neural responses in parietal cortex. Proc. Natl. Acad. Sci. U. S. A. 93:1996;11956-11961.
59. Schmitt M. On the complexity of computing and learning with multiplicative neural networks. Neural Comput. 14:2002;241-301.
60. Schultz W. Getting formal with dopamine and reward. Neuron. 36:2002;241-263.
61. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23:2000;473-500.
62. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science. 275:1997;1593-1599.
63. Shadlen M.N., Newsome W.T. Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. U. S. A. 93:1996;628-633.
64. Shadlen M.N., Newsome W.T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86:2001;1916-1936.
65. Suri R.E., Schultz W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience. 91:1999;871-890.
66. Suri R.E., Schultz W. Temporal difference model reproduces anticipatory neural activity. Neural Comput. 13:2001;841-862.
67. Tapert S.F., Cheung E.H., Brown G.G., Frank L.R., Paulus M.P., Schweinsburg A.D., Meloy M.J., Brown S.A. Neural response to alcohol stimuli in adolescents with alcohol use disorder. Arch. Gen. Psychiatry. 60:2003;727-735.
68. Tversky A., Kahneman D. The framing of decisions and the psychology of choice. Science. 211:1981;453-458.
69. Volkow N.D., Fowler J.S. Addiction, a disease of compulsion and drive: involvement of the orbitofrontal cortex. Cereb. Cortex. 10:2000;318-325.
70. Wang G.J., Volkow N.D., Fowler J.S., Cervany P., Hitzemann R.J., Pappas N.R., Wong C.T., Felder C. Regional brain metabolic activation during craving elicited by recall of previous drug experiences. Life Sci. 64:1999;775-784.
71. Ward, B.D., Deconvolution analysis of FMRI time series data. 5-1-2002. Medical College of Wisconsin. AFNI 3dDeconvolve Documentation. (GENERIC) Ref Type: Report.
72. Wasserman E.A., Miller R.R. What's elementary about associative learning? Annu. Rev. Psychol. 48:1997;573-607.
73. Zysset S., Huber O., Ferstl E., von Cramon D.Y. The anterior frontomedian cortex and evaluative judgment: an fMRI study. NeuroImage. 15:2002;983-991.