Using ε{lunate}-greedy reinforcement learning methods to further understand ventromedial prefrontal patients' deficits on the Iowa Gambling Task

Neural Networks

Volumn 20, Issue 6, 2007, Pages 676-689

  Kalidindi, Kiran ^a   Bowman, Howard ^a  

^a UNIVERSITY OF KENT   (United Kingdom)

Author keywords

Decision making; Iowa Gambling Task; Orbitofrontal cortex; Reinforcement learning; Ventromedial prefrontal cortex

Indexed keywords

DECISION MAKING; MATHEMATICAL MODELS; PATIENT MONITORING;

IOWA GAMBLING TASK; ORBITOFRONTAL CORTEX; VENTROMEDIAL PREFRONTAL CORTEX;

REINFORCEMENT LEARNING;

ARTICLE; BRAIN FUNCTION; BRAIN REGION; DECISION MAKING; HUMAN; MATHEMATICAL MODEL; NERVE CELL NETWORK; NONHUMAN; ORBITAL CORTEX; PATHOLOGICAL GAMBLING; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; TASK PERFORMANCE; THALAMUS VENTRAL NUCLEUS; COGNITIVE DEFECT; EMOTION; GAME; PATHOLOGY; PATHOPHYSIOLOGY; PHYSIOLOGY; PSYCHOLOGICAL MODEL;

COGNITION DISORDERS; DECISION MAKING; EMOTIONS; GAMES, EXPERIMENTAL; HUMANS; MODELS, PSYCHOLOGICAL; PREFRONTAL CORTEX; REINFORCEMENT (PSYCHOLOGY);

EID: 34547768134     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2007.04.026     Document Type: Article

References (59)

1. Arsten A.F.T., and Robbins T.W. Neurochemical modulation of prefrontal cortical function in humans and animals. In: Stuss D.T., and Knight R.T. (Eds). Principles of frontal lobe function (2002), Oxford University Press 51-84
2. Bechara A., Damasio A.R., Damasio H., and Anderson S.W. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50 (1994) 7-15
3. Bechara A., and Damasio H. Decision-making and addiction (part I): Imparied activation of somatic states in substance dependent individuals when pondering decisions with negative consequences. Neuropsychologia 40 (2002) 1675-1689
4. Bechara A., Damasio H., Damasio A.R., and Lee G.D. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. The Journal of Neuroscience 19 13 (1999) 5473-5481
5. Bechara A., Damasio H., Tranel D., and Anderson S.W. Dissociation of working memory from decision making within the human prefrontal cortex. Journal of Neuroscience 18 1 (1998) 428-437
6. Bechara A., Damasio H., Tranel D., and Damasio A.R. Deciding advantageously before knowing the advantageous strategy. Science 275 5304 (1997) 1293-1295
7. Bechara A., Damasio H., Tranel D., and Damasio A.R. The Iowa Gambling Task and the somatic marker hypothesis: Some questions and answers. Trends in Cognitive Science 9 4 (2005) 159-162
8. Bechara A., Dolan S., and Damasio H. Decision-making and addiction (part II): Myopia for future or hypersensitivity to reward?. Neuropsychologia 40 (2002) 1690-1705
9. Bechara A., Tranel D., and Damasio H. Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. Brain 123 (2000) 2189-2202
10. Busemeyer J.R., and Stout J.C. A contribution of cognitive decision models to clinical assessment: Decomposing performance on the bechara gambling task. Psychological Assessment 14 3 (2002) 253-262
11. Clark L., Cools R., and Robbins T.W. The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning. Brain and Cognition 55 (2004) 41-53
12. Clark L., Manes F., Antoun N., Sahakian B.J., and Robbins T.W. The contributions of lesion laterality and lesion volume to decision-making impairment following frontal lobe damage. Neuropsychologia 41 (2003) 1474-1483
13. Damasio A.R. Descartes error: Emotion, reason and the human brain (1994), Grosset/Putnam, New York
14. Daum I., Schugens M.M., Channon S., Polkey C.E., and Gray J.A. T-maze discrimination and reversal learning after unilateral temporal or frontal lobe lesions in man. Cortex 27 4 (1991) 613-622
15. Deco G., and Rolls E.T. Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. Cerebral Cortex 15 (2005) 15-30
16. Dias R., Robbins T.W., and Roberts A.C. Dissociable forms of inhibitory control within prefrontal cortex with an analog of wisconsin card sort test : Restriction to novel situations and independence from on-line processing. The Journal of Neuroscience 17 23 (1997) 9285-9297
17. Doya K. Metalearning and neuromodulation. Neural Networks 15 (2002) 495-506
18. Dunn B.D., Dalgleish T., and Lawrence A. The somatic marker hypothesis: A critical evaluation. Neuroscience and Biobehavioural Reviews 30 2 (2006) 239-271
19. Fellows L.K., and Farah M.J. Ventromedial frontal cortex mediates affective shifting in humans: Evidence from a reversal learning paradigm. Brain 126 (2003) 1830-1837
20. Fellows L.K., and Farah M.J. Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cerebral Cortex 15 1 (2005) 58-63
21. Frank M.J., and Claus E.D. Anatomy of a decision: Striato-orbitofrontal interactions in reinforcement learning, decision making and reversal. Psychological Review 113 (2006) 300-326
22. Goldman-Rakic P.S., and Leung H. Functional architecture of the dorsolateral prefrontal cortex in monkeys and humans. Principles of frontal lobe function (2002), Oxford University Press 85-95
23. Hinson J.M., Jameson T.L., and Whitney P. Somatic markers, working memory, and decision making. Cognitive, Affective, and Behavioral Neuroscience 2 4 (2002) 341-353
24. Hornak J., O'Doherty J., Bramham J., Rolls E.T., Morris R.G., Bullock P.R., et al. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience 16 3 (2004) 463-478
25. Jacobs A.M., and Grainger J. Models of visual word recognition - sampling the state of the art. Journal of Experimental Psychology: Human Perception and Performance 29 (1994) 1311-1334
26. Kalidindi, K., Bowman, H., & Wyble, B. (2005). Using ε{lunate}-greedy reinforcement methods to further understand ventromedial prefrontal patients' deficits on the Iowa gambling task. Technical report 12-05. Computing Laboratory, University of Kent. http://www.cs.kent.ac.uk/people/rpg/kk49/
27. Knight R.T., and Scabini D. Anatomic bases of event-related potentials and their relationship to novelty detection in humans. Journal of Clinical Neurophysiology 15 (1998) 3-13
28. Knight R.T., and Stuss D. Prefrontal cortex: The present and the future. In: Stuss D.T., and Knight R.T. (Eds). Principles of frontal lobe function (2002), Oxford University Press 573-597
29. Lawrence A.D., Sahakian B.J., Rogers R.D., Hodges J.R., and Robbins T.W. Discrimination, reversal, and shift learning in huntington's disease: Mechanisms of impaired response selection. Neuropsychologia 37 (1999) 1359-1374
30. Lhermitte F. 'Utilization behaviour' and its relation to lesions of the frontal lobes. Brain 106 (1983) 237-255
31. Lhermitte F. Human autonomy and the frontal lobes. Part II: Patient behaviour in complex and social situations: The "environmental dependency syndrome". Annals of Neurology 19 4 (1986) 335-343
32. Maia T.V., and McClelland J.L. A reexamination of the evidence for the somatic marker hypothesis: What participants really know in Iowa Gambling Task. Proceedings of the National Academy of Sciences 101 (2004) 16075-16080
33. Manes F., Sahakian B.J., Clark L., Rogers R.D., Antoun N., Aitken M., et al. Decision-making processes following damage to the prefrontal cortex. Brain 125 (2002) 624-639
34. Mavaddat N., Kirkpatrick P., Rogers R.D., and Sahakin B.J. Deficits in decision-making with aneurysms of the anterior communicating artery. Brain 123 10 (2000) 2109-2117
35. Mesulam M. The human frontal lobes: Transcending the default mode through contingent encoding. Principles of frontal lobe function (2002), Oxford University Press 8-30
36. Mobini S., Body S., Ho M.Y., Bradshaw C.M., Szabadi E., Deakin J.F., et al. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probablisitic reinforcement. Psychopharmacology 160 (2002) 290-298
37. O'Reilly R., and Munakata Y. Computational explorations in cognitive neuroscience (2000), MIT Press
38. Peters E., and Slovic P. The springs of action: Affective and analytical information processing in choice. Personality and Social Psychology Bulletin 26 12 (2000) 1465-1475
39. Pitt M.A., Myung I.J., and Zhang S. Toward a method of selecting among computational models of cognition. Psychological Review 109 3 (2002) 472-491
40. Roesch M., and Schoenbaum G. From associations to expectancies: Orbitofrontal cortex as gateway between the limbic system and representational memory. The orbitofrontal cortex (2006), Oxford University Press 199-236
41. Rogers R.D., Everitt B.J., Baldacchino A., Blackshaw A.J., Swainson R., Wynne K., et al. Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: Evidence for monoaminergic mechanisms. Neuopsychopharmachology 20 (1999) 322-339
42. Rolls E.T. The orbitofrontal cortex and reward. Cerebral Cortex 10 (2000) 284-294
43. Rolls E.T. The function of the orbitofrontal cortex. Brain and Cognition 55 (2004) 11-29
44. Rolls E.T. Emotion explained (2005), Oxford University Press
45. Rolls E.T., Hornak J., Wade D., and McGrath J. Emotion related learning in patients with social and emotional changes associated with frontal lobe damage. Journal of Neurology, Neurosurgery and Psychiatry 57 (1994) 1518-1524
46. Rule R.R., Shimamura A.P., and Knight R.T. Orbitofrontal cortex and dynamic filtering of emotional stimuli. Cognitive, Affective and Behavioural Neuroscience 2 3 (2002) 264-270
47. Sacks O. The man who mistook his wife for a hat. Touchstone (1998)
48. Sanfey A.G., Hastie R., Colvin M.K., and Grafman J. Phineas gauged: Decision-making and the frontal lobes. Neuropsychologia 41 (2003) 1218-1229
49. Shallice T., Burgess P.W., Schon F., and Baxter D.M. The origins of utiliztion behaviour. Brain 112 (1989) 1587-1598
50. Shurman B., Horan W.P., and Nuechterlein K.H. Schizophrenia patients demonstrate a distinctive pattern of decision-making impairment in the Iowa Gambling Task. Schizophrenia Research 72 (2005) 215-224
51. Sloman, A. (2004). What are emotion theories about? In Invited talk at cross-disciplinary workshop on architectures for modeling emotion at the AAAI spring symposium at Stanford University in March 2004. See: http://www.cs.bham.ac.uk/research/cogaff/sloman-aaai04-emotions.pdf
52. Stone V.E., Baron-Cohen S., and Knight R.T. Frontal lobe contributions to theory of mind. Journal of Cognitive Neuroscience 10 (1998) 640-656
53. Stout J.C., Rodawalt W.C., and Siemers E.R. Risky decision making in huntington's diesease. Journal of the International Neuropsychological Society 7 (2001) 92-101
54. Sutton R.S., and Barto A.G. A bradford book. Reinforcement learning: An introduction (1998), The MIT Press
55. Tranel D., Bechara A., and Denburg N.L. Asymmetric functional roles of right and left ventromedial prefrontal corticies in social conduction, decision-making, and emotional processing. Cortex 38 (2002) 589-612
56. Tremblay L., and Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. The Journal of Neurophysiology 83 4 (2000) 1877-1885
57. Wagar B.M., and Thagard P. Spiking phineas gage: A neurocomputational theory of cognitive affective integration in decision making. Psychological Review 111 1 (2004) 67-79
58. Yechiam E., Busemeyer J.R., Stout J.C., and Bechara A. Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychological Science 16 12 (2005) 973-978
59. Zald D.H. Neuropsychological assessment of the orbitofrontal cortex. The orbitofrontal cortex (2006), Oxford University Press 449-480