A Comparison of reinforcement learning models for the Iowa gambling task using parameter space partitioning

Journal of Problem Solving

Volumn 5, Issue 2, 2013, Pages 1-32

  Steingroever, Helen ^a   Wetzels, Ruud ^b   Wagenmakers, Eric Jan ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b European Foundation for the Improvement of Living and Working Conditions   (Ireland)

Author keywords

Decision making; Expectancy valence model; Loss aversion; Prospect valence learning model

Indexed keywords

EID: 84877681288     PISSN: None     EISSN: 19326246     Source Type: Journal    
DOI: 10.7771/1932-6246.1150     Document Type: Article

References (63)

1. Agay, N., Yechiam, E., Carmel, Z., & Levkovitz, Y. (2010). Non-specific effects of methylphenidate (Ritalin) on cognitive ability and decision-making of ADHD and healthy adults. Psychopharmacology, 210(4), 511-19. http://dx.doi.org/10.1007/s00213-010-1853-4
2. Ahn, W.-Y., Busemeyer, J. R., Wagenmakers, E.-J., & Stout, J. C. (2008). Comparison of decision learning models using the generalization criterion method. Cognitive Science, 32(8), 1376-1402. http://dx.doi.org/10.1080/0364021080235299
3. Ahn, W.-Y., Krawitz, A., Kim, W., Busemeyer, J. R., & Brown, J. W. (2011). A model-based fMRI analysis with hierarchical Bayesian parameter estimation. Journal of Neuroscience, Psychology, and Economics, 4(2), 95-110. http://dx.doi.org/10.1037/a0020684
4. Bass, S., & Nussbaum, D. (2010). Decision making and aggression in forensic psychiatric inpatients. Criminal Justice and Behavior, 37(4), 365-83. http://dx.doi. org/10.1177/0093854809360043
5. Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1-3), 7-15.
6. Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997). Deciding advantageously before knowing the advantageous strategy. Science, 275(5304), 1293-95. http://dx.doi. org/10.1126/science.275.5304.1293
7. Bishara, A. J., Pleskac, T. J., Fridberg, D. J., Yechiam, E., Lucas, J., Busemeyer, J. R., et al. (2009). Similar processes despite divergent behavior in two commonly used measures of risky decision making. Journal of Behavioral Decision Making, 22(4), 435-54. http:// dx.doi.org/10.1002/bdm.641
8. Blair, R. J. R., Colledge, E., & Mitchell, D. G. V. (2001). Somatic markers and response reversal: Is there orbitofrontal cortex dysfunction in boys with psychopathic tendencies? Journal of Abnormal Child Psychology, 29(6), 499-511. http://dx.doi. org/10.1023/A:1012277125119
9. Bowman, C. H., Evans, C. E. Y., & Turnbull, O. H. (2005). Artificial time constraints on the Iowa gambling task: The effects on behavioural performance and subjective experience. Brain and Cognition, 57, 21-25. http://dx.doi.org/10.1016/j.bandc.2004.08.015
10. Brambilla, P., Perlini, C., Bellani, M., Tomelleri, L., Ferro, A., Cerruti, S., et al. (2012). Increased salience of gains versus decreased associative learning differentiate bipolar disorder from schizophrenia during incentive decision making. Psychological Medicine, 1, 1-10. http://dx.doi.org/10.1017/S0033291712001304
11. Brown, W., Anderson, L., Symington, M., & Paul, L. (2012). Decision-making in individuals with agenesis of the corpus callosum: Expectancy-Valence in the Iowa gambling task. Archives of Clinical Neuropsychology, 27, 532-44. http://dx.doi.org/10.1093/arclin/acs052
12. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task. Psychological Assessment, 14, 253-62. http://dx.doi.org/10.1037/1040-3590.14.3.253
13. Caroselli, J. S., Hiscock, M., Scheibel, R. S., & Ingram, F. (2006). The simulated gambling paradigm applied to young adults: An examination of university students' performance. Applied Neuropsychology, 13, 203-12. http://dx.doi.org/10.1207/s15324826an1304_1
14. Cavedini, P., Riboldi, G., D'Annucci, A., Belotti, P., Cisima, M., & Bellodi, L. (2002). Decisionmaking heterogeneity in obsessive-compulsive disorder: Ventromedial prefrontal cortex function predicts different treatment outcomes. Neuropsychologia, 40, 205-11. http://dx.doi.org/10.1016/S0028-3932(01)00077-X
15. Cavedini, P., Riboldi, G., Keller, R., D'Annucci, A., & Bellodi, L. (2002). Frontal lobe dysfunction in pathological gambling patients. Biological Psychiatry, 51, 334-41. http://dx.doi. org/10.1016/S0006-3223(01)01227-6
16. Cella, M., Dymond, S., Cooper, A., & Turnbull, O. H. (2012). Cognitive decision modelling of emotion-based learning impairment in schizophrenia: The role of awareness. Psychiatry Research, 196, 15-19. http://dx.doi.org/10.1016/j.psychres.2011.08.015
17. Chiu, Y.-C., & Lin, C.-H. (2007). Is deck C an advantageous deck in the Iowa gambling task? Behavioral and Brain Functions, 3, 1-11. http://dx.doi.org/10.1186/1744-9081-3-37
18. Chiu, Y.-C., Lin, C.-H., Huang, J.-T., Lin, S., Lee, P.-L., & Hsieh, J.-C. (2008). Immediate gain is long-term loss: Are there foresighted decision makers in the Iowa gambling task? Behavioral and Brain Functions, 4, 1-10. http://dx.doi.org/10.1186/1744-9081-4-13
19. de Visser, L., van der Knaap, L., van de Loo, A., van der Weerd, C., Ohl, F., & van den Bos, R. (2010). Trait anxiety affects decision-making differently in healthy men and women: Towards gender-specific endophenotypes of anxiety. Neuropsychologia, 48, 1598-1606. http://dx.doi.org/10.1016/j.neuropsychologia.2010.01.027
20. Dunn, B. D., Dalgleish, T., & Lawrence, A. D. (2006). The somatic marker hypothesis: A critical evaluation. Neuroscience & Biobehavioral Reviews, 30, 239-71. http://dx.doi. org/10.1016/j.neubiorev.2005.07.001
21. Escartin, G., Junqué, C., Juncadella, M., Gabarrós, A., Miquel, M. de, & Rubio, F. (2012). Decision-making impairment on the Iowa gambling task after endovascular coiling or neurosurgical clipping for ruptured anterior communicating artery aneurysm. Neuropsychology, 26, 172-80. http://dx.doi.org/10.1037/a0024336
22. Farah, H., Yechiam, E., Bekhor, S., Toledo, T., & Polus, A. (2008). Association of risk proneness in overtaking maneuvers with impaired decision making. Transportation Research Part F: Traffic Psychology and Behaviour, 11, 313-23. http://dx.doi.org/10.1016/j. trf.2008.01.005
23. Fernie, G., & Tunney, R. J. (2006). Some decks are better than others: The effect of reinforcer type and task instructions on learning in the Iowa gambling task. Brain and Cognition, 60, 94-102.
24. Fridberg, D. J., Queller, S., Ahn, W.-Y., Kim, W., Bishara, A. J., Busemeyer, J. R., et al. (2010). Cognitive mechanisms underlying risky decision-making in chronic cannabis users. Journal of Mathematical Psychology, 54, 28-38. http://dx.doi.org/10.1016/j.jmp.2009.10.002
25. Gullo, M. J., & Stieger, A. A. (2011). Anticipatory stress restores decision-making deficits in heavy drinkers by increasing sensitivity to losses. Drug and Alcohol Dependence, 117, 204-10. http://dx.doi.org/10.1016/j.drugalcdep.2011.02.002
26. Hochman, G., Yechiam, E., & Bechara, A. (2010). Recency gets larger as lesions move from anterior to posterior locations within the ventromedial prefrontal cortex. Behavioural Brain Research, 213, 27-34. http://dx.doi.org/10.1016/j.bbr.2010.04.023
27. Isella, V., Mapelli, C., Morielli, N., Pelati, O., Franceschi, M., & Appollonio, I. (2008). Agerelated quantitative and qualitative changes in decision making ability. Behavioural Neurology, 19, 59-63.
28. Johnson, S. A., Yechiam, E., Murphy, R. R., Queller, S., & Stout, J. C. (2006). Motivational processes and autonomic responsivity in Asperger's disorder: Evidence from the Iowa gambling task. Journal of the International Neuropsychological Society, 12, 668-76. http://dx.doi.org/10.1017/S1355617706060802
29. Kester, H. M., Sevy, S., Yechiam, E., Burdick, K. E., Cervellione, K. L., & Kumra, S. (2006). Decision-making impairments in adolescents with early-onset schizophrenia. Schizophrenia Research, 85, 113-23. http://dx.doi.org/10.1016/j.schres.2006.02.028
30. Kjome, K. L., Lane, S. D., Schmitz, J. M., Green, C., Ma, L., Prasla, I., et al. (2010). Relationship between impulsivity and decision making in cocaine dependence. Psychiatry Research, 178, 299-304. http://dx.doi.org/10.1016/j.psychres.2009.11.024
31. Lane, S. D., Yechiam, E., & Busemeyer, J. R. (2006). Application of a computational decision model to examine acute drug effects on human risk taking. Experimental and Clinical Psychopharmacology, 14, 254-64. http://dx.doi.org/10.1037/1064-1297.14.2.254
32. Lev, D., Hershkovitz, E., & Yechiam, M. (2008). Decision making and personality in traffic offenders: A study of Israeli drivers. Accident Analysis and Prevention, 40, 223-30. http://dx.doi.org/10.1016/j.aap.2007.05.009
33. Lin, C.-H., Chiu, Y.-C., Lee, P.-L., & Hsieh, J.-C. (2007). Is deck B a disadvantageous deck in the Iowa gambling task? Behavioral and Brain Functions, 3, 1-10. http://dx.doi.org/10.1186/1744-9081-3-16
34. Lin, C.-H., Chiu, Y.-C., Lee, P.-L., & Hsieh, J.-C. (2007). Is deck B a disadvantageous deck in the Iowa gambling task? Behavioral and Brain Functions, 3, 16.
35. Lovallo, W. R., Yechiam, E., Sorocco, K. H., Vincent, A. S., & Collins, F. L. (2006). Working memory and decision-making biases in young adults with a family history of alcoholism: Studies from the Oklahoma family health patterns project. Alcoholism: Clinical and Experimental Research, 30, 763-73. http://dx.doi.org/10.1111/j.1530-0277.2006.00089.x
36. Luce, R. (1959). Individual choice behavior. New York: Wiley.
37. MacPherson, S. E., Phillips, L. H., & Della Sala, S. (2002). Age, executive function, and social decision making: A dorsolateral prefrontal theory of cognitive aging. Psychology and Aging, 17, 598-609. http://dx.doi.org/10.1037/0882-7974.17.4.598
38. Martino, D. J., Bucay, D., Butman, J. T., & Allegri, R. F. (2007). Neuropsychological frontal impairments and negative symptoms in schizophrenia. Psychiatry Research, 152, 121-28. http://dx.doi.org/10.1016/j.psychres.2006.03.002
39. Peatfield, N., Parkinson, J., & Intriligator, J. (2012). Emotion-based learning is biased by brand logos. Applied Cognitive Psychology, 26, 694-701. http://dx.doi.org/10.1002/acp.2847
40. Pitt, M., Kim, W., Navarro, D., & Myung, J. (2006). Global model analysis by parameter space partitioning. Psychological Review, 113, 57-83. http://dx.doi.org/10.1037/0033-295X.113.1.57
41. Pitt, M., Myung, J., Montenegro, M., & Pooley, J. (2008). Measuring model flexibility with parameter space partitioning: An introduction and application example. Cognitive Science, 32, 1285-303. http://dx.doi.org/10.1080/03640210802477534
42. Premkumar, P., Fannon, D., Kuipers, E., Simmons, A., Frangou, S., & Kumari, V. (2008). Emotional decision-making and its dissociable components in schizophrenia and schizoaffective disorder: A behavioural and MRI investigation. Neuropsychologia, 46, 2002-12. http://dx.doi.org/10.1016/j.neuropsychologia.2008.01.022
43. Rodríguez-Sánchez, J. M., Crespo-Facorro, B., Iglesias, R. P., Bosch, C. G.-B., Álvarez, M., Llorca, J., & Vázquez-Barquero, J. L. (2005). Prefrontal cognitive functions in stabilized first-episode patients with schizophrenia spectrum disorders: A dissociation between dorsolateral and orbitofrontal functioning. Schizophrenia Research, 77, 279-88.
44. Sevy, S., Burdick, K. E., Visweswaraiah, H., Abdelmessih, S., Lukin, M., Yechiam, E., et al. (2007). Iowa gambling task in schizophrenia: A review and new data in patients with schizophrenia and co-occurring cannabis use disorders. Schizophrenia Research, 92, 74-84. http://dx.doi.org/10.1016/j.schres.2007.01.005
45. Sevy, S., Hassoun, Y., Bechara, A., Yechiam, E., Napolitano, B., Burdick, K., et al. (2006). Emotion-based decision-making in healthy subjects: short-term effects of reducing dopamine levels. Psychopharmacology, 188, 228-35. http://dx.doi.org/10.1007/ s00213-006-0450-z
46. Steingroever, H., Wetzels, R., Horstmann, A., Neumann, J., & Wagenmakers, E.-J. (2013). Performance of healthy participants on the Iowa gambling task. Psychological Assessment, 25, 180-93.
47. Stout, J., Busemeyer, J., Lin, A., Grant, S., & Bonson, K. (2004). Cognitive modeling analysis of decision-making processes in cocaine abusers. Psychonomic Bulletin & Review, 11, 742-47. http://dx.doi.org/10.3758/BF03196629
48. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: The MIT Press.
49. Toplak, M., Jain, U., & Tannock, R. (2005). Executive and motivational processes in adolescents with attention-deficit-hyperactivity disorder (ADHD). Behavioral and Brain Functions, 1, 1-12. http://dx.doi.org/10.1186/1744-9081-1-8
50. Toplak, M., Sorge, G., Benoit, A., West, R., & Stanovich, K. (2010). Decision-making and cognitive abilities: A review of associations between Iowa gambling task performance, executive functions, and intelligence. Clinical Psychology Review, 30, 562-81. http:// dx.doi.org/10.1016/j.cpr.2010.04.002
51. Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5, 297-323. http://dx.doi.org/10.1007/ BF00122574
52. van den Bos, R., Homberg, J., Gijsbers, E., Heijer, E. den, & Cuppen, E. (2009). The effect of COMT V al158 Met genotype on decision-making and preliminary findings on its interaction with the 5-HTTLPR in healthy females. Neuropharmacology, 56, 493-98. http://dx.doi.org/10.1016/j.neuropharm.2008.10.002
53. rd Annual Conference on Neural Information Processing Systems (p. 1919-27). Red Hook, NY: Curran Associates Inc.
54. Wilder, K. E., Weinberger, D. R., & Goldberg, T. E. (1998). Operant conditioning and the orbitofrontal cortex in schizophrenic patients: Unexpected evidence for intact functioning. Schizophrenia Research, 30, 169-74. http://dx.doi.org/10.1016/S0920-9964(97)00135-7
55. Wood, S., Busemeyer, J., Koling, A., Cox, C. R., & Davis, H. (2005). Older adults as adaptive decision makers: Evidence from the Iowa gambling task. Psychology and Aging, 20, 220-25. http://dx.doi.org/10.1037/0882-7974.20.2.220
56. Yechiam, E., Arshavsky, O., Shamay-Tsoory, S. G., Yaniv, S., & Aharon, J. (2010). Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions. Brain and Cognition, 72, 317-24. http://dx.doi.org/10.1016/j.bandc.2009.10.005
57. Yechiam, E., & Busemeyer, J. (2005). Comparison of basic assumptions embedded in learning models for experience-based decision making. Psychonomic Bulletin & Review, 12, 387-402. http://dx.doi.org/10.3758/BF03193783
58. Yechiam, E., & Busemeyer, J. R. (2008). Evaluating generalizability and parameter consistency in learning models. Games and Economic Behavior, 63, 370-94. http://dx.doi. org/10.1016/j.geb.2007.08.011
59. Yechiam, E., Busemeyer, J. R., Stout, J. C., & Bechara, A. (2005). Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychological Science, 16, 973-78. http://dx.doi.org/10.1111/j.1467-9280.2005.01646.x
60. Yechiam, E., & Ert, E. (2007). Evaluating the reliance on past choices in adaptive learning models. Journal of Mathematical Psychology, 51, 75-84. http://dx.doi.org/10.1016/j. jmp.2006.11.002
61. Yechiam, E., Hayden, E. P., Bodkins, M., O'Donnell, B. F., & Hetrick, W. P. (2008). Decision making in bipolar disorder: A cognitive modeling approach. Psychiatry Research, 161, 142-52. http://dx.doi.org/10.1016/j.psychres.2007.07.001
62. Yechiam, E., Kanz, J. E., Bechara, A., Stout, J. C., Busemeyer, J. R., Altmaier, E. M., et al. (2008). Neurocognitive deficits related to poor decision making in people behind bars. Psychonomic Bulletin & Review, 15, 44-51. http://dx.doi.org/10.3758/PBR.15.1.44
63. Zhao, J., & Costello, F. (2007). Computational modelling of switching behaviour in repeated gambles. Artificial Intelligence Review, 27, 209-22. http://dx.doi.org/10.1007/s10462-008-9083-4