Psychological models of human and optimal performance in bandit problems

Cognitive Systems Research

Volumn 12, Issue 2, 2011, Pages 164-174

  Lee, Michael D ^a   Zhang, Shunan ^a   Munro, Miles ^a   Steyvers, Mark ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bandit problem; Exploration versus exploitation; Heuristic models; Latent state models; Reinforcement learning

Indexed keywords

BANDIT PROBLEMS; BENCHMARK MODELS; DECISION MAKERS; EXPLORATION AND EXPLOITATION; EXPLORATION VERSUS EXPLOITATION; HEURISTIC MODELS; HUMAN DATA; LATENT STATE MODELS; OPTIMAL DECISIONS; OPTIMAL PERFORMANCE; PSYCHOLOGICAL MODEL;

DECISION MAKING; OPTIMIZATION;

REINFORCEMENT LEARNING;

ARTICLE; CONTROLLED STUDY; DECISION MAKING; EXPLORATORY BEHAVIOR; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; MENTAL PERFORMANCE; MODEL; NORMAL HUMAN; PRIORITY JOURNAL; PROBLEM SOLVING; REINFORCEMENT;

EID: 79952189388     PISSN: 13890417     EISSN: 13890417     Source Type: Journal    
DOI: 10.1016/j.cogsys.2010.07.007     Document Type: Article

References (19)

1. Batchelder W.H. An all-or-none theory for learning on both the paired-associate and concept levels. Journal of Mathematical Psychology 1970, 7:97-117.
2. Bernado J.M., Smith A.F.M. Bayesian theory 2000, Wiley, Chichester, UK.
3. Berry D.A., Fristedt B. Bandit problems: Sequential allocation of experiments 1985, Chapman & Hall, London.
4. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? Exploration versus exploitation. Philosophical Transactions of the Royal Society B: Biological Sciences 2007, 362:933-942.
5. Congdon P. Bayesian statistical modeling 2006, Wiley, Chichester, UK.
6. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
7. Gelman A., Carlin J.B., Stern H.S., Rubin D.B. Bayesian data analysis 2004, Chapman & Hall, Boca Raton, FL. second ed.
8. Gittins J.C. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B 1979, 41:148-177.
9. Griffiths T.L., Steyvers M., Blei D.M., Tenenbaum J.B. Integrating topics and syntax. Advances in Neural Information Processing Systems 2005, 17.
10. Kaebling L.P., Littman M.L., Moore A.W. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 1996, 4:237-285.
11. Lee M.D. Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review 2008, 15(1):1-15.
12. Lunn D.J., Thomas A., Best N., Spiegelhalter D. WinBUGS - a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing 2000, 10:325-337.
13. Macready W.G., Wolpert D.H. Bandit problems and the exploration/exploitation tradeoff. IEEE Transactions on evolutionary computation 1998, 2(1):2-22.
14. Pitt M.A., Myung I.J., Zhang S. Toward a method of selecting among computational models of cognition. Psychological Review 2002, 109:472-491.
15. Shiffrin R.M., Lee M.D., Kim W.-J., Wagenmakers E.-J. A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science 2008, 32(8):1248-1284.
16. Steyvers M., Lee M.D., Wagenmakers E.-J. A Bayesian analysis of human decision-making on bandit problems. Journal of Mathematical Psychology 2009, 53:168-179.
17. Sutton R.S., Barto A.G. Reinforcement learning: An introduction 1998, The MIT Press, Cambridge, MA.
18. Vandekerckhove J., Tuerlinckx F. Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review 2007, 14:1011-1026.
19. Zhang, S., Lee, M. D., & Munro, M. N. (2009). Human and optimal exploration and exploitation in bandit problem. In Proceedings of the 9th International conference on cognitive modeling (ICCM), Manchester, UK.