Structure learning in human sequential decision-making

PLoS Computational Biology

Volumn 6, Issue 12, 2010, Pages

  Acuña, Daniel E ^a   Schrater, Paul ^a,b  

^a University of Minnesota   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; REINFORCEMENT LEARNING;

BAYESIAN REINFORCEMENT LEARNING; DECISION TASK; GENERATIVE MODEL; HUMAN FACES; LEARNING AGENTS; LEARNING PROBLEM; SEQUENTIAL DECISION MAKING; SEQUENTIAL DECISIONS; STRUCTURE-LEARNING; SUB-OPTIMAL PERFORMANCE;

DECISION MAKING;

ARTICLE; BAYESIAN LEARNING; BEHAVIOR CHANGE; CONTROLLED STUDY; DECISION MAKING; EXPERIENTIAL LEARNING; EXPLORATORY BEHAVIOR; HUMAN; HUMAN EXPERIMENT; LEARNING ALGORITHM; LEARNING ENVIRONMENT; LEARNING STYLE; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MENTAL TASK; QUALITATIVE ANALYSIS; QUANTITATIVE ANALYSIS; REINFORCEMENT; REWARD; TASK PERFORMANCE; ALGORITHM; BAYES THEOREM; LEARNING; PHYSIOLOGY; THEORETICAL MODEL;

ALGORITHMS; BAYES THEOREM; DECISION MAKING; HUMANS; LEARNING; MODELS, THEORETICAL; REWARD; TASK PERFORMANCE AND ANALYSIS;

EID: 78651226963     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1001003     Document Type: Article

References (45)

1. Bellman RE (1956) A problem in the sequential design of experiments. Sankhyā 16: 221-229.
2. Gittins JC (1989) Multi-armed bandit allocation indices. Chichester [West Sussex]; New York: Wiley.
3. Whittle P (1988) Restless bandits: activity allocation in a changing world. J Appl Probab 25: 287-298.
4. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature 441: 876-879.
5. Yi MS, Steyvers M, Lee M (2009) Modeling human performance in restless bandits with particle filters. The Journal of Problem Solving 2: Available: http://docs.lib.purdue.edu/jps/vol2/iss2/5/.
6. Yu AJ, Cohen JD (2009) Sequential effects: Superstition or rational behavior? In: Advances in Neural Information Processing Systems, 21. Cambridge, MA: MIT Press. pp 1873-1880.
7. Sakai Y, Fukai T (2008) When does reward maximization lead to matching law? PLoS One 3: e3795.
8. Sakai Y, Fukai T (2008) The actor-critic learning is behind the matching law: Matching vs. optimal behaviors. Neural Comput 20: 227-251.
9. Kaelbling L, Littman M, Cassandra A (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101: 99-134.
10. Dearden R, Friedman N, Russell S (1998) Bayesian Q-learning. In: Fifteenth National Conf. on Artificial Intelligence (AAAI). pp 761-768.
11. Strens MJA (2000) A bayesian framework for reinforcement learning. In: Proceedings of the Seventeenth International Conference on Machine Learning Morgan Kaufmann Publishers Inc. pp 943-950.
12. Poupart P, Vlassis N, Hoey J, Regan K (2006) An analytic solution to discrete bayesian reinforcement learning. In: 23rd International Conference on Machine Learning. Pittsburgh, Penn. pp 697-704.
13. Tenenbaum JB, Griffiths TL (2001) Structure learning in human causal induction. In: Advances in Neural Information Processing Systems 13. Cambridge, MA: MIT Press. pp 59-65.
14. Heckerman D, Geiger D, Chickering DM (1995) Learning bayesian networks: The combination of knowledge and statistical data. Mach Learn 20: 197-243.
15. Neapolitan RE (2004) Learning Bayesian networks. Upper Saddle River, NJ: Pearson Prentice Hall.
16. Tenenbaum JB, Griffiths TL, Kemp C (2006) Theory-based bayesian models of inductive learning and reasoning. Trends Cogn Sci 10: 309-318.
17. Bellman RE (1957) Dynamic programming. Princeton: Princeton University Press.
18. Gittins JC, Jones DM (1974) A dynamic allocation index for the sequential design of experiments. In: Gani J, Sarkadi K, Vincze I, eds. Progress in statistics. Amsterdam: North-Holland Pub. Co. pp 241-266.
19. Watkins C, Dayan P (1992) Technical note: Q-learning. Mach Learn 8: 279-292.
20. Sutton RS, Barto AG (1998) Reinforcement learning: An introduction MIT Press.
21. Schultz W, Dayan P, Montague P (1997) A neural substrate of prediction and reward. Science 275: 1593-1599.
22. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1-27.
23. Gelman A, Carlin JB, Stern HS, Rubin DB (2003) Bayesian Data Analysis Chapman & Hall/CRC.
24. Meyer RJ, Shi Y (1995) Sequential choice under ambiguity: Intuitive solutions to the armed-bandit problem. Manage Sci 41: 817-834.
25. Banks J, Olson M, Porter D (1997) An experimental analysis of the bandit problem. Econ Theory 10: 55-77.
26. Anderson C (2001) Behavioral Models of Strategies in Multi-Armed Bandit Problems. Ph.D. thesis, California Institute of Technology, Pasadena, CA.
27. Gans N, Knox G, Croson R (2007) Simple models of discrete choice and their performance in bandit experiments. Manuf Serv Oper Manag 9: 383-408.
28. Edwards W (1956) Reward probability, amount, and information as determiners of sequential two-alternative decisions. J Exp Psychol 52: 177-88.
29. Edwards W (1961) Probability learning in 1000 trials. J Exp Psychol 62: 385-394.
30. Brackbill Y, Bravos A (1962) Supplementary report: The utility of correctly predicting infrequent events. J Exp Psychol 64: 648-649.
31. Horowitz AD (1973) Experimental Study of the Two-Armed Bandit Problem. Ph.D. Dissertation. Chapel Hill, NC: University of North Carolina, Chapel Hill.
32. Gershman SJ, Niv Y (2010) Learning latent structure: carving nature at its joints. Curr Opin Neurobiol 20: 251-256.
33. Courville AC, Daw ND, Gordon GJ, Touretzky DS (2004) Model uncertainty in classical conditioning. In: Advances in Neural Information Processing Systems 16. Cambridge, MA: MIT Press. pp 977-986.
34. Braun DA, Mehring C, Wolpert DM (2009) Structure learning in action. Behav Brain Res 206: 157-165.
35. Acuna D, Schrater P (2008) Bayesian modeling of human sequential decisionmaking on the multi-armed bandit problem. In: Sloutsky V, Love B, McRae K, eds. 30th Annual Conference of the Cognitive Science Society. AustinTX: Cognitive Science Society. pp 2065-2070.
36. Lee MD (2006) A hierarchical Bayesian model of human decision-making on an optimal stopping problem. Cogn Sci 30: 1-26.
37. Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS (2007) Learning the value of information in an uncertain world. Nat Neurosci 10: 1214-1221.
38. Steyvers M, Brown S (2006) Prediction and change detection. In: NIPS 2006. pp 1281-1288.
39. Anderson J (2000) Learning and memory. Wiley New York.
40. Erev I, Roth AE (1998) Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88: 848-881.
41. Smith A, Li M, Becker S, Kapur S (2006) Dopamine, prediction error and associative learning: A model-based account. Network 17: 61-84.
42. Johnson A, van der Meer M, Redish A (2007) Integrating hippocampus and striatum in decision-making. Curr Opin Neurobiol 17: 692-697.
43. Steyvers M, Lee MD, Wagenmakers E (2009) A bayesian analysis of human decision-making on bandit problems. J Math Psychol 53: 168-179.
44. Howard R (1960) Dynamic Programming. Cambridge, MA: MIT Press.
45. Fel'dbaum A (1965) Optimal Control Systems Academic Press.