The nature of belief-directed exploratory choice in human decision-making

Frontiers in Psychology

Volumn 2, Issue JAN, 2011, Pages

  Knox, W Bradley ^a   Otto, A Ross ^b   Stone, Peter ^a   Love, Bradley C ^c  

^a University of Texas at Austin   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Decision making; Exploitation; Exploration; Ideal actor; Ideal observer; Planning; POMDP; Reinforcement learning

Indexed keywords

EID: 84866086627     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2011.00398     Document Type: Article

References (25)

1. Biele, G., Erev, I., and Ert, E. (2009). Learning, risk attitude and hot stoves in restless bandit problems. J. Math. Psychol. 53, 155-167.
2. Boorman, E. D., Behrens, T. E., Woolrich, M. W., and Rushworth, M. F. (2009). How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733-743.
3. Botvinick, M. M., and Rosen, Z. B. (2009). Anticipation of cognitive demand during decision-making. Psychol. Res. 73, 835-842.
4. Cassandra, A., Littman, M., and Zhang, N. (1997). "Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes," in Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence, Providence, 54-61.
5. Cohen, J. D., McClure, S. M., and Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933-942.
6. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., and Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204-1215.
7. Daw, N. D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711.
8. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., and Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876-879.
9. De Neys, W., and Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition 106, 1248-1299.
10. Gureckis, T. M., and Love, B. C. (2009). Short-term gains, long-term pains: how cues about state aid learning in dynamic environments. Cognition 113, 293-313.
11. Jepma, M., and Nieuwenhuis, S. (2011). Pupil diameter predicts changes in the exploration-exploitation tradeoff: evidence for the adaptive gain theory. J. Cogn. Neurosci. 23, 1587-1596.
12. Kaelbling, L., Littman, M., and Cassandra, A. (1998). Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99-134.
13. Lee, M. D., Zhang, S., Munro, M., and Steyvers, M. (2011). Psychological models of human and optimal performance in bandit problems. Cogn. Syst. Res. 12, 164-174.
14. Levinthal, D.A., and March, J. G. (1993). The myopia of learning. Strategic Manage. J. 14, 95-112.
15. Najemnik, J., and Geisler, W. S. (2005). Optimal eye movement strategies in visual search. Nature 434, 387-391.
16. Otto, A. R., Markman, A. B., Gureckis, T. M., and Love, B. C. (2010). Regulatory fit and systematic exploration in a dynamic decision-making environment. J. Exp. Psychol. Learn. Mem. Cogn. 36, 797-804.
17. Pearson, J. M., Hayden, B. Y., Raghavachari, S., and Platt, M. L. (2009). Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr. Biol. 19, 1532-1537.
18. Pinheiro, J. C., and Bates, D. M. (2000). Mixed-Effects Models in S and S- PLUS. New York: Springer.
19. Ratcliff, R., and Rouder, J. (1998). Modeling response times for two-choice decisions. Psychol. Sci. 9, 347.
20. Schwartz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6, 461-464.
21. Stankiewicz, B. J., Legge, G. E., Mansfield, J. S., and Schlicht, E. J. (2006). Lost in virtual space: studies in human and ideal spatial navigation. J. Exp. Psychol. Hum. Percept. Perform. 32, 688-704.
22. Steyvers, M., and Brown, S. (2005). "Prediction and change detection," in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vancouver.
23. Sutton, R., and Barto, A. G. (1998). Reinforcement Learning. Cambridge, MA: MIT Press.
24. Walton, M. E., Devlin, J. T., and Rushworth, M. F. S. (2004). Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259-1265.
25. Worthy, D. A., Maddox, W. T., and Markman, A. B. (2007). Regulatory fit effects in a choice task. Psychon. Bull. Rev. 14, 1125-1132.