Forgetful Bayes and myopic planning: Human learning and decision-making in a bandit setting

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Zhang, Shunan ^a   Yu, Angela J ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; COMPUTATIONAL COMPLEXITY;

COGNITIVE SCIENCE; EXPLORATION AND EXPLOITATION; ITERATIVE LEARNING; KNOWN ENVIRONMENTS; MULTI ARMED BANDIT; NOISY OBSERVATIONS; RISK MINIMIZATION; UNCERTAIN ENVIRONMENTS;

OPTIMIZATION;

EID: 84898954453     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. J. Banks, M. Olson, and D. Porter. An experimental analysis of the bandit problem. Economic Theory, 10:55-77, 2013.
2. R. Bellman. On the theory of dynamic programming. Proceedings of the National Academy of Sciences, 1952.
3. R. Cho, L. Nystrom, E. Brown, A. Jones, T. Braver, P. Holmes, and J. D. Cohen. Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task. Cognitive, Affective and Behavioral Neuroscience, 2:283-299, 2002.
4. J. D. Cohen, S. M. McClure, and A. J. Yu. Should I stay or should I go? Exploration versus exploitation. Philosophical Transactions of the Royal Society B: Biological Sciences, 362:933-942, 2007.
5. N. D. Daw, J. P. O'Doherty, P. Dayan, B. Seymour, and R. J. Dolan. Cortical substrates for exploratory decisions in humans. Nature, 441:876-879, 2006.
6. A. Ejova, D. J. Navarro, and A. F. Perfors. When to walk away: The effect of variability on keeping options viable. In N. Taatgen, H. van Rijn, L. Schomaker, and J. Nerbonne, editors, Proceedings of the 31st Annual Conference of the Cognitive Science Society, Austin, TX, 2009.
7. P. Frazier,W. Powell, and S. Dayanik. A knowledge-gradient policy for sequential information collection. SIAM Journal on Control and Optimization, 47:2410-2439, 2008.
8. W. R. Garner. An informational analysis of absolute judgments of loudness. Journal of Experimental Psychology, 46:373-380, 1953.
9. A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian data analysis. Chapman &Hall/CRC, Boca Raton, FL, 2 edition, 2004.
10. J. C. Gittins. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, 41:148-177, 1979.
11. L. P. Kaebling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285, 1996.
12. M. D. Lee, S. Zhang, M. Munro, and M. Steyvers. sychological models of human and optimal performance in bandit problems. Cognitive Systems Research, 12:164-174, 2011.
13. M. I. Posner and Y. Cohen. Components of visual orienting. Attention and Performance Vol. X, 1984.
14. W. Powell and I. Ryzhov. Optimal Learning. Wiley, 1 edition, 2012.
15. H. Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58:527-535, 1952.
16. I. Ryzhov, W. Powell, and P. Frazier. The knowledge gradient algorithm for a general class of online learning problems. Operations Research, 60:180-195, 2012.
17. 17] J. Shin and D. Ariely. Keeping doors open: The effect of unavailability on incentives to keep options viable. MANAGEMENT SCIENCE, 50:575-586, 2004.
18. M. Steyvers, M. D. Lee, and E.-J. Wagenmakers. A bayesian analysis of human decisionmaking on bandit problems. Journal of Mathematical Psychology, 53:168-179, 2009.
19. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
20. M. C. Treisman and T. C. Williams. A theory of criterion setting with an application to sequential dependencies. Psychological Review, 91:68-111, 1984.
21. A. J. Yu and J. D. Cohen. Sequential effects: Superstition or rational behavior? In Advances in Neural Information Processing Systems, volume 21, pages 1873-1880, Cambridge, MA., 2009. MIT Press.
22. S. Zhang and A. J. Yu. Cheap but clever: Human active learning in a bandit setting. In Proceedings of the Cognitive Science Society Conference, 2013.