Optimal experimental design for a class of bandit problems

Journal of Mathematical Psychology

Volumn 54, Issue 6, 2010, Pages 499-508

  Zhang, Shunan ^a   Lee, Michael D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 78149413946     PISSN: 00222496     EISSN: 10960880     Source Type: Journal    
DOI: 10.1016/j.jmp.2010.08.002     Document Type: Article

References (33)

1. Amzal B., Bois F.Y., Parent E., Robert C.P. Bayesian-optimal design via interacting particle systems. Journal of the American Statistical Association 2006, 101(474):773-785.
2. Atkinson A.C., Donev A.N. Optimum experimental designs 1992, Oxford University Press.
3. Berry D.A., Fristedt B. Bandit problems: sequential allocation of experiments 1985, Chapman & Hall, London.
4. Bingham, D., & Chipman, H. (2002). Optimal design for model selection. Technical report 388. University of Michigan.
5. Box G.E.B., Hill W.J. Discrimination among mechanistic models. Technometrics 1967, 9:5-21.
6. Brand H., Sakoda J.M., Woods P.J. Effects of a random versus pattern reinforcement instructional set in a contingent partial reinforcement situation. Psychological Reports 1957, 3:473-479.
7. Brand H., Wood P.J., Sakoda J.M. Anticipation of reward as a function of partial reinforcement. Journal of Experimental Psychology 1956, 52(1):18-22.
8. Brooks S.P., Gelman A. General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics 1997, 7:434-455.
9. Cavagnaro D.R., Myung J.I., Pitt M.A., Kujala J.V. Adaptive design optimization: a mutual information based approach to model discrimination in cognitive science. Neural Computation 2010, 22:831-886.
10. Chaloner K., Verdinelli I. Bayesian experimental design: a review. Statistical Science 1995, 10:274-304.
11. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? Exploration versus exploitation. Philosophical Transactions of the Royal Society B: Biological Sciences 2007, 362:933-942.
12. Cover T., Thomas J. Elements of information theory 1991, Wiley.
13. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
14. Geweke J. Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. Bayesian Statistics 1992, 4:169-193.
15. Gittins J.C. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B 1979, 41:148-177.
16. Han J., Kamber M. Data mining 2006, Morgan Kauffman, New York. second ed.
17. Kaebling L.P., Littman M.L., Moore A.W. Reinforcement learning: a survey. Journal of Artificial Intelligence Research 1996, 4:237-285.
18. Kass R.E., Raftery A.E. Bayes factors. Journal of the American Statistical Association 1995, 90:377-395.
19. Kiefer J. Optimum experimental designs. Journal of the Royal Statistical Society: Series B 1959, 21:272-319.
20. Koller D., Friedman N., Getoor L., Taskar B. Graphical models in a nutshell. Introduction to statistical relational learning 2007, MIT Press, Cambridge, MA. L. Getoor, B. Taskar (Eds.).
21. Lee M.D. Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review 2008, 15(1):1-15.
22. Lee, M. D., Zhang, S., Munro, M. N., & Steyvers, M. (in press). Psychological models of human and optimal performance on bandit problems. Cognitive Systems Research.
23. Lindley D. On a measure of the information provided by an experiment. Annals of Mathematical Statistics 1956, 27:986-1005.
24. Mackay D.J.C. Information-based objective functions for active data selection. Neural Computation 1992, 4:590-604.
25. Macready W.G., Wolpert D.H. Bandit problems and the exploration/exploitation tradeoff. IEEE Transactions on Evolutionary Computation 1998, 2(1):2-22.
26. Müller P. Simulation-based optimal design. Bayesian Statistics 1999, 6:459-474.
27. Müller P., Sansó B., De Iorio M. Optimal Bayesian design by inhomogeneous Markov chain simulation. Journal of the American Statistical Association 2004, 99(467):788-798.
28. Myung J., Pitt M. Optimal experimental design for model discrimination. Psychological Review 2009, 116:499-518.
29. Paninski L. Asymptotic theory of information-theoretic experimental design. Neural Computation 2005, 17:1480-1507.
30. Robbins H. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society 1952, 58:527-535.
31. Shiffrin R.M., Lee M.D., Kim W.-J., Wagenmakers E.-J. A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science 2008, 32:1248-1284.
32. Steyvers M., Lee M.D., Wagenmakers E.-J. A Bayesian analysis of human decision-making on bandit problems. Journal of Mathematical Psychology 2009, 53:168-179.
33. Sutton R.S., Barto A.G. Reinforcement learning: an introduction 1998, The MIT Press, Cambridge, MA.