Optimal Experimental Design for Model Discrimination

Psychological Review

Volumn 116, Issue 3, 2009, Pages 499-518

  Myung, Jay I ^a   Pitt, Mark A ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

categorization; formal modeling; Markov chain Monte Carlo; model discrimination; retention

Indexed keywords

ALGORITHM; ANIMAL; ARTICLE; ASSOCIATION; CONCEPT FORMATION; HUMAN; LONG TERM MEMORY; METHODOLOGY; MONTE CARLO METHOD; PERCEPTIVE DISCRIMINATION; PROBABILITY; PSYCHOLOGICAL MODEL; STATISTICAL MODEL; STATISTICS; UNCERTAINTY;

ALGORITHMS; ANIMALS; ASSOCIATION; CONCEPT FORMATION; DISCRIMINATION (PSYCHOLOGY); HUMANS; LIKELIHOOD FUNCTIONS; MARKOV CHAINS; MODELS, PSYCHOLOGICAL; MONTE CARLO METHOD; RESEARCH DESIGN; RETENTION (PSYCHOLOGY); UNCERTAINTY;

EID: 67651052215     PISSN: 0033295X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0016104     Document Type: Article

References (67)

1. Akaike H. Information theory and an extension of the maximum likelihood principle. In: Petrox B.N., and Caski F. (Eds). Second International Symposium on Information Theory (1973), Akademia Kiado, Budapest, Hungary 267-281
2. Allen T.T., Yu L., and Schmitz J. An experimental design criterion for minimizing meta-model prediction errors applied to die casting process design. Journal of the Royal Statistical Society: Applied Statistics 52 C (2003) 103-117
3. Amzal B., Bois F.Y., Parent E., and Robert C.P. Bayesian-optimal design via interacting particle systems. Journal of the American Statistical Association 101 (2006) 773-785
4. Anderson J.R., Fincham J.M., and Douglass S. Practice and retention. Journal of Experimental Psychology: Learning, Memory, and Cognition 25 (1999) 1120-1136
5. Anderson J.R., and Lebiere C. The atomic components of thought (1998), Erlbaum, Mahwah, NJ
6. Atkinson A.C., and Donev A.N. Optimum experimental designs (1992), Oxford University Press, Oxford, England
7. Atkinson C.V., and Fedorov V.V. The design of experiments for discriminating between two rival models. Biometrika 62 (1975) 57-70
8. Balota D.A., and Chumbley J.I. Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage. Journal of Experimental Psychology: Human Perception and Performance 10 (1984) 340-357
9. Bardsley W.G., Wood R.M.W., and Melikhova E.M. Optimal design: A computer program to study the best possible spacing of design points for model discrimination. Computers & Chemistry 20 (1996) 145-157
10. Bingham D., and Chipman H. Optimal design for model selection (Tech. Rep. No. 388) (2002), University of Michigan, Ann Arbor
11. Box G.E.B., and Hill W.J. Discrimination among mechanistic models. Technometrics 9 (1967) 57-71
12. Busemeyer J.R., and Townsend J.T. Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review 100 (1991) 432-459
13. Casella G., and Berger R.L. Statistical inference. 2nd ed. (2002), Thomson Learning, Pacific Grove, CA
14. Chaloner K., and Verdinelli I. Bayesian experimental design: A review. Statistical Science 10 (1995) 274-304
15. Del Moral P., Doucet A., and Jasra A. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society: Statistical Methodology 68 B (2006) 411-436
16. Doucet A., de Freitas N., and Gordon N. Sequential Monte Carlo methods in practice (2001), Springer, New York
17. Ebbinghaus H. Uber das Gedachtnis [Memory: A contribution to experimental psychology] (1885), Duncker Humbolt, Leipzig, Germany
18. El-Gamal M.A., and Palfrey T.R. Economical experiments: Bayesian efficient experimental design. International Journal of Game Theory 25 (1996) 495-517
19. Gilks W.R., Richardson S., and Spiegelhalter D.J. Markov chain Monte Carlo in practice (1996), Chapman & Hall, Boca Raton, FL
20. Grunwald P. Model selection based on minimum description length. Journal of Mathematical Psychology 44 (2000) 133-152
21. Grunwald P., Myung I.J., and Pitt M.A. Advances in minimum description length: Theory and applications (2005), MIT Press, Cambridge, MA
22. Jeffreys H. Theory of probability. 3rd ed. (1961), Oxford University Press, London
23. Kass R.E., and Raftery A.E. Bayes factors. Journal of the American Statistical Association 90 (1995) 773-795
24. Kiefer J. Optimum experimental designs. Journal of the Royal Statistical Society: Statistical Methodology 21 B (1959) 272-319
25. Kirkpatrick S., Gelatt C.D., and Vecchi M.P. Optimization by simulated annealing. Science 220 (1983, May 13) 671-680
26. Kueck, H., de Freitas, N., & Doucet, A. (2006, September). SMC samplers for Bayesian optimal nonlinear design. Paper presented at the Nonlinear Statistical Signal Processing Workshop, Cambridge, England.
27. Kujala J.V., and Lukka T.J. Bayesian adaptive estimation: The next dimension. Journal of Mathematical Psychology 50 (2006) 369-389
28. Kujala, J. V., Richardson, U., & Lyytinen, H. (in press). A Bayesian-optimal principle for a child-friendly adaptation in learning games. Journal of Mathematical Psychology
29. Lee M.D. On the complexity of additive clustering models. Journal of Mathematical Psychology 45 (2001) 131-148
30. Lee M.D., and Pope K.J. Model selection for the rate problem: A comparison of significance testing, Bayesian and minimum description length statistical inference. Journal of Mathematical Psychology 44 (2006) 190-204
31. Lesmes L.A., Jeon S.-T., Lu Z.-L., and Dosher B.A. Bayesian adaptive estimation of threshold versus contrast external noise functions: The quick TvC method. Vision Research 46 (2006) 3160-3176
32. Lewi, J., Butera, R., & Paninski, L. (in press). Sequential optimal design of neurophysiology experiment. Neural Computation
33. Loredo T.J. Bayesian adaptive estimation. In: Erickson G.J., and Zhai Y. (Eds). Bayesian inference and maximum entropy methods in science and engineering (2004), American Institute of Physics, Melville, NY 330-346
34. Medin D.L., and Schaffer M.M. Context theory of classification learning. Psychological Review 85 (1978) 207-238
35. Muller P. Simulation-based optimal design. In: Bernardo J.M., Berger J.O., Dawid A.P., and Smith A.F.M. (Eds). Bayesian statistics 6 (1999), Oxford University Press, Oxford, England 459-474
36. Muller P., Berry D.A., Grieve A.P., Smith M., and Krams M. Simulation-based sequential Bayesian design. Journal of Statistical Planning and Inference 137 (2007) 3140-3150
37. Muller P., Sanso B., and De Iorio M. Optimal Bayesian design by inhomogeneous Markov chain simulation. Journal of the American Statistical Association 99 (2004) 788-798
38. Myung I.J., Forster M.R., and Browne M.W. Guest editors' introduction: Special issue on model selection. Journal of Mathematical Psychology 44 (2000) 1-2
39. Myung I.J., and Pitt M. Applying Occam's razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review 4 (1997) 79-95
40. Myung J.I., Navarro D.J., and Pitt M.A. Model selection by normalized maximum likelihood. Journal of Mathematical Psychology 50 (2006) 167-179
41. Navarro D.J., and Lee M.D. Common and distinctive features in stimulus representation: A modified version of the contrast model. Psychonomic Bulletin & Review 11 (2004) 961-974
42. Navarro D.J., Pitt M.A., and Myung I.J. Assessing the distinguishability of models and the informativeness of data. Cognitive Psychology 49 (2004) 47-84
43. Nosofsky R.M. Attention, similarity, and identification-categorization relationship. Journal of Experimental Psychology: General 115 (1986) 39-57
44. Nosofsky R.M., and Zaki S.R. Exemplar and prototype models revisited: Response strategies, selective attention, and stimulus generalization. Journal of Experimental Psychology: Learning, Memory, and Cognition 28 (2002) 924-940
45. Pitt M.A., and Myung I.J. When a good fit can be bad. Trends in Cognitive Sciences 6 (2002) 421-425
46. Pitt M.A., Myung I.J., and Zhang S. Toward a method of selecting among computational models of cognition. Psychological Review 109 (2002) 472-491
47. Ponce de Leon A.C., and Atkinson A.C. Optimum experimental design for discriminating between two rival models in the presence of prior information. Biometrika 78 (1991) 601-608
48. Reed S.K. Pattern recognition and categorization. Cognitive Psychology 3 (1972) 382-407
49. Rissanen J.J. Fisher information and stochastic complexity. IEEE Transactions on Information Theory 42 (1996) 40-47
50. Rissanen J.J. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory 47 (2001) 1712-1717
51. Robert C.P. The Bayesian choice. 2nd ed. (2001), Springer, New York
52. Robert C.P., and Casella G. Monte Carlo methods. 2nd ed. (2004), Springer, New York
53. Rubin D.C., Hinton S., and Wenzel A. The precise course of retention. Journal of Experimental Psychology: Learning, Memory, and Cognition 25 (1999) 1161-1176
54. Rubin D.C., and Wenzel A. One hundred years of forgetting: A quantitative description of retention. Psychological Review 103 (1996) 734-760
55. Shiffrin R.M., and Steyvers M. A model for recognition memory: REM-retrieving effectively from memory. Psychonomic Bulletin & Review 4 (1997) 145-166
56. Slamecka N.J., and McElree B. Normal forgetting of verbal lists as a function of their degree of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition 9 (1983) 384-397
57. Smith J.D., and Minda J.P. Prototypes in the mist: The early epochs of category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition 24 (1998) 1411-1436
58. Squire L.R. On the course of forgetting in very long term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition 15 (1989) 241-245
59. Townsend J.T., and Wenger M.W. A theory of interactive parallel processing: New capacity measures and predictions for a response time inequality series. Psychological Review 111 (2004) 1003-1035
60. Ucinski D., and Bogacka B. T-optimum design for discrimination between two multiresponse dynamic models. Journal of the Royal Statistical Society: Statistical Methodology 67 B (2005) 3-18
61. Vanpaemel W., and Storms G. In search of abstraction: The varying abstraction model of categorization. Psychonomic Bulletin & Review 15 (2008) 732-749
62. Wagenmakers E.-J., Ratcliff R., Gomez P., and Iverson G.J. Assessing model mimicry using the parametric bootstrap. Journal of Mathematical Psychology 48 (2004) 28-50
63. Wagenmakers E.-J., and Waldorp L. Editors' introduction. Journal of Mathematical Psychology 50 (2006) 99-100
64. Waugh N.C., and Norman D.A. Primary memory. Psychological Review 72 (1965) 89-104
65. Wickelgren W.A. Sparing of short-term memory in an amnesiac patient: Implications of strength theory of memory. Neuropsychologica 6 (1968) 235-244
66. Wickens T.D. On the form of forgetting function: Comment on Rubin and Wenzel (1996): A quantitative description of retention. Psychological Review 105 (1998) 379-386
67. Wixted J.T., and Ebbesen E.B. On the form of forgetting. Psychological Science 2 (1991) 409-415