A model-averaging approach to replication: The case of prep

Psychological Methods

Volumn 15, Issue 2, 2010, Pages 172-181

  Iverson, Geoffrey J ^a   Wagenmakers, Eric Jan ^b   Lee, Michael D ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Bayesian analysis; Model averaging; Prediction; Statistical hypothesis testing

Indexed keywords

ARTICLE; BAYES THEOREM; EPIDEMIOLOGY; HUMAN; MATHEMATICAL COMPUTING; PROBABILITY; PSYCHOLOGICAL MODEL; PSYCHOLOGY; REPRODUCIBILITY; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICS; STATISTICAL BIAS; STATISTICS AND NUMERICAL DATA;

BAYES THEOREM; BIAS (EPIDEMIOLOGY); DATA INTERPRETATION, STATISTICAL; HUMANS; MATHEMATICAL COMPUTING; MODELS, PSYCHOLOGICAL; MODELS, STATISTICAL; PROBABILITY; PSYCHOLOGY, EXPERIMENTAL; REPRODUCIBILITY OF RESULTS;

EID: 77955583936     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0017182     Document Type: Article

References (70)

1. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 1974, 19:716-723.
2. rep statistic as a measure of confidence in model fitting. Psychonomic Bulletin & Review 2008, 15:16-27.
3. Berger J.O. Statistical decision theory and Bayesian analysis 1985, Springer, New York, NY. 2nd ed.
4. Berger J.O. Robust Bayesian analysis: Sensitivity to the prior. Journal of Statistical Planning and Inference 1990, 25:303-328.
5. Berger J.O. Could Fisher, Jeffreys and Neyman have agreed on testing?. Statistical Science 2003, 18:1-32.
6. Berger J.O., Boukai B., Wang Y. Unified frequentist and Bayesian testing of a precise hypothesis (with discussion). Statistical Science 1997, 12:133-160.
7. Berger J.O., Brown L., Wolpert R. A unified conditional frequentist and Bayesian test for fixed and sequential hypothesis testing. The Annals of Statistics 1994, 22:1787-1807.
8. Berger J.O., Delampady M. Testing precise hypotheses. Statistical Science 1987, 2:317-352.
9. Berger J.O., Pericchi L.R. The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association 1996, 91:109-122.
10. Bernardo J.M., Smith A.F.M. Bayesian theory 1994, Wiley, New York, NY.
11. Bowers J.S., Vigliocco G., Haan R. Orthographic, phonological, and articulatory contributions to masked letter and word priming. Journal of Experimental Psychology: Human Perception and Performance 1998, 24:1705-1719.
12. Browne M. Cross-validation methods. Journal of Mathematical Psychology 2000, 44:108-132.
13. Burnham K.P., Anderson D.R. Model selection and multimodel inference: A practical information-theoretic approach 2002, Springer, New York, NY. 2nd ed.
14. Cohen J. The earth is round (p < .05). American Psychologist 1994, 49:997-1003.
15. Congdon P. Applied Bayesian modelling 2003, Wiley, Chichester, England.
16. Cumming G. Understanding the average probability of replication: Comment on Killeen (2005). Psychological Science 2005, 16:1002-1004.
17. Dawid A.P. Statistical theory: The prequential approach. Journal of the Royal Statistical Society: Series A: Statistics in Society 1984, 147:278-292.
18. Dennis S., Humphreys M.S. A context noise model of episodic word recognition. Psychological Review 2001, 108:452-477.
19. Dennis S.J., Lee M.D., Kinnell A. Bayesian analysis of recognition memory: The case of the list-length effect. Journal of Memory and Language 2008, 59:361-376.
20. Doros G., Geier A.B. Probability of replication revisited: Comment on " an alternative to null-hypothesis significance tests." Psychological Science 2005, 16:1005-1006.
21. Draper D. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society: Series B: Statistical Methodology 1995, 57:45-97.
22. Edwards W., Lindman H., Savage L.J. Bayesian statistical inference for psychological research. Psychological Review 1963, 70:193-242.
23. Efron B., Tibshirani R. Improvements on cross-validation: The .632+ bootstrap method. Journal of the American Statistical Association 1997, 92:548-560.
24. Forster K.I., Mohan K., Hector J. The mechanics of masked priming. Masked priming: The state of the art 2003, 3-38. Psychology Press, New York, NY. S. Kinoshita, S.J. Lupker (Eds.).
25. Gamerman D., Lopes H.F. Markov chain Monte Carlo: Stochastic simulation for Bayesian inference 2006, Taylor & Francis, Boca Raton, FL. 2nd ed.
26. Geisser S. The predictive sample reuse method with applications. Journal of the American Statistical Association 1975, 70:320-328.
27. Gill J. Bayesian methods: A social and behavioral sciences approach 2002, Chapman & Hall/CRC Press, Boca Raton, FL.
28. Gillund G., Shiffrin R.M. A retrieval model for both recognition and recall. Psychological Review 1984, 91:1-67.
29. Good I.J. Good thinking: The foundations of probability and its applications 1983, University of Minnesota Press, Minneapolis.
30. Good I.J. Weight of evidence: A brief survey. Bayesian statistics 2 1985, 249-269. Elsevier, New York, NY. J.M. Bernardo, M.H. DeGroot, D.V. Lindley, A.F.M. Smith (Eds.).
31. Grünwald P. Model selection based on minimum description length. Journal of Mathematical Psychology 2000, 44:133-152.
32. Grünwald P.D., Myung I.J., Pitt M.A. Advances in minimum description length: Theory and applications 2005, MIT Press, Cambridge, MA.
33. Hoeting J.A., Madigan D., Raftery A.E., Volinsky C.T. Bayesian model averaging: A tutorial. Statistical Science 1999, 14:382-417.
34. Hoijtink H., Klugkist I., Boelen P. Bayesian evaluation of informative hypotheses that are of practical value for social scientists 2008, Springer, New York, NY.
35. rep misestimates the probability of replication. Psychonomic Bulletin & Review 2009, 16:424-429.
36. rep: An agony in five fits. Journal of Mathematical Psychology 2009, 53:195-202.
37. Jeffreys H. Theory of probability 1961, Oxford University Press, Oxford, England. 3rd ed.
38. Kass R.E., Raftery A.E. Bayes factors. Journal of the American Statistical Association 1995, 90:773-795.
39. Killeen P.R. An alternative to null-hypothesis significance tests. Psychological Science 2005, 16:345-353.
40. Killeen P.R. Replicability, confidence, and priors. Psychological Science 2005, 16:1009-1012.
41. Killeen P. Tea-tests. General Psychologist 2005, 40:15-18.
42. Killeen P.R. Beyond statistical inference: A decision theory for science. Psychonomic Bulletin & Review 2006, 13:549-562.
43. Killeen P.R. Replication statistics. Best practices in quantitative methods 2007, 103-125. Sage, Thousand Oaks, CA. J.W. Osborne (Ed.).
44. Klugkist I., Laudy O., Hoijtink H. Inequality constrained analysis of variance: A Bayesian approach. Psychological Methods 2005, 10:477-493.
45. Lee M.D., Pope K.J. Model selection for the rate problem: A comparison of significance testing, Bayesian, and minimum description length statistical inference. Journal of Mathematical Psychology 2006, 50:193-202.
46. Lee M.D., Wagenmakers E.-J. Bayesian statistical inference in psychology: Comment on Trafimow (2003). Psychological Review 2005, 112:662-668.
47. Lindley D.V. The philosophy of statistics. The Statistician 2000, 49:293-337.
48. Lindsay R.M., Ehrenberg A.S.C. The design of replicated studies. The American Statistician 1993, 47:217-228.
49. Macdonald R.R. Why replication probabilities depend on prior probability distributions: A rejoinder to Killeen (2005). Psychological Science 2005, 16:1007-1008.
50. Madigan D., Raftery A.E. Model selection and accounting for model uncertainty in graphical models using Occam's window. Journal of the American Statistical Association 1994, 89:1535-1546.
51. Meyer D.E., Schvaneveldt R.W. Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology 1971, 90:227-234.
52. Myung I.J., Forster M.R., Browne M.W. Model selection [Special issue]. Journal of Mathematical Psychology 2000, 44(1-2).
53. Myung I.J., Pitt M.A. Applying Occam's razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review 1997, 4:79-95.
54. Nickerson R.S. Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods 2000, 5:241-301.
55. O'Hagan A. Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society: Series B: Statistical Methodology 1995, 57:99-138.
56. O'Hagan A., Forster J. Kendall's advanced theory of statistics: Vol. 2B. Bayesian inference 2004, Arnold, London, England. 2nd ed.
57. Pecher D., Zeelenberg R., Raaijmakers J.G.W. Does pizza prime coin? Perceptual priming in lexical decision and pronunciation. Journal of Memory and Language 1998, 38:401-418.
58. Pitt M.A., Myung I.J., Zhang S. Toward a method of selecting among computational models of cognition. Psychological Review 2002, 109:472-491.
59. Raftery A.E. Bayesian model selection in social research. Sociological methodology 1995 1996, 111-196. Blackwell, Cambridge, England. P.V. Marsden (Ed.).
60. Richard F.D., Bond C.F.J., Stokes-Zoota J.J. One hundred years of social psychology quantitatively described. Review of General Psychology 2003, 7:331-363.
61. Rissanen J. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory 2001, 47:1712-1717.
62. Rouder J.N., Speckman P.L., Sun D., Morey R.D., Iverson G. Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review 2009, 16:225-237.
63. Sanabria F., Killeen P.R. Better statistics for better decisions: Rejecting null hypotheses statistical tests in favor of replication statistics. Psychology in the Schools 2007, 44:471-481.
64. Smith A.F.M., Spiegelhalter D.J. Bayes factors and choice criteria for linear models. Journal of the Royal Statistical Society: Series B: Statistical Methodology 1980, 42:213-220.
65. Stone M. Cross-validatory choice and assessment of statistical predictions (with discussion). Journal of the Royal Statistical Society: Series B: Statistical Methodology 1974, 36:111-147.
66. Wagenmakers E.-J. A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review 2007, 14:779-804.
67. Wagenmakers E.-J., Grünwald P. A Bayesian perspective on hypothesis testing. Psychological Science 2006, 17:641-642.
68. Wagenmakers E.-J., Grünwald P., Steyvers M. Accumulative prediction error and the selection of time series models. Journal of Mathematical Psychology 2006, 50:149-166.
69. Wagenmakers E.-J., Waldorp L. Model selection: Theoretical developments and applications [Special issue]. Journal of Mathematical Psychology 2006, 50(2).
70. Zellner A., Siow A. Posterior odds ratios for selected regression hypotheses. Bayesian statistics 1980, 585-603. University Press, Valencia, Spain. J.M. Bernardo, M.H. DeGroot, D.V. Lindley, A.F.M. Smith (Eds.).