Model selection by normalized maximum likelihood

Journal of Mathematical Psychology

Volumn 50, Issue 2, 2006, Pages 167-179

  Myung, Jay I ^a   Navarro, Daniel J ^b   Pitt, Mark A ^a  

^a OHIO STATE UNIVERSITY   (United States)

^b UNIVERSITY OF ADELAIDE   (Australia)

Author keywords

Cognitive modeling; Inductive inference; Minimum Description Length; Model complexity

Indexed keywords

EID: 33645030440     PISSN: 00222496     EISSN: 10960880     Source Type: Journal    
DOI: 10.1016/j.jmp.2005.06.008     Document Type: Article

References (58)

1. H. Akaike Information theory and an extension of the maximum likelihood principle B.N. Petrov F. Csaki Second international symposium on information theory 1973 Akademiai Kiado Budapest 267-281
2. V. Balasubramanian Statistical inference, Occam's razor and statistical mechanics on the space of probability distributions Neural Computation 9 1997 349-368
3. V. Balasubramanian MDL, Bayesian inference and the geometry of the space of probability distributions P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge MA 81-98
4. A. Barron J. Rissanen B. Yu The minimum description length principle in coding and modeling IEEE Transactions on Information Theory 44 1998 2743-2760
5. J.O. Berger R.L. Wolpert The likelihood principle 2nd ed. 1988 Institute of Mathematical Statistics Hayward, CA
6. T. Cover J. Thomas Elements of information theory 1991 Wiley Interscience New York, NY
7. P. Dawid Present position and potential developments: Some personal views, statistical theory, the prequential approach Journal of the Royal Statistical Society, Series A 147 1984 278-292
8. P. Dawid Prequential analysis stochastic complexity and Bayesian inference J. Bernardo J. Berger A. Dawid A. Smith Bayesian statistics Vol. 4 1992 Oxford University Press Oxford, UK 109-125
9. W. Edwards H. Lindman L.J. Savage Bayesian statistical inference for psychological research Psychological Review 70 1963 193-242
10. B. Efron Estimating the error rate of a prediction rule: Some improvements on cross-validation Journal of the American Statistical Society 78 1983 316-331
11. D.P. Foster R.A. Stine The contribution of parameters to stochastic complexity P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge, MA 195-214
12. S. Geisser Predictive inference: An introduction 1993 Chapman & Hall New York, NY
13. S. Geisser W.F. Eddy A predictive approach to model selection Journal of the American Statistical Society 74 1979 153-160
14. A. Gelman J.B. Carlin H.S. Stern D.O. Rubin Bayesian data analysis 2nd ed. 2004 Chapman & Hall New York, NY
15. Grünwald, P. (1998). The minimum description length principle and reasoning under uncertainty. Ph.D. thesis, ILLC Dissertation Series DS 1998-03, CWI, The Netherlands.
16. P. Grünwald Model selection based on minimum description length Journal of Mathematical Psychology 44 2000 133-170
17. P. Grünwald Minimum description length tutorial P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge, MA 23-80
18. Grünwald, P., & de Rooij, S. (2005). Asymptotic log-loss prequential maximum likelihood codes. Proceedings of the 18th annual conference on computational learning theory (COLT 2005).
19. Grünwald, P., Myung, I. J., & Pitt, M. A. (Eds.). (2005). Advances in minimum description length: Theory and applications. Cambridge, MA: MIT Press.
20. M.H. Hansen B. Yu Model selection and the principle of minimum description length Journal of the American Statistical Association 96 2001 746-774
21. T. Hastie R. Tibshirani J. Friedman The elements of statistical learning 2001 Springer New York
22. B.M. Hill The validity of the likelihood principle American Statistician 41 1987 95-100
23. R.E. Kass A.E. Raftery Bayes factors Journal of the American Statistical Society 90 1995 773-795
24. P. Kontkanen P. Myllymäki W. Buntine J. Rissanen H. Tirri An MDL framework for data clustering P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge, MA 323-354
25. S. Kullback Information theory and statistics 2nd ed. 1968 Dover New York
26. A.D. Lanterman Schwarz, Wallace, and Rissanen: Intertwining themes in theories of model selection International Statistical Review 69 2001 185-212
27. A.D. Lanterman Hypothesis testing for Poisson vs. geometric distributions using stochastic complexity P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge, MA 99-125
28. M.D. Lee On the complexity of additive clustering models Journal of Mathematical Psychology 45 2001 131-148
29. M.D. Lee D.J. Navarro Minimum description length and psychological clustering models P. Grünwald I.J. Myung M.A. Pitt Advances in minimum description length: Theory and applications 2005 MIT Press Cambridge, MA 355-384
30. M. Li P.M.B. Vitányi An introduction to Kolmogorov complexity and its applications 2nd ed. 1997 Springer New York, NY
31. H. Linhart W. Zucchini Model selection 1986 Wiley New York, NY
32. I.J. Myung The importance of complexity in model selection Journal of Mathematical Psychology 44 2000 190-204
33. I.J. Myung M.A. Pitt Applying Occam's razor in modeling cognition: A Bayesian approach Psychonomic Bulletin & Review 4 1997 79-95
34. D.J. Navarro A note on the applied use of MDL approximations Neural Computation 16 2004 1763-1768
35. D.J. Navarro M.D. Lee Common and distinctive features in stimulus representation: A modified version of the contrast model Psychonomic Bulletin and Review 11 2004 961-974
36. R.M. Nosofsky M.A. Gluck T.J. Palmeri S.C. McKinley P. Glauthier Comparing models of rule-based classification learning: A replication and extension of Shepard, Hovland and Jenkins (1961) Memory & Cognition 22 1994 352-369
37. M.A. Pitt I.J. Myung S. Zhang Toward a method of selecting among computational models of cognition Psychological Review 109 2002 472-491
38. S.J. Press Subjective and objective Bayesian statistics 2nd ed. 2003 Wiley New York
39. J. Rissanen Modeling by the shortest data description Automatica 14 1978 465-471
40. J. Rissanen A universal prior for integers and estimation by minimum description length Annals of Statistics 11 1983 416-431
41. J. Rissanen Stochastic complexity and modeling Annals of Statistics 14 1986 1080-1100
42. J. Rissanen Stochastic complexity Journal of the Royal Statistical Society, Series B 49 1987 223-239
43. J. Rissanen Stochastic complexity in statistical inquiry 1989 World Scientific Publishing Singapore
44. J. Rissanen Fisher information and stochastic complexity IEEE Transactions on Information Theory 42 1996 40-47
45. J. Rissanen MDL denoising IEEE Transactions on Information Theory 46 2000 2537-2543
46. J. Rissanen Strong optimality of the normalized ML models as universal codes and information in data IEEE Transactions on Information Theory 47 2001 1712-1717
47. Rissanen, J. (2003). Lectures on statistical modeling theory. October 2003. Available online at www.mdl-research.org.
48. M.J. Schervish Theory of statistics 1995 Springer New York
49. G. Schwarz Estimating the dimension of a model The Annals of Statistics 6 1978 461-464
50. R.N. Shepard C.I. Hovland H.M. Jenkins Learning and memorization of classification Psychological Monographs 75 13 1961
51. Y.M. Shtarkov Universal sequential coding of single messages Problems in Information Transmission 23 1987 3-17
52. M. Stone Cross-validatory choice and assessment of statistical predictions Journal of the Royal Statistical Society, Series B 36 1974 111-147
53. P.M.B. Vitányi M. Li Minimum description length induction, Bayesianism and Kolmogorov complexity IEEE Transactions on Information Theory 46 2000 446-464
54. Wagenmakers, E.-J., Grünwald, P., & Steyvers, M. (2005). Accumulative prediction error and the selection of time series models. Journal of Mathematical Psychology, this issue.
55. C.S. Wallace D.M. Boulton An information measure for classification Computer Journal 11 1968 185-194
56. C.S. Wallace D.L. Dowe Minimum message length and Kolmogorov complexity Computer Journal 42 1999 270-287
57. C.S. Wallace D.L. Dowe Refinements of MDL and MML coding Computer Journal 42 1999 330-337
58. W. Zucchini An introduction to model selection Journal of Mathematical Psychology 44 2000 41-61