A Bayesian view of model complexity

Statistica Neerlandica

Volumn 66, Issue 3, 2012, Pages 253-271

  van der Linde, Angelika ^a  

^a UNIVERSITY OF BREMEN   (Germany)

Author keywords

Effective number of parameters; Generalized degrees of freedom; Information criteria; Model comparison; Mutual information

Indexed keywords

EID: 84864023779     PISSN: 00390402     EISSN: 14679574     Source Type: Journal    
DOI: 10.1111/j.1467-9574.2011.00518.x     Document Type: Article

References (43)

1. Akaike, H. (1973), Information theory and an extension of the maximum likelihood principle, in: B. Petrov and F. Csaki (eds), Second International Symposium on Information Theory, Akademiai Kaido, Budapest, pp. 267-281.
2. Amari, S., O. E. Barndorff-Nielsen, R. E. Kass, S. L. Lauritzen and C. R. Rao (1987), Differential geometry in statistical inference. Springer, Berlin.
3. Ando, T. (2007), Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayesian models. Biometrika 94, 443-458.
4. Berger, J. O. (2006), The case for objective Bayesian analysis. Bayesian Analysis 1, 385-464. With discussion.
5. Berger, J. O., J. M. Bernardo and D. Sun (2009), The formal definition of reference priors. Annals of Statistics 37, 905-938.
6. Bernardo, J. M. (1997), Noninformative priors do not exist. Journal of Statistical Planning and Inference 65, 159-189. With discussion.
7. Bernardo, J. M. and A. F. M. Smith (1994), Bayesian theory. Wiley, Chichester.
8. Bernardo, J. M. (2003), Bayesian statistics. in R. Viertl (ed.), Encyclopedia of life support systems (EOLLS). UNESCO, Oxford. Available at.
9. Blyth, S. (1994), Local divergence and association. Biometrika 81, 579-584.
10. Bozdogan, H. (2010), A new class of information complexity (ICOMP) criteria with an application to customer profiling and segmentation. Istanbul University Journal of the School of Business Administration 39, 370-398.
11. Chaloner, K. and I. Verdinelli (1995), Bayesian experimental design. A review. Statistical Science 10, 273-304.
12. Chib, S. (1995), Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90, 1313-1321.
13. Csiszar, I. (1967), Information type measures of difference of probability distributions and indirect observations. Studia Scientarum Mathematicum Hungarica 2, 299-318.
14. Degroot, M. H. (1962), Uncertainty, information and sequential experiments. Annals of Mathematical Statistics 33, 404-419.
15. Didelot, X., R. G. Everitt, A. M. Johansen and D. J. Lawson (2011), Likelihood-free estimation of model evidence. Bayesian Analysis 6, 1-30.
16. Efron, B. (1996), How biased is the apparent error rate of a prediction rule? Journal of the American Statistical Association 74, 461-470.
17. Gelfand, A. E. and S. K. Ghosh (1998), Model choice: a minimum posterior predictive loss approach. Biometrika 85, 1-11.
18. Goel, P. (1983), Information measures and Bayesian hierarchical models. Journal of the American Statistical Association 78, 408-410.
19. Green, P. J. and B. W. Silverman (1994), Nonparametric regression and generalized linear models. Chapman and Hall, London.
20. Gutierrez-Pena, E. and S. Walker (2001), A Bayesian predictive approach to model selection. Journal of Statistical Planning and Inference 93, 259-276.
21. Gutierrez-Pena, E. (1997), A Bayesian predictive semiparametric approach to variable selection and model comparison in regression, in: Proceedings of the 51st session of the ISI, Istanbul, pp. 17-29.
22. Hastie, T. J. and R. J. Tibshirani (1990), Generalized additive models. Chapman and Hall, London.
23. Konishi, S. and G. Kitagawa (1996), Generalised information criteria in model selection. Biometrika 83, 875-890.
24. Kullback, S. (1968), Information theory and statistics. Dover, Mineola, New York.
25. Lanterman, A. D. (2001), Schwarz, Wallace and Rissanen: intertwining themes in theories of model selection. International Statistical Review 69, 185-212.
26. Mcculloch, R. E. (1988), Information and the likelihood function in exponential families. The American Statistician 42, 73-75.
27. Micheas, A. C. and K. Zografos (2006), Measuring stochastic dependence using φ{symbol}- divergence. Multivariate Analysis 97, 765-784.
28. Murray, A. C. and J. W. Rice (1993), Differential geometry and statistics. Chapman and Hall, London.
29. Neal, R. (2001), Annealed importance sampling. Statistics and Computing 11, 125-139.
30. Plummer, M. (2002), Discussion of 'Bayesian measures of model complexity and fit'. Journal of the Royal Statistical Society B 64, 620-621.
31. Plummer, M. (2008), Penalized loss functions for Bayesian model comparison. Biostatistics 9, 523-539.
32. San Martini, A. and F. Spezzaferi (1984), A predictive model selection criterion. Journal of the Royal Statistical Society B 46, 296-303.
33. Schwarz, G. (1978), Estimating the dimension of a model. Annals of Statistics 6, 461-464.
34. Shibata, R. (1997), Bootstrap estimate of Kullback-Leibler information for model selection. Statistica Sinica 7, 375-394.
35. Slonim, N., G. S. Atwal, G. Tkacik and W. Bialek (2005), Estimating mutual information and multi-information in large networks.
36. Spiegelhalter, D., N. Best, B. P. Carlin and A. Van Der Linde (2002), Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society B 64, 583-639. With discussion.
37. Sprenger, J. (2010), The weight of simplicity in statistical model comparison. Technical report, Center for Logic and Philosophy of Science, Tilburg University, The Netherlands,.
38. Stone, M. (1977), An asymptotic equivalence of choice of model cross-validation and Akaike's criterion. Journal of the Royal Statistical Society B 36, 44-47.
39. Van Der Linde, A. (2004), On the association between a random parameter and an observable. Test 13, 85-111.
40. Wahba, G. (1990), Spline models for observational data. SIAM, Philadelphia, PA.
41. Watanabe, S. (2010), Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research 1, 1-48.
42. Ye, J. (1998), On measuring and correcting the effects of data mining and model selection. Journal of the American Statistical Association 93, 120-131.
43. Ye, J. and W. Wong (1998), Evaluation of highly complex modeling procedures with binomial and Poisson data. Technical report, Graduate School of Business, University of Chicago.