Methods and criteria for model selection

Journal of the American Statistical Association

Volumn 99, Issue 465, 2004, Pages 279-290

  Kadane, Joseph B ^a   Lazar, Nicole A ^b  

^a Leonard J Savage University   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

AIC; Bayes factors; BIC; Mallow's Cp; Model averaging; Subset selection; Variable selection

Indexed keywords

EID: 2142781609     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1198/016214504000000269     Document Type: Article

References (91)

1. Agresti, A. (1990), Categorical Data Analysis, New York: Wiley.
2. Aitkin, M. (1991), “Posterior Bayes Factors” (with discussion), Journal of the Royal Statistical Society, Ser. B, 53, 111–142.
3. Akaike, H. (1973), “Information Theory and an Extension of the Maximum Likelihood Principle,” in Second International Symposium on Information Theory, eds. B. N. Petrox and F. Caski, Budapest: Akademiai Kiado, pp. 267–281.
4. Akaike, H. (1983), “Information Measures and Model Selection,” Bulletin of the International Statistical Institute, 50, 277–290.
5. Allen, D. M. (1974), “The Relationship Between Variable Selection and Prediction,” Technometrics, 16, 125–127.
6. Bayarri, M. J. (1999), Discussion of “Bayesian Model Choice: What and Why?”, by J. T. Key, L. R. Pericchi, and A. F.M. Smith, in Bayesian Statistics (Vol. 6), eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Cambridge, MA; New York: Oxford University Press, pp. 357–359.
7. Berger, J. O., and Pericchi, L. R. (1996), “The Intrinsic Bayes Factor for Model Selection and Prediction,” Journal of the American Statistical Association, 91, 109–122.
8. Berger, J. O., and Pericchi, L. R. (1997), “On Criticisms and Comparisons of Default Bayes Factors for Model Selection and Hypothesis Testing” (with discussion), in Proceedings of the Workshop on Model Selection, ed. W. Racugno, Bologna: Pitagora Editrice, pp. 1–50.
9. Berger, J. O., and Pericchi, L. R. (1998), “Accurate and Stable Bayesian Model Selection: The Median Intrinsic Bayes Factor,” Sankhyā, Ser. B, 60, 1–18.
10. Bernardo, J. M., and Rueda, R. (2002), “Bayesian Hypothesis Testing: A Reference Approach,” International Statistical Review, 70, 351–372.
11. Bernardo, J. M., and Smith, A. F. M. (1994), Bayesian Theory, New York: Wiley.
12. Bertolino, F., and Racugno, W. (1997), “Is the Intrinsic Bayes Factor Intrinsic?” Metron, LIV, 5–15.
13. Box, G. E. P. (1980), “Sampling and Bayes’ Inference in Scientific Modelling and Robustness” (with discussion), Journal of the Royal Statistical Society, Ser. A, 143, 383–430.
14. Box, G. E. P., and Tiao, G. C. (1992), Bayesian Inference in Statistical Analysis, New York:Wiley.
15. Breiman, L. (1992), “The Little Bootstrap and OtherMethods for Dimensionality Selection in Regression: X-Fixed Prediction Error,” Journal of the American Statistical Association, 87, 738–754.
16. Breiman, L. (1995), “Better Subset Selection Using the Nonnegative Garrote,” Technometrics, 37, 373–384.
17. Brown, P. J., Vannucci, M., and Fearn, T. (1998), “Multivariate Bayesian Variable Selection and Prediction,” Journal of the Royal Statistical Society, Ser. B, 60, 627–641.
18. Brown, P. J., Fearn, T., and Vannucci, M. (1999), “The Choice of Variables in Multivariate Regression: A Non-Conjugate Bayesian Decision Theory Approach,” Biometrika, 86, 635–648.
19. Carlin, B. P., and Chib, S. (1995), “Bayesian Model Choice via Markov Chain Monte Carlo Methods,” Journal of the Royal Statistical Society, Ser. B, 57, 473–484.
20. Carlin, B. P., and Polson, N. G. (1991), “Inference for Nonconjugate Bayesian Models Using the Gibbs Sampler,” Canadian Journal of Statistics, 19, 399–405.
21. Chib, S. (1995), “Marginal Likelihood From the Gibbs Output,” Journal of the American Statistical Association, 90, 1313–1321.
22. DiCiccio, T. J., Kass, R. E., Raftery, A., andWasserman, L. (1997), “Computing Bayes Factors by Combining Simulation and Asymptotic Approximations,” Journal of the American Statistical Association, 92, 903–915.
23. Dickey, J. M., Dawid, A. P., and Kadane, J. B. (1986), “Subjective-Probability Assessment Methods for Multivariate-t and Matrix-t Models,” in Bayesian Inference and Decision Techniques, eds. P. K. Goel and A. Zellner, Amsterdam: Elsevier, pp. 177–195.
24. Donoho, D. L., and Johnstone, I. M. (1994), “Ideal Spatial Adaptation by Wavelet Shrinkage,” Biometrika, 81, 425–455.
25. Draper, D. (1995), “Assessment and Propagation of Model Uncertainty” (with discussion), Journal of the Royal Statistical Society, Ser. B, 57, 45–97.
26. Draper, N., and Smith, H. (1981), Applied Regression Analysis (2nd ed.), New York: Wiley.
27. Efron, B. (1979), “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, 7, 1–26.
28. Efron, B. (1982), The Jackknife, the Bootstrap and Other Resampling Plans, Philadelphia: Society for Industrial and Applied Mathematics.
29. Foster, D. P., and George, E. I. (1994), “The Risk Inflation Criterion for Multiple Regression,” The Annals of Statistics, 22, 1947–1975.
30. Furnival, G., and Wilson, R. (1974), “Regression by Leaps and Bounds,” Technometrics, 16, 499–511.
31. Garthwaite, P. H., and Al-Awadhi, S. A. (2001), “Non-Conjugate Prior Distribution Assessment for Multivariate Normal Sampling,” Journal of the Royal Statistical Society, Ser. B, 63, 95–110.
32. Garthwaite, P. H., and Dickey, J. M. (1992), “Elicitation of Prior Distributions for Variable-Selection Problems in Regression,” The Annals of Statistics, 20, 1697–1719.
33. Geisser, S., and Eddy, W. F. (1979), “A Predictive Approach to Model Selection,” Journal of the American Statistical Association, 74, 153–160.
34. Gelfand, A. E., and Dey, D. K. (1994), “Bayesian Model Choice: Asymptotics and Exact Calculations,” Journal of the Royal Statistical Society, Ser. B, 56, 501–514.
35. Gelfand, A. E., and Ghosh, S. K. (1998), “Model Choice: A Minimum Posterior Predictive Loss Approach,” Biometrika, 85, 1–11.
36. Gelman, A., and Meng, X. L. (1998), “Simulation Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling,” Statistical Science, 13, 163–185.
37. George, E. I. (2000), “The Variable Selection Procedure,” Journal of the American Statistical Association, 95, 1304–1308.
38. George, E. I., and Foster, D. P. (2000), “Calibration and Empirical Bayes Variable Selection,” Biometrika, 87, 731–747.
39. George, E. I., and McCullogh, R. E. (1993), “Variable Selection via Gibbs Sampling,” Journal of the American Statistical Association, 88, 881–889.
40. George, E. I., and McCullogh, R. E. (1997), “Approaches for Bayesian Variable Selection,” Statistica Sinica, 7, 339–373.
41. Geweke, J. F., and Meese, R. A. (1981), “Estimating Regression Models of Finite but Unknown Order,” International Economics Review, 22, 55–70.
42. Goldstein, M. (1991), Discussion of “Posterior Bayes Factors,” by M. Aitkin, Journal of the Royal Statistical Society, Ser. B, 53, 134.
43. Good, I. J. (1950), Probability and the Weighting of Evidence, London: Charles W. Griffin.
44. Graybill, F. A. (1976), Theory and Application of the Linear Model, Belmont, CA: Wadsworth.
45. Green, P. J. (1995), “Reversible JumpMarkov ChainMonte Carlo Computation and Bayesian Model Determination,” Biometrika, 82, 711–732.
46. Han, C., and Carlin, B. P. (2001), “Markov Chain Monte Carlo Methods for Computing Bayes Factors: A Comparative Review,” Journal of the American Statistical Association, 96, 1122–1132.
47. Hastie, T. J., and Pregibon, D. (1992), “Generalized Linear Models,” in Statistical Models in S, eds. J. M. Chambers and T. J. Hastie, Belmont, CA: Wadsworth.
48. Jeffreys, H. (1961), Theory of Probability (3rd ed.), Cambridge, MA; New York: Oxford University Press.
49. Kadane, J. B., and Dickey, J. M. (1980), “Bayesian Decision Theory and the Simplification of Models,” in Evaluation of Econometric Models, eds. J. Kmenta and J. Ramsey, New York: Academic Press, pp. 245–268.
50. Kadane, J. B., and Wolfson, L. J. (1998), “Experiences in Elicitation,” The Statistician, 47, 3–19.
51. Kadane, J. B., Dickey, J. M., Winkler, R. L., Smith, W. S., and Peters, S. C. (1980), “Interactive Elicitation of Opinion for a Normal Linear Model,” Journal of the American Statistical Association, 75, 845–854.
52. Kass, R. E., and Raftery, A. E. (1995), “Bayes Factors,” Journal of the American Statistical Association, 90, 773–795.
53. Katz, R. W. (1981),”On Some Criteria for Estimating the Order of a Markov Chain,” Technometrics, 23, 243–249.
54. Kennard, R. W. (1971), “A Note on the Cp Statistic,” Technometrics, 13, 899–900.
55. Key, J. T., Pericchi, L. R., and Smith, A. F. M. (1999), “Bayesian Model Choice: What and Why?” (with discussion), in Bayesian Statistics (Vol. 6), eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Cambridge, MA; New York: Oxford University Press, pp. 343–370.
56. Koehler, A. B., and Murphree, E. S. (1988), “A Comparison of the Akaike and Schwarz Criteria for Selecting Model Order,” Applied Statistics, 37, 187–195.
57. Kuo, L., and Mallick, B. (1998), “Variable Selection for Regression Models,” Sankhyā, 60, 65–81.
58. Laud, P. W., and Ibrahim, J. G. (1995), “Predictive Model Selection,” Journal of the Royal Statistical Society, Ser. B, 57, 247–262.
59. Lawless, J. F., and Singhal, K. (1978), “Efficient Screening of Non-Normal Regression Models,” Biometrics, 43, 318–327.
60. Leamer, E. E. (1978), Specification Searches, New York: Wiley.
61. Lindley, D. V. (1957), “A Statistical Paradox,” Biometrika, 44, 187–192.
62. Lindley, D. V. (1968), “The Choice of Variables in Multiple Regression” (with discussion), Journal of the Royal Statistical Association, Ser. B, 30, 31–66.
63. Lindley, D. V. (1976), “A Class of Utility Functions,” The Annals of Statistics, 4, 1–10.
64. Lindley, D. V. (1991), Discussion of “Posterior Bayes Factors,” by M. Aitken, Journal of the Royal Statistical Society, Ser. B, 53, 130–131.
65. Lindley, D. V. (1997), “Some Comments on Bayes Factors,” Journal of Statistical Planning and Inference, 61, 181–189.
66. MacKay, D. J. C. (1992), “Bayesian Interpolation,” Neural Computation, 4, 415–447.
67. Madigan, D., and Raftery, A. E. (1994), “Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam’s Window,” Journal of the American Statistical Association, 89, 1535–1546.
68. Madigan, D., and York, J. (1995), “Bayesian Graphical Models for Discrete Data,” International Statistical Review, 63, 215–232.
69. Mallows, C. L. (1973), “Some Comments on Cp,” Technometrics, 15, 661–675.
70. Mallows, C. L. (1995), “More comments on Cp,” Technometrics, 37, 362–372.
71. Meng, X. L., andWong, W. H. (1996), “Simulating Ratios of Normalizing Constants via a Simple Identity: A Theoretical Exploration,” Statistica Sinica, 6, 831–860.
72. Miller, A. J. (2002), Subset Selection in Regression (2nd ed.), London; New York: Chapman & Hall.
73. Mitchell, T. J., and Beauchamp, J. J. (1988), “Bayesian Variable Selection in Linear Regression” (with discussion), Journal of the American Statistical Association, 83, 1023–1036.
74. O’Hagan, A. (1991), Discussion of “Posterior Bayes Factors,” by M. Aitkin, Journal of the Royal Statistical Society, Ser. B, 53, 136.
75. O’Hagan, A. (1995), “Fractional Bayes Factors for Model Comparisons” (with discussion), Journal of the Royal Statistical Society, Ser. B, 57, 99–138.
76. O’Hagan, A. (1997), “Properties of Intrinsic and Fractional Bayes Factors,” Test, 6, 101–118.
77. O’Hagan, A. (1998), “Eliciting Expert Beliefs in Substantial Practical Applications,” Statistician, 47, 21–35.
78. Petrone, S. (1997), Discussion of “Choosing Among Models When None of Them Are True,” in Proceedings of the Workshop on Model Selection, ed. W. Racugno, Bologna: Pitagora Editrice, pp. 355–358.
79. Piccinato, L. (1997), Discussion of “Choosing Among Models When None of Them Are True,” in Proceedings of the Workshop on Model Selection, ed. W. Racugno, Bologna: Pitagora Editrice, pp. 350–354.
80. Raftery, A. E., Madigan, D., and Hoeting, J. A. (1997), “Bayesian Model Averaging for Linear Regression Models,” Journal of the American Statistical Association, 92, 179–191.
81. Richardson, S., and Green, P. J. (1997), “On Bayesian Analysis of Mixtures With an Unknown Number of Components” (with discussion), Journal of the Royal Statistical Society, Ser. B, 59, 731–792.
82. San Martini, A., and Spezzaferri, F. (1984), “A Predictive Model Selection Criterion,” Journal of the Royal Statistical Society, Ser. B, 46, 296–303.
83. Schwarz, G. (1978), “Estimating the Dimension of a Model,” The Annals of Statistics, 6, 461–464.
84. Shafer, G. (1982), “Lindley’s Paradox,” Journal of the American Statistical Association, 77, 325–351.
85. Smith, A. F. M., and Spiegelhalter, D. J. (1980), “Bayes Factors and Choice Criteria for Linear Models,” Journal of the Royal Statistical Society, Ser. B, 42, 213–220.
86. Stone, M. (1974), “Cross-Validatory Choice and Assessment of Statistical Predictions” (with discussion), Journal of the Royal Statistical Society, Ser. B, 36, 111–147.
87. Tibshirani, R. (1996), “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society, Ser. B, 58, 267–288.
88. Tibshirani, R., and Knight, K. (1999), “The Covariance Inflation Criterion for Model Selection,” Journal of the Royal Statistical Society, Ser. B, 61, 529–546.
89. Weisberg, S. (1985), Applied Linear Regression (2nd ed.), New York: Wiley.
90. Whittaker, J. (1990), Graphical Models in Applied Multivariate Statistics, New York:Wiley.
91. Winkler, R. L. (1999), Discussion of “Bayesian Model Choice: What and Why?” by J. T. Key, L. R. Pericchi, and A. F. M. Smith, in Bayesian Statistics (Vol. 6), eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Cambridge, MA; New York: Oxford University Press, pp. 367–368.