Automatic Bayesian model averaging for linear regression and applications in Bayesian curve fitting

Statistica Sinica

Volumn 11, Issue 4, 2001, Pages 1005-1029

  Liang, Faming ^a   Truong, Young K ^a   Wong, Wing Hung ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b HARVARD SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

Bayesian model averaging; Curve fitting; Evolutionary Monte Carlo; Mallows' Cp; Markov chain Monte Carlo

Indexed keywords

EID: 0347666339     PISSN: 10170405     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (60)

1. Akaike, H. (1973). Information theory and the extension of the maximum likelihood principle. In Proc. Int. Symp. Information Theory (Edited by B. N. Petrov and F. Czáki), 267-281. Akademia Kiadoó, Budapest.
2. Allen, D. M. (1974). The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16, 125-127.
3. Berger, J. O. and Pericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction. J. Amer. Statist. Assoc. 91, 109-122.
4. Carlin, B. P. and Chib, S. (1995). Bayesian model choice via Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 57, 473-484.
5. Clyde, M. (1999). Bayesian model averaging and model search strategies. In Bayesian Statistics 6 (Edited by J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), University Press, Oxford.
6. David, A. P. (1984). Statistical theory - the frequential approach. J. Roy. Statist. Soc. Ser. A 147, 278-292.
7. Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998). Automatic Bayesian curve fitting. J. Roy. Statist. Soc. Ser. B 60, 333-350.
8. Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion). J. Roy. Statist. Soc. Ser. B 57, 45-97.
9. Ehrlich, L. (1973). Participation in illegitimate activities: a theoretical and empirical investigation. J. Political Economy 81, 521-567.
10. Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing. 2nd edition. Marcel Dekker, New York.
11. Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57, 371-394.
12. Fernández, G., Ley, E. and Steel, M. F. J. (2001). Benchmark priors for Bayesian model averaging. J. Econometrics, 100, 381-427.
13. Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22, 1947-1975.
14. Friedman, J. H. (1991). Multivariate adaptive regression splines. Ann. Statist. 19, 1-141.
15. Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 31, 3-21.
16. Geisser, S. and Eddy, W. (1979). A predictive approach to model selection. J. Amer. Statist. Assoc. 74, 153-160.
17. Gelfand, A. E. (1995). Model determination using sampling-based methods. In Markov Chain Monte Carlo in Practice (Edited by W. R. Gilks, S. Richardson and D. J. Spiegelhalter), 145-161. Chapman and Hall, London.
18. Gelfand, A. E., Dey, D. K. and Chang, H. (1992). Model determination using predictive distributions with implementation via sampling-based methods. In Bayesian Statistics 4 (Edited by J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), 147-167. Oxford Press.
19. Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721-741.
20. George, E. I. (1999). Bayesian model selection. In Encyclopedia of Statistical Science Update 3 (Edited by S. Kotz, C. Read and D. Banks), 39-46. Wiley, New York.
21. George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88, 881-889.
22. George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7, 339-373.
23. Geweke, J. (1996). Variable selection and model comparison in regression. In Bayesian Statistics 5 (Edited by J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), 609-620. Oxford Press.
24. Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface (Edited by E. M. Keramigas), 153-163. Interface Foundation, Fairfax Station.
25. Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization, & Machine Learning. Addison Wesley.
26. Good, I. J. (1950). Probability and the Weighting of Evidence. Charles Griffin, London.
27. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711-732.
28. Hansen, M. and Kooperberg, C. (2000). Spline adaptation in extended linear models. Statist. Sci., tentatively accepted.
29. Hansen, M., Kooperberg, C. and Sardy, S. (1998). Triogram models. J. Amer. Statist. Assoc. 93, 101-119.
30. Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall, London.
31. Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97-109.
32. Hocking, R. R. (1976). The analysis and selection of variables in linear regression. Biometrics 32, 1-49.
33. Hoeting, J. A., Madigan, D., Raftery, A. E. and Volinsky, C. (1999). Bayesian model averaging: a tutorial (with discussion). Statist. Sci. 14, 382-417.
34. Holland, J. H. (1975). Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor.
35. Hukushima K. and Nemoto, K. (1996). Exchange Monte Carlo method and application to spin glass simulations. J. Phys. Soc. Jpn. 65, 1604-1608.
36. Kass, R. E. and Raftery, A. E. (1995). Bayes factors. J. Amer. Statist. Assoc. 90, 773-795.
37. Kirkpatrick, S., Gelatt, Jr., C. D. and Vecchi, M. P. (1983). Optimization by simulated annealing. Science 220, 671-680.
38. Kohn, R. and Ansley, C. (1987). A new algorithm for spline smoothing based on smoothing a stochastic process. SIAM J. Sci. Statist. Comput. 8, 33-48.
39. Kohn, R., Ansley, C. and Tharm, D. (1991). The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Amer. Stat. Assoc. 86, 1042-1050.
40. Koo, J. Y. (1997). Spline estimation of discontinuous regression functions. J. Comput. Graph. Statist. 6, 266-284.
41. Kooperberg, C. and Stone, C. (1992). Logspline density estimation for censored data. J. Comput. Graph. Statist. 1, 301-328.
42. Laud, P. W. and Ibrahim, J. G. (1995). Predictive model selection. J. Roy. Statist. Soc. Ser. B 57, 247-262.
43. p model sampling and change point problem. Statist. Sinica 10, 317-342.
44. Madigan, D. and Raftery, A. E. (1994). Model selection and accounting for model uncertainty in graphical models using Occam's window. J. Amer. Statist. Assoc. 89, 1535-1546.
45. Madigan, D. and York, J. (1995). Bayesian graphical models for discrete data. Internat. Statist. Rev. 63, 215-232.
46. p. Technometrics 15, 661-676.
47. p. Technometrics 37, 362-372.
48. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087-1091.
49. Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian variable selection in linear regression (with discussion). J. Amer. Statist. Assoc. 83, 1023-1036.
50. Müller, H. G. and Stadtmüller, U. (1987). Variable bandwidth kernel estimators of regression curves. Ann. Statist. 15, 182-201.
51. O'Hagan, A. (1995). Fractional Bayes factor for model comparison (with discussion). J. Roy. Statist. Soc. Ser. B 57, 99-138.
52. Phillips, D. B. and Smith, A. F. M. (1995). Bayesian model comparison via jump diffusions. In Markov Chain Monte Carlo in Practice (Edited by W. R. Gilks, S. Richardson and D. J. Spiegelhalter), 215-239. Chapman and Hall, London.
53. Raftery, A. E. (1996). Approximate Bayes factor and accounting for model uncertainty in generalized linear models. Biometrika 83, 251-266.
54. Raftery, A. E., Madigan, D. and Hoeting, J. A. (1997). Bayesian model averaging for linear regression models. J. Amer. Statist. Assoc. 92, 179-191.
55. San Martini, A. and Spezzaferri, F. (1984). A predictive model selection criterion. J. Roy. Statist. Soc. Ser. B 46, 296-303.
56. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6, 461-464.
57. Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Ann. Statist. 22, 1701-1762.
58. Vandaele, W. (1978). Participation in illegitimate activities: Ehrlich revisited. In Dererrence and Incapacitation (Edited by A. Blumstein, J. Cohen and D. Nagin), 270-335. National Academy of Science Press, Washington.
59. Wang, Y. (1995). Jump and sharp cusp detection by wavelets. Biometrika 82, 385-397.
60. Zeller, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti (Edited by P. K. Goel and A. Zellner), 233-243. North-Holland, Amsterdam.