Objective Bayesian Model Selection in Generalized Additive Models With Penalized Splines

Journal of Computational and Graphical Statistics

Volumn 24, Issue 2, 2015, Pages 394-415

  Bové, Daniel Sabanés ^a   Held, Leonhard ^b   Kauermann, Göran ^c  

^a F HOFFMANN LA ROCHE LTD   (Switzerland)

^b UNIVERSITY OF ZURICH   (Switzerland)

^c UNIVERSITY OF MUNICH   (Germany)

Author keywords

Function selection; g prior; Shrinkage; Stochastic search; Variable selection

Indexed keywords

EID: 84931068928     PISSN: 10618600     EISSN: 15372715     Source Type: Journal    
DOI: 10.1080/10618600.2014.912136     Document Type: Article

References (72)

1. M.Abrahamowicz,, T.MacKenzie,, and J.M.Esdaile, (1996), “Time-Dependent Hazard Ratio: Modeling and Hypothesis Testing With Application in Lupus Nephritis,” Journal of the American Statistical Association, 91, 1432–1439.
2. M.Abramowitz,, and I.A.Stegun, (1964), Handbook of Mathematical Functions With Formulas, Graphs, and Mathematical Tables (9th Dover printing, 10th GPO printing), New York: Dover.
3. M.Aerts,, G.Claeskens,, and M.P.Wand, (2002), “Some Theory for Penalized Spline Generalized Additive Models,” Journal of Statistical Planning and Inference, 103, 455–470.
4. M.P.Appell, (1925), “Sur les Fonctions Hypergeometriques de Plusieurs Variables, les Polynomes d’Hermite et Autres Fonctions Spheriques dans l’Hyperespace,” Memorial des Sciences Mathematiques, 3, 1–75.
5. M.M.Barbieri,, and J.O.Berger, (2004), “Optimal Predictive Model Selection,” The Annals of Statistics, 32, 870–897.
6. M.J.Bayarri,, J.O.Berger,, A.Forte,, and G.García-Donato, (2012), “Criteria for Bayesian Model Choice With Application to Variable Selection,” The Annals of Statistics, 40, 1550–1577.
7. C.Belitz,, and S.Lang, (2008), “Simultaneous Selection of Variables and Smoothing Parameters in Structured Additive Regression Models,” Computational Statistics and Data Analysis, 53, 61–81.
8. J.O.Berger,, and L.R.Pericchi, (2001), “Objective Bayesian Methods for Model Selection: Introduction and Comparison,” in Model Selection, ed. P.Lahiri, Beachwood, OH: Institute of Mathematical Statistics, vol. 38 of IMS Lecture Notes, pp. 135–207.
9. J.M.Bernardo, (1979), “Reference Posterior Distributions for Bayesian Inference,” Journal of the Royal Statistical Society, Series B, 41, 113–147.
10. J.Besag,, P.Green,, D.Higdon,, and K.Mengersen, (1995), “Bayesian Computation and Stochastic Systems” (with discussion), Statistical Science, 10, 3–66.
11. Å.Björck, (1967), “Solving Linear Least Squares Problems by Gram–Schmidt Orthogonalization,” BIT Numerical Mathematics, 7, 1–21.
12. N.E.Breslow,, and D.G.Clayton, (1993), “Approximate Inference in Generalized Linear Mixed Models,” Journal of the American Statistical Association, 88, 9–25.
13. A.Brezger,, and S.Lang, (2008), “Simultaneous Probability Statements for Bayesian P-Splines,” Statistical Modelling, 8, 141–168.
14. E.Cantoni,, and T.Hastie, (2002), “Degrees-of-Freedom Tests for Smoothing Splines,” Biometrika, 89, 251–263.
15. G.Celeux,, M.E.Anbari,, J.-M.Marin,, and C.P.Robert, (2012), “Regularization in Regression: Comparing Bayesian and Frequentist Methods in a Poorly Informative Situation,” Bayesian Analysis, 7, 477–502.
16. S.Chib,, and I.Jeliazkov, (2001), “Marginal Likelihood from the Metropolis-Hastings Output,” Journal of the American Statistical Association, 96, 270–281.
17. F.Colavecchia,, and G.Gasaneo, (2004), “f1: A Code to Compute Appell’s F1 Hypergeometric Function,” Computer Physics Communications, 157, 32–38.
18. R.Cottet,, R.J.Kohn,, and D.J.Nott, (2008), “Variable Selection and Model Averaging in Semiparametric Overdispersed Generalized Linear Models,” Journal of the American Statistical Association, 103, 661–671.
19. W.Cui,, and E.I.George, (2008), “Empirical Bayes vs. Fully Bayes Variable Selection,” Journal of Statistical Planning and Inference, 138, 888–900.
20. D.G.T.Denison,, C.C.Holmes,, B.K.Mallick,, and A.F.M.Smith, (2002), Bayesian Methods for Nonlinear Classification and Regression, Wiley Series in Probability and Statistics, Chichester: Wiley.
21. D.G.T.Denison,, B.K.Mallick,, and A.F.M.Smith, (1998), “Automatic Bayesian Curve Fitting,” Journal of the Royal Statistical Society, Series B, 60, 333–350.
22. P.H.C.Eilers,, and B.D.Marx, (2010), “Splines, Knots, and Penalties,” Wiley Interdisciplinary Reviews Computational Statistics, 2, 637–653.
23. L.Fahrmeir,, T.Kneib,, and S.Konrath, (2010), “Bayesian Regularisation in Structured Additive Regression: A Unifying Perspective on Shrinkage, Smoothing and Predictor Selection,” Statistics and Computing, 20, 203–219.
24. L.Fahrmeir,, T.Kneib,, and S.Lang, (2004), “Penalized Structured Additive Regression for Space-Time Data: A Bayesian Perspective,” Statistica Sinica, 14, 715–745.
25. Y.Fong,, H.Rue,, and J.Wakefield, (2010), “Bayesian Inference for Generalized Linear Mixed Models,” Biostatistics, 11, 397–412.
26. J.Forster,, R.Gill,, and A.Overstall, (2012), “Reversible Jump Methods for Generalized Linear Models and Generalized Linear Mixed Models,” Statistics and Computing, 22, 107–120.
27. A.Forte, (2011), “Objective Bayes Criteria for Variable Selection,” Ph.D. thesis, Universitat de Valencia. Available at https://www.educacion.gob.es/teseo/imprimirFicheroTesis.do?fichero=22234
28. A.Frank,, and A.Asuncion, (2010), “UCI Machine Learning Repository,” available at http://archive.ics.uci.edu/ml
29. J.H.Friedman, (2001), “Greedy Function Approximation: A Gradient Boosting Machine,” The Annals of Statistics, 29, 1189–1232.
30. D.Gamerman, (1997), “Sampling from the Posterior Distribution in Generalized Linear Mixed Models,” Statistics and Computing, 7, 57–68.
31. E.I.George,, and R.E.McCulloch, (1993), “Variable Selection via Gibbs Sampling,” Journal of the American Statistical Association, 88, 881–889.
32. T.J.Hastie,, and R.J.Tibshirani, (1990), Generalized Additive Models, London: Chapman and Hall.
33. L.Held, (2004), “Simultaneous Posterior Probability Statements from Monte Carlo Output,” Journal of Computational and Graphical Statistics, 13, 20–35.
34. H.V.Henderson,, and S.R.Searle, (1981), “On Deriving the Inverse of a Sum of Matrices,” SIAM Review, 23, 53–60.
35. J.A.Hoeting,, D.Madigan,, A.E.Raftery,, and C.T.Volinsky, (1999), “Bayesian Model Averaging: A Tutorial,” Statistical Science, 14, 382–417.
36. C.C.Holmes,, and L.Held, (2006), “Bayesian Auxiliary Variable Models for Binary and Multinomial Regression,” Bayesian Analysis, 1, 145–168.
37. G.Kauermann,, T.Krivobokova,, and L.Fahrmeir, (2009), “Some Asymptotic Results on Generalized Penalized Spline Smoothing,” Journal of the Royal Statistical Society, Series B, 71, 487–503.
38. G.Kauermann,, and G.Tutz, (2001), “Testing Generalized Linear and Semiparametric Models Against Smooth Alternatives,” Journal of the Royal Statistical Society, Series B, 63, 147–166.
39. T.Kneib,, T.Hothorn,, and G.Tutz, (2009), “Variable Selection and Model Choice in Geoadditive Regression Models,” Biometrics, 65, 626–634.
40. E.Ley,, and M.F.Steel, (2009), “On the Effect of Prior Assumptions in Bayesian Model Averaging With Applications to Growth Regression,” Journal of Applied Econometrics, 24, 651–674.
41. E.Ley,, and M.F.Steel, (2012), “Mixtures of g-Priors for Bayesian Model Averaging With Economic Applications,” Journal of Econometrics, 171, 251–266.
42. F.Liang,, R.Paulo,, G.Molina,, M.A.Clyde,, and J.O.Berger, (2008), “Mixtures of g Priors for Bayesian Variable Selection,” Journal of the American Statistical Association, 103, 410–423.
43. D.Madigan,, and J.York, (1995), “Bayesian Graphical Models for Discrete Data,” International Statistical Review, 63, 215–232.
44. G.Marra,, and S.N.Wood, (2011), “Practical Variable Selection for Generalized Additive Models,” Computational Statistics and Data Analysis, 55, 2372–2387.
45. P.McCullagh,, and J.A.Nelder, (1989), Generalized Linear Models (2nd ed.), no. 37 in Monographs on Statistics and Applied Probability, New York: Chapman and Hall.
46. L.Meier,, S.van de Geer,, and P.Bühlmann, (2009), “High-Dimensional Additive Modeling,” The Annals of Statistics, 37, 3779–3821.
47. A.M.Overstall,, and J.J.Forster, (2010), “Default Bayesian Model Determination Methods for Generalized Linear Mixed Models,” Computational Statistics and Data Analysis, 54, 3269–3288.
48. D.K.Pauler, (1998), “The Schwarz Criterion and Related Methods for Normal Linear Models,” Biometrika, 85, 13–27.
49. P.Peduzzi,, J.Concato,, E.Kemper,, T.Holford,, and A.Feinstein, (1996), “A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis,” Journal of Clinical Epidemiology, 49, 1373–1379.
50. P.Ravikumar,, H.Liu,, J.Lafferty,, and L.Wasserman, (2008), “SpAM: Sparse Additive Models,” in Advances in Neural Information Processing Systems (Vol. 20), eds. J.Platt, D.Koller, Y.Singer, and S.Roweis, Cambridge, MA: MIT Press, pp. 1201–1208.
51. B.D.Ripley, (1996), Pattern Recognition and Neural Networks, Cambridge: Cambridge University Press.
52. C.Ritter,, and M.A.Tanner, (1992), “Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy–Gibbs Sampler,” Journal of the American Statistical Association, 87, 861–868.
53. H.Rue,, S.Martino,, and N.Chopin, (2009), “Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations,” Journal of the Royal Statistical Society, Series B, 71, 319–392.
54. D.Ruppert,, M.P.Wand,, and R.J.Carroll, (2003), Semiparametric Regression, Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge: Cambridge University Press.
55. Sabanés Bové, D. (2012), appell: Compute Appell’s F1 hypergeometric function, R package version 0.0-3. Available at http://cran.r-project.org/web/packages/appell/
56. L.Held, (2011a), “Bayesian Fractional Polynomials,” Statistics and Computing, 21, 309–324.
57. L.Held, (2011b), “Hyper-g Priors for Generalized Linear Models,” Bayesian Analysis, 6, 387–410.
58. F.Scheipl,, L.Fahrmeir,, and T.Kneib, (2012), “Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models,” Journal of the American Statistical Association, 107, 1518–1532.
59. F.Scheipl,, T.Kneib,, and L.Fahrmeir, (2013), “Penalized Likelihood and Bayesian Function Selection in Regression Models,” Advances in Statistical Analysis, 97, 349–385.
60. J.G.Scott,, and J.O.Berger, (2010), “Bayes and Empirical-Bayes Multiplicity Adjustment in the Variable-Selection Problem,” The Annals of Statistics, 38, 2587–2619.
61. M.Smith,, and R.Kohn, (1996), “Nonparametric Regression Using Bayesian Variable Selection,” Journal of Econometrics, 75, 317–343.
62. G.Tutz,, and H.Binder, (2006), “Generalized Additive Modeling With Implicit Variable Selection by Likelihood-Based Boosting,” Biometrics, 62, 961–971.
63. M.P.Wand, (2003), “Smoothing and Mixed Models,” Computational Statistics, 18, 223–249.
64. M.P.Wand,, and J.T.Ormerod, (2008), “On Semiparametric Regression With O’Sullivan Penalized Splines,” Australian and New Zealand Journal of Statistics, 50, 179–198.
65. M.West, (1985), “Generalized Linear Models: Scale Parameters, Outlier Accommodation and Prior Distributions,” in Bayesian Statistics (Vol. 2), eds. J.M.Bernardo, M.H.DeGroot, D.V.Lindley, and A.F.M.Smith, Amsterdam: North-Holland, pp. 531–558.
66. S.N.Wood, (2006), Generalized Additive Models: An Introduction With R, Boca Raton: Chapman & Hall/ CRC.
67. S.N.Wood, (2011), “Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models,” Journal of the Royal Statistical Society, Series B, 73, 3–36.
68. P.Yau,, R.J.Kohn,, and S.Wood, (2003), “Bayesian Variable Selection and Model Averaging in High-Dimensional Multinomial Nonparametric Regression,” Journal of Computational and Graphical Statistics, 12, 23–54.
69. A.Zellner, (1986), “On Assessing Prior Distributions and Bayesian Regression Analysis With g-Prior Distributions,” in Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, eds. P.K.Goel, and A.Zellner, Amsterdam: North-Holland, Vol. 6 of Studies in Bayesian Econometrics and Statistics, chap. 5, pp. 233–243.
70. A.Zellner,, and A.Siow, (1980), “Posterior Odds Ratios for Selected Regression Hypotheses,” in Bayesian Statistics: Proceedings of the First International Meeting Held in Valencia, eds. J.M.Bernardo, M.H.DeGroot, D.V.Lindley, and A.F.M.Smith, Valencia: University of Valencia Press, pp. 585–603.
71. H.H.Zhang,, and Y.Lin, (2006), “Component Selection and Smoothing for Nonparametric Regression in Exponential Families,” Statistica Sinica, 16, 1021–1041.
72. H.Zou,, and T.Hastie, (2005), “Regularization and Variable Selection via the Elastic Net,” Journal of the Royal Statistical Society, Series B, 67, 301–320.