Smoothing Parameter and Model Selection for General Smooth Models

Journal of the American Statistical Association

Volumn 111, Issue 516, 2016, Pages 1548-1563

  Wood, Simon N ^a   Pya, Natalya ^b,c   Säfken, Benjamin ^d  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b NAZARBAYEV UNIVERSITY   (Kazakhstan)

^c KIMEP UNIVERSITY   (Kazakhstan)

^d UNIVERSITY OF GÖTTINGEN   (Germany)

Author keywords

Additive model; AIC; Distributional regression; GAM; Location scale and shape model; Ordered categorical regression; Penalized regression spline; REML; Smooth Cox model; Smoothing parameter uncertainty; Statistical algorithm; Tweedie distribution

Indexed keywords

EID: 85010676878     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1080/01621459.2016.1180986     Document Type: Article

References (44)

1. Adam, B.-L., Qu, Y., Davis, J. W., Ward, M. D., Clements, M. A., Cazares, L. H., Semmes, O. J., Schellhammer, P. F., Yasui, Y., Feng, Z., et al. (2002), “Serum Protein Fingerprinting Coupled With a Pattern-Matching Algorithm Distinguishes Prostate Cancer From Benign Prostate Hyperplasia and Healthy Men,” Cancer Research, 62, 3609–3614.
2. Akaike, H., (1973), “Information Theory and an Extension of the Maximum Likelihood Principle,” in International Symposium on Information Theory, eds. B. Petran, and F. Csaaki, Budapest:Akadeemiai Kiadi, pp. 267–281.
3. Anderssen, R., and Bloomfield, P., (1974), “A Time Series Approach to Numerical Differentiation,” Technometrics, 16, 69–75.
4. Augustin, N. H., Sauleau, E.-A., and Wood, S. N., (2012), “On Quantile Quantile Plots for Generalized Linear Models,” Computational Statistics & Data Analysis, 56, 2404–2409.
5. Bache, K., and Lichman, M., (2013), “UCI Machine Learning Repository,” available at
6. Belitz, C., Brezger, A., Kneib, T., Lang, S., and Umlauf, N., (2015), “Bayesx:Software for Bayesian Inference in Structured Additive Regression Models,” available at http://www.statistik.lmu.de/~bayesx/bayesx.html.
7. Breslow, N. E., and Clayton, D. G., (1993), “Approximate Inference in Generalized Linear Mixed Models,” Journal of the American Statistical Association, 88, 9–25.
8. Brezger, A., and Lang, S., (2006), “Generalized Structured Additive Regression Based on Bayesian p-Splines,” Computational Statistics & Data Analysis, 50, 967–991.
9. Cline, A. K., Moler, C. B., Stewart, G. W., and Wilkinson, J. H., (1979), “An Estimate for the Condition Number of a Matrix,” SIAM Journal on Numerical Analysis, 16, 368–375.
10. Cox, D., (1972), “Regression Models and Life Tables,” Journal of the Royal Statistical Society, Series B, 34, 187–220.
11. Davison, A. C., (2003), Statistical Models, Cambridge, UK:Cambridge University Press.
12. Eilers, P. H. C., and Marx, B. D., (1996), “Flexible Smoothing With B-Splines and Penalties,” Statistical Science 11, 89–121.
13. Fahrmeir, L., Kneib, T., and Lang, S., (2004), “Penalized Structured Additive Regression for Space-Time Data:A Bayesian Perspective,” Statistica Sinica, 14, 731–761.
14. Fahrmeir, L., Kneib, T., Lang, S., and Marx, B., (2013), “Regression Models, New York:Springer.
15. Fahrmeir, L., and Lang, S., (2001), “Bayesian Inference for Generalized Additive Mixed Models Based on Markov Random Field Priors,” Applied Statistics, 50, 201–220.
16. Greven, S., and Kneib, T., (2010), “On the Behaviour of Marginal and Conditional AIC in Linear Mixed Models,” Biometrika, 97, 773–789.
17. Hastie, T., and Tibshirani, R., (1986), “Generalized Additive Models” (with discussion), Statistical Science, 1, 297–318.
18. Hastie, T., and Tibshirani, R., (1990), Generalized Additive Models, London:Chapman & Hall.
19. Kass, R. E., and Steffey, D., (1989), “Approximate Bayesian Inference in Conditionally Independent Hierarchical Models (Parametric Empirical Bayes Models),” Journal of the American Statistical Association, 84, 717–726.
20. Klein, N., Kneib, T., Klasen, S., and Lang, S., (2014), “Bayesian Structured Additive Distributional Regression for Multivariate Responses,” Journal of the Royal Statistical Society, Series C, 64, 569–591.
21. Klein, N., Kneib, T., Lang, S., and Sohn, A., (2015), “Bayesian Structured Additive Distributional Regression With an Application to Regional Income Inequality in Germany,” Annals of Applied Statistics, 9, 1024–1052.
22. Laird, N. M., and Ware, J. H., (1982), “Random-Effects Models for Longitudinal Data,” Biometrics, 38, 963–974.
23. Liang, H., Wu, H., and Zou, G., (2008), “A Note on Conditional AIC for Linear Mixed-Effects Models,” Biometrika, 95, 773–778.
24. Marra, G., and Wood, S. N., (2011), “Practical Variable Selection for Generalized Additive Models,” Computational Statistics & Data Analysis, 55, 2372–2387.
25. Marra, G., and Wood, S. N., (2012), “Coverage Properties of Confidence Intervals for Generalized Additive Model Components,” Scandinavian Journal of Statistics, 39, 53–74.
26. Marx, B. D., and Eilers, P. H., (1998), “Direct Generalized Additive Modeling With Penalized Likelihood,” Computational Statistics and Data Analysis, 28, 193–209.
27. Nocedal, J., and Wright, S., (2006), “Numerical Optimization (2nd ed.), New York:Springer Verlag.
28. Nychka, D., (1988), “Bayesian Confidence Intervals for Smoothing Splines,” Journal of the American Statistical Association, 83, 1134–1143.
29. Reiss, P. T., and Ogden, T. R., (2009), “Smoothing Parameter Selection for a Class of Semiparametric Linear Models,” Journal of the Royal Statistical Society, Series B, 71, 505–523.
30. Rigby, R., and Stasinopoulos, D. M., (2005), “Generalized Additive Models for Location, Scale and Shape,” Journal of the Royal Statistical Society, Series C, 54, 507–554.
31. Rigby, R. A., and Stasinopoulos, D. M., (2014), “Automatic Smoothing Parameter Selection in GAMLSS With an Application to Centile Estimation,” Statistical Methods in Medical Research, 23, 318–332.
32. Rue, H., Martino, S., and Chopin, N., (2009), “Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations,” Journal of the Royal Statistical Society, Series B, 71, 319–392.
33. Ruppert, D., Wand, M. P., and Carroll, R. J., (2003), Semiparametric Regression, New York:Cambridge University Press.
34. Säfken, B., Kneib, T., van Waveren, C.-S., and Greven, S., (2014), “A Unifying Approach to the Estimation of the Conditional Akaike Information in Generalized Linear Mixed Models,” Electronic Journal of Statistics, 8, 201–225.
35. Shun, Z., and McCullagh, P., (1995), “Laplace Approximation of High Dimensional Integrals,” Journal of the Royal Statistical Society, Series B, 57, 749–760.
36. Silverman, B. W., (1985), “Some Aspects of the Spline Smoothing Approach to Non-Parametric Regression Curve Fitting,” Journal of the Royal Statistical Society, Series B, 47, 1–53.
37. Umlauf, N., Adler, D., Kneib, T., Lang, S., and Zeileis, A., (2015), “Structured Additive Regression Models:An r Interface to Bayesx,” Journal of Statistical Software, 63, 1–46.
38. Wahba, G., (1983), “Bayesian Confidence Intervals for the Cross Validated Smoothing Spline,” Journal of the Royal Statistical Society, Series B, 45, 133–150.
39. Wahba, G., (1985), “A Comparison of GCV and GML for Choosing the Smoothing Parameter in the Generalized Spline Smoothing Problem,” The Annals of Statistics, pp. 1378–1402.
40. Wood, S. N., (2000), “Modelling and Smoothing Parameter Estimation With Multiple Quadratic Penalties,” Journal of the Royal Statistical Society, Series B, 62, 413–428.
41. Wood, S. N., (2006), “Low-Rank Scale-Invariant Tensor Product Smooths for Generalized Additive Mixed Models,” Biometrics, 62, 1025–1036.
42. Wood, S. N., (2011), “Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models,” Journal of the Royal Statistical Society, Series B, 73, 3–36.
43. Yee, T. W., and Wild, C., (1996), “Vector Generalized Additive Models,” Journal of the Royal Statistical Society, Series B, 481–493.
44. Yu, D., and Yau, K. K., (2012), “Conditional Akaike Information Criterion for Generalized Linear Mixed Models,” Computational Statistics & Data Analysis, 56, 629–644.