Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 73, Issue 1, 2011, Pages 3-36

  Wood, Simon N ^a  

^a UNIVERSITY OF BATH   (United Kingdom)

Author keywords

Adaptive smoothing; Generalized additive mixed model; Generalized additive model; Generalized cross validation; Marginal likelihood; Model selection; Penalized generalized linear model; Penalized regression splines; Restricted maximum likelihood; Scalar on function regression; Stable computation

Indexed keywords

EID: 78650862532     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/j.1467-9868.2010.00749.x     Document Type: Article

References (65)

1. Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Donngarra, J., Du Croz, J., Greenbaum, A., Hammerling, S., McKenney, A. and Sorenson, D. (1999) LAPACK Users' Guide, 3rd edn. Philadelphia: Society for Industrial and Applied Mathematics.
2. Anderssen, R. S. and Bloomfield, P. (1974) A time series approach to numerical differentiation. Technometrics, 16, 69-75.
3. Breslow, N. E. and Clayton, D. G. (1993) Approximate inference in generalized linear mixed models. J. Am. Statist. Ass., 88, 9-25.
4. Brezger, A., Kneib, T. and Lang, S. (2007) BayesX 1.5.0. University of Munich, Munich. (Available from.)
5. Brezger, A. and Lang, S. (2006) Generalized structured additive regression based on Bayesian P-splines. Computnl Statist. Data Anal., 50, 967-991.
6. Cline, A. K., Moler, C. B., Stewart, G. W. and Wilkinson, J. H. (1979) An estimate for the condition number of a matrix. SIAM J. Numer. Anal., 13, 293-309.
7. Craven, P. and Wahba, G. (1979) Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross validation. Numer. Math., 31, 377-403.
8. Davison, A. C. (2003) Statistical Models. Cambridge: Cambridge University Press.
9. Demidenko, E. (2004) Mixed Models: Theory and Applications. Hoboken: Wiley.
10. Dunn, P. K. and Smith, G. K. (2005) Series evaluation of Tweedie exponential dispersion model densities. Statist. Comput., 15, 267-280.
11. Efron, B. and Hinkley, D. V. (1978) Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information. Biometrika, 65, 457-487.
12. Eilers, P. H. C. and Marx, B. D. (1996) Flexible smoothing with B-splines and penalties. Statist. Sci., 11, 89-121.
13. Eilers, P. H. C. and Marx, B. D. (2002) Generalized linear additive smooth structures. J. Computnl Graph. Statist., 11, 758-783.
14. Escabias, M., Aguilera, A. M. and Valderrama, M. J. (2004) Principal component estimation of functional logistic regression: discussion of two different approaches. Nonparam. Statist., 16, 365-384.
15. Fahrmeir, L., Kneib, T. and Lang, S. (2004) Penalized structured additive regression for space time data: a Bayesian perspective. Statist. Sin., 14, 731-761.
16. Fahrmeir, L. and Lang, S. (2001) Bayesian inference for generalized additive mixed models based on Markov random field priors. Appl. Statist., 50, 201-220.
17. Golub, G. H. and van Loan, C. F. (1996) Matrix Computations, 3rd edn. Baltimore: Johns Hopkins University Press.
18. Green, P. J. and Silverman, B. W. (1994) Nonparametric Regression and Generalized Linear Models. London: Chapman and Hall.
19. Gu, C. (1992) Cross validating non-Gaussian data. J. Computnl Graph. Statist., 1, 169-179.
20. Gu, C. (2002) Smoothing Spline ANOVA Models. New York: Springer.
21. Gu, C. and Kim, Y.-J. (2002) Penalized likelihood regression: general formulation and efficient approximation. Can. J. Statist., 30, 619-628.
22. Hall, P. and Opsomer, J. D. (2005) Theory for penalised spline regression. Biometrika, 92, 105-118.
23. Härdle, W., Hall, P. and Marron, J. S. (1988) How far are automatically chosen regression smoothing parameters from their optimum? J. Am. Statist. Ass., 83, 86-95.
24. Harville, D. A. (1977) Maximum likelihood approaches to variance component estimation and to related problems. J. Am. Statist. Ass., 72, 320-338.
25. Harville, D. A. (1997) Matrix Algebra from a Statistician's Perspective. New York: Springer.
26. Hastie, T. and Tibshirani, R. (1986) Generalized additive models (with discussion). Statist. Sci., 1, 297-318.
27. Hastie, T. and Tibshirani, R. (1990) Generalized Additive Models. London: Chapman and Hall.
28. Hastie, T. and Tibshirani, R. (1993) Varying-coefficient models (with discussion). J. R. Statist. Soc. B, 55, 757-796.
29. Hurvich, C. M., Simonoff, J. S. and Tsai, C.-L. (1998) Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Statist. Soc. B, 60, 271-293.
30. Kalivas, J. H. (1997) Two data sets of near infrared spectra. Chemometr. Intell. Lab. Syst., 37, 255-259.
31. Kauermann, G. (2005) A note on smoothing parameter selection for penalized spline smoothing. J. Statist. Planng Inf., 127, 53-69.
32. Kauermann, G., Krivobokova, T. and Fahrmeir, L. (2009) Some asymptotic results on generalized penalized spline smoothing. J. R. Statist. Soc. B, 71, 487-503.
33. Kimeldorf, G. and Wahba, G. (1970) A correspondence between Bayesian estimation of stochastic processes and smoothing by splines. Ann. Math. Statist., 41, 495-502.
34. Kohn, R., Ansley, C. F. and Tharm, D. (1991) The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Am. Statist. Ass., 86, 1042-1050.
35. Krivobokova, T., Crainiceanu, C. M. and Kauermann, G. (2008) Fast adaptive penalized splines. J. Computnl Graph. Statist., 17, 1-20.
36. Laird, N. M. and Ware, J. H. (1982) Random-effects models for longitudinal data. Biometrics, 38, 963-974.
37. Lang, S. and Brezger, A. (2004) Bayesian P-splines. J. Computnl Graph. Statist., 13, 183-212.
38. Marx, B. D. and Eilers, P. H. (1998) Direct generalized additive modeling with penalized likelihood. Computnl Statist. Data Anal., 28, 193-209.
39. Marx, B. D. and Eilers, P. H. (1999) Generalized linear regression on sampled signals and curves: a P-spline approach. Technometrics, 41, 1-13.
40. Monahan, J. F. (2001) Numerical Methods of Statistics. Cambridge: Cambridge University Press.
41. Nelder, J. A. and Wedderburn, R. W. M. (1972) Generalized linear models. J. R. Statist. Soc. A, 135, 370-384
42. Nocedal, J. and Wright, S. J. (2006) Numerical Optimization, 2nd edn. New York: Springer.
43. Parker, R. L. and Rice, J. A. (1985) Discussion on 'Some aspects of the spline smoothing approach to non-parametric regression curve fitting' (by B. W. Silverman). J. R. Statist. Soc. B, 47, 40-42.
44. Patterson, H. D. and Thompson, R. (1971) Recovery of interblock information when block sizes are unequal. Biometrika, 58, 545-554.
45. Ramsay, J. O. and Silverman, B. W. (2005) Functional Data Analysis. New York: Springer.
46. R Development Core Team (2008) R 2.8.1: a Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.
47. Reiss, P. T. and Ogden, R. T. (2007) Functional principal component regression and functional partial least squares. J. Am. Statist. Ass., 102, 984-996.
48. Reiss, P. T. and Ogden, R. T. (2009) Smoothing parameter selection for a class of semiparametric linear models. J. R. Statist. Soc. B, 71, 505-523.
49. Rue, H., Martino, S. and Chopin, N. (2009) Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion). J. R. Statist. Soc. B, 71, 319-392.
50. Ruppert, D., Wand, M. P. and Carroll, R. J. (2003) Semiparametric Regression. Cambridge: Cambridge University Press.
51. Silverman, B. W. (1985) Some aspects of the spline smoothing approach to non-parametric regression curve fitting (with discussion). J. R. Statist. Soc. B, 47, 1-52.
52. Tutz, G. and Binder, H. (2006) Generalized additive modeling with implicit variable selection by likelihood-based boosting. Biometrics, 62, 961-971.
53. Tweedie, M. C. K. (1984) An index which distinguishes between some important exponential families. In Statistics: Applications and New Directions: Proc. Indian Statistical Institute Golden Jubilee Int. Conf. (eds J. K. Ghosh and J. Roy), pp. 579-604. Calcutta: Indian Statistical Institute.
54. Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S, 4th edn. New York: Springer.
55. Wahba, G. (1980) Spline bases, regularization and generalized cross validation for solving approximation problems with large quantities of noisy data. In Approximation Theory III (ed. E. Cheney). London: Academic Press.
56. Wahba, G. (1983) Bayesian ''confidence intervals'' for the cross-validated smoothing spline. J. R. Statist. Soc. B, 45, 133-150.
57. Wahba, G. (1985) A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Statist., 13, 1378-1402.
58. Wahba, G. (1990) Spline Models for Observational Data. Philadelphia: Society for Industrial and Applied Mathematics.
59. Wahba, G. and Wold, S. (1975) A completely automatic French curve: fitting spline functions by cross-validation. Communs Statist. Theor. Meth., 4, 125-141.
60. Watkins, D. S. (1991) Fundamentals of Matrix Computations. New York: Wiley.
61. Wehrens, R. and Mevik, B.-H. (2007) pls: Partial Least Squares Regression (PLSR) and Principal Component Regression (PCR). R Package Version 2.1-0. (Available from.)
62. Wood, S. N. (2003) Thin plate regression splines. J. R. Statist. Soc. B, 65, 95-114.
63. Wood, S. N. (2004) Stable and efficient multiple smoothing parameter estimation for generalized additive models. J. Am. Statist. Ass., 99, 673-686.
64. Wood, S. N. (2006) Generalized Additive Models: an Introduction with R. Boca Raton: CRC-Chapman and Hall.
65. Wood, S. N. (2008) Fast stable direct fitting and smoothness selection for generalized additive models. J. R. Statist. Soc. B, 70, 495-518.