Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 71, Issue 2, 2009, Pages 319-392

  Rue, Håvard ^a   Martino, Sara ^a   Chopin, Nicolas ^b  

^a NORWEGIAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Norway)

^b Ecole Nationale de la Statistique et de l'Administration Economique   (France)

Author keywords

Approximate Bayesian inference; Gaussian Markov random fields; Generalized additive mixed models; Laplace approximation; Parallel computing; Sparse matrices; Structured additive regression models

Indexed keywords

EID: 62849120031     PISSN: 13697412     EISSN: 14679868     Source Type: Journal    
DOI: 10.1111/j.1467-9868.2008.00700.x     Document Type: Article

References (98)

1. Ainsworth, L. M. Dean, C. B. (2006) Approximate inference for disease mapping. Computnl Statist. Data Anal., 50, 2552 2570.
2. Albert, J. H. Chib, S. (1993) Bayesian analysis of binary and polychotomous response data. J. Am. Statist. Ass., 88, 669 679.
3. Attias, H. (1999) Inferring parameters and structure of latent variable models by variational Bayes. In Proc. 15th Conf. Uncertainty in Artificial Intelligence, vol. 2, pp. 21 30. San Francisco : Morgan Kaufmann.
4. Attias, H. (2000) A variational Bayesian framework for graphical models. Adv. Neur. Informn Process. Syst., 12, 209 215.
5. Azzalini, A. Capitanio, A. (1999) Statistical applications of the multivariate skew normal distribution. J. R. Statist. Soc. B, 61, 579 602.
6. Banerjee, S., Carlin, B. P. Gelfand, A. E. (2004) Hierarchical Modeling and Analysis for Spatial Data. London : Chapman and Hall.
7. Banerjee, S., Gelfand, A. E., Finley, A. O. Sang, H. (2008) Gaussian predictive process models for large spatial data sets. J. R. Statist. Soc. B, 70, 825 848.
8. Beal, M. J. (2003) Variational algorithms for approximate Bayesian inference. PhD Thesis. University College London, London.
9. Besag, J., Green, P. J., Higdon, D. Mengersen, K. (1995) Bayesian computation and stochastic systems (with discussion). Statist. Sci., 10, 3 66.
10. Besag, J., York, J. Mollié, A. (1991) Bayesian image restoration with two applications in spatial statistics (with discussion). Ann. Inst. Statist. Math., 43, 1 59.
11. Beskos, A., Papaspiliopoulos, O., Roberts, G. O. Fearnhead, P. (2006) Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J. R. Statist. Soc. B, 68, 333 382.
12. Bishop, C. M. (2006) Pattern Recognition and Machine Learning. New York : Springer.
13. Box, G. E. P. Wilson, K. B. (1951) On the experimental attainment of optimum conditions (with discussion). J. R. Statist. Soc. B, 13, 1 45.
14. Breslow, N. E. Clayton, D. G. (1993) Approximate inference in generalized linear mixed models. J. Am. Statist. Ass., 88, 9 25.
15. Brix, A. Diggle, P. J. (2001) Spatiotemporal prediction for log-Gaussian Cox processes. J. R. Statist. Soc. B, 63, 823 841.
16. Brix, A. Møller, J. (2001) Space-time multi type log Gaussian Cox processes with a view to modelling weeds. Scand. J. Statist., 28, 471 488.
17. Carter, C. K. Kohn, R. (1994) On Gibbs sampling for state space models. Biometrika, 81, 541 543.
18. Chib, S., Nardari, F. Shephard, N. (2002) Markov chain Monte Carlo methods for stochastic volatility models. J. Econometr., 108, 281 316.
19. Chu, W. Ghahramani, Z. (2005) Gaussian processes for ordinal regression. J. Mach. Learn. Res., 6, 1019 1041.
20. Cressie, N. A. C. (1993) Statistics for Spatial Data, revised edn. New York : Wiley.
21. Cressie, N. Johannesson, G. (2008) Fixed rank kriging for very large spatial data sets. J. R. Statist. Soc. B, 70, 209 226.
22. Czado, C., Gneiting, T. Held, L. (2007) Predictive model assessment for count data. Technical Report 518. Department of Statistics, University of Washington, Seattle.
23. Dey, D. K., Ghosh, S. K. Mallick, B. K. (eds 2000) Generalized Linear Models: a Bayesian Perspective. Boca Raton : Chapman and Hall-CRC.
24. Diggle, P. J. Ribeiro, P. J. (2006) Model-based Geostatistics. New York : Springer.
25. Durbin, J. Koopman, S. J. (2000) Time series analysis of non-Gaussian observations based on state space models from both classical and Bayesian perspectives (with discussion). J. R. Statist. Soc. B, 62, 3 56.
26. Eidsvik, J., Martino, S. Rue, H. (2009) Approximate Bayesian inference in spatial generalized linear mixed models. Scand. J. Statist., to be published.
27. Fahrmeir, L. Lang, S. (2001) Bayesian inference for generalized additive mixed models based on Markov random field priors. Appl. Statist., 50, 201 220.
28. Fahrmeir, L. Tutz, G. (2001) Multivariate Statistical Modelling based on Generalized Linear Models, 2nd edn. Berlin : Springer.
29. Finkenstadt, B., Held, L. Isham, V. (eds 2006) Statistical Methods for Spatio-temporal Systems. Boca Raton : Chapman and Hall-CRC.
30. Friel, N. Rue, H. (2007) Recursive computing and simulation-free inference for general factorizable models. Biometrika, 94, 661 672.
31. Frühwirth-Schnatter, S. Frühwirth, R. (2007) Auxiliary mixture sampling with applications to logistic models. Computnl Statist. Data Anal., 51, 3509 3528.
32. Frühwirth-Schnatter, S. Wagner, H. (2006) Auxiliary mixture sampling for parameter-driven models of time series of small counts with applications to state space modelling. Biometrika, 93, 827 841.
33. Gamerman, D. (1997) Sampling from the posterior distribution in generalized linear mixed models. Statist. Comput., 7, 57 68.
34. Gamerman, D. (1998) Markov chain Monte Carlo for dynamic generalised linear models. Biometrika, 85, 215 227.
35. Gelfand, A. E. (1996) Model determination using sampling-based methods. In Markov Chain Monte Carlo in Practice (eds W. R. Gilks, S. Richardson D. J. Spiegelhalter pp. 145 161. London : Chapman and Hall.
36. Gelman, A., Carlin, J. B., Stern, H. S. Rubin, D. B. (2004) Bayesian Data Analysis, 2nd edn. Boca Raton : Chapman and Hall-CRC.
37. Gneiting, T. (2002) Nonseparable, stationary covariance functions for space-time data. J. Am. Statist. Ass., 97, 590 600.
38. Gneiting, T. Raftery, A. E. (2007) Strictly proper scoring rules, prediction, and estimation. J. Am. Statist. Ass., 102, 359 378.
39. Gschløssl, S. Czado, C. (2008) Modelling count data with overdispersion and spatial effects. Statist. Pap., 49, 531 552.
40. Held, L., Natario, I., Fenton, S., Rue, H. Becker, N. (2005) Towards joint disease mapping. Statist. Meth. Med. Res., 14, 61 82.
41. Hinton, G. E. van Camp, D. (1993) Keeping the neural networks simple by minimizing the description length of the weights. In Proc. 6th A. Conf. Computational Learning Theory, Santa Cruz, pp. 5 13. New York : Association for Computing Machinery Press.
42. Holmes, C. C. Held, L. (2006) Bayesian auxiliary variable models for binary and multinomial regression. Bayes. Anal., 1, 145 168.
43. Hsiao, C. K., Huang, S. Y. Chang, C. W. (2004) Bayesian marginal inference via candidate's formula. Statist. Comput., 14, 59 66.
44. Humphreys, K. Titterington, D. M. (2000) Approximate Bayesian inference for simple mixtures. In Proc. Computational Statistics 2000 (eds J. G. Bethlehem P. G. M. van der Heijden pp. 331 336. Heidelberg : Physica.
45. Jordan, M. I. (2004) Graphical models. Statist. Sci., 19, 140 155.
46. Kammann, E. E. Wand, M. P. (2003) Geoadditive models. Appl. Statist., 52, 1 18.
47. Kass, R. E., Tierney, L. Kadane, J. B. (1999) The validity of posterior expansions based on Laplace's method. In Essays in Honor of George Bernard (eds S. Geisser, J. S. Hodges, S. J. Press A. Zellner pp. 473 488. Amsterdam : North-Holland.
48. Kass, R. E. Vaidyanathan, S. K. (1992) Approximate Bayes factors and orthogonal parameters, with application to testing equality of two binomial proportions. J. R. Statist. Soc. B, 54, 129 144.
49. Kitagawa, G. Gersch, W. (1996) Smoothness priors analysis of time series. Lect. Notes Statist., 116.
50. Knorr-Held, L. (1999) Conditional prior proposals in dynamic models. Scand. J. Statist., 26, 129 144.
51. Knorr-Held, L., Raßer, G. Becker, N. (2002) Disease mapping of stage-specific cancer incidence data. Biometrics, 58, 492 501.
52. Knorr-Held, L. Rue, H. (2002) On block updating in Markov random field models for disease mapping. Scand. J. Statist., 29, 597 614.
53. Kohn, R. Ansley, C. F. (1987) A new algorithm for spline smoothing based on smoothing a stochastic process. SIAM J. Scient. Statist. Comput., 8, 33 48.
54. Kuss, M. Rasmussen, C. E. (2005) Assessing approximate inference for binary Gaussian process classification. J. Mach. Learn. Res., 6, 1679 1704.
55. Lang, S. Brezger, A. (2004) Bayesian P-splines. J. Computnl Graph. Statist., 13, 183 212.
56. Mackay, D. J. C. (1995) Ensemble learning and evidence maximization. Technical Report. Cavendish Laboratory University of Cambridge, Cambridge.
57. Mackay, D. J. C. (1997) Ensemble learning for hidden Markov models. Technical Report. Cavendish Laboratory, University of Cambridge, Cambridge.
58. Marroquin, J. L., Velasco, F. A., Rivera, M. Nakamura, M. (2001) Gauss-Markov measure field models for low-level vision. IEEE Trans. Pattn Anal. Mach. Intell., 23, 337 348.
59. Martino, S. (2007) Approximate Bayesian inference for latent Gaussian models. PhD Thesis. Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim.
60. Martino, S. Rue, H. (2008) Implementing approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations: a manual for the inla-program. Technical Report 2. Department of Mathematical Sciences, Norwegian University of Science and Technology, Trondheim.
61. Minka, T. P. (2001) Expectation propagation for approximate Bayesian inference. Uncertnty Artif. Intell., 17, 362 369.
62. Møller, J., Syversveen, A. R. Waagepetersen, R. P. (1998) Log Gaussian Cox processes. Scand. J. Statist., 25, 451 482.
63. Møller, J. Waagepetersen, R. (2003) Statistical Inference and Simulation for Spatial Point Processes. London : Chapman and Hall.
64. Neal, R. M. (1998) Regression and classification using Gaussian process priors. In Bayesian Statistics 6 (eds J. M. Bernardo, J. O. Berger, A. P. Dawid A. F. M. Smith pp. 475 501. New York : Oxford University Press.
65. O'Hagan, A. (1978) Curve fitting and optimal design for prediction (with discussion). J. R. Statist. Soc. B, 40, 1 42.
66. Papaspiliopoulos, O., Roberts, G. O. Sköld, M. (2007) A general framework for the parameterization of hierarchical models. Statist. Sci., 22, 59 73.
67. Pettit, L. I. (1990) The conditional predictive ordinate for the normal distribution. J. R. Statist. Soc. B, 52, 175 184.
68. Rasmussen, C. E. Williams, C. K. I. (2006) Gaussian Processes for Machine Learning. Cambridge : MIT Press.
69. R Development Core Team (2007) R: a Language and Environment for Statistical Computing. Vienna : R Foundation for Statistical Computing.
70. Rellier, G., Descombes, X., Zerubia, J. Falzon, F. (2002) A Gauss-Markov model for hyperspectral texture analysis of urban areas. In Proc. 16th Int. Conf. Pattern Recognition, pp. 692 695. Washington DC : Institute of Electrical and Electronics Engineers Computer Society.
71. Robert, C. P. Casella, G. (1999) Monte Carlo Statistical Methods. New York : Springer.
72. Rue, H. (2001) Fast sampling of Gaussian Markov random fields. J. R. Statist. Soc. B, 63, 325 338.
73. Rue, H. Held, L. (2005) Gaussian Markov Random Fields: Theory and Applications. London : Chapman and Hall-CRC Press.
74. Rue, H. Martino, S. (2007) Approximate Bayesian inference for hierarchical Gaussian Markov random fields models. J. Statist. Planng Inf., 137, 3177 3192.
75. Rue, H., Steinsland, I. Erland, S. (2004) Approximating hidden Gaussian Markov random fields. J. R. Statist. Soc. B, 66, 877 892.
76. Sanchez, S. M. Sanchez, P. J. (2005) Very large fractional factorials and central composite designs. ACM Trans. Modlng Comput. Simuln, 15, 362 377.
77. Schervish, M. J. (1995) Theory of Statistics, 2nd edn. New York : Springer.
78. Shephard, N. (1994) Partial non-Gaussian state space. Biometrika, 81, 115 131.
79. Shephard, N. Pitt, M. K. (1997) Likelihood analysis of non-Gaussian measurement time series. Biometrika, 84, 653 667.
80. Shun, Z. McCullagh, P. (1995) Laplace approximation of high dimensional integrals. J. R. Statist. Soc. B, 57, 749 760.
81. Smith, A. F. M., Skene, A. M., Shaw, J. E. H. Naylor, J. C. (1987) Progress with numerical and graphical methods for practical Bayesian statistics. Statistician, 36, 75 82.
82. Spiegelhalter, D. J., Best, N. G., Carlin, B. P. van der Linde, A. (2002) Bayesian measures of model complexity and fit (with discussion). J. R. Statist. Soc. B, 64, 583 639.
83. Terzopoulos, D. (1988) The computation of visible-surface representations. IEEE Trans. Pattn Anal. Mach. Intell., 10, 417 438.
84. Thall, P. F. Vail, S. C. (1990) Some covariance models for longitudinal count data with overdispersion. Biometrics, 46, 657 671.
85. Thomas, A., O'Hara, B., Ligges, U. Sturtz, S. (2006) Making BUGS open. R News, 6, 12 16.
86. Tierney, L. Kadane, J. B. (1986) Accurate approximations for posterior moments and marginal densities. J. Am. Statist. Ass., 81, 82 86.
87. Titterington, D. M. (2004) Bayesian methods for neural networks and related models. Statist. Sci., 19, 128 139.
88. Waagepetersen, R. P. (2007) An estimating function approach to inference for inhomogeneous Neyman-Scott processes. Biometrics, 63, 252 258.
89. Wahba, G. (1978) Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. R. Statist. Soc. B, 40, 364 372.
90. Wakefield, J. (2007) Disease mapping and spatial regression with count data. Biostatistics, 8, 158 183.
91. Wakefield, J. C., Best, N. G. Waller, L. A. (2000) Bayesian approaches to disease mapping. In Spatial Epidemiology: Methods and Applications (eds P. Elliot, J. C. Wakefield, N. G. Best D. J. Briggs pp. 104 107. Oxford : Oxford University Press.
92. Wang, B. Titterington, D. M. (2005) Inadequacy of interval estimates corresponding to variational Bayesian approximations. In Proc. 10th Int. Wrkshp Artificial Intelligence and Statistics (eds R. G. Cowell Z. Ghahramani pp. 373 380. Society for Artificial Intelligence and Statistics.
93. Wang, B. Titterington, D. M. (2006) Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model. Bayes. Anal., 1, 625 650.
94. Wecker, W. E. Ansley, C. F. (1983) The signal extraction approach to nonlinear regression and spline smoothing. J. Am. Statist. Ass., 78, 81 89.
95. Weir, I. S. Pettitt, A. N. (2000) Binary probability maps using a hidden conditional autoregressive Gaussian process with an application to Finnish common toad data. Appl. Statist., 49, 473 484.
96. West, M. Harrison, J. (1997) Bayesian Forecasting and Dynamic Models, 2nd edn. New York : Springer.
97. Williams, C. K. I. Barber, D. (1998) Bayesian classification with Gaussian processes. IEEE Trans. Pattn Anal. Mach. Intell., 20, 1342 1351.
98. Zoeter, O. Heskes, T. (2005) Gaussian quadrature based expectation propagation. In Proc. 10th Int. Wrkshp Artificial Intelligence and Statistics (eds R. G. Cowell Z. Ghahramani pp. 445 452. Society for Artificial Intelligence and Statistics.