Gaussian variational approximate inference for generalized linear mixed models

Journal of Computational and Graphical Statistics

Volumn 21, Issue 1, 2012, Pages 2-17

  Ormerod, J T ^a   Wand, M P ^b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

Best prediction; Likelihood based inference; Longitudinal data analysis; Machine learning; Variance components

Indexed keywords

EID: 84859847512     PISSN: 10618600     EISSN: None     Source Type: Journal    
DOI: 10.1198/jcgs.2011.09118     Document Type: Article

References (42)

1. Archambeau, C., Cornford, D., Opper, M., and Shawe-Taylor, J. (2007), "Gaussian Process Approximations of Stochastic Differential Equations," Journal of Machine Learning Research: Workshop and Conference Proceedings, 1, 1-16.[4]
2. Barber, D., and Bishop, C. M. (1998), "Ensemble Learning for Multi-Layer Networks," in Advances in Neural Information Processing Systems, eds. M. I. Jordan, K. J. Kearns, and S. A. Solla, Vol. 10, Cambridge, MA: MIT Press, pp. 395-401.[4]
3. Bates, D., and Maechler, M. (2009), "lme4 0.999375. Linear Mixed-Effects Models Using S4 Classes," R package, available at http://cran.r-project.org.[10]
4. Bishop, C. M. (2006), Pattern Recognition and Machine Learning, New York: Springer.[3,7,13]
5. Breslow, N. E., and Clayton, D. G. (1993), "Approximate Inference in Generalized Linear Mixed Models," Journal of the American Statistical Association, 88, 9-25.[3,13,14]
6. Carlin, B. P., and Louis, T. A. (2008), Bayes and Empirical Bayes Methods for Data Analysis (3rd ed.), New York: Chapman & Hall.[2]
7. De Backer, M., De Vroey, C., Lesaffre, E., Scheys, I., and De Keyser, P. (1998), "Twelve Weeks of Continuous Oral Therapy for Toenail Onychomycosis Caused by Dermatophytes: A Double-Blind Comparative Trial of Terbinafine 250 mg/Day versus Itraconazole 200 mg/Day," Journal of the American Academy of Dermatology, 38 (5), S57-S63.[10,14]
8. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004), Bayesian Data Analysis, Boca Raton, FL: Chapman & Hall.[2]
9. Hall, P., Ormerod, J. T., and Wand, M. P. (2011), "Theory of Gaussian Variational Approximation for a Poisson Linear Mixed Model," Statistica Sinica, 21, 369-389.[3,9]
10. Honkela, A., and Valpola, H. (2005), "Unsupervised Variational Bayesian Learning of Nonlinear Models," in Advances in Neural Information Processing Systems, Vol. 17, Cambridge, MA: MIT Press, pp. 593-600.[4]
11. Jaakkola, T. S., and Jordan, M. I. (2000), "Bayesian Parameter Estimation via Variational Methods," Statistics and Computing, 10, 25-37.[4]
12. Jordan, M. I. (2004), "Graphical Models," Statistical Science, 19, 140-155.[3]
13. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. (1999), "An Introduction to Variational Methods for Graphical Models," Machine Learning, 37, 183-233.[3]
14. Kullback, S., and Leibler, R. A. (1951), "On Information and Sufficiency," The Annals of Mathematical Statistics, 22, 79-86.[8]
15. Leach, A. (2000), "Menzies School of Health Research 1999-2000 Annual Report," Menzies School of Health Research, pp. 18-21.[5]
16. Lee, Y., and Nelder, J. A. (1996), "Hierarchical Generalized Linear Models," Journal of the Royal Statistical Society, Ser. B, 58, 619-656.[3]
17. Lesaffre, E., and Spiessens, B. (2001), "On the Effect of the Number of Quadrature Points in a Logistic Random-Effects Model: An Example," Applied Statistics, 50, 325-335.[10,15]
18. Little, R. J., and Rubin, D. B. (2004), Statistical Analysis With Missing Data (2nd ed.), New York: Wiley.[2]
19. Liu, Q., and Pierce, D. A. (1994), "A Note on Gauss-Hermite Quadrature," Biometrika, 81, 624-629.[7,10]
20. Marin, J.-M., and Robert, C. P. (2007), Bayesian Core: A Practical Approach to Computational Bayesian Statistics, New York: Springer.[2]
21. McCulloch, C. E. (1997), "Maximum Likelihood for Algorithms for Generalized Linear Mixed Models," Journal of the American Statistical Association, 92, 162-170.[10]
22. McCulloch, C. E., Searle, S. R., and Neuhaus, J. M. (2008), Generalized, Linear, and Mixed Models (2nd ed.), New York: Wiley.[2,9]
23. Neal, R. M., and Hinton, G. E. (1999), "A New View of the EM Algorithm That Justifies Incremental and Other Variants," in Learning in Graphical Models, ed. M. I. Jordan, Cambridge, MA: MIT Press, pp. 355-368.[7]
24. Noh, M., and Lee, Y. (2007), "REML Estimation for Binary Data in GLMMs," Journal of Multivariate Analysis, 98, 896-915.[10-12]
25. Opper, M., and Archambeau, C. (2009), "Variational Gaussian Approximation Revisited," Neural Computation, 21, 786-792.[4,9]
26. Ormerod, J. T. (2008), "On Semiparametric Regression and Data Mining," Ph.D. thesis, School of Mathematics and Statistics, The University of New South Wales, Sydney, Australia.[4,7]
27. Ormerod, J. T., andWand, M. P. (2008), "Variational Approximations for Logistic MixedModels," in Proceedings of the Ninth Iranian Statistics Conference, Department of Statistics, University of Isfahan, Isfahan, Iran, pp. 450-467.[4]
28. Ormerod, J. T., andWand, M. P. (2010), "Explaining Variational Approximation," The American Statistician, 64, 140-153.[3]
29. Parisi, G. (1988), Statistical Field Theory, Redwood City, CA: Addison-Wesley.[4]
30. Pinheiro, J. C., and Bates, D. M. (1995), "Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model," Journal of Computational and Graphical Statistics, 4, 12-35.[10]
31. R Development Core Team (2009), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. Available at www.R-project.org.[10]
32. Raudenbush, S. W., Yang, M.-L., and Yosef, M. (2000), "Maximum Likelihood for Generalized Linear Models With Nested Random Effects via High-Order, Multivariate Laplace Approximation," Journal of Computational and Graphical Statistics, 9, 141-157.[3]
33. Rue, H., Martino, S., and Chopin, N. (2009), "Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations," Journal of the Royal Statistical Society, Ser. B, 7, 319-392.[3,4]
34. Seeger, M. (2000), "Bayesian Model Selection for Support Vector Machines, Gaussian Processes and Other Kernel Classifiers," in Advances in Neural Information Processing Systems, Vol. 12, Cambridge,MA: MIT Press, pp. 603-609.[4]
35. Skrondal, A., and Rabe-Hesketh, S. (2004), Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models, Boca Raton, FL: Chapman & Hall.[2]
36. Thall, P. F., and Vail, S. C. (1990), "Some Covariance Models for Longitudinal Count DataWith Overdispersion," Biometrics, 46, 657-671.[10,13]
37. Titterington, D. M. (2004), "Bayesian Methods for Neural Networks and Related Models," Statistical Science, 19, 128-139.[7]
38. Venables, W. N., and Ripley, B. D. (2009), "VR 7.2. Functions and Datasets to Support Venables and Ripley 'Modern Applied Statistics With S' (4th ed.)," R package, available at http://cran.r-project.org.[10]
39. Wang, B., and Titterington, D. M. (2005), "Inadequacy of Interval Estimates Corresponding to Variational Bayesian Approximations," in Proceedings of the 10th International Workshop on Artificial Intelligence, eds.
40. R. G. Cowell and Z. Ghahramani, Barbados: Society for Artificial Intelligence and Statistics, pp. 373-380.[4]
41. Zeger, S. L., Liang, K. Y., and Albert, P. S. (1988), "Models for Longitudinal Data: A Generalized Estimating Equation Approach," Biometrics, 44, 1049-1060.[9]
42. Zhao, Y., Staudenmayer, J., Coull, B. A., and Wand, M. P. (2006), "General Design Bayesian Generalized Linear Mixed Models," Statistical Science, 21, 35-51.[3,6]