Variational inference for generalized linear mixed models using partially noncentered parametrizations

Statistical Science

Volumn 28, Issue 2, 2013, Pages 168-188

  Tan, Linda S L ^a   Nott, David J ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Hierarchical centering; Longitudinal data analysis.; Nonconjugate models; Variational bayes; Variational message passing

Indexed keywords

EID: 84878988007     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-STS418     Document Type: Article

References (50)

1. ATTIAS, H. (1999). Inferring parameters and structure of latent variable models by variational Bayes. In Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence 21-30. Morgan Kaufmann, San Francisco, CA.
2. ATTIAS, H. (2000). A variational Bayesian framework for graphical models. In Advances in Neural Information Processing Systems 12 209-215. MIT Press, Cambridge, MA.
3. BISHOP, C. M. (2006). Pattern Recognition and Machine Learning. Springer, New York. MR2247587
4. BLOCKER, A. W. (2011). Fast Rcpp implementation of Gauss-Hermite quadrature. R package "fastGHQuad" version 0.1-1. Available at http://cran.r-project.org/.
5. BRAUN, M. and MCAULIFFE, J. (2010). Variational inference for large-scale models of discrete choice. J. Amer. Statist. Assoc. 105 324-335. MR2757203
6. BRESLOW, N. E., and CLAYTON, D. G. (1993). Approximate inference in generalized linear mixed models. J. Amer. Statist. Assoc. 88 9-25.
7. BROWN, P., and ZHOU, L. (2010). MCMC for generalized linear mixed models with glmmBUGS. The R Journal 2 13-16.
8. BROWNE, W. J., and DRAPER, D. (2006). A comparison of Bayesian, and likelihood-based methods for fitting multilevel models. Bayesian Anal. 1 473-513 (electronic). MR2221283
9. CAI, B. and DUNSON, D. B. (2008). Bayesian variable selection in generalized linear mixed models. In Random Effect, and Latent Variable Model Selection. Lecture Notes in Statistics 192 63-91. Springer, New York.
10. CHRISTENSEN, O. F., ROBERTS, G. O., and SKÖLD, M. (2006). Robust Markov chain Monte Carlo methods for spatial generalized linear mixed models. J. Comput. Graph. Statist. 15 1-17. MR2269360
11. CORDUNEANU, A., and BISHOP, C. M. (2001). Variational Bayesian model selection for mixture distributions. In Artificial Intelligence, and Statistics 27-34. Morgan Kaufmann, San Francisco, CA.
12. DE BACKER, M., DE VROEY, C., LESAFFRE, E., SCHEYS, I., and DE KEYSER, P. (1998). Twelve weeks of continuous oral therapy for toenail onychomycosis caused by dermatophytes: A double-blind comparative trial of terbinafine 250 mg/day versus itraconazole 200 mg/day. Journal of the American Academy of Dermatology 38 57-63.
13. FITZMAURICE, G., and LAIRD, N. (1993). A likelihood-based method for analysing longitudinal binary responses. Biometrika 80 141-151.
14. FONG, Y., RUE, H., and WAKEFIELD, J. (2010). Bayesian inference for generalised linear mixed models. Biostatistics 11 397-412.
15. GELFAND, A. E., SAHU, S. K., and CARLIN, B. P. (1995). Efficient parameterisations for normal linear mixed models. Biometrika 82 479-488. MR1366275
16. GELFAND, A. E., SAHU, S. K., and CARLIN, B. P. (1996). Efficient parametrizations for generalized linear mixed models. In Bayesian Statistics 5 (Alicante, 1994) 165-180. Oxford Univ. Press, New York. MR1425405
17. GELMAN, A., CARLIN, J. B., STERN, H. S., and RUBIN, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL. MR2027492
18. GHAHRAMANI, Z., and BEAL, M. J. (2001). Propagation algorithms for variational Bayesian learning. In Advances in Neural Information Processing Systems 13 507-513. MIT Press, Cambridge, MA.
19. HOFFMAN, M. D., BLEI, D. M., WANG, C., and PAISLEY, J. (2012). Stochastic variational inference. Available at arXiv:1206.7051.
20. JAAKKOLA, T. S., and JORDAN, M. I. (2000). Bayesian parameter estimation via variational methods. Statist. Comput. 10 25-37.
21. KASS, R. E., and NATARAJAN, R. (2006). A default conjugate prior for variance components in generalized linear mixed models (comment on article by Browne, and Draper). Bayesian Anal. 1 535-542 (electronic). MR2221285
22. KNOWLES, D. A., and MINKA, T. P. (2011). Non-conjugate variational message passing for multinomial, and binary regression. In Advances in Neural Information Processing Systems 24 1701-1709. Available at http://books.nips.cc/papers/files/nips24/NIPS2011_0962.pdf.
23. LIU, Q., and PIERCE, D. A. (1994). A note on Gauss-Hermite quadrature. Biometrika 81 624-629. MR1311107
24. LIU, J. S., and WU, Y. N. (1999). Parameter expansion for data augmentation. J. Amer. Statist. Assoc. 94 1264-1274. MR1731488
25. LUNN, D. J., THOMAS, A., BEST, N., and SPIEGELHALTER, D. (2000). WinBUGS-A Bayesian modelling framework: Concepts, structure,, and extensibility. Statist. Comput. 10 325-337.
26. MAGNUS, J. R., and NEUDECKER, H. (1988). Matrix Differential Calculus with Applications in Statistics, and Econometrics. Wiley, Chichester. MR0940471
27. MENG, X.-L., and VAN DYK, D. (1997). The EM algorithm-An old folk-song sung to a fast new tune (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 59 511-567. MR1452025
28. MENG, X.-L., and VAN DYK, D. A. (1999). Seeking efficient data augmentation schemes via conditional, and marginal augmentation. Biometrika 86 301-320. MR1705351
29. O'HAGAN, A., and FORSTER, J. (2004). Kendall's Advanced Theory of Statistics V. 2B: Bayesian Inference, 2nd ed. Arnold, London.
30. ORMEROD, J. T. and WAND, M. P. (2010). Explaining variational approximations. Amer. Statist. 64 140-153. MR2757005
31. ORMEROD, J. T., and WAND, M. P. (2012). Gaussian variational approximate inference for generalized linear mixed models. J. Comput. Graph. Statist. 21 2-17. MR2913353
32. OVERSTALL, A. M., and FORSTER, J. J. (2010). Default Bayesian model determination methods for generalised linear mixed models. Comput. Statist. Data Anal. 54 3269-3288. MR2727751
33. PAPASPILIOPOULOS, O., ROBERTS, G. O., and SKöLD, M. (2003). Non-centered parameterizations for hierarchical models, and data augmentation. In Bayesian Statistics 7 (Tenerife, 2002) 307-326. Oxford Univ. Press, New York. MR2003180
34. PAPASPILIOPOULOS, O., ROBERTS, G. O., and SKöLD, M. (2007). A general framework for the parametrization of hierarchical models. Statist. Sci. 22 59-73. MR2408661
35. QI, Y., and JAAKKOLA, T. S. (2006). Parameter expanded variational Bayesian methods. In Advances in Neural Information Processing Systems 19 1097-1104. MIT Press, Cambridge, MA.
36. RAUDENBUSH, S. W., YANG, M.-L., and YOSEF, M. (2000). Maximum likelihood for generalized linear models with nested random effects via high-order, multivariate Laplace approximation. J. Comput. Graph. Statist. 9 141-157. MR1826278
37. RIJMEN, F., and VOMLEL, J. (2008). Assessing the performance of variational methods for mixed logistic regression models. J. Stat. Comput. Simul. 78 765-779. MR2528235
38. ROOS, M., and HELD, L. (2011). Sensitivity analysis in Bayesian generalized linear mixed models for binary data. Bayesian Anal. 6 259-278. MR2806244
39. ROULIN, A., and BERSIER, L. F. (2007). Nestling barn owls beg more intensely in the presence of their mother than in the presence of their father. Animal Behaviour 74 1099-1106.
40. SAUL, L. K., and JORDAN (1998). A mean field learning algorithm for unsupervised neural networks. In Learning in Graphical Models 541-554. Kluwer Academic, Dordrecht.
41. STURTZ, S., LIGGES, U. and GELMAN, A. (2005). R2WinBUGS: A package for runningWinBUGS from R. Journal of Statistical Software 12 1-16.
42. TAN, S. L., and NOTT, D. J. (2013). Variational approximation for mixtures of linear mixed models. J. Comput. Graph. Statist. To appear. DOI:10.1080/10618600.2012.761138.
43. THALL, P. F., and VAIL, S. C. (1990). Some covariance models for longitudinal count data with overdispersion. Biometrics 46 657-671. MR1085814
44. VENABLES, W. N., and RIPLEY, B. D. (2002). Modern Applied Statistics with S, 4th ed. Springer, New York.
45. WAND, M. P. (2013). Fully simplified multivariate normal updates in non-conjugate variational message passing. Unpublished manuscript. Available at http://www.uow.edu.au/~mwand/fsupap.pdf.
46. WINN, J., and BISHOP, C. M. (2005). Variational message passing. J. Mach. Learn. Res. 6 661-694. MR2249835
47. YU, Y. and MENG, X.-L. (2011). To center or not to center: That is not the question-An ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC efficiency. J. Comput. Graph. Statist. 20 531-570. MR2878987
48. YU, D., and YAU, K. K. W. (2012). Conditional Akaike information criterion for generalized linear mixed models. Comput. Statist. Data Anal. 56 629-644. MR2853760
49. ZHAO, Y., STAUDENMAYER, J., COULL, B. A. and WAND, M. P. (2006). General design Bayesian generalized linear mixed models. Statist. Sci. 21 35-51. MR2275966
50. ZUUR, A. F., IENO, E. N.,WALKER, N. J., SAVELIEV, A. A., and SMITH, G. M. (2009). Mixed Effects Models, and Extensions in Ecology with R. Springer, New York. MR2722501