PASS-GLM: Polynomial approximate sufficient statistics for scalable Bayesian GLM inference

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 3612-3622

  Huggins, Jonathan H ^a   Adams, Ryan P ^b   Broderick, Tamara ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN NETWORKS; HIERARCHICAL SYSTEMS; MAXIMUM LIKELIHOOD; STOCHASTIC SYSTEMS; UNCERTAINTY ANALYSIS;

COMPETITIVE PERFORMANCE; GENERALIZED LINEAR MODEL; LAPLACE APPROXIMATION; PROBABILISTIC APPROACHES; QUADRATIC APPROXIMATION; STOCHASTIC GRADIENT DESCENT; THEORETICAL GUARANTEES; UNCERTAINTY ESTIMATES;

REGRESSION ANALYSIS;

EID: 85047005558     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (41)

1. S. Ahn, A. Korattikara, and M. Welling. Bayesian posterior sampling via stochastic gradient Fisher scoring. In International Conference on Machine Learning, 2012.
2. P. Alquier, N. Friel, R. Everitt, and A. Boland. Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels. Statistics and Computing, 26:29-47, 2016.
3. E. Angelino, M. J. Johnson, and R. P. Adams. Patterns of scalable Bayesian inference. Foundations and Trends® in Machine Learning, 9(2-3):119-247, 2016.
4. O. Bachem, M. Lucic, and A. Krause. Practical coreset constructions for machine learning. arXiv.org, Mar. 2017.
5. R. Bardenet, A. Doucet, and C. C. Holmes. Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach. In International Conference on Machine Learning, pages 405-413, 2014.
6. R. Bardenet, A. Doucet, and C. C. Holmes. On Markov chain Monte Carlo methods for tall data. Journal of Machine Learning Research, 18:1-43, 2017.
7. M. J. Betancourt. The fundamental incompatibility of Hamiltonian Monte Carlo and data subsampling. In International Conference on Machine Learning, 2015.
8. J. Bierkens, P. Fearnhead, and G. O. Roberts. The zig-zag process and super-efficient sampling for Bayesian analysis of big data. arXiv.org, July 2016.
9. A. Bouchard-Côté, S. J. Vollmer, and A. Doucet. The bouncy particle sampler: A non-reversible rejection-free Markov chain Monte Carlo method. arXiv.org, pages 1-37, Jan. 2016.
10. T. Broderick, N. Boyd, A. Wibisono, A. C. Wilson, and M. I. Jordan. Streaming variational Bayes. In Advances in Neural Information Processing Systems, Dec. 2013.
11. T. Campbell, J. Straub, J. W. Fisher, III, and J. P. How. Streaming, distributed variational inference for Bayesian nonparametrics. In Advances in Neural Information Processing Systems, 2015.
12. R. Entezari, R. V. Craiu, and J. S. Rosenthal. Likelihood inflating sampling algorithm. arXiv.org, May 2016.
13. D. Feldman, M. Faulkner, and A. Krause. Scalable training of mixture models via coresets. In Advances in Neural Information Processing Systems, pages 2142-2150, 2011.
14. W. Fithian and T. Hastie. Local case-control sampling: Efficient subsampling in imbalanced data sets. The Annals of Statistics, 42(5):1693-1724, Oct. 2014.
15. A. Gelman, A. Vehtari, P. Jylänki, T. Sivula, D. Tran, S. Sahai, P. Blomstedt, J. P. Cunningham, D. Schiminovich, and C. Robert. Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data. arXiv.org, Dec. 2014.
16. L. Han, T. Yang, and T. Zhang. Local uncertainty sampling for large-scale multi-class logistic regression. arXiv.org, Apr. 2016.
17. L. Hasenclever, S. Webb, T. Lienart, S. Vollmer, B. Lakshminarayanan, C. Blundell, and Y. W. Teh. Distributed Bayesian learning with stochastic natural-gradient expectation propagation and the posterior server. Journal of Machine Learning Research, 18:1-37, 2017.
18. M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley. Stochastic variational inference. Journal of Machine Learning Research, 14:1303-1347, 2013.
19. J. H. Huggins, T. Campbell, and T. Broderick. Coresets for scalable Bayesian logistic regression. In Advances in Neural Information Processing Systems, May 2016.
20. T. Jaakkola and M. I. Jordan. A variational approach to Bayesian logistic regression models and their extensions. In Sixth International Workshop on Artificial Intelligence and Statistics, Volume 82, 1997.
21. A. Korattikara, Y. Chen, and M. Welling. Austerity in MCMC land: Cutting the Metropolis-Hastings budget. In International Conference on Machine Learning, 2014.
22. A. Kucukelbir, R. Ranganath, A. Gelman, and D. M. Blei. Automatic variational inference in Stan. In Advances in Neural Information Processing Systems, June 2015.
23. P. Li, T. J. Hastie, and K. W. Church. Very sparse random projections. In SIGKDD Conference on Knowledge Discovery and Data Mining, 2006.
24. M. Lucic, M. Faulkner, A. Krause, and D. Feldman. Training mixture models at scale via coresets. arXiv.org, Mar. 2017.
25. D. Maclaurin and R. P. Adams. Firefly Monte Carlo: Exact MCMC with subsets of data. In Uncertainty in Artificial Intelligence, Mar. 2014.
26. J. C. Mason and D. C. Handscomb. Chebyshev Polynomials. Chapman and Hall/CRC, New York, 2003.
27. T. P. Minka. Expectation propagation for approximate Bayesian inference. In Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc, Aug. 2001.
28. R. Nishihara, P. Moritz, S. Wang, A. Tumanov, W. Paul, J. Schleier-Smith, R. Liaw, M. Niknami, M. I. Jordan, and I. Stoica. Real-time machine learning: The missing pieces. In Workshop on Hot Topics in Operating Systems, 2017.
29. A. Pakman, D. Gilboa, D. Carlson, and L. Paninski. Stochastic bouncy particle sampler. In International Conference on Machine Learning, Sept. 2017.
30. N. S. Pillai and A. Smith. Ergodicity of approximate MCMC chains with applications to large data sets. arXiv.org, May 2014.
31. M. Pollock, P. Fearnhead, A. M. Johansen, and G. O. Roberts. The scalable Langevin exact algorithm: Bayesian inference for big data. arXiv.org, Sept. 2016.
32. M. Rabinovich, E. Angelino, and M. I. Jordan. Variational consensus Monte Carlo. arXiv.org, June 2015.
33. A. Rahimi and B. Recht. Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning. In Advances in Neural Information Processing Systems, pages 1313-1320, 2009.
34. S. L. Scott, A. W. Blocker, F. V. Bonassi, H. A. Chipman, E. I. George, and R. E. McCulloch. Bayes and big data: The consensus Monte Carlo algorithm. In Bayes 250, 2013.
35. S. Srivastava, V. Cevher, Q. Tran-Dinh, and D. Dunson. WASP: Scalable Bayes via barycenters of subset posteriors. In International Conference on Artificial Intelligence and Statistics, 2015.
36. M. Stephanou, M. Varughese, and I. Macdonald. Sequential quantiles via Hermite series density estimation. Electronic Journal of Statistics, 11(1):570-607, 2017.
37. G. Szegö. Orthogonal Polynomials. American Mathematical Society, 4th edition, 1975.
38. Y. W. Teh, A. H. Thiery, and S. Vollmer. Consistency and fluctuations for stochastic gradient Langevin dynamics. Journal of Machine Learning Research, 17(7):1-33, Mar. 2016.
39. L. Tierney and J. B. Kadane. Accurate approximations for posterior moments and marginal densities. Journal of the American Statistical Association, 81(393):82-86, 1986.
40. A. W. van der Vaart. Asymptotic Statistics. University of Cambridge, 1998.
41. M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In International Conference on Machine Learning, 2011.