The Variational Garrote

Machine Learning

Volumn 96, Issue 3, 2014, Pages 269-294

  Kappen, Hilbert J ^a   Gómez, Vicenç ^a  

^a RADBOUD UNIVERSITY   (Netherlands)

Author keywords

Mean field; Nonnegative garrote; Sparse regression; Spike and slab; Variational approximation

Indexed keywords

INDUSTRIAL HEATING; LAGRANGE MULTIPLIERS; REGRESSION ANALYSIS;

MEAN-FIELD; NON NEGATIVES; SPARSE REGRESSION; SPIKE-AND-SLAB; VARIATIONAL APPROXIMATION;

NUMERICAL METHODS;

EID: 84905499203     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-013-5427-7     Document Type: Article

References (31)

1. Titsias, M., & Lázaro-Gredilla, M. (2011). Spike and slab variational inference for multi-task and multiple kernel learning. In Advances in neural information processing systems (Vol. 24, pp. 2339-2347).
2. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, Methodological, 58(1), 267-288.
3. Friedman, J. H., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1-22.
4. Frank, I. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109-135.
5. George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88(423), 881-889.
6. Mitchell, T. J., & Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404), 1023-1032.
7. Brown, P. J., Vannucci, M., & Fearn, T. (1998). Multivariate Bayesian variable selection and prediction. Journal of the Royal Statistical Society, Series B, Statistical Methodology, 60(3), 627-641.
8. Clyde, M., & George, E. I. (2004). Model uncertainty. Statistical Science, 19(1), 81-94.
9. Ishwaran, H., & Rao, J. S. (2005). Spike and slab variable selection: frequentist and Bayesian strategies. The Annals of Statistics, 33(2), 730-773.
10. Carbonetto, P., & Stephens, M. (2012). Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies. Bayesian Analysis, 7(1), 73-108.
11. Logsdon, B. A., Hoffman, G. E., & Mezey, J. G. (2010). A variational Bayes algorithm for fast and accurate multiple locus genome-wide association analysis. BMC Bioinformatics, 11, 58.
12. Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37(2), 183-233.
13. Kappen, H. J., & Spanjers, J. J. (2000). Mean field theory for asymmetric neural networks. Physical Review E, 61, 5658-5663.
14. Murphy, K. P., Weiss, Y., & Jordan, M. I. (1999). Loopy Belief Propagation for approximate inference: an empirical study. In Proceedings of the 15th annual conference on uncertainty in artificial intelligence (pp. 467-475). San Francisco: Morgan Kaufmann.
15. Opper, M., & Saad, D. (2001). Advanced mean field methods: theory and practice. Cambridge: MIT Press.
16. Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1-2), 1-305.
17. Barber, D., & Wiegerinck, W. (1999). Tractable variational structures for approximating graphical models. In Advances in neural information processing systems II (pp. 183-189). Cambridge: MIT Press.
18. Breiman, L. (1995). Better subset regression using the nonnegative garrote. Technometrics, 37(4), 373-384.
19. O'Hara, R. B., & Sillanpää, M. J. (2009). A review of Bayesian variable selection methods: what, how and which. Bayesian Analysis, 4(1), 85-118.
20. Attias, H. (1999). Independent factor analysis. Neural Computation, 11(4), 803-851.
21. Yoshida, R., & West, M. (2010). Bayesian learning in sparse graphical factor models via variational mean-field annealing. Journal of Machine Learning Research, 99, 1771-1798.
22. Hernández-Lobato, D., Hernández-Lobato, J. M., Helleputte, T., & Dupont, P. (2010). Expectation propagation for Bayesian multi-task feature selection. In Proceedings of the 2010 European conference on machine learning and knowledge discovery in databases: part I (pp. 522-537). New York: Springer.
23. Tibshirani, R. (2011). Regression shrinkage and selection via the lasso: a retrospective. Journal of the Royal Statistical Society, Series B, Statistical Methodology, 73(3), 273-282.
24. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348-1360.
25. Mazumder, R., Friedman, J. H., & Hastie, T. (2011). SparseNet: coordinate descent with nonconvex penalties. Journal of the American Statistical Association, 106(495), 1125-1138.
26. Park, T., & Casella, G. (2008). The Bayesian lasso. Journal of the American Statistical Association, 103(482), 681-686.
27. Griffin, J. E., & Brown, P. J. (2011). Bayesian hyper-lassos with non-convex penalization. Australian & New Zealand Journal of Statistics, 53(4), 423-442.
28. Zhao, P., & Yu, B. (2006). On model selection consistency of lasso. Journal of Machine Learning Research, 7, 2541-2563.
29. Candes, E. J., & Tao, T. (2005). Decoding by linear programming. IEEE Transactions on Information Theory, 51(12), 4203-4215.
30. Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289-1306.
31. p -norm minimization. Journal of Statistical Mechanics: Theory and Experiment, 2009(09), L09003.