Bayesian structure learning for markov random fields with a spike and slab prior

Uncertainty in Artificial Intelligence - Proceedings of the 28th Conference, UAI 2012

Volumn , Issue , 2012, Pages 174-184

  Chen, Yutian ^a   Welling, Max ^a  

^a Irvine   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN STRUCTURE LEARNING; CONNECTIVITY STRUCTURES; FULLY BAYESIAN APPROACHES; OPTIMAL REGULARIZATION; PREDICTIVE PERFORMANCE; REGULARIZATION PARAMETERS; REGULARIZED OPTIMIZATIONS; REVERSIBLE JUMP MCMC;

ARTIFICIAL INTELLIGENCE; MARKOV PROCESSES; MODEL STRUCTURES; PARAMETERIZATION; UNCERTAINTY ANALYSIS;

BAYESIAN NETWORKS;

EID: 84886056511     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. G. Andrew and J. Gao. Scalable training of L1-regularized log-linear models. In Proceedings of the 24th international conference on Machine learning, pages 33-40. ACM, 2007.
2. G. Bresler, E. Mossel, and A. Sly. Reconstruction of Markov random fields from samples: Some observations and algorithms. Approximation, Randomization and Combinatorial Optimization. Algorithms and Techniques, pages 343-356, 2008.
3. A. Courville, J. Bergstra, and Y. Bengio. A spike and slab restricted Boltzmann machine. Journal of Machine Learning Research, W&CP, 15, 2011.
4. S. Della Pietra, V. Della Pietra, and J. Lafferty. Inducing features of random fields. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(4):380-393, 1997.
5. M. Dudik, S. Phillips, and R. Schapire. Performance guarantees for regularized maximum entropy density estimation. Learning Theory, pages 472-486, 2004.
6. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2010. URL http://www.jstatsoft.org/v33/i01/.
7. M. Girolami and B. Calderhead. Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2):123-214, 2011.
8. P.J. Green. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4):711-732, 1995.
9. D. Hernandez-Lobato, J. Hernandez-Lobato, T. Helleputte, and P. Dupont. Expectation propagation for Bayesian multi-task feature selection. Machine Learning and Knowledge Discovery in Databases, pages 522-537, 2010.
10. H. Hofling and R. Tibshirani. Estimation of sparse binary pairwise Markov networks using pseudo-likelihoods. The Journal of Machine Learning Research, 10:883-906, 2009.
11. A.M. Horowitz. A generalized guided Monte Carlo algorithms. Physics Letters B, 268(2):247-252, 1991.
12. J. Hu, A. Joshi, and V.E. Johnson. Log-linear models for gene association. Journal of the American Statistical Association, 104(486):597-607, 2009.
13. H. Ishwaran and J.S. Rao. Spike and slab variable selection: frequentist and Bayesian strategies. The Annals of Statistics, 33(2):730-773, 2005.
14. B. Jones, C. Carvalho, A. Dobra, C. Hans, C. Carter, and M. West. Experiments in stochastic computation for high-dimensional graphical models. Statistical Science, 20(4):388-400, 2005.
15. S.I. Lee, V. Ganapathi, and D. Koller. Efficient structure learning of Markov networks using L1 regularization. In In NIPS. Citeseer, 2006.
16. S.Z. Li. Markov random field modeling in image analysis. Springer-Verlag New York Inc, 2009.
17. Y. Lin and D.D. Lee. Bayesian L1-norm sparse learning. In Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on, volume 5, pages V-V. IEEE, 2006.
18. T.J. Mitchell and J.J. Beauchamp. Bayesian variable selection in linear regression. Journal of the American Statistical Association, pages 1023-1032, 1988.
19. S. Mohamed, K. Heller, and Z. Ghahramani. Bayesian and L1 approaches to sparse unsupervised learning. Arxiv preprint arXiv:1106.1157, 2011.
20. I. Murray and Z. Ghahramani. Bayesian learning in undirected graphical models: approximate MCMC algorithms. In Proceedings of the 14th Annual Conference on Uncertainty in AI, pages 392-399, 2004.
21. I. Murray, Z. Ghahramani, and D.J.C. MacKay. MCMC for doubly-intractable distributions. In Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-06), Pittsburgh, PA, 2006.
22. R.M. Neal. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo: Methods and Applications, page 113, 2010.
23. S. Parise a.M.Welling. Structure learning in Markov random fields. Advances in Neural Information Processing Systems, 2006.
24. T. Park and G. Casella. The Bayesian lasso. Journal of the American Statistical Association, 103(482):681-686, 2008.
25. Y. Qi, M. Szummer, and T.P. Minka. Bayesian conditional random fields. In Artificial Intelligence and Statistics, 2005.
26. P. Ravikumar, M.J. Wainwright, and J.D. Lafferty. Highdimensional ising model selection using L1-regularized logistic regression. The Annals of Statistics, 38(3):1287-1319, 2010.
27. S. Riezler and A. Vasserman. Incremental feature selection and L1 regularization for relaxed maximum-entropy modeling. In Proceedings of EMNLP, volume 4, 2004.
28. G. Robins, P. Pattison, Y. Kalish, and D. Lusher. An introduction to exponential random graph models for social networks. Social networks, 29(2):173-191, 2007.
29. F. Sha and F. Pereira. Shallow parsing with conditional random fields. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 134-141. Association for Computational Linguistics, 2003.
30. T. Tieleman. Training restricted Boltzmann machines using approximations to the likelihood gradients. In Proceedings of the International Conference on Machine Learning, volume 25, pages 1064-1071, 2008.
31. Y. Tsuruoka, J. Tsujii, and S. Ananiadou. Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 477-485. Association for Computational Linguistics, 2009.
32. M.J. Wainwright, P. Ravikumar, and J.D. Lafferty. High-dimensional graphical model selection using L1-regularized logistic regression. Advances in neural information processing systems, 19:1465, 2007.
33. M.Welling and Y.W. Teh. Bayesian learning via stochastic gradient langevin dynamics. 2011.
34. J. Zhu, N. Lao, and E.P. Xing. Grafting-light: fast, incremental feature selection and structure learning of Markov random fields. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 303-312. ACM, 2010.