Gibbs max-margin topic models with data augmentation

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 1073-1110

  Zhu, Jun ^a   Chen, Ning ^a   Perkins, Hugh ^a   Zhang, Bo ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Gibbs classifiers; Max margin learning; Regularized Bayesian inference; Supervised topic models; Support vector machines

Indexed keywords

BAYESIAN NETWORKS; INFERENCE ENGINES; SEMANTICS;

ACCURATE PREDICTION; BAYESIAN INFERENCE; CLASSIFICATION PERFORMANCE; CONDITIONAL DISTRIBUTION; MAX-MARGIN LEARNING; MULTI-LABEL CLASSIFICATIONS; POSTERIOR DISTRIBUTIONS; TOPIC MODEL;

SUPPORT VECTOR MACHINES;

EID: 84899824631     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (52)

1. Amr Ahmed, Mohamed Aly, Joseph Gonzalez, Shravan Narayanamurthy, and Alex Smola. Scalable inference in latent variable models. In International Conference on Web Search and Data Mining (WSDM), pages 123-132, 2012.
2. Edoardo M. Airoldi, David M. Blei, Stephen E. Fienberg, and Eric P. Xing. Mixed mem- bership stochastic blockmodels. Journal of Machine Learning Research (JMLR), 9:1981- 2014, 2008.
3. Rie K. Ando and Tong Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research (JMLR), (6):1817-1853, 2005.
4. Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. Multi-task feature learning. In Advances in Neural Information Processing Systems (NIPS), pages 41-48, 2007.
5. David M. Blei and John D. McAulié. Supervised topic models. In Advances in Neural Information Processing Systems (NIPS), pages 121-128, 2007.
6. David M. Blei, Andrew Ng, and Michael I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research (JMLR), 3:993-1022, 2003.
7. Olivier Catoni. PAC-Bayesian supervised classification: The thermodynamics of statistical learning. Monograph Series of the Institute of Mathematical Statistics, 2007.
8. Jonathan Chang and David Blei. Relational topic models for document networks. In Artificial Intelligence and Statistics (AISTATS), pages 81-88, 2009.
9. Ning Chen, Jun Zhu, Fuchun Sun, and Eric P. Xing. Large-margin predictive latent subspace learning for multiview data analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 34(12):2365-2378, 2012.
10. Ning Chen, Jun Zhu, Fei Xia, and Bo Zhang. Generalized relational topic models with data augmentation. In International Joint Conference on Artificial Intelligence (IJCAI), pages 1273-1279, 2013.
11. Koby Crammer and Yoram Singer. On the algorithmic implementation of multiclass kernel- based vector machines. Journal of Machine Learning Research (JMLR), (2):265-292, 2001.
12. Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Ser. B, (39):1-38, 1977.
13. Luc Devroye. Non-uniform Random Variate Generation. Springer-Verlag, 1986.
14. Li Fei-Fei and Pietro Perona. A Bayesian hierarchical model for learning natural scene categories. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 524-531, 2005.
15. Kuzman Ganchev, Joao Graca, Jennifer Gillenwater, and Ben Taskar. Posterior regular- ization for structured latent variable models. Journal of Machine Learning Research (JMLR), 11:2001-2049, 2010.
16. Pascal Germain, Alexandre Lacasse, Francois Laviolette, and Mario Marchand. PAC- Bayesian learning of linear classifiers. In International Conference on Machine Learning (ICML), pages 353-360, 2009.
17. Thomas Griths and Mark Steyvers. Finding scientific topics. Proceedings of National Academy of Science (PNAS), pages 5228-5235, 2004. (Pubitemid 38469131)
18. James P. Hobert. The Data Augmentation Algorithm: Theory and Methodology. In Hand- book of Markov Chain Monte Carlo (S. Brooks, A. Gelman, G. Jones and X.-L. Meng, eds.). Chapman & Hall/CRC Press, Boca Raton, FL., 2011.
19. Chris C. Holmes and Leonhard Held. Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Analysis, 1(1):145-168, 2006.
20. Tommi Jaakkola, Marina Meila, and Tony Jebara. Maximum entropy discrimination. In Advances in Neural Information Processing Systems (NIPS), 1999.
21. Tony Jebara. Discriminative, Generative and Imitative Learning. PhD thesis, Media Lab- oratory, MIT, Dec 2001.
22. Qixia Jiang, Jun Zhu, Maosong Sun, and Eric P. Xing. Monte Carlo methods for maximum margin supervised topic models. In Advances in Neural Information Processing Systems (NIPS), pages 1601-1609, 2012.
23. Thorsten Joachims. Making Large-scale SVM Learning Practical. MIT press, 1999.
24. Simons Lacoste-Jullien, Fei Sha, and Michael I. Jordan. DiscLDA: Discriminative learn- ing for dimensionality reduction and classification. In Advances in Neural Information Processing Systems (NIPS), pages 897-904, 2009.
25. David McAllester. PAC-Bayesian stochastic model selection. Machine Learning, 51:5-21, 2003.
26. Xiao-Li Meng and David A. van Dyk. The EM algorithm - an old folk song sung to a fast new tune. Journal of the Royal Statistical Society, Ser. B, (59):511-567, 1997. (Pubitemid 127686967)
27. John R. Michael, William R. Schucany, and Roy W. Haas. Generating random variates using transformations with multiple roots. The American Statistician, 30(2):88-90, 1976.
28. Kevin Miller, M. Pawan Kumar, Ben Packer, Danny Goodman, and Daphne Koller. Max- margin min-entropy models. In Artificial Intelligence and Statistics (AISTATS), pages 779-787, 2012.
29. Radford M. Neal. Markov chain Monte Carlo methods based on 'slicing' the density function. Technical Report No. 9722, Department of Statistics, University of Toronto, 1997.
30. David Newman, Arthur Asuncion, Padhraic Smyth, and Max Welling. Distributed algo- rithms for topic models. Journal of Machine Learning Research (JMLR), (10):1801-1828, 2009.
31. Nicholas G. Polson and Steven L. Scott. Data augmentation for support vector machines. Bayesian Analysis, 6(1):1-24, 2011.
32. Ryan Rifkin and Aldebaro Klautau. In defense of one-vs-all classification. Journal of Machine Learning Research (JMLR), (5):101-141, 2004.
33. Alex Smola and Shravan Narayanamurthy. An architecture for parallel topic models. Very Large Data Base (VLDB), 3(1-2):703-710, 2010.
34. Alex Smola and Bernhard Scholkopf. A tutorial on support vector regression. Statistics and Computing, 14(3):199-222, 2003. (Pubitemid 39063488)
35. Robert H. Swendsen and Jian-Sheng Wang. Nonuniversal critical dynamics in Monte Carlo simulations. Physical Review Letters, (58):86-88, 1987.
36. Aixin Tan. Convergence Rates and Regeneration of the Block Gibbs Sampler for Bayesian Random Effects Models. PhD thesis, Department of Statistics, University of Florida, 2009.
37. Martin A. Tanner and Wing-Hung Wong. The calculation of posterior distributions by data augmentation. Journal of the Americal Statistical Association (JASA), 82(398):528-540, 1987.
38. Ben Taskar, Carlos Guestrin, and Daphne Koller. Max-margin Markov networks. In Ad- vances in Neural Information Processing Systems (NIPS), 2003.
39. Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. Mining multi-label data. Data Mining and Knowledge Discovery Handbook, 2nd ed., pages 667-685, 2010.
40. David van Dyk and Xiao-Li Meng. The art of data augmentation. Journal of Computational and Graphical Statistics (JCGS), 10(1):1-50, 2001. (Pubitemid 33721606)
41. Vladimir N. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1995.
42. Minjie Xu, Jun Zhu, and Bo Zhang. Bayesian nonparametric maximum margin matrix factorization for collaborative prediction. In Advances in Neural Information Processing Systems (NIPS), pages 64-72, 2012.
43. Minjie Xu, Jun Zhu, and Bo Zhang. Fast max-margin matrix factorization with data augmentation. In International Conference on Machine Learning (ICML), pages 978- 986, 2013.
44. Shuanghong Yang, Jiang Bian, and Hongyuan Zha. Hybrid generative/discriminative learning for automatic image annotation. In Uncertainty in Artificial Intelligence (UAI), pages 683-690, 2010.
45. Chun-Nam Yu and Thorsten Joachims. Learning structural SVMs with latent variables. In International Conference on Machine Learning (ICML), pages 1169-1176, 2009.
46. Jun Zhu. Max-margin nonparametric latent feature models for link prediction. In International Conference on Machine Learning (ICML), pages 719-726, 2012.
47. Jun Zhu and Eric P. Xing. Conditional topic random fields. In International Conference on Machine Learning (ICML), pages 1239-1246, 2010.
48. Jun Zhu, Ning Chen, and Eric P. Xing. Infinite SVM: Dirichlet process mixtures of large- margin kernel machines. In International Conference on Machine Learning (ICML), pages 617-624, 2011.
49. Jun Zhu, Amr Ahmed, and Eric. P. Xing. MedLDA: maximum margin supervised topic models. Journal of Machine Learning Research (JMLR), (13):2237-2278, 2012.
50. Jun Zhu, Ning Chen, Hugh Perkins, and Bo Zhang. Gibbs max-margin topic models with fast inference algorithms. In International Conference on Machine Learning (ICML), pages 124-132, 2013a.
51. Jun Zhu, Xun Zheng, Li Zhou, and Bo Zhang. Scalable inference in max-margin topic mod- els. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 964-972, 2013b.
52. Jun Zhu, Ning Chen, and Eric P. Xing. Bayesian inference with posterior regularization and applications to infinite latent SVMs. Journal of Machine Learning Research (JMLR, in press) (Technical Report, arXiv:1210.1766v3), 2014.