Provable tensor methods for learning mixtures of generalized linear models

Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016

Volumn , Issue , 2016, Pages 1223-1231

  Sedghi, Hanie ^a   Janzamin, Majid ^b   Anandkumar, Anima ^b  

^a Allen Institute for AI ^*  (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Mixture of generalized linear models; Score function; Spectral tensor decomposition

Indexed keywords

ARTIFICIAL INTELLIGENCE; MAXIMUM PRINCIPLE; MIXTURES; TENSORS;

CROSS CORRELATIONS; EXPECTATION MAXIMIZATION; FEATURE TRANSFORMATIONS; GENERALIZED LINEAR MODEL; LOW-ORDER POLYNOMIALS; REGRESSION PROBLEM; SCORE FUNCTION; TENSOR DECOMPOSITION;

LEARNING SYSTEMS;

EID: 85048938500     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Alekh Agarwal, Sham M Kakade, Nikos Karampatziakis, Le Song, and Gregory Valiant. Least squares revisited: Scalable approaches for multi-class prediction. In Proc. of ICML, 2014.
2. A. Anandkumar, R. Ge, D. Hsu, and S. M. Kakade. A Tensor Spectral Approach to Learning Mixed Membership Community Models. In COLT, June 2013.
3. A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models. J. of Machine Learning Research, 15:2773–2832, 2014a.
4. Anima Anandkumar, Rong Ge, and Majid Janzamin. Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods. arXiv preprint arXiv:1408.0553, Aug. 2014b.
5. Animashree Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. Tensor decompositions for learning latent variable models. Journal of Machine Learning Research, 15:2773–2832, 2014c.
6. Animashree Anandkumar, Rong Ge, and Majid Janzamin. Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates. arXiv preprint arXiv:1402.5180, Feb. 2014d.
7. Pranjal Awasthi, Avrim Blum, Or Sheffet, and Aravindan Vijayaraghavan. Learning mixtures of ranking models. In Proc. of NIPS, 2014.
8. Christopher M Bishop and Markus Svensen. Bayesian hierarchical mixtures of experts. In Proc. of Uncertainty in Artificial Intelligence, 2003.
9. Arun Chaganty and Percy Liang. Spectral experts for estimating mixtures of linear regressions. In Proc. of The 30th International Conference on Machine Learning, 2013.
10. Yudong Chen, Xinyang Yi, and Constantine Caramanis. A convex formulation for mixed regression with two components: Minimax optimal rates. In Conf. on Learning Theory, 2014.
11. F. Huang, U.N. Niranjan, M. Hakeem, and A. Anandkumar. Online Tensor Methods for Learning Latent Variable Models. Accepted to JMLR, 2014.
12. Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. In Journal of Machine Learning Research, pages 695–709, 2005.
13. Tommi Jaakkola, David Haussler, et al. Exploiting generative models in discriminative classifiers. In Advances in neural information processing systems, pages 487–493, 1999.
14. Majid Janzamin, Hanie Sedghi, and Anima Anandkumar. Score Function Features for Discriminative Learning: Matrix and Tensor Frameworks. arXiv preprint arXiv:1412.2863, Dec. 2014.
15. Michael I Jordan and Robert A Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural computation, 6(2):181–214, 1994.
16. Sham M Kakade, Adam Kalai, Varun Kanade, and Ohad Shamir. Efficient learning of generalized linear and single index models with isotonic regression. In NIPS, pages 927–935, 2011.
17. Nikos Karampatziakis and Paul Mineiro. Discriminative features via generalized eigenvectors. In Proceedings of The 31st International Conference on Machine Learning, pages 494–502, 2014.
18. Ker-Chau Li. Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414):316–327, 1991.
19. Ker-Chau Li. On principal hessian directions for data visualization and dimension reduction: another application of stein’s lemma. Journal of the American Statistical Association, 87(420):1025–1039, 1992.
20. Percy Liang, Alexandre Bouchard-Côté, Dan Klein, and Ben Taskar. An end-to-end discriminative approach to machine translation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 761–768. Association for Computational Linguistics, 2006.
21. Zhang Liu and Lieven Vandenberghe. Interior-point method for nuclear norm approximation with application to system identification. SIAM Journal on Matrix Analysis and Applications, 31(3):1235–1256, 2009.
22. Seewong Oh and Devavrat Shah. Learning mixed multinomial logit model from ordinal data. In Proc. of NIPS, 2014.
23. K. Pearson. Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society, London, A., page 71, 1894.
24. Slav Petrov and Dan Klein. Discriminative log-linear grammars with latent variables. In Advances in Neural Information Processing Systems, pages 1153–1160, 2007.
25. Ariadna Quattoni, Michael Collins, and Trevor Darrell. Conditional random fields for object recognition. In Advances in neural information processing systems, pages 1097–1104, 2004.
26. Charles Stein. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory