A convex formulation for mixed regression with two components: Minimax optimal rates

Journal of Machine Learning Research

Volumn 35, Issue , 2014, Pages 560-604

  Chen, Yudong ^a   Yi, Xinyang ^b   Caramanis, Constantine ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b The University of Texas at Austin   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX OPTIMIZATION; INFORMATION THEORY; RECOVERY; STOCHASTIC SYSTEMS;

ARBITRARY NOISE; OPTIMAL RATE; OPTIMIZATION FORMULATIONS; REGRESSION PROBLEM; SAMPLE COMPLEXITY; STOCHASTIC NOISE; TRACTABLE ALGORITHMS; TWO-COMPONENT;

OPTIMIZATION;

EID: 84939617201     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (28)

1. Anima Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. Tensor decompositions for learning latent variable models. CoRR, abs/1210.7559, 2012.
2. Martin Azizyan, Aarti Singh, and Larry Wasserman. Minimax theory for high-dimensional gaussian mixtures with sparse mean separation. arXiv preprint arXiv:1306.2035, 2013.
3. Lucien Birgé. Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrsch. verw. Gebiete, 65(2):181-237, 1983.
4. T. Tony Cai and Anru Zhang. ROP: Matrix recovery via rank-one projections. arXiv preprint arXiv:1310.5791, 2013.
5. Emmanuel Candès and Yaniv Plan. Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements. IEEE Transactions on Information Theory, 57(4):2342-2359, 2011.
6. Arun Chaganty and Percy Liang. Spectral experts for estimating mixtures of linear regressions. In International Conference on Machine Learning (ICML), 2013.
7. Jiahua Chen. Optimal rate of convergence for finite mixture models. The Annals of Statistics, pages 221-233, 1995.
8. Yuxin Chen, Yuejie Chi, and Andrea Goldsmith. Exact and stable covariance estimation from quadratic sampling via convex programming. arXiv preprint arXiv:1310.0807, 2013.
9. Thomas M Cover and Joy A Thomas. Elements of information theory. Wiley, 2012.
10. Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), pages 1-38, 1977.
11. Ehsan Elhamifar and René Vidal. Sparse subspace clustering. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 2790-2797. IEEE, 2009.
12. Bettina Grün and Friedrich Leisch. Applications of finite mixtures of regression models. URL: http://cran. r-project.org/web/packages/flexmix/vignettes/regression-examples.pdf, 2007.
13. Daniel Hsu and Sham M. Kakade. Learning gaussian mixture models: Moment methods and spectral decompositions. CoRR, abs/1206.5766, 2012.
14. Geoffrey McLachlan and David Peel. Finite Mixture Models. Wiley series in probability and statistics: Applied probability and statistics. Wiley, 2004. ISBN 9780471654063. URL http://books.google.com/books?id=7M5vK8OpXZ4C.
15. Benjamin Recht, Maryam Fazel, and Pablo A. Parrilo. Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization. SIAM Review, 52(471), 2010.
16. Mark Rudelson and Roman Vershynin. Hanson-wright inequality and sub-gaussian concentration. arXiv preprint arXiv:1306.2872, 2013.
17. Mahdi Soltanolkotabi, Ehsan Elhamifar, and Emmanuel Candes. Robust subspace clustering. arXiv preprint arXiv:1301.2603, 2013.
18. Nicolas Stadler, Peter Buhlmann, and Sara Geer. L1-penalization for mixture regression models. TEST, 19(2):209-256, 2010. ISSN 1133-0686.
19. Yuekai Sun, Stratis Ioannidis, and Andrea Montanari. Learning mixtures of linear classifiers. arXiv preprint arXiv:1311.2547, 2013.
20. J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12(4):389-434, 2012.
21. Alexandre B. Tsybakov. Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, 2009.
22. Roman Vershynin. Introduction to the non-asymptotic analysis of random matrices. Arxiv preprint arxiv:1011.3027, 2010.
23. Kert Viele and Barbara Tong. Modeling with mixtures of linear regressions. Statistics and Computing, 12(4), 2002. ISSN 0960-3174. URL http://dx.doi.org/10.1023/A% 3A1020779827503.
24. Yu-Xiang Wang and Huan Xu. Noisy sparse subspace clustering. In Proceedings of The 30th International Conference on Machine Learning, pages 89-97, 2013.
25. C. F. Wu. On the convergence properties of the EM algorithm. The Annals of Statistics, 11(1):95-103, 1983.
26. Y. Yang and A. Barron. Information-theoretic determination of minimax rates of convergence. The Annals of Statistics, 27(5):1564-1599, 1999.
27. Xinyang Yi, Constantine Caramanis, and Sujay Sanghavi. Alternating minimization for mixed linear regression. Arxiv preprint arxiv:1310.3745, 2013.
28. Bin Yu. Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam, pages 423-435. Springer, 1997.