Learning mixtures of arbitrary distributions over large discrete domains

ITCS 2014 - Proceedings of the 2014 Conference on Innovations in Theoretical Computer Science

Volumn , Issue , 2014, Pages 207-223

  Rabani, Yuval ^a   Schulman, Leonard J ^b   Swamy, Chaitanya ^c  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Convex geometry; Linear programming; Mixture learning; Moment methods; Randomized algorithms; Spectral techniques; Topic models

Indexed keywords

COMPUTER SCIENCE; INFORMATION THEORY; LEARNING ALGORITHMS; LINEAR PROGRAMMING; METHOD OF MOMENTS; OPTIMIZATION;

ARBITRARY DISTRIBUTION; CONVEX GEOMETRY; DIMENSION REDUCTION; MIXTURE DISTRIBUTIONS; RANDOMIZED ALGORITHMS; SPECTRAL TECHNIQUES; TOPIC DISTRIBUTIONS; TOPIC MODEL;

MIXTURES;

EID: 84893305245     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2554797.2554818     Document Type: Conference Paper

References (43)

1. D. Achlioptas and F. McSherry. On spectral learning of mixtures of distributions. In Proc. 18th COLT, pages 458-469, 2005.
2. A. Anandkumar, D.P. Foster, D. Hsu, S.M. Kakade, and Y.-K. Liu. Two SVDs suffice: Spectral decompositions for probabilistic topic modeling and latent Dirichlet allocation. CoRR, abs/1204.6703, 2012. http://arxiv.org/abs/1204.6703.
3. A. Anandkumar, D. Hsu, and S.M. Kakade. A method of moments for mixture models and hidden Markov models. In Proc. 25th COLT, pages 33.1-33.34, 2012. http://arxiv.org/abs/1203.0683.
4. S. Arora, R. Ge, and A. Moitra. Learning topic models - going beyond SVD. In Proc. 53rd FOCS, pages 1-10, 2012.
5. S. Arora and R. Kannan. Learning mixtures of separated nonspherical Gaussians. Ann. Appl. Prob., 15: 69-92, 2005.
6. Y. Azar, A. Fiat, A. R. Karlin, F. McSherry, and J. Saia. Spectral analysis of data. In Proc. 33rd STOC, pages 619-626, 2001.
7. Z. Bar-Yossef, R. Kumar, and D. Sivakumar. Sampling algorithms: lower bounds and applications. In Proc. 33rd STOC, pages 266-275, 2001.
8. A. Barvinok. A Course in Convexity, Volume 54 of Graduate Studies in Mathematics. AMS, 2002.
9. T. Batu, S. Guha, and S. Kannan. Inferring mixtures of Markov chains. In Proc. 17th COLT, pages 186-199, 2004.
10. M. Belkin and K. Sinha. Polynomial learning of distribution families. In Proc. 51st FOCS, pages 103-112, 2010.
11. D.M. Blei. Probabilistic topic models. Commun. ACM, 55(4): 77-84, 2012.
12. D.M. Blei, A.Y. Ng, and M.I. Jordan. Latent Dirichlet allocation. J. Machine Learning Res., 3: 993-1022, 2003.
13. S.C. Brubaker and S. Vempala. Isotropic PCA and affine-invariant clustering. In Proc. 49th FOCS, pages 551-560, 2008.
14. K. Chaudhuri, E. Halperin, S. Rao, and S. Zhou. A rigorous analysis of population stratification with limited data. In Proc. 18th SODA, pages 1046-1055, 2007.
15. K. Chaudhuri and S. Rao. Beyond Gaussians: Spectral methods for learning mixtures of heavy-tailed distributions. In Proc. 21st COLT, pages 21-32, 2008.
16. K. Chaudhuri and S. Rao. Learning mixtures of product distributions using correlations and independence. In Proc. 21st COLT, pages 9-20, 2008.
17. S. Chung and K. Murty. Polynomially bounded ellipsoid algorithms for convex quadratic programming. In O. Mangasarian, R. Meyer, and S. Robinson, editors, Nonlinear Programming, Volume 4, pages 439-485. Academic Press, Orlando, Florida, 1981.
18. M. Cryan, L. Goldberg, and P. Goldberg. Evolutionary trees can be learned in polynomial time in the two state general Markov model. SICOMP, 31(2): 375-397, 2002.
19. A. Dasgupta, J. Hopcroft, J. Kleinberg, and M. Sandler. On learning mixtures of heavy-tailed distributions. In Proc. 46th FOCS, pages 491-500, 2005.
20. S. Dasgupta. Learning mixtures of Gaussians. In Proc. 40th FOCS, pages 634-644, 1999.
21. S. Dasgupta and L.J. Schulman. A probabilistic analysis of EM for mixtures of separated, spherical Gaussians. Journal of Machine Learning Research, 8: 203-226, 2007.
22. C. Daskalakis, I. Diakonikolas, and R.A. Servedio. Learning k-modal distributions via testing. In Proc. 23rd SODA, pages 1371-1385, 2012.
23. D. Dubhashi and A. Panconesi. Concentration of measure for the analysis of randomized algorithms. Cambridge University Press, 2009.
24. J. Feldman, R. O'Donnell, and R.A. Servedio. PAC learning mixtures of axis-aligned Gaussians with no separation assumption. In Proc. 19th COLT, pages 20-34, 2006.
25. J. Feldman, R. O'Donnell, and R.A. Servedio. Learning mixtures of product distributions over discrete domains. SIAM J. Comput., 37(5): 1536-1564, 2008.
26. Y. Freund and Y. Mansour. Estimating a mixture of two product distributions. In Proc. 12th COLT, pages 183-192, 1999.
27. N. Gravin, J. Lasserre, D. Pasechnik, and S. Robins. The inverse moment problem for convex polytopes. J. Discrete and Computational Geometry, 48(3): 596-621, 2012.
28. T. Hofmann. Probabilistic latent semantic analysis. In Proc. 15th UAI, pages 289-296, 1999.
29. T. Hofmann and J. Puzicha. Latent class models for collaborative filtering. In Proc. IJCAI, pages 688-693, 1999.
30. R. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1985.
31. A.T. Kalai, A. Moitra, and G. Valiant. Efficiently learning mixtures of two Gaussians. In Proc. 42nd STOC, pages 553-562, June 2010.
32. R. Kannan, H. Salmasian, and S. Vempala. The spectral method for general mixture models. SIAM J. Computing, 38(3): 1141-1156, 2008.
33. M. Kearns, Y. Mansour, D. Ron, R. Rubinfeld, R. Schapire, and L. Sellie. On the learnability of discrete distributions. In Proc. 26th STOC, pages 273-282, 1994.
34. J. Kleinberg and M. Sandler. Using mixture models for collaborative filtering. J. Comput. Syst. Sci., 74: 49-69, 2008.
35. F. McSherry. Spectral partitioning of random graphs. In Proc. 42nd FOCS, pages 529-537, October 2001.
36. A. Moitra and G. Valiant. Settling the polynomial learnability of mixtures of Gaussians. In Proc. 51st FOCS, pages 93-102, 2010.
37. E. Mossel and S. Roch. Learning nonsingular phylogenies and hidden Markov models. In Prof. 46th STOC, pages 366-375, 2005.
38. V. Y. Pan. Optimal and nearly optimal algorithms for approximating polynomial zeros. Computers & Mathematics with Applications, 31(12): 97-138, 1996.
39. C. Papadimitriou, P. Raghavan, H. Tamaki, and S. Vempala. Latent semantic indexing: A probabilistic analysis. J. Comput. Syst. Sci., 61(2): 217-235, 2000.
40. Y. Rabani, L.J. Schulman, and C. Swamy. Inference from sparse sampling. http://www.cs.technion.ac.il/~rabani/Papers/RabaniSS-manuscript.pdf, 2008.
41. S. Vempala and G. Wang. A spectral algorithm for learning mixtures of distributions. J. Comput. Syst. Sci., 68(4): 841-860, 2004.
42. C. Villani. Topics in optimal transportation, Volume 58 of Graduate Studies in Mathematics. AMS, 2003.
43. V. H. Vu. Spectral norm of random matrices. In Proc. 37th STOC, pages 423-430, 2005.