Learning mixtures of spherical gaussians: Moment methods and spectral decompositions

ITCS 2013 - Proceedings of the 2013 ACM Conference on Innovations in Theoretical Computer Science

Volumn , Issue , 2013, Pages 11-19

  Hsu, Daniel ^a   Kakade, Sham M ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

method of moments; mixture models; mixtures of gaussians; spectral decomposition

Indexed keywords

COMPUTATIONALLY EFFICIENT; EFFICIENT ESTIMATION; ESTIMATION PROBLEM; GAUSSIANS; MIXTURE COMPONENTS; MIXTURE MODEL; MIXTURES OF GAUSSIANS; MOMENT-BASED ESTIMATORS; PARAMETER ESTIMATE; SPECTRAL DECOMPOSITION; SPECTRAL DECOMPOSITION TECHNIQUE;

COMPUTATIONAL EFFICIENCY; COMPUTER SCIENCE; INDEPENDENT COMPONENT ANALYSIS; INFORMATION THEORY; METHOD OF MOMENTS; PARAMETER ESTIMATION; SPHERES;

MIXTURES;

EID: 84873361882     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2422436.2422439     Document Type: Conference Paper

References (27)

1. E. S. Allman, C. Matias, and J. A. Rhodes. Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics, 37(6A):3099-3132, 2009.
2. A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models, 2012. Manuscript.
3. A. Anandkumar, D. Hsu, and S. M. Kakade. A method of moments for mixture models and hidden Markov models. In COLT, 2012.
4. S. Arora, R. Ge, A. Moitra, and S. Sachdeva. Provable ICA with unknown Gaussian noise, and implications for Gaussian mixtures and autoencoders. In NIPS, 2012.
5. S. Arora and R. Kannan. Learning mixtures of arbitrary Gaussians. In STOC, 2001.
6. M. Belkin and K. Sinha. Polynomial learning of distribution families. In FOCS, 2010.
7. A. Bunse-Gerstner, R. Byers, and V. Mehrmann. Numerical methods for simultaneous diagonalization. SIAM Journal on Matrix Analysis and Applications, 14(4):927-949, 1993.
8. J.-F. Cardoso and P. Comon. Independent component analysis, a survey of some algebraic methods. In IEEE International Symposium on Circuits and Systems Circuits and Systems Connecting the World, 1996.
9. J. T. Chang. Full reconstruction of Markov models on evolutionary trees: Identifiability and consistency. Mathematical Biosciences, 137:51-73, 1996.
10. K. Chaudhuri and S. Rao. Learning mixtures of product distributions using correlations and independence. In COLT, 2008.
11. P. Comon. Independent component analysis, a new concept? Signal Processing, 36(3):287-314, 1994.
12. P. Comon and C. Jutten. Handbook of Blind Source Separation: Independent Component Analysis and Applications. Academic Press. Elsevier, 2010.
13. S. Dasgupta. Learning mixutres of Gaussians. In FOCS, 1999.
14. S. Dasgupta and L. Schulman. A probabilistic analysis of EM for mixtures of separated, spherical Gaussians. Journal of Machine Learning Research, 8(Feb):203-226, 2007.
15. A. M. Frieze, M. Jerrum, and R. Kannan. Learning linear transformations. In FOCS, 1996.
16. D. Hsu, S. M. Kakade, and T. Zhang. An analysis of random design linear regression, 2011. arXiv:1106.2363.
17. D. Hsu, S. M. Kakade, and T. Zhang. A spectral algorithm for learning hidden Markov models. Journal of Computer and System Sciences, 78(5):1460-1480, 2012.
18. A. Hyvärinen and E. Oja. Independent component analysis: algorithms and applications. Neural Networks, 13(4-5):411-430, 2000.
19. A. T. Kalai, A. Moitra, and G. Valiant. Efficiently learning mixtures of two Gaussians. In STOC, 2010.
20. B. G. Lindsay and P. Basak. Multivariate normal mixtures: a fast consistent method. Journal of the American Statistical Association, 88(422):468-476, 1993.
21. J. B. MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 281-297. University of California Press, 1967.
22. A. Moitra and G. Valiant. Settling the polynomial learnability of mixtures of Gaussians. In FOCS, 2010.
23. E. Mossel and S. Roch. Learning nonsingular phylogenies and hidden Markov models. Annals of Applied Probability, 16(2):583-614, 2006.
24. P. Q. Nguyen and O. Regev. Learning a parallelepiped: Cryptanalysis of GGH and NTRU signatures. Journal of Cryptology, 22(2):139-160, 2009.
25. K. Pearson. Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society, London, A., page 71, 1894.
26. D. M. Titterington, A. F. M. Smith, and U. E. Makov. Statistical analysis of finite mixture distributions. Wiley, 1985.
27. S. Vempala and G. Wang. A spectral algorithm for learning mixtures of distributions. In FOCS, 2002.