The more, the merrier: The blessing of dimensionality for learning large Gaussian Mixtures

Journal of Machine Learning Research

Volumn 35, Issue , 2014, Pages 1135-1164

  Anderson, Joseph ^a   Belkin, Mikhail ^a   Goyal, Navin ^b   Rademacher, Luis ^a   Voss, James ^a  

^a The Ohio State University   (United States)

^b MICROSOFT RESEARCH INDIA   (India)

Author keywords

Blessing of dimensionality; Gaussian mixture models; Independent Component Analysis; Smoothed analysis; Tensor methods

Indexed keywords

COVARIANCE MATRIX; GAUSSIAN DISTRIBUTION; INFORMATION THEORY; MATHEMATICAL TRANSFORMATIONS; TENSORS;

BLESSING OF DIMENSIONALITY; COMPUTATIONAL ASPECTS; GAUSSIAN MIXTURE MODEL; INDEPENDENT COMPONENT ANALYSIS(ICA); INFORMATION-THEORETIC LOWER BOUNDS; MIXTURES OF GAUSSIANS; SMOOTHED ANALYSIS; TENSOR METHOD;

INDEPENDENT COMPONENT ANALYSIS;

EID: 84936853791     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (31)

1. D. Achlioptas and F. McSherry. On spectral learning of mixture of distributions. In The 18th Annual Conference on Learning Theory, 2005.
2. L. Albera, A. Ferreol, P. Comon, and P. Chevalier. Blind Identification of Overcomplete MixturEs of sources (BIOME). Lin. Algebra Appl., 391:1-30, 2004.
3. Noga Alon and Joel H Spencer. The probabilistic method. Wiley, 2004.
4. S. Arora and R. Kannan. Learning Mixtures of Arbitrary Gaussians. In 33rd ACM Symposium on Theory of Computing, 2001.
5. Sanjeev Arora, Rong Ge, Ankur Moitra, and Sushant Sachdeva. Provable ICA with unknown Gaussian noise, with implications for Gaussian mixtures and autoencoders. In NIPS, pages 2384-2392, 2012.
6. Mikhail Belkin and Kaushik Sinha. Polynomial learning of distribution families. In FOCS, pages 103-112. IEEE Computer Society, 2010. ISBN 978-0-7695-4244-7.
7. Mikhail Belkin, Luis Rademacher, and James Voss. Blind signal separation in the presence of Gaussian noise. In JMLR W&CP, volume 30: COLT, pages 270-287, 2013.
8. Aditya Bhaskara, Moses Charikar, Ankur Moitra, and Aravindan Vijayaraghavan. Smoothed analysis of tensor decompositions. CoRR, abs/1311.3651v4, 2014.
9. Anthony Carbery and James Wright. Distributional and Lq norm inequalities for polynomials over convex bodies in Rn. Mathematical Research Letters, 8:233-248, 2001.
10. J-F Cardoso. Super-symmetric decomposition of the fourth-order cumulant tensor. Blind identification of more sources than sensors. In Acoustics, Speech, and Signal Processing, 1991. ICASSP-91., 1991 International Conference on, pages 3109-3112. IEEE, 1991.
11. J.-F. Cardoso and A. Souloumiac. Blind beamforming for non-gaussian signals. In Radar and Signal Processing, IEE Proceedings F, volume 140, pages 362-370, 1993.
12. Pierre Comon and Christian Jutten, editors. Handbook of Blind Source Separation. Academic Press, 2010.
13. A. Dasgupta. Probability for Statistics and Machine Learning. Springer, 2011.
14. S. Dasgupta. Learning Mixture of Gaussians. In 40th Annual Symposium on Foundations of Computer Science, 1999.
15. S. Dasgupta and L. Schulman. A Two Round Variant of EM for Gaussian Mixtures. In 16th Conference on Uncertainty in Artificial Intelligence, 2000.
16. J. Feldman, R. A. Servedio, and R. O'Donnell. PAC Learning Axis Aligned Mixtures of Gaussians with No Separation Assumption. In The 19th Annual Conference on Learning Theory, 2006.
17. Navin Goyal, Santosh Vempala, and Ying Xiao. Fourier PCA. CoRR, http://arxiv.org/abs/1306.5825, 2013.
18. Daniel Hsu and Sham M. Kakade. Learning mixtures of spherical Gaussians: moment methods and spectral decompositions. In ITCS, pages 11-20, 2013.
19. Maurice Kendall, Alan Stuart, and J. Keith Ord. Kendall's advanced theory of statistics. Vol. 1. Halsted Press, sixth edition, 1994. Distribution theory.
20. A. Moitra and G. Valiant. Settling the polynomial learnability of mixtures of Gaussians. In 51st Annual IEEE Symposium on Foundations of Computer Science (FOCS 2010), 2010.
21. Elchanan Mossel, Ryan O'Donnell, and Krzysztof Oleszkiewicz. Noise stability of functions with low influences: Invariance and optimality. Annals of Math., 171:295-341, 2010.
22. Ole A Nielsen. An Introduction to Integration Theory and Measure Theory. Wiley, 1997.
23. B. C. Rennie and A. J. Dobson. On Stirling numbers of the second kind. Journal of Combinatorial Theory, 7(2):116-121, 1969. ISSN 0021-9800. doi: http://dx.doi.org/10. 1016/S0021-9800 (69) 80045-1. URL http://www.sciencedirect.com/science/article/pii/S0021980069800451.
24. Christian Rieger and Barbara Zwicknagl. Sampling inequalities for infinitely smooth functions, with applications to interpolation and machine learning. Advances in Computational Mathematics, 32(1):103-129, 2010.
25. John Riordan. Moment recurrence relations for binomial, poisson and hypergeometric frequency distributions. Annals of Mathematical Statistics, 8:103-111, 1937.
26. Halsey Lawrence Royden, Patrick Fitzpatrick, and Prentice Hall. Real analysis, volume 4. Prentice Hall New York, 1988.
27. Mark Rudelson and Roman Vershynin. Smallest singular value of a random rectangular matrix. Comm. Pure Appl. Math., 62(12):1707-1739, 2009.
28. Paul Valiant and Gregory Valiant. Estimating the unseen: Improved estimators for entropy and other properties. In NIPS, pages 2157-2165, 2013.
29. S. Vempala and G. Wang. A Spectral Algorithm for Learning Mixtures of Distributions. In 43rd Annual Symposium on Foundations of Computer Science, 2002.
30. Holger Wendland. Scattered data approximation, volume 17. Cambridge University Press Cambridge, 2005.
31. A. Winkelbauer. Moments and Absolute Moments of the Normal Distribution. ArXiv e-prints, September 2012.