Learning mixtures of product distributions over discrete domains

SIAM Journal on Computing

Volumn 37, Issue 5, 2007, Pages 1536-1564

  Feldman, Jon ^a   O'Donnell, Ryan ^b   Servedio, Rocco A ^c  

^a GOOGLE INC   (United States)

^b 7121 Wean Hall ^*  (United States)

^c Department of Earth and Environmental Sciences   (United States)

Author keywords

Computational learning theory; Mixture distributions; PAC learning; Poduct distributions

Indexed keywords

ANNUAL CONFERENCES; COMPUTATIONAL LEARNING THEORIES; COMPUTATIONAL LEARNING THEORY; DISCRETE DOMAINS; EVOLUTIONARY TREES; LEARNING FRAMEWORKS; MIXTURE DISTRIBUTIONS; NEW YORK; OPEN PROBLEMS; PAC LEARNING; PODUCT DISTRIBUTIONS; PROBABLY APPROXIMATELY CORRECT; PRODUCT DISTRIBUTIONS; TIME ALGORITHMS; UNIVERSITY OF WARWICK; WARWICK;

ALGORITHMS; APPROXIMATION ALGORITHMS; BOOLEAN FUNCTIONS; EDUCATION; GRAPH THEORY; IMAGE SEGMENTATION; LEARNING SYSTEMS; MIXTURES; POLYNOMIAL APPROXIMATION; TREES (MATHEMATICS);

LEARNING ALGORITHMS;

EID: 55249099345     PISSN: 00975397     EISSN: None     Source Type: Journal    
DOI: 10.1137/060670705     Document Type: Article

References (24)

1. S. AARONSON, Multilinear formulas and skepticism, of quantum computation, in Proceedings of the 36th Annual Symposium on Theory of Computing (STOC), Chicago, IL, 2004, ACM, New York, pp. 118-127.
2. S. ARORA AND R. KANNAN, Learning mixtures of arbitrary Gaussians, in Proceedings of the 33rd Annual Symposium on Theory of Computing (STOC), Heraklion, Crete, 2001, ACM, New York, pp. 247-257.
3. A. BLUM, Rank-r decision trees are a subclass of r-decision lists, Inform. Process. Lett., 42 (1992), pp. 183-185.
4. A. BLUM, Learning a function of r relevant variables (open problem), in Proceedings of the 16th Annual Conference on Learning Theory and 7th Kernel Workshop, Washington, DC, 2003, pp. 731-733.
5. A. BLUM, M. FURST, J. JACKSON, M. KEARNS, Y. MANSOUR, AND S. RUDICH, Weakly learning DNF and characterizing statistical query learning using Fourier analysis, in Proceedings of the 26th Annual Symposium on Theory of Computing (STOC), Montreal, QC, 1994, ACM, New York, pp. 253-262.
6. S. CHAWLA, C. DWORK, F. MCSHERRY, A. SMITH, AND H. WEE, Towards privacy in public databases, in Proceedings of the 2nd Theory of Cryptography Conference (TCC), 2005, pp. 363-385.
7. T. COVER AND J. THOMAS, Elements of Information Theory, Wiley, New York, 1991.
8. M. CRYAN, Learning and Approximation Algorithms for Problems Motivated by Evolutionary Trees, Ph.D. thesis, Computer Science, University of Warwick, Warwick, UK, 1999.
9. M. CRYAN, L. A. GOLDBERG, AND P. W. GOLDBERG, Evolutionary trees can be learned in polynomial time in the two-state general Markov model, SIAM J, Comput., 31 (2001), pp. 375-397.
10. S. DASGUPTA, Learning mixtures of Gaussians, in Proceedings of the 40th Annual Symposium on Foundations of Computer Science (FOCS), New York, 1999, IEEE Computer Society Press, Piscataway, NJ, pp. 634-644.
11. S. DASGUPTA AND L. SCHULMAN, A two-round variant of EM for Gaussian mixtures, in Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, Seattle, WA, 2000, Morgan Kaufmann, San Francisco, pp. 143-151.
12. A. EHRENFEUCHT AND D. HAUSSLER, Learning decision trees from random examples, Inform. and Comput., 82 (1989), pp. 231-246.
13. J. FELDMAN, R. O'DONNELL, AND R. SERVEDIO, PAC learning mixtures of axis-aligned Gaussians with no separation assumption, in Proceedings of the 19th Annual Conference on Learning Theory (COLT), Pittsburgh, PA, 2006, Springer, New York, pp. 20-34; available online at http://arxiv.org/abs/cs.LG/0609093.
14. Y. FREUND AND Y. MANSOUR, Estimating a mixture of two product distributions, in Proceedings of the 12th Annual Conference on Computational Learning Theory, Santa Cruz, CA, 1999, ACM, New York, pp. 183-192.
15. S. GOREINOV, E. TYRTYSHNIKOV, AND N. ZAMARASHKIN, A theory of pseudoskeleton approximations, Linear Algebra Appl., 261 (1997), pp. 1-21.
16. W. HOEFFDING, Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc, 58 (1963), pp. 13-30.
17. M. KEARNS, Efficient noise-tolerant learning from statistical queries, J. ACM, 45 (1998), pp. 983-1006.
18. M. KFARNS, Y. MANSOUR, D. RON, R. RUBINFELD, R. SCHAPIRE, AND L. SELLIE, On the learnability of discrete distributions, in Proceedings of the 26th Annual Symposium on Theory of Computing (STOC), Montréal, QC, 1994, ACM, New York, pp. 273-282.
19. B. LINDSAY, Mixture Models: Theory, Geometry, and Applications, Institute for Mathematical Statistics, Beachwood, OH, 1995.
20. E. MOSSEL AND S. ROCH, Learning nonsingular phylogenies and hidden Markov models, in Proceedings of the 37th Annual Symposium on Theory of Computing (STOC), Baltimore, MD, 2005, pp. 366-375.
21. R. A. REDNER, AND H. F. WALKER, Mixture densities, maximum likelihood, and the EM algorithm, SIAM Rev., 26 (1984), pp. 195-239.
22. A. SMITH, Personal communication, 2005.
23. D. M. TITTERINGTON, A. F. M. SMITH, AND U. E. MAKOV, Statistical Analysis of Finite Mixture Distributions, Wiley, New York, 1985.
24. S. VEMPALA AND G. WANG, A spectral algorithm for learning mixtures of distributions, in Proceedings of the 43rd Annual Symposium on Foundations of Computer Science (FOCS), Vancouver, BC, 2002, IEEE Computer Society Press, Piscataway, NJ, pp. 113-122.