A Comparison of New and Old Algorithms for a Mixture Estimation Problem

Machine Learning

Volumn 27, Issue 1, 1997, Pages 97-119

  Helmbold, David P ^a   Schapire, Robert E ^b   Singer, Yoram ^b   Warmuth, Manfred K ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

EM; Exponentiated gradient algorithms; Maximum likelihood; Mixture models

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; ESTIMATION; ITERATIVE METHODS; MATHEMATICAL MODELS; POLYNOMIALS; THEOREM PROVING; VECTORS;

EXPONENTIAL GRADIENT ALGORITHMS; MAXIMUM LIKELIHOOD (ML) ESTIMATION;

LEARNING ALGORITHMS;

EID: 0031122868     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007301011561     Document Type: Review

References (17)

1. Abe, N., Takeuchi, J., and Warmuth, M. (1991). Polynomial learnability of probablistic concepts with respect to the Kullback-Leibler divergence. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 277-289. Morgan Kaufmann.
2. Bridle, J. (1989). Probabilistic interpretation of feedforward classification network outputs with relationships to statistical pattern recognition. In Fogelman-Souli, F. and Hérault, J., editors, Neuro-Computing: Algorithms, Architectures, and Applications. New York: Springer Verlag.
3. Cover, T. (1991). Universal portfolios. Mathematical Finance, 1(1):1-29.
4. Dempster, A., Laird, N., and Rubin, D. (1977). Maximum-likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B39:1-38.
5. Duda, R. and Hart, P. (1973). Pattern Classification and Scene Analysis. Wiley.
6. Golub, G. and Van Loan, C. (1989). Matrix Computations. Johns-Hopkins University Press.
7. Helmbold, D., Schapire, R. E., Singer, Y., and Warmuth, M. K. (1996). On-line portfolio selection using multiplicative updates. In Proc. 13th International Conference on Machine Learning, pages 243-251. Morgan Kaufmann, San Francisco.
8. Kivinen, J. and Warmuth, M. (1995a). Additive versus exponentiated gradient updates. In Proceedings of the Twenty-Seventh Annual ACM Symposium on Theory of Computing.
9. Kivinen, J. and Warmuth, M. (1995b). The perceptron algorithm vs. winnow: linear vs. logarithmic mistake bounds when few input variables are relevant. In Proceedings of the Eighth Annual Workshop on Computational Learning Theory.
10. Littlestone, N. (1988). Learning when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2:285-318.
11. Luenberger, D. (1984). Linear and Nonlinear Programming. Addison-Wesley.
12. Meng, X. and Rubin, D. (1992). Recent extensions of the EM algorithm (with discussion). In Bernardo, J., Berger, J., Dawid, A., and Smith, A., editors, Bayesian Statistics, 4. Oxfod: Clarendon Press.
13. Neal, R. and Hinton, G. (1993). A new view of the EM algorithm that justifies incremental and other variants. Unpublished manuscript.
14. Peters, B. and Walker, H. (1978a). An iterative procedure for obtaining maximum-likelihood estimates of the parameters for a mixture of normal distributions. SIAM Journal of Applied Mathematics, 35:362-378.
15. Peters, B. and Walker, H. (1978b). The numerical evaluation of the maximum-likelihood estimates of a subset of mixture proportions. SIAM Journal of Applied Mathematics, 35:447-452.
16. Redner, R. and Walker, H. (1984). Mixture densities, maximum likelihood, and the EM algorithm. Siam Review, 26:195-239.
17. Singer, Y. and Warmuth, M. (1996). Training algorithms for hidden markov models using entropy based distance functions. To appear in Advances in Neural Information Processing Systems, 8.