Learning continuous latent variable models with Bregman divergences

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 2842, Issue , 2003, Pages 190-204

  Wang, Shaojun ^a   Schuurmans, Dale ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Alternating minimization; Backward projection; Bregman divergence; Forward projection; Information geometry; Iterative scaling; Statistical machine learning; Unsupervised learning

Indexed keywords

CONSTRAINED OPTIMIZATION; ITERATIVE METHODS; MACHINE LEARNING; UNSUPERVISED LEARNING; FUNCTIONS; GEOMETRY; INFORMATION ANALYSIS; LEARNING ALGORITHMS; STATISTICAL METHODS;

ALTERNATING MINIMIZATION; BACKWARD PROJECTION; BREGMAN DIVERGENCES; FORWARD PROJECTION; INFORMATION GEOMETRY; ITERATIVE SCALING; STATISTICAL MACHINE LEARNING;

LEARNING ALGORITHMS; ARTIFICIAL INTELLIGENCE;

CONVEX FUNCTIONS; LATENT STRUCTURES; NOVEL TRAINING ALGORITHMS; STATISTICAL LEARNING ALGORITHMS;

EID: 0242278267     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-39624-6_16     Document Type: Conference Paper

References (25)

1. H. Bauschke and J. Borwein, “Joint and Separate Convexity of the Bregman Distance,” in: Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, Elsevier, 2001, pp. 23–36
2. J. Borwein and A. Lewis, “Duality relationships for entropy-like minimization problems,” SIAM J. Control Optim., Vol. 29, No. 2, pp. 325-338, 1991
3. J. Borwein; A. Lewis, Convex Analysis and Nolinear Optimization, Springer 2000
4. L. Bregman, “The relaxation method of finding the common point of convex sets and its applications to the solution of problems in convex programming,” USSR Computational Mathematics and Mathematical Physics, Vol. 7, pp. 200-217, 1967
5. A. Buja and W. Stuetzle, “Degrees of Boosting,” manuscript, 2002
6. C. Byrne and Y. Censor, “Proximity function minimization using multiple Bregman projections with applications to split feasibility and Kullback-Leibler distance minimization,” Annals of Operations Research, Vol. 105, pp. 77-98, 2001
7. Y. Censor and S. Zenios, Parallel Optimization: Theory, Algorithms, and Applications, Oxford University Press, 1997
8. M. Collins, R. Schapire and Y. Singer, “Logistic regression, AdaBoost and Bregman distances,” Machine Learning, Vol. 48, No. 1-3, pp. 253-285, 2002
9. I. Csiszar and G. Tusnady, “Information geometry and alternating minimization procedures,” Statistics and Decisions, Supplement Issue 1, pp. 205-237, 1984
10. I. Csiszar, “Why least squares and maximum entropy?” The Annals of Statisics, Vol. 19, No. 4, pp. 2032-2066, 1991
11. I. Csiszar, “Generalized projections for non-negative functions,” Acta Mathematica Hungarica, Vol. 68, No. 1-2, pp. 161-185, 1995
12. I. Csiszar, “Maxent, mathematics, and information theory,” Maximum Entropy and Bayesian Methods, Edited by K. Hanson and R. Silver, pp. 35-50, Kluwer, 1996
13. S. Della Pietra, V. Della Pietra and J. Lafferty, “Duality and auxiliary functions for Bregman distances,” Technical Report CMU-CS-01-109, CMU, 2001
14. A. Dempster, N. Laird and D. Rubin, “Maximum likelihood estimation from incomplete data via the EM algorithm,” J. Royal Stat. Soc. B, Vol. 39, pp 1-38, 1977
15. P. Eggermont and V. LaRiccia, “On EM-like algorithms for minimum distance estimation,” Technical Report, Mathematical Sciences, University of Delaware, 1998
16. J. Friedman, T. Hastie and R. Tibshirani, “Additive logistic regression: A statistical view of boosting,” Annals of Statistics, Vol. 28, No. 2, pp. 337-407, 2000
17. R. Johnson and J. Shore, “Which is the better entropy expression for speech processing:-S log S or log S?,” IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 32, No. 1, pp. 129-137, 1984
18. J. Lafferty, S. Della Pietra and V. Della Pietra, “Statistical learning algorithms based on Bregman distances,” Canadian Workshop on Info. Theory, pp. 77-80, 1997
19. J. Lafferty, “Additive models, boosting, and inference for generalized divergences,” Annual Conference on Computational Learning Theory, pp. 125-133, 1999
20. G. Lebanon and J. Lafferty, “Boosting and maximum likelihood for exponential models,” In Advances in Neural Information Processing Systems (NIPS), 14, 2002
21. D. Luenberger, Optimization by Vector Space Methods, John Wiley & Sons, 1969
22. V. Vapnik, The Natural of Statistical Learning Theory, Springer, 2000
23. T. Zhang, “Statistical behavior and consistency of classification methods based on convex risk minimization,” to appear in Annals of Statistics, 2004
24. S. Wang, D. Schuurmans and Y. Zhao, “The latent maximum entropy principle,” manuscript, 2002
25. S. Wang, D. Schuurmans, A. Ghodsi and J. Rosenthal, “Unsupervised Boosting with the Latent Maximum Entropy Principle,” manuscript, 2003