Maximum entropy distribution estimation with generalized regularization

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4005 LNAI, Issue , 2006, Pages 123-138

  Dudík, Miroslav ^a   Schapire, Robert E ^a  

^a Princeton University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONSTRAINT THEORY; INFORMATION ANALYSIS; PROBLEM SOLVING; RECURSIVE FUNCTIONS; USER INTERFACES;

ENTROPY DISTRIBUTION; MAXENT PROBLEMS; REGULARIZATION FUNCTIONS;

ALGORITHMS;

EID: 33746050365     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11776420_12     Document Type: Conference Paper

References (21)

1. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106 (1957) 620-630
2. Berger, A.L., Della Pietra, S.A., Della Pietra, V.J.: A maximum entropy approach to natural language processing. Computational Linguistics 22(1) (1996) 39-71
3. Della Pietra, S., Della Pietra, V., Lafferty, J.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4) (1997) 1-13
4. Phillips, S.J., Dudík, M., Schapire, R.E.: A ME approach to species distribution modeling. In: Proceedings of the Twenty-First International Conference on Machine Learning. (2004)
5. Lau, R.: Adaptive statistical language modeling. Master's thesis, MIT Department of Electrical Engineering and Computer Science (1994)
6. Chen, S.F., Rosenfeld, R.: A survey of smoothing techniques for ME models. IEEE Transactions on Speech and Audio Processing 8(1) (2000) 37-50
7. Lebanon, G., Lafferty, J.: Boosting and maximum likelihood for exponential models. Technical Report CMU-CS-01-144, CMU School of Computer Science (2001)
8. Zhang, T.: Class-size independent generalization analysis of some discriminative multicategory classification. In: Advances in Neural Information Processing Systems 17. (2005)
9. Goodman, J.: Exponential priors for maximum entropy models. In: Conference of the North American Chapter of the Association for Computational Linguistics. (2004)
10. Kazaina, J., Tsujii, J.: Evaluation and extension of ME models with inequality constraints. In: Conference on Empirical Methods in Natural Language Processing. (2003) 137-144
11. Dudík, M., Phillips, S.J., Schapire, R.E.: Performance guarantees for regularized maximum entropy density estimation. In: COLT 2004. (2004) 472-486
12. Altun, Y., Smola, A.: Unifying divergence minimization and statistical inference via convex duality. In: COLT 2006. (2006)
13. Dudík, M., Schapire, R.E., Phillips, S.J.: Correcting sample selection bias in ME density estimation. In: Advances in Neural Information Processing Systems 18. (2006)
14. Collins, M., Schapire, R.E., Singer, Y.: Logistic regression, AdaBoost and Bregman distances. Machine Learning 48(1) (2002) 253-285
15. Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. The Annals of Mathematical Statistics 43(5) (1972) 1470-1480
16. Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: Proceedings of the Sixth Conference on Natural Language Learning. (2002) 49-55
17. Krishnapuram, B., Carin, L., Figueiredo, M.A.T., Hartemink, A.J.: Sparse multinomial logistic regression: Fast algorithms and generalization bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(6) (2005) 957-968
18. 2 regularization, and rotational invariance. In: Proceedings of the Twenty-First International Conference on Machine Learning. (2004)
19. Newman, W.: Extension to the ME method. IEEE Trans. on Inf. Th. IT-23(1) (1977) 89-93
20. Rockafellar, R.T.: Convex Analysis. Princeton University Press (1970)
21. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)