Performance guarantees for regularized maximum entropy density estimation

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

Volumn 3120, Issue , 2004, Pages 472-486

  Dudik, Miroslav ^a   Phillips, Steven J ^b   Schapire, Robert E ^a  

^a Princeton University   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ENTROPY; MATHEMATICAL MODELS; PROBABILITY DENSITY FUNCTION; PROBLEM SOLVING; STATISTICAL METHODS; GEOGRAPHICAL DISTRIBUTION;

ENTROPY DENSITY ESTIMATION; GIBBS DISTRIBUTION; SENSITIVITY CURVES; STATISTICAL LEARNING; DENSITY ESTIMATES; EMPIRICAL AVERAGES; NON-ASYMPTOTIC BOUNDS; PERFORMANCE GUARANTEES; PRINCIPLE OF MAXIMUM ENTROPY; UNDERLYING DISTRIBUTION; UNIFORM CONVERGENCE;

COMPUTER SCIENCE; PROBABILITY DISTRIBUTIONS;

EID: 9444278508     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/978-3-540-27819-1_33     Document Type: Conference Paper

References (20)

1. Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39-71, 1996.
2. S. F. Chen and R. Rosenfeld. A survey of smoothing techniques for ME models. IEEE Transactions on Speech and Audio Processing, 8(1):37-50, January 2000.
3. Michael Collins, Robert E. Schapire, and Yoram Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48(1):253-285, 2002.
4. J. N. Darroch and D. Ratcliff. Generalized iterative scaling for log-linear models. The Annals of Mathematical Statistics, 43(5): 1470-1480, 1972.
5. Ofer Dekel, Shai Shalev-Shwartz, and Yoram Singer. Smooth e-insensitive regression by loss symmetrization. In Proceedings of the Sixteenth Annual Conference on Computational Learning Theory, pages 433-447. Springer, 2003.
6. Stephen Della Pietra, Vincent Della Pietra, and John Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):1-13, April 1997.
7. Luc Devroye. Bounds for the uniform deviation of empirical measures. Journal of Multivariate Analysis, 12:72-79, 1982.
8. Joshua Goodman. Exponential priors for maximum entropy models. Technical report, Microsoft Research, 2003, (Available from http://research.microsoft.com/ ~joshuago/longexponentialprior.ps).
9. E. T. Jaynes. Information theory and statistical mechanics. Physics Reviews, 106:620-630, 1957.
10. Jun'ichi Kazama and Jun'ichi Tsujii. Evaluation and extension of maximum entropy models with inequality constraints. In Conference on Empirical Methods in Natural Language Processing, pages 137-144, 2003.
11. Robert Malouf. A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning, pages 49-55, 2002.
12. Mark New, Mike Hulme, and Phil Jones. Representing twentieth-century space-time climate variability. Part 1: Development of a 1961-90 mean monthly terrestrial climatology. Journal of Climate, 12:829-856, 1999.
13. Steven J. Phillips, Miroslav Dudik, and Robert E. Schapire. A maximum entropy approach to species distribution modeling. In Proceedings of the Twenty-First International Conference on Machine Learning, 2004.
14. R. Tyrrell Rockafellar. Convex Analysis. Princeton University Press, 1970.
15. Saharon Rosset and Eran Segal. Boosting density estimation. In Advances in Neural Information Processing Systems 15, pages 641-648. MIT Press, 2003.
16. Ruslan Salakhutdinov, Sam T. Roweis, and Zoubin Ghahramani. On the convergence of bound optimization algorithms. In Uncertainty in Artificial Intelligence 19, pages 509-516, 2003.
17. J. R. Sauer, J. E. Hines, and J. Fallon. The North American breeding bird survey, results and analysis 1966-2000, Version 2001.2. http://www.mbr-pwrc. usgs.gov/bbs/bbs.html, 2001. USGS Patuxent Wildlife Research Center, Laurel, MD.
18. USGS. HYDRO 1k, elevation derivative database. Available at http://edcdaac.usgs.gov/gtopo30/hydro/, 2001. United States Geological Survey, Sioux Falls, South Dakota.
19. Max Welling, Richard S. Zemel, and Geoffrey E. Hinton. Self supervised boosting. In Advances in Neural Information Processing Systems 15, pages 665-672. MIT Press, 2003.
20. Peter M. Williams. Bayesian regularization and pruning using a Laplace prior. Neural Computation, 7(1):117-143, 1995.