Maximum likelihood in cost-sensitive learning: Model specification, approximations, and upper bounds

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 3313-3332

  Dmochowski, Jacek P ^a   Sajda, Paul ^b   Parra, Lucas C ^a  

^a City University of New York   (United States)

^b Columbia University ^*  (United States)

Author keywords

Cost sensitive learning; Empirical risk minimization; Imbalanced data sets; Loss function

Indexed keywords

ASYMPTOTICALLY OPTIMAL; CONVEX UPPER BOUND; COST-SENSITIVE; COST-SENSITIVE LEARNING; EMPIRICAL RESULTS; EMPIRICAL RISK MINIMIZATION; IMBALANCED DATA SETS; LOG LIKELIHOOD; LOSS FUNCTION; MISCLASSIFICATION COSTS; MISSPECIFICATION; MODEL SET; MODEL SPECIFICATIONS; PARAMETRIC ESTIMATION; PATTERN CLASSIFICATION; REAL WORLD DATA; RISK MINIMIZATION; THRESHOLDING; UPPER BOUND;

MAXIMUM LIKELIHOOD ESTIMATION; OPTIMIZATION; RISK PERCEPTION; SPECIFICATIONS;

COSTS;

EID: 79551493545     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. A. Asuncion and D. J. Newman. Uci machine learning repository, 2007. URL http://archive. ics.uci.edu/ml/.
2. L. Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996.
3. N. V. Chawla and N. Japkowicz. Editorial: Special issue on learning from imbalanced data sets. SIGKDD Explorations, 6:2004, 2004.
4. P. Domingos. Metacost: A general method for making classifiers cost-sensitive. In In Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, pages 155-164. ACM Press, 1999.
5. C. Drummond and R. C. Holte. Explicitly representing expected cost: An alternative to roc representation. In In Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 198-207. ACM Press, 2000.
6. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. Wiley, New York, 2001.
7. M. Dudik and S. J. Phillips. Generative and discriminative learning with unknown labeling bias. Advances in Neural information Processing Systems, 21, 2009.
8. J. P. Egan. Signal Detection Theory and ROC Analysis. Academic Press, New York, NY, 1975.
9. C. Elkan. The foundations of cost-sensitive learning. In In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pages 973-978, 2001.
10. G. Elliott and R. P. Lieli. Predicting binary outcomes. Technical report, 2007.
11. T. Fawcett. Roc graphs: notes and practical considerations for researchers. Technical Report HPL- 2003-4, 2004.
12. R. Fletcher. Practical Methods of Optimization. Wiley, West Sussex, 2000.
13. A. Guerrero-Curieses, J. Cid-Sueiro, R. Alaiz-Rodriguez, and A. R. Figureras-Vidal. Local estimation of posterior class probabilities to minimize classification errors. IEEE Transactions on Neural Networks, 15:309-317, 2004.
14. J. L. Horowitz. A smoothed maximum score estimator for the binary response model. Econometrica, 60:505-531, 1992.
15. S. M. Kay. Fundamentals of statistical signal processing: estimation theory. 1993.
16. R. P. Lieli and H. White. The construction of empirical credit scoring rules based on maximization principles. Journal of Econometrics, 157:110-119, 2010.
17. Y. Lin, Y. Lee, and G. Wahba. Support vector machines for classification in nonstandard situations. Machine Learning, 46:191-202, 2002. (Pubitemid 34129968)
18. M. A. Maloof. Learning when data sets are imbalanced and when costs are unequal and unknown. In ICML-2003 Workshop on Learning from Imbalanced Data Sets II, 2003.
19. D. D. Margineantu. On class-probability estimates and cost-sensitive evaluation of classifiers. In In Workshop on Cost-Sensitive Learning at the Seventeenth International Conference on Machine Learning (WCSL at ICML2000), 2000.
20. H. Masnadi-Shirazi and N. Vasconcelos. Risk minimization, probability elicitation, and costsensitive svms. In Proceedings of the International Conference on Machine Learning, pages 204-213. ACM Press, 2010.
21. P. McCullagh and J. A. Nelder. Generalized Linear Models, 2nd ed. Chapman and Hall, London, 1989.
22. L. Parra, C. Spence, A. Gerson, and P. Sajda. Recipes for the linear analysis of eeg. NeuroImage, 28:326-341, 2005.
23. L. Parra, C. Christoforou, A. Gerson,M. Dyrholm, A. Luo,M.Wagner,M. Philiastides, and P. Sajda. Spatio-temporal linear decoding of brain state: Application to performance augmentation in highthroughput tasks. IEEE Signal Processing Magazine, 25:95-115, 2008.
24. F. Provost and T. Fawcett. Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pages 43-48. AAAI Press, 1997.
25. H. Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Statistical Planning and Inference, 90:227-244, 2000.
26. V. N. Vapnik. Statistical Learning Theory. Wiley, New York, NY, 1998.
27. V. N. Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5):988-999, 1999.
28. H. White. Maximum likelihood estimation of misspecified models. Econometrica, 50:1-25, 1982.
29. B. Zadrozny and C. Elkan. Learning and making decisions when costs and probabilities are both unknown. In In Proceedings of the Seventh International Conference on Knowledge Discovery and Data Mining, pages 204-213. ACM Press, 2001.
30. B. Zadrozny, J. Langford, and N. Abe. Cost-sensitive learning by cost-proportionate example weighting. In n Proceedings of the Third IEEE International Conference on Data Mining, ICDM 2003., pages 435-442. IEEE, 2003.