Detecting and correcting for label shift with black box predictors

35th International Conference on Machine Learning, ICML 2018

Volumn 7, Issue , 2018, Pages 4887-4897

  Lipton, Zachary C ^a,b   Wang, Yu Xiang ^b,c   Smola, Alexander J ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b AMAZON   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; DIAGNOSIS; IMAGE ENHANCEMENT; LEARNING SYSTEMS;

BLACK BOXES; CONFUSION MATRICES; HIGH DIMENSIONAL DATASETS; NATURAL IMAGES; SHIFT ESTIMATIONS; TEST SETS;

CLASSIFICATION (OF INFORMATION);

EID: 85057277412     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. Bickel, S., Bruckner, M., & Scheffer, T. (2009). Discriminative learning under covariate shift. Journal of Machine Learning Research, 10(Sep), 2137-2155.
2. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford university press.
3. Buck, A., Gart, J., et al. (1966). Comparison of a screening test and a reference test in epidemiologic studies, ii. a probabilistic model for the comparison of diagnostic tests. American Journal of Epidemiology.
4. Chan, Y. S., & Ng, H. T. (2005). Word sense disambiguation with distribution estimation. In Proceedings of the 19th international joint conference on Artificial intelligence, (pp. 1010-1015). Morgan Kaufmann Publishers Inc.
5. Deng, I., Dong, W., Socher, R., Li, L.-J., Li, K. & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In CVPR.
6. Elkan, C. (2001). The foundations of cost-sensitive learning. In IJCAI.
7. Forman, G. (2008). Quantifying counts and costs via classification. Data Mining and Knowledge Discovery.
8. Gretton, A., Borgwardt, K. M., Rasch, M. J., Scholkopf, B., & Smola, A. (2012). A kernel two-sample test. Journal of Machine Learning Research, 13(Mar), 723-773.
9. Gretton, A., Smola, A. J., Huang, J., Schmittfull, M., Borgwardt, K. M., & Scholkopf, B. (2009). Covariate shift by kernel mean matching. Journal of Machine Learning Research.
10. Heckman, J. I. (1977). Sample selection bias as a specification error (with an application to the estimation of labor supply functions).
11. Huang, J., Gretton, A., Borgwardt, K. M., Scholkopf, B., & Smola, A. J. (2007). Correcting sample selection bias by unlabeled data. In Advances in neural information processing systems.
12. Kang, J. D. & Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical science, 22(4), 523-539.
13. Manski, C. F. & Lerman, S. R. (1977). The estimation of choice probabilities from choice based samples. Econometrica: Journal of the Econometric Society.
14. Ramdas, A., Reddi, S. J., Poczos, B., Singh, A., & Wasserman, L. A. (2015). On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions. In AAAI, (pp. 3571-3577).
15. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55.
16. Saerens, M., Latinne, P., & Decaestecker, C. (2002). Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. Neural computation, 14(1), 21-41.
17. Scholkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K., & Mooij, J. (2012). On causal and anticausal learning. In International Coference on International Conference on Machine Learning (ICML-12), (pp. 459-466). Omni- press.
18. Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference.
19. Storkey, A. (2009). When training and test sets are different: Characterizing learning transfer. Dataset shift in machine learning.
20. Sugiyama, M., Nakajima, S., Kashima, H., Buenau, P. V., & Kawanabe, M. (2008). Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in neural information processing systems.
21. Zadrozny, B. (2004). Learning and evaluating classifiers under sample selection bias. In Proceedings of the twenty- first international conference on Machine learning, (p. 114). ACM.
22. Zaremba, W., Gretton, A., & Blaschko, M. (2013). B-test: A non-parametric, low variance kernel two-sample test. In Advances in neural information processing systems, (pp. 755-763).
23. Zhang, K., Scholkopf, B., Muandet, K & Wang, Z. (2013). Domain adaptation under target and conditional shift. In International Conference on Machine Learning, (pp. 819- 827).
24. Zhu, X., Gibson, B. R., Jun, K.-S., Rogers, T. T., Harrison, J., & Kalish, C. (2010). Cognitive models of test-item effects in human category learning. In ICML.