Classification with Noisy Labels by Importance Reweighting

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 3, 2016, Pages 447-461

  Liu, Tongliang ^a   Tao, Dacheng ^a  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

Classification; consistency; importance reweighting; label noise; noise rate estimation

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION);

CONDITIONAL PROBABILITIES; CONSISTENCY; LOSS FUNCTIONS; NOISE RATE; OPTIMAL CLASSIFIERS; RE-WEIGHTING; REAL DATA SETS; UNOBSERVABLE;

COMPUTER VISION;

EID: 84962090090     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2015.2456899     Document Type: Article

References (52)

1. D. Angluin and P. Laird, "Learning from noisy examples," Mach. Learn., vol. 2, no. 4, pp. 343-370, 1988.
2. L. G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
3. M. Kearns, "Efficient noise-tolerant learning from statistical queries," J. ACM, vol. 45, no. 6, pp. 983-1006, 1998.
4. N. D. Lawrence and B. Schölkopf, "Estimating a kernel Fisher discriminant in the presence of label noise," in Proc. 8th Int. Conf. Mach. Learn., 2001, pp. 306-313.
5. B. Biggio, B. Nelson, and P. Laskov, "Support vector machines under adversarial label noise," in Proc. Asian Conf. Mach. Learn., 2011, pp. 97-112.
6. T. Yang, M. Mahdavi, R. Jin, L. Zhang, and Y. Zhou, "Multiple kernel learning from noisy labels by stochastic programming," in Proc. 29th Int. Conf. Mach. Learn., 2012, pp. 233-240.
7. B. Frenay and M. Verleysen, "Classification in the presence of label noise: A survey," IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 845-869, May 2014.
8. J. A. Aslam and S. E. Decatur, "On the sample complexity of noise-tolerant learning," Inf. Process. Lett., vol. 57, no. 4, pp. 189-195, 1996.
9. N. Manwani and P. Sastry, "Noise tolerance under risk minimization," IEEE Trans. Cybern., vol. 43, no. 3, pp. 1146-1151, Jun. 2013.
10. N. Natarajan, I. Dhillon, P. Ravikumar, and A. Tewari, "Learning with noisy labels," in Proc. Neural Inf. Process. Syst., 2013, pp. 1196-1204.
11. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe, "Convexity, classification, and risk bounds," J. Am. Statist. Assoc., vol. 101, no. 473, pp. 138-156, 2006.
12. C. Scott, "Calibrated asymmetric surrogate losses," Electron. J. Statist., vol. 6, pp. 958-992, 2012.
13. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, 2006.
14. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Roy. Statist. Soc.: Series B (Statist. Methodol.), vol. 67, no. 2, pp. 301-320, 2005.
15. T. K. Pong, P. Tseng, S. Ji, and J. Ye, "Trace norm regularization: reformulations, algorithms, and multi-task learning," SIAM J. Optim., vol. 20, no. 6, pp. 3465-3489, 2010.
16. Y. Nesterov, "Smooth minimization of non-smooth functions," Math. Program., vol. 103, no. 1, pp. 127-152, 2005.
17. P. Gong, C. Zhang, Z. Lu, J. Huang, and J. Ye, "A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems," in Proc. Int. Conf. Mach. Learn., 2013, pp. 37-45.
18. M. Kearns and M. Li, "Learning in the presence of malicious errors," SIAM J. Comput., vol. 22, no. 4, pp. 807-837, 1993.
19. N. Cesa-Bianchi, E. Dichterman, P. Fischer, E. Shamir, and H. U. Simon, "Sample-efficient strategies for learning in the presence of noise," J. ACM, vol. 46, no. 5, pp. 684-719, 1999.
20. N. H. Bshouty, N. Eiron, and E. Kushilevitz, "PAC learning with nasty noise," Theoretical Comput. Sci., vol. 288, no. 2, pp. 255-275, 2002.
21. P. M. Long and R. A. Servedio, "Learning large-margin halfspaces with more malicious noise," in Proc. 25th Annu. Conf. Neural Inf. Process. Syst., 2011, pp. 91-99.
22. A. R. Klivans, P. M. Long, and R. A. Servedio, "Learning halfspaces with malicious noise," J. Mach. Learn. Res., vol. 10, pp. 2715-2740, 2009.
23. N. Cesa-Bianchi, S. Shalev-Shwartz, and O. Shamir, "Online learning of noisy data," IEEE Trans. Inf. Theory, vol. 57, no. 12, pp. 7907-7931, 2011.
24. T. Bylander, "Learning linear threshold functions in the presence of classification noise," in Proc. 7th Annu. Conf. Comput. Learn. Theory, 1994, pp. 340-347.
25. E. Cohen, "Learning noisy perceptrons by a perceptron in polynomial time," in Proc. 38th Annu. Symp. Found. Comput. Sci., 1997, pp. 514-523.
26. A. Blum, A. Frieze, R. Kannan, and S. Vempala, "A polynomialtime algorithm for learning noisy linear threshold functions," Algorithmica, vol. 22, no. 1-2, pp. 35-52, 1998.
27. G. Stempfel and L. Ralaivola, "Learning kernel perceptrons on noisy data using random projections," in Proc. 18th Int. Conf. Algorithmic Learn. Theory, 2007, pp. 328-342.
28. R. Khardon and G. Wachman, "Noise tolerant variants of the perceptron algorithm," J. Mach. Learn. Res., vol. 8, pp. 227-248, 2007.
29. H. Moore and N. P. S. M. CALIF. (1977). Robust Regression Using Maximum-Likelihood Weighting and Assuming Cauchy-Distributed Random Error. Defense Technical Information Center [Online]. Available: http://books.google.com.au/books?id=zbl8NwAACAAJ
30. W. Liu, P. P. Pokharel, and J. C. Prncipe, "Correntropy: Properties and applications in non-Gaussian signal processing," IEEE Trans. Signal Process., vol. 55, no. 11, pp. 5286-5298, Nov. 2007.
31. R. He, W. Zheng, and B. Hu, "Maximum correntropy criterion for robust face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1561-1576, Aug. 2011.
32. K. Crammer and D. D. Lee, "Learning via Gaussian herding," in Proc. Adv. Neural Inf. Process. Syst. 23, 2010, pp. 451-459.
33. C. Scott, G. Blanchard, and G. Handy, "Classification with asymmetric label noise: Consistency and maximal denoising," in Proc. Conf. Learn. Theory, 2013, pp. 489-511.
34. G. Blanchard, G. Lee, and C. Scott, "Semi-supervised novelty detection," J. Mach. Learn. Res., vol. 11, pp. 2973-3009, 2010.
35. C. Scott, "A rate of convergence for mixture proportion estimation, with application to learning from noisy labels," in Proc. 18th Int. Conf. Artif. Intell. Statist., 2015, pp. 838-846.
36. V. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer, 2000.
37. M. Anthony and P. L. Bartlett, Neural Network Learning: Theoretical Foundations. Cambridge, U.K.: Cambridge Univ. Press, 2009.
38. L. Bruzzone and M. Marconcini, "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 770-787, May 2010.
39. A. Gretton, A. Smola, J. Huang, M. Schmittfull, K. Borgwardt, and B. Schölkopf, "Covariate shift by kernel mean matching," in Dataset Shift in Machine Learning, J. Quiñonero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence, Eds. Cambridge, MA, USA: MIT Press, 2009, ch. 8, pp. 131-160.
40. M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations of Machine Learning. Cambridge, MA, USA: MIT Press, 2012.
41. P. L. Bartlett and S. Mendelson, "Rademacher and Gaussian complexities: Risk bounds and structural results," J. Mach. Learn. Res., vol. 3, pp. 463-482, 2003.
42. P. L. Bartlett, O. Bousquet, and S. Mendelson, "Local Rademacher complexities," The Ann. Statist., vol. 33, no. 4, pp. 1497-1537, 2005.
43. I. Steinwart, "Support vector machines are universally consistent," J. Complexity, vol. 18, no. 3, pp. 768-791, 2002.
44. M. Yamada, T. Suzuki, T. Kanamori, H. Hachiya, and M. Sugiyama, "Relative density-ratio estimation for robust distribution comparison," in Proc. Adv. Neural Inf. Process. Syst. 24, 2011, pp. 594-602.
45. V. Vapnik, I. Braga, and R. Izmailov, "Constructive setting of the density ratio estimation problem and its rigorous solution," arXiv preprint arXiv:1306.0407, 2013.
46. M. Sugiyama, T. Suzuki, and T. Kanamori, "Density ratio estimation: Acomprehensive review," RIMSKokyuroku., Kyoto University Press, 2010, pp. 10-31.
47. T. Kanamori, T. Suzuki, and M. Sugiyama, "Theoretical analysis of density ratio estimation," IEICE Trans. Fundam. Electron., Commun. Comput. Sci., vol. 93, no. 4, pp. 787-798, 2010.
48. J. Huang, A. Gretton, K. M. Borgwardt, B. Schölkopf, and A. J. Smola, "Correcting sample selection bias by unlabeled data," in Proc. Adv. Neural Inf. Process. Syst. 19, 2006, pp. 601-608.
49. T. Kanamori, S. Hido, and M. Sugiyama, "A least-squares approach to direct importance estimation," J. Mach. Learn. Res., vol. 10, pp. 1391-1445, 2009.
50. Q. Sun, R. Chattopadhyay, S. Panchanathan, and J. Ye, "A twostage weighting framework for multi-source domain adaptation," in Proc. Adv. Neural Inf. Process. Syst. 24, 2011, pp. 505-513.
51. R. Nock and F. Nielsen, "Bregman divergences and surrogates for learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 2048-2059, Nov. 2009.
52. M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe, "Direct importance estimation with model selection and its application to covariate shift adaptation," in Proc. Adv. Neural Inf. Process. Syst. 20, 2008, pp. 1433-1440.