A new PAC-Bayesian perspective on domain adaptation

33rd International Conference on Machine Learning, ICML 2016

Volumn 2, Issue , 2016, Pages 1368-1379

  Germain, Pascal ^a,b   Habrard, Amaury ^c   Laviolette, François ^d   Morvant, Emilie ^c  

^a INRIA   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c UNIVERSITÉ JEAN MONNET   (France)

^d UNIVERSITÉ LAVAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ECONOMIC AND SOCIAL EFFECTS; LEARNING SYSTEMS; RISK ASSESSMENT;

BAYESIAN DOMAINS; BAYESIAN PERSPECTIVE; DOMAIN ADAPTATION; ERROR MEASURES; GENERALIZATION BOUND; LINEAR CLASSIFIERS; MAJORITY-VOTE MODEL; RATIO CONTROL;

LEARNING ALGORITHMS;

EID: 84999040133     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (49)

1. Ambroladze, A., Parrado-Hernández, E., and Shawe-Taylor, J. Tighter PAC-Bayes bounds. In NIPS, pp. 9-16, 2006.
2. Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F. Analysis of representations for domain adaptation. In NIPS, pp. 137-144, 2006.
3. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. Wortman. A theory of learning from different domains. Mach. Learn., 79 (1-2). T51-175, 2010.
4. Ben-David, S., Shalev-Shwartz, S., and Urner, R. Domain adaptation-can quantity compensate for quality? In ISAIM, 2012.
5. Berlind, C. and Urner, R. Active nearest neighbors in changing environments. In ICML, pp. 1870-1879, 2015.
6. Blitzer, J. Domain adaptation of natural language processing systems. PhD thesis, UPenn, 2007.
7. Blitzer, J., McDonald, R., and Pereira, F. Domain adaptation with structural correspondence learning. In EMNLP, pp. 120-128, 2006.
8. Bruzzone, L. and Marconcini, M. Domain adaptation problems: A DASVM classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Inrel., 32(5):770-787, 2010.
9. Catoni, O. PAC-Bayesian supervised classification: the thermodynamics of statistical learning, volume 56. Inst, of Mathematical Statistic, 2007.
10. Chen, M., Weinberger, K. Q., and Blitzer, J. Co-training for domain adaptation. In NIPS, pp. 2456-2464, 2011.
11. Chen, M., Xu, Z. E., Weinberger, K. Q., and Sha, F. Marginalized denoising autoencoders for domain adaptation. In ICML, pp. 767-774, 2012.
12. Cortes, C. and Mohri, M. Domain adaptation and sample bias correction theory and algorithm for regression. Theor. Comput. Sci., 519:103-126, 2014.
13. Cortes, C, Mansour, Y., and Mohri, M. Learning bounds for importance weighting. In NIPS, pp. 442-450, 2010.
14. Cortes, C, Mohri, M., and Medina, A. Muñoz. Adaptation algorithm and theory based on generalized discrepancy. In ACM SIGKDD, pp. 169-178, 2015.
15. Daumé III, H. Frustratingly easy domain adaptation. In ACL, 2007.
16. Ganin, Y and Lempitsky, V. S. Unsupervised domain adaptation by backpropagation. In ICML, pp. 1180-1189, 2015.
17. Ganin, Y, Ustinova, E., H, Ajakan, Germain, P., Larochelle, H., Laviolette, F, Marchand, M., and Lempitsky, V. Domain-adversarial training of neural networks. JMLR, 17(59):1-35, 2016.
18. Germain, P., Laçasse, A., Laviolette, F, and Marchand, M. PAC-Bayesian learning of linear classifiers. In ICML, pp. 353-360, 2009.
19. Germain, P., Habrard, A., Laviolette, F., and Morvant, E. A PAC-Bayesian approach for domain adaptation with specialization to linear classifiers. In ICML, pp. 738-746, 2013.
20. Germain, P., Lacasse, A., Laviolette, F., Marchand,., and Roy, J.-F. Risk bounds for the majority vote: From a PAC-Bayesian analysis to a learning algorithm. JMLR, 16:787-860, 2015.
21. Herbrich, R. and Graepel, T. A PAC-Bayesian margin bound for linear classifiers: Why svms work. In NIPS, pp. 224-230, 2000.
22. Huang, J., Smola, A., Gretton, A., Borgwardt, K., and Schölkopf, B. Correcting sample selection bias by unlabeled data. In NIPS, pp. 601-608, 2006.
23. Jiang, J. A literature survey on domain adaptation of statistical classifiers, 2008.
24. Laçasse, A., Laviolette, F., Marchand, M., Germain, P., and Usunier, N. PAC-Bayes bounds for the risk of the majority vote and the variance of the Gibbs classifier. In NIPS, pp. 769-776, 2006.
25. Langford, J. and Shawe-Taylor, J. PAC-Bayes & margins. In NIPS, pp. 439-446, 2002.
26. Li, X. and Bilmes, J. A Bayesian divergence prior for classiffier adaptation. In AISTATS, pp. 275-282, 2007.
27. Liu, Q., Mackey, A. J., Roos, D. S., and Pereira, F. Evigan: a hidden variable model for integrating gene evidence for eukaryotic gene prediction. Bioinformatics, 24(5):597-605, 2008.
28. Mansour, Y, Mohri, M., and Rostamizadeh, A. Domain adaptation: Learning bounds and algorithms. In COLT, 2009.
29. Margolis, A. A literature review of domain adaptation with unlabeled data, 2011.
30. Maurer, A. A note on the PAC-Bayesian theorem. CoRR, cs. LG/0411099, 2004.
31. McAllester, D. A. Some PAC-Bayesian theorems. Mach. Learn., 37:355-363, 1999.
32. McAllester, D. A. and Keshet, J. Generalization bounds and consistency for latent structural probit and ramp loss. In NIPS, pp. 2205-2212, 2011.
33. Morvant, E., Habrard, A., and Ayache, S. Parsimonious Unsupervised and Semi-Supervised Domain Adaptation with Good Similarity Functions. KAIS, 33(2):309-349, 2012.
34. Pan, S. J. and Yang, Q. A survey on transfer learning. T. Knowl. Data En., 22(10):1345-1359, 2010.
35. Parrado-Hernández, E., Ambroladze, A., Shawe-Taylor, J., and Sun, S. PAC-Bayes bounds with data dependent priors. JMLR, 13:3507-3531, 2012.
36. Patel, V. M., Gopalan, R., Li, R., and Chellappa, R. Visual domain adaptation: A survey of recent advances. IEEE Signal Proc. Mag., 32(3):53-69, 2015.
37. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V, Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V, Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. JMLR, 12:2825-2830, 2011.
38. Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset shift in machine learning. The MIT Press, 2009.
39. Shimodaira, H. Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Statist. Plann. Inference, 90(2):227-244, 2000.
40. Sugiyama, M., Nakajima, S., Kashima, H., von Biinau, P., and Kawanabe, M. Direct importance estimation with model selection and its application to covariate shift adaptation. In NIPS, 2007.
41. Urner, R., Shalev-Shwartz, S., and Ben-David, S. Access to unlabeled data can speed up prediction time. In ICML, pp. 641-648, 2011.
42. Zhang, C, Zhang, L., and Ye, J. Generalization bounds for domain adaptation. In NIPS, pp. 3320-3328, 2012.
43. Zhong, E., Fan, W., Yang, Q., Verscheure, O., and Ren, J. Cross validation framework to choose amongst models and datasets for transfer learning. In ECML-PKDD, pp. 547-562, 2010.
44. Germain, P., Lacasse, A., Laviolette, F., and Marchand, M. PAC-Bayesian learning of linear classifiers. In ICML, pp. 353-360, 2009.
45. Germain, P., Habrard, A., Laviolette, F., and Morvant, E. A PAC-Bayesian approach for domain adaptation with specialization to linear classifiers. In ICML, pp. 738-746, 2013.
46. Jones, E., Oliphant, T., Peterson, P., et al. SciPy: Open source scientific tools for Python, 2001-. URL http://www.sc ipy.org/.
47. Maurer, A. A note on the PAC-Bayesian theorem. CoRR, cs. LG/0411099, 2004.
48. McAllester, D. A PAC-Bayesian tutorial with a dropout bound. CoRR, abs/1307.2118, 2013.
49. Seidin, Y. and Tishby, N. PAC-Bayesian analysis of co-clustering and beyond. JMLR, 11:3595-3646, 2010.