Domain adaptation from multiple sources: A domain-dependent regularization approach

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 3, 2012, Pages 504-518

  Duan, Lixin ^a   Xu, Dong ^a   Tsang, Ivor Wai Hung ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Domain adaptation machine; domain dependent regularizer; multiple source domain adaptation

Indexed keywords

DOCUMENT RETRIEVAL; DOMAIN ADAPTATION; GENERALIZATION ABILITY; LABEL PREDICTIONS; REGULARIZATION APPROACH; REGULARIZER; SUPPORT VECTOR REGRESSION (SVR); VIDEO CONCEPT DETECTION;

DATA PROCESSING; LEAST SQUARES APPROXIMATIONS;

SUPPORT VECTOR MACHINES;

EID: 84862192949     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2011.2178556     Document Type: Article

References (49)

1. P. Wu and T. G. Dietterich, "Improving SVM accuracy by training on auxiliary data sources," in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, Jul. 2004, pp. 871-878.
2. J. Blitzer, R. McDonald, and F. Pereira, "Domain adaptation with structural correspondence learning," in Proc. Conf. Emp. Methods Natural Lang. Process., Sydney, Australia, Jul. 2006, pp. 120-128.
3. H. Daumé, III, "Frustratingly easy domain adaptation," in Proc. Annu. Meeting Assoc. Comput. Linguist., Prague, Czech Republic, Jun. 2007, pp. 256-263.
4. L. Duan, I. W. Tsang, D. Xu, and T.-S. Chua, "Domain adaptation from multiple sources via auxiliary classifiers," in Proc. Int. Conf. Mach. Learn., Montreal, QC, Canada, Jun. 2009, pp. 289-296.
5. J. Yang, R. Yan, and A. G. Hauptmann, "Cross-domain video concept detection using adaptive SVMs," in Proc. Int. Conf. Multimedia, Augsburg, Germany, Sep. 2007, pp. 188-197.
6. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. Annu. Conf. Comput. Learn. Theory, Madison, WI, Jul. 1998, pp. 92-100.
7. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, Dec. 2006.
8. L. Bruzzone and M. Marconcini, "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 770-787, May 2010.
9. L. Duan, I. W. Tsang, D. Xu, and S. J. Maybank, "Domain transfer SVM for video concept detection," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Miami, FL, Jun. 2009, pp. 1375-1381.
10. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. Int. Conf. Mach. Learn., Bled, Slovenia, Jun. 1999, pp. 200-209.
11. S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, "Domain adaptation via transfer component analysis," IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 199-210, Feb. 2011.
12. X. Zhu, "Semi-supervised learning literature survey," Dept. Comput. Sci., Univ. Wisconsin-Madison, Madison, Tech. Rep. 1530, Jul. 2008.
13. J. Huang, A. J. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf, "Correcting sample selection bias by unlabeled data," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 601-608.
14. A. J. Storkey and M. Sugiyama, "Mixture regression for covariate shift," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 1337-1344.
15. M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe, "Direct importance estimation with model selection and its application to covariate shift adaptation," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2008, pp. 1433-1440.
16. S. Bickel, C. Sawade, and T. Scheffer, "Transfer learning by distribution matching for targeted advertising," in Advances in Neural Information Processing Systems 21. Cambridge, MA: MIT Press, 2009.
17. P. Luo, F. Zhuang, H. Xiong, Y. Xiong, and Q. He, "Transfer learning from multiple source domains via consensus regularization," in Proc. ACM Conf. Inf. Knowl. Manage., Napa Valley, CA, Oct. 2008, pp. 103-112.
18. G. Schweikert, C. Widmer, B. Schölkopf, and G. Rätsch, "An empirical analysis of domain adaptation algorithms for genomic sequence analysis," in Advances in Neural Information Processing Systems 21. Cambridge, MA: MIT Press, 2009, pp. 1433-1440.
19. M. T. Rosenstein, Z. Marx, and L. P. Kaelbling, "To transfer or not to transfer," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, Dec. 2005.
20. S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, "Analysis of representations for domain adaptation," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 137-144.
21. J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman, "Learning bounds for domain adaptation," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2008, pp. 129-136.
22. K. Crammer, M. Kearns, and J. Wortman, "Learning from multiple sources," J. Mach. Learn. Res., vol. 9, pp. 1757-1774, Aug. 2008.
23. Y. Mansour, M. Mohri, and A. Rostamizadeh, "Domain adaptation with multiple sources," in Advances in Neural Information Processing Systems 21. Cambridge, MA: MIT Press, 2009, pp. 1041-1048.
24. Y. Mansour, M. Mohri, and A. Rostamizadeh, "Domain adaptation: Learning bounds and algorithms," Comput. Res. Reposit., vol. abs/0902.3, no. 2007, pp. 1-12, 2009.
25. S. Ben-David, T. Luu, T. Lu, and D. Pál, "Impossibility theorems for domain adaptation," J. Mach. Learn. Res., vol. 9, pp. 129-136, May 2010.
26. D. Kifer, S. Ben-David, and J. Gehrke, "Detecting change in data streams," in Proc. Int. Conf. Very Large Data Bases, Toronto, ON, Canada, Aug. 2004, pp. 180-191.
27. S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, "A theory of learning from different domains," Mach. Learn., vol. 79, nos. 1-2, pp. 151-175, May 2010.
28. T. Evgeniou, C. Micchelli, and M. Pontil, "Learning multiple tasks with kernel methods," J. Mach. Learn. Res., vol. 6, pp. 615-637, Apr. 2005.
29. T. Kato, H. Kashima, M. Sugiyama, and K. Asai, "Multi-task learning via conic programming," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2008, pp. 737-744.
30. D. Sheldon, "Graphical multi-task learning," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, Dec. 2008.
31. K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001.
32. F. H. Sinz, O. Chapelle, A. Agarwal, and B. Shölkopf, "An analysis of inference with the universum," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2008, pp. 1369-1376.
33. J. Weston, R. Collobert, F. Sinz, L. Bottou, and V. Vapnik, "Inference with the universum," in Proc. Int. Conf. Mach. Learn., Pittsburgh, PA, Jun. 2006, pp. 1009-1016.
34. V. Vapnik, "Transductive inference and semi-supervised learning," in Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006, pp. 454-472.
35. T. Van Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. de Moor, and J. Vandewalle, "Benchmarking least squares support vector machine classifiers," Mach. Learn., vol. 54, no. 1, pp. 5-32, Jan. 2004.
36. V. Sindhwani, P. Niyogi, and M. Belkin, "Beyond the point cloud: From transductive to semi-supervised learning," in Proc. Int. Conf. Mach. Learn., Bonn, Germany, Jun. 2005, pp. 824-831.
37. A. J. Smola and B. Schölkopf, "A tutorial on support vector regression," Stat. Comput., vol. 14, no. 3, pp. 199-222, Aug. 2004.
38. I. W. Tsang and J. T. Kwok, "Large-scale sparsified manifold regularization," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 1401-1408.
39. C.-C. Chang and C.-J. Lin. (2001). LIBSVM: A Library for Support Vector Machines [Online]. Available: http://www.csie.ntu.edu.tw/∼cjlin/libsvm
40. T. D. Bie and N. Cristianini, "Semi-supervised learning using semidefinite programming," in Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006, pp. 119-135.
41. L. Duan, D. Xu, I. W. Tsang, and J. Luo, "Visual event recognition in videos by learning from web data," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., San Francisco, CA, Jun. 2010, pp. 1959-1966.
42. S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
43. S. Ozawa, A. Roy, and D. Roussinov, "A multitask learning model for online pattern recognition," IEEE Trans. Neural Netw., vol. 20, no. 3, pp. 430-445, Mar. 2009.
44. M. Naphade, J. R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis, "Large-scale concept ontology for multimedia," IEEE Multimedia Mag., vol. 13, no. 3, pp. 86-91, Jul.-Sep. 2006.
45. S.-F. Chang, J. He, Y.-G. Jiang, E. E. Khoury, C.-W. Ngo, A. Yanagawa, and E. Zavesky, "Columbia University/VIREO-cityU/IRIT TRECVID2008 high-level feature extraction and interactive video search," in Proc. NIST TRECVID Workshop, Gaithersburg, MD, Nov. 2008, pp. 1-16.
46. A. Yanagawa, W. Hsu, and S.-F. Chang, "Columbia University's baseline detectors for 374 LSCOM semantic visual concepts," Dept. Electr. Eng., Columbia University, New York, ADVENT Tech. Rep. 222-2006-8, Mar. 2007.
47. A. F. Smeaton, P. Over, and W. Kraaij, "Evaluation campaigns and TRECVID," in Proc. ACM Int. Workshop Multimedia Inf. Retrieval, Santa Barbara, CA, Oct. 2006, pp. 321-330.
48. J. C. Platt, "Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods," in Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 1999, pp. 61-74.
49. K. M. Borgwardt, A. Gretton, M. J. Rasch, H.-P. Kriegel, B. Schölkopf, and A. J. Smola, "Integrating structured biological data by kernel maximum mean discrepancy," Bioinformatics, vol. 22, no. 14, pp. 49-57, Jul. 2006.