Transfer learning using computational intelligence: A survey

Knowledge-Based Systems

Volumn 80, Issue , 2015, Pages 14-23

  Lu, Jie ^a   Behbood, Vahid ^a   Hao, Peng ^a   Zuo, Hua ^a   Xue, Shan ^a   Zhang, Guangquan ^a  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

Bayes; Computational intelligence; Fuzzy sets and systems; Genetic algorithm; Neural network; Transfer learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; FUZZY INFERENCE; GENETIC ALGORITHMS; INTELLIGENT COMPUTING; KNOWLEDGE MANAGEMENT; LEARNING SYSTEMS; NEURAL NETWORKS; PREDICTIVE ANALYTICS; SURVEYS;

BAYES; FUZZY SETS AND SYSTEMS; LEARNING TECHNIQUES; MACHINE LEARNING METHODS; PREDICTIVE MODELING; RESEARCH AND APPLICATION; TECHNIQUE DEVELOPMENT; TRANSFER LEARNING METHODS;

TRANSFER LEARNING;

EID: 84926155931     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2015.01.010     Document Type: Article

References (104)

1. X. Zhu Semi-Supervised Learning Literature Survey 2005 University of Wisconsin Madison, USA
2. K. Nigam, A.K. Mccallum, S. Thrun, and T. Mitchell Text classification from labeled and unlabeled documents using EM Mach. Learn. 39 2-3 2000 103 134
3. A. Blum, T. Mitchell, Combining labeled and unlabeled data with co-training, in: Eleventh Annual Conference on Computational Learning Theory, Madison, USA, 1998.
4. T. Joachims, Transductive inference for text classification using support vector machines, in: Sixteenth International Conference on Machine Learning, Bled, USA, 1999.
5. G.P.C. Fung, J.X. Yu, H.J. Lu, and P.S. Yu Text classification without negative examples revisit IEEE Trans. Knowl. Data Eng. 18 1 2006 6 20
6. X. Yin, J. Han, J. Yang, and P.S. Yu Efficient classification across multiple database relations: a CrossMine approach IEEE Trans. Knowl. Data Eng. 18 6 2006 770 783
7. L.I. Kuncheva, and J.J. Rodriguez Classifier ensembles with a random linear oracle IEEE Trans. Knowl. Data Eng. 19 4 2007 500 508
8. E. Baralis, S. Chiusano, and P. Garza A lazy approach to associative classification IEEE Trans. Knowl. Data Eng. 20 2 2008 156 171
9. S.J. Pan, and Q. Yang A survey on transfer learning IEEE Trans. Knowl. Data Eng. 22 10 2010 1345 1359
10. R. Klinkenberg, T. Joachims, Detecting concept drift with support vector machines, in: Seventeenth International Conference on Machine Learning, Stanford, USA, 2000.
11. A. Margolis, A Literature Review of Domain Adaptation with Unlabeled Data, Rapport Technique, University of Washington, 2011.
12. J.J. Heckman Sample selection bias as a specification error Econometrica 47 1 1979 153 162
13. J. Huang, A.J. Smola, A. Gretton, K.M. Borgwardt, and B. Scholkopf Correcting sample selection bias by unlabeled data Adv. Neural Inf. Process. Syst. 19 2007 601 608
14. M. Sugiyama, S. Nakajima, H. Kashima, P. Bunau, and M. Kawanabe Direct importance estimation with model selection and its application to covariate shift adaptation Adv. Neural Inf. Process. Syst. 20 2008 1433 1440
15. Y. Tsuboi, H. Kashima, S. Hido, S. Bickel, and M. Sugiyama Direct density ratio estimation for large-scale covariate shift adaptation Inf. Media Technol. 4 2 2009 529 546
16. S. Tan, X. Cheng, Y. Wang, H. Xu, Adapting naive Bayes to domain adaptation for sentiment analysis, in: 31st European Conference on Advances in Information Retrieval, Toulouse, France, 2009.
17. W. Dai, G. Xue, Q. Yang, Y. Yu, Transferring naive Bayes classifiers for text classification, in: 22nd National Conference on Artificial Intelligence, Vancouver, Canada, 2007.
18. D. McClosky, E. Charniak, M. Johnson, Reranking and self-training for parser adaptation, in: 21st International Conference on Computational Linguistics, Sydney, Australia, 2006.
19. B. Roark, M. Bacchiani, Supervised and unsupervised PCFG adaptation to novel domains, in: Conference of the North American Chapter of the Association for Computational Linguistics on Human Language, Edmonton, Canada, 2003.
20. K. Sagae, J. Tsujii, Dependency parsing and domain adaptation with LR models and parser ensembles, in: 11th Conference on Computational Natural Language Learning, Prague, Czech Republic, 2007.
21. K. Sagae, Self-training without reranking for parser domain adaptation and its impact on semantic role labeling, in: Workshop on Domain Adaptation for Natural Language Processing, Uppsala, Sweden, 2010.
22. J. Jiang, C.X. Zhai, Instance weighting for domain adaptation in NLP, in: 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, 2007.
23. O. Sandu, G. Carenini, G. Murray, R. Ng, Domain adaptation to summarize human conversations, in: Workshop on Domain Adaptation for Natural Language Processing, Uppsala, Sweden, 2010.
24. J. Jiang, C.X. Zhai, A two-stage approach to domain adaptation for statistical classifiers, in: Sixteenth ACM Conference on Information and Knowledge Management, Lisbon, Portugal, 2007.
25. M. Ciaramita, O. Chapelle, Adaptive parameters for entity recognition with perceptron HMMs, in: Workshop on Domain Adaptation for Natural Language Processing, Uppsala, Sweden, 2010.
26. S. Tan, Y. Wang, G. Wu, X. Cheng, Using unlabeled data to handle domain-transfer problem of semantic detection, in: ACM Symposium on Applied Computing, Fortaleza, Brazil, 2008.
27. L. Rigutini, M. Maggini, B. Liu, An EM based training algorithm for cross-language text categorization, in: IEEE/WIC/ACM International Conference on Web Intelligence, Compiegne, France, 2005.
28. L. Shi, R. Mihalcea, M. Tian, Cross language text classification by model translation and semi-supervised learning, in: Conference on Empirical Methods in Natural Language Processing, Cambridge, USA, 2010.
29. M. Jeong, C.Y. Lin, G.G. Lee, Semi-supervised speech act recognition in emails and forums, in: Conference on Empirical Methods in Natural Language Processing, Singapore, Singapore, 2009.
30. A. Arnold, R. Nallapati, W.W. Cohen, A comparative study of methods for transductive transfer learning, in: Seventh IEEE International Conference on Data Mining Workshops, Omaha, USA, 2007.
31. A. Aue, M. Gamon, Customizing sentiment classifiers to new domains: A case study, in: International Conference on Recent Advances in Natural Language Processing, Borovets, Bulgaria, 2005.
32. S. Satpal, S. Sarawagi, Domain adaptation of conditional probability models via feature subsetting, in: 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, 2007.
33. B. Chen, W. Lam, I. Tsang, T.L. Wong, Extracting discriminative concepts for domain adaptation in text mining, in: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, 2009.
34. S.J. Pan, J.T. Kwok, Q. Yang, Transfer learning via dimensionality reduction, in: 23rd National Conference on Artificial Intelligence, Chicago, USA, 2008.
35. S.J. Pan, I.W. Tsang, J.T. Kwork, and Q. Yang Domain adaptation via transfer component analysis IEEE Trans. Neural Networks 22 2 2011 199 210
36. J. Blitzer, R. McOnald, F. Pereira, Domain adaptation with structural correspondence learning, in: Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, 2006.
37. J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, J. Wortman, Learning bounds for domain adaptation, in: Twenty-First Annual Conference on Neural Information Processing Systems, Cambridge, USA, 2007.
38. S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J.W. Vaughan A theory of learning from different domains Mach. Learn. 79 1 2010 151 175
39. F. Huang, A. Yates, Exploring representation-learning approaches to domain adaptation, in: Workshop on Domain Adaptation for Natural Language Processing, Uppsala, Sweden, 2010.
40. F. Huang, A. Yates, Open-domain semantic role labeling by modeling word spans, in: 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, 2010.
41. F. Huang, A. Yates, Distributional representations for handling sparsity in supervised sequence-labeling, in: Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Conference on Natural Language Processing, Singapore, Singapore, 2009.
42. S.J. Pan, X. Ni, J. Sun, Q. Yang, Z. Chen, Cross-domain sentiment classification via spectral feature alignment, in: 19th International Conference on World Wide Web, Raleigh, USA, 2010.
43. J. Gao, W. Fan, J. Jiang, J. Han, Knowledge transfer via multiple model local structure mapping, in: 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, USA, 2008.
44. G.-R. Xue, W. Dai, Q. Yang, Y. Yu, Topic-bridged PLSA for cross-domain text classification, in: 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Singapore, Singapore, 2008.
45. D.H. Hubel, and T.N. Wiesel Receptive fields, binocular interaction and functional architecture in the cat's visual cortex J. Physiol. 160 1 1962 106 154
46. A. Ahmed, K. Yu, W. Xu, Y. Gong, and E. Xing Training Hierarchical Feed-Forward Visual Recognition Models Using Transfer Learning from Pseudo-Tasks, in Computer Vision-ECCV 2008 2008 Springer pp. 69-82
47. J.-T. Huang, J. Li, D. Yu, L. Deng, Y. Gong, Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, Canada, 2013.
48. D.C. Ciresan, U. Meier, J. Schmidhuber, Transfer learning for Latin and Chinese characters with deep neural networks, in: International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia, 2012.
49. R. Collobert, J. Weston, A unified architecture for natural language processing: deep neural networks with multitask learning, in: 25th International Conference on Machine Learning, Helsinki, Finland, 2008.
50. C. Kandaswamy, L.M. Silva, L.A. Alexandre, J.M. Santos, J.M. de Sa, Improving deep neural network performance by reusing features trained with transductive transference, in: Proceedings of the 24th International Conference on Artificial Neural Networks, Hamburg, Germany, 2014.
51. S. Chopra, S. Balakrishnan, R. Gopalan, DLID: deep learning for domain adaptation by interpolating between domains, in: ICML Workshop on Challenges in Representation Learning, Atlanta, USA, 2013.
52. P. Swietojanski, A. Ghoshal, S. Renals, Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR, in: IEEE Workshop on Spoken Language Technology (SLT), Miami, USA, 2012.
53. R. Caruana, Multitask learning: a knowledge-based source of inductive bias, in: Tenth International Conference of Machine Learning, MA, USA, 1993.
54. R. Caruana Multitask learning Mach. Learn. 28 1997 41 75
55. D. Silver, R. Mercer, Selective functional transfer: Inductive bias from related tasks, in: International Conference on Artificial Intelligence and Soft Computing (ASC), Cancun, Mexico, 2001.
56. D.L. Silver, and R.E. Mercer The task rehearsal method of life-long learning: overcoming impoverished data Advances in Artificial Intelligence 2002 Springer 90 101
57. D.L. Silver, R.E. Mercer, Sequential inductive transfer for coronary artery disease diagnosis, in: International Joint Conference on Neural Networks (IJCNN), Orlando, USA, 2007.
58. D.L. Silver, R. Poirier, Context-sensitive MTL networks for machine lifelong learning, in: 20th Florida Artificial Intelligence Research Society (FLAIRS) Conference, Key West, USA, 2007.
59. D.L. Silver, R. Poirier, and D. Currie Inductive transfer with context-sensitive neural networks Mach. Learn. 73 3 2008 313 336
60. D.L. Silver, and L. Tu Image transformation: inductive transfer between multiple tasks having multiple outputs Advances in Artificial Intelligence 2008 Springer 296 307
61. K. Yamauchi Covariate shift and incremental learning Advances in Neuro-Information Processing 2009 Springer 1154 1162
62. H. Shimodaira Improving predictive inference under covariate shift by weighting the log-likelihood function J. Stat. Plann. Inference 90 2 2000 227 244
63. K. Yamauchi Optimal incremental learning under covariate shift Memetic Comput. 1 4 2009 271 279
64. W. Liu, H. Zhang, J. Li, Inductive transfer through neural network error and dataset regrouping, in: IEEE International Conference on Intelligent Computing and Intelligent Systems (ICIS), Shanghai, China, 2009.
65. K. Ueki, M. Sugiyama, Y. Ihara, Perceived age estimation under lighting condition change by covariate shift adaptation, in: 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey, 2010.
66. L.A. Celiberto Jr., L.A., J.P. Matsuura, R.L. de Mantaras, R.A.C. Bianchi, Using cases as heuristics in reinforcement learning: a transfer learning application, in: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, Barcelona, Spain, 2011.
67. D.D. Lewis, Representation and Learning in Information Retrieval, University of Massachusetts, 1992.
68. I. Kononenko Inductive and Bayesian learning in medical diagnosis Appl. Artif. Intell. Int. J. 7 4 1993 317 337
69. I. Androutsopoulos, J. Koutsias, K.V. Chandrinos, C.D. Spyropoulos, An experimental comparison of naive Bayesian and keyword-based anti-spam filtering with personal e-mail messages, in: 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Pisa, Italy, 2000.
70. F. Sebastiani Machine learning in automated text categorization ACM Comput. Surveys (CSUR) 34 1 2002 1 47
71. D.M. Roy, L.P. Kaelbling, Efficient Bayesian task-level transfer learning, in: International Joint Conference on Artificial Intelligence, Hyderabad, India, 2007.
72. Y. Ma, G. Luo, X. Zeng, and A. Chen Transfer learning for cross-company software defect prediction Inf. Softw. Technol. 54 3 2012 248 256
73. D. Heckerman A Tutorial on Learning with Bayesian Networks 1998 Springer
74. W.L. Buntine A guide to the literature on learning probabilistic networks from data IEEE Trans. Knowl. Data Eng. 8 2 1996 195 210
75. A. Niculescu-Mizil, R. Caruana, Inductive transfer for Bayesian network structure learning, in: Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS), San Juan, Puerto Rico, 2007.
76. R. Luis, L.E. Sucar, and E.F. Morales Inductive transfer for learning Bayesian networks Mach. Learn. 79 1-2 2010 227 255
77. M. Richardson, P. Domingos, Learning with knowledge from multiple experts, in: International Conference on Machine learning (ICML), Washington, USA, 2003.
78. D. Oyen, T. Lane, Leveraging domain knowledge in multitask Bayesian network structure learning, in: 26th Association for the Advancement of Artificial Intelligence (AAAI) Conference, Toronto, Canada, 2012.
79. D. Oyen, T. Lane, Bayesian discovery of multiple Bayesian networks via transfer learning, in: 13th IEEE International Conference on Data Mining (ICDM), Dalla, USA, 2013.
80. N. Friedman, and D. Koller Being Bayesian about network structure: a Bayesian approach to structure discovery in Bayesian networks Mach. Learn. 50 1-2 2003 95 125
81. M. Koivisto, and K. Sood Exact Bayesian structure discovery in Bayesian networks J. Mach. Learn. Res. 5 2004 549 573
82. A. Wilson, A. Fern, S. Ray, P. Tadepalli, Multi-task reinforcement learning: a hierarchical Bayesian approach, in: 24th International Conference on Machine Learning, Corvallis, USA, 2007.
83. A. Wilson, A. Fern, P. Tadepalli, Transfer learning in sequential decision problems: a hierarchical Bayesian approach, in: International Conference of Machine Learning, Edinburgh, Scotland, 2012.
84. P. Yang, Q. Tan, Y. Ding, Bayesian task-level transfer learning for non-linear regression, in: International Conference on Computer Science and Software Engineering, Wuhan, China, 2008.
85. V.C. Raykar, B. Krishnapuram, J. Bi, M. Dundar, R.B. Rao, Bayesian multiple instance learning: Automatic feature selection and inductive transfer, in: 25th International Conference on Machine Learning, Helsinki, Finland, 2008.
86. J.R. Finkel, C.D. Manning, Hierarchical Bayesian domain adaptation, in: Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, USA, 2009.
87. F. Wood, Y.W. Teh, A hierarchical nonparametric Bayesian approach to statistical language model domain adaptation, in: International Conference on Artificial Intelligence and Statistics, Las Vegas, USA, 2009.
88. R. Salakhutdinov, J. Tenenbaum, A. Torralba, One-shot learning with a hierarchical nonparametric Bayesian model, MIT-CSAIL-TR-2010-052, Editor, MIT, 2010.
89. L.A. Zadeh Fuzzy sets Inf. Control 8 3 1965 338 353
90. R.E. Bellman, and L.A. Zadeh Decision-making in a fuzzy environment Manage. Sci. 17 4 1970 141 164
91. V. Behbood, J. Lu, G. Zhang, Long term bank failure prediction using fuzzy refinement-based transductive transfer learning, in: IEEE International Conference on Fuzzy Systems (FUZZ), Taipei, Taiwan, 2011.
92. V. Behbood, J. Lu, and G. Zhang Fuzzy bridged refinement domain adaptation: long-term bank failure prediction Int. J. Comput. Intell. Appl. 12 01 2013
93. V. Behbood, J. Lu, and G. Zhang Fuzzy refinement domain adaptation for long term prediction in banking ecosystem IEEE Trans. Industr. Inf. 10 2 2014 1637 1646
94. J. Shell, S. Coupland, Towards fuzzy transfer learning for intelligent environments, in: Ambient Intelligence, vol. 7683, 2012, pp. 145-160.
95. J. Shell, and S. Coupland Fuzzy transfer learning: methodology and application Inf. Sci. 293 2015 59 79
96. Z. Deng, K. Choi, and Y. Jiang Generalized hidden-mapping ridge regression, knowledge-leveraged inductive transfer learning for neural networks, fuzzy systems and kernel method IEEE Trans. Cybern. 44 12 2014 2585 2599
97. B. Koçer, and A. Arslan Genetic transfer learning Expert Syst. Appl. 37 10 2010 6997 7002
98. V. Behbood, J. Lu, G. Zhang, Text categorization by fuzzy domain adaptation, in: IEEE International Conference on Fuzzy Systems, Hyderabad, India, 2013.
99. S. Thrun Is learning the n-th thing any easier than learning the first? Adv. Neural Inf. Process. Syst. 1996 640 646
100. S. Thrun, A lifelong learning perspective for mobile robot control, in: IEEE/RSJ/GI International Conference on Intelligent Robots and Systems, Munich, Germany, 1994.
101. M.E. Taylor, P. Stone, and Y. Liu Transfer learning via inter-task mappings for temporal difference learning J. Mach. Learn. Res. 8 1 2007 2125 2167
102. W. Zou, S. Zhu, A.Y. Ng, K. Yu, Deep learning of invariant features via simulated fixations in video, in: Advances in Neural Information Processing Systems, Lake Tahoe, USA, 2012.
103. F. Jin, S. Sun, Neural network multitask learning for traffic flow forecasting, in: IEEE International Joint Conference on Neural Networks (IJCNN), Hong Kong, China, 2008.
104. J. Shell, Fuzzy transfer learning, Ph.D. Thesis, De Montfort University, 2013.