Generalized hidden-mapping ridge regression, knowledge-leveraged inductive transfer learning for neural networks, fuzzy systems and kernel methods

IEEE Transactions on Cybernetics

Volumn 44, Issue 12, 2014, Pages 2585-2599

  Deng, Zhaohong ^a,b   Choi, Kup Sze ^c   Jiang, Yizhang ^a   Wang, Shitong ^b  

^a JIANGNAN UNIVERSITY   (China)

^b University of California   (United States)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Classification; Fuzzy systems; Generalized hidden mapping ridge regression (GHRR); Inductive transfer learning; Kernel methods; Knowledge leverage; Neural networks; Regression

Indexed keywords

CLASSIFICATION (OF INFORMATION); FUZZY SYSTEMS; KNOWLEDGE MANAGEMENT; LEARNING SYSTEMS; MAPPING; NEURAL NETWORKS; REGRESSION ANALYSIS;

INDUCTIVE TRANSFER; KERNEL METHODS; KNOWLEDGE-LEVERAGE; REGRESSION; RIDGE REGRESSION;

FUZZY NEURAL NETWORKS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; FUZZY LOGIC; PROCEDURES; REGRESSION ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; FUZZY LOGIC; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS;

EID: 84911938464     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2311014     Document Type: Article

References (49)

1. S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowledge Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
2. X. Liao, Y. Xue, and L. Carin, "Logistic regression with an auxiliary data source," in Proc. 21st Int. Conf. Mach. Learning, Aug. 2005, pp. 505-512.
3. J. Huang, A. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf, "Correcting sample selection bias by unlabeled data," in Proc. 19th Ann. Conf. Neural Inform. Process. Syst., 2007, pp. 601-608.
4. S. Bickel, M. Brückner, and T. Scheffer, "Discriminative learning for differing training and test distributions," in Proc. 24th Int. Conf. Mach. Learning, 2007, pp. 81-88.
5. M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe, "Direct importance estimation with model selection and its application to covariate shift adaptation," in Proc. 20th Ann. Conf. Neural Inform. Process. Syst., Dec. 2008, pp. 1433-1440.
6. N. D. Lawrence and J. C. Platt, "Learning to learn with the informative vector machine," in Proc. 21st Int. Conf. Mach. Learning, Jul. 2004, p. 65.
7. A. Schwaighofer, V. Tresp, and K. Yu, "Learning Gaussian process kernels via hierarchical Bayes," in Proc. 17th Ann. Conf. Neural Inform. Process. Syst., 2005, pp. 1209-1216.
8. J. Gao, W. Fan, J. Jiang, and J. Han, "Knowledge transfer via multiple model local structure mapping," in Proc. 14th ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining, Aug. 2008, pp. 283-291.
9. L. Mihalkova, T. Huynh, and R. J. Mooney, "Mapping and revising Markov logic networks for transfer learning," in Proc. 22nd AAAI Conf. Artif. Intell., Jul. 2007, pp. 608-614.
10. L. Mihalkova and R. J. Mooney, "Transfer learning by mapping with minimal target data," in Proc. AAAI'08, Jul. 2008, pp. 1-5.
11. J. Davis and P. Domingos, "Deep transfer via second-order Markov logic," in Proc. Workshop Transfer Learning Complex Tasks AAAI'08, Jul. 2008, pp. 217-224.
12. S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, "Domain adaptation via transfer component analysis," IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 199-210, Feb. 2011.
13. L. X. Duan, D. Xu, and I. W. Tsang, "Domain adaptation from multiple sources: A domain-dependent regularization approach," IEEE Trans. Neural Netw. Learning Syst., vol. 23, no. 3, pp. 504-518, Mar. 2012.
14. L. X. Duan, I. W. Tsang, and D. Xu, "Domain transfer multiple kernel learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 465-479, Mar. 2012.
15. H. DaumKe, III and D. Marcu, "Domain adaptation for statistical classifiers," J. Artif. Intell. Res., vol. 26, no. 1, pp. 101-126, May 2006.
16. P. Yang, Q. Tan, and Y. Ding, "Bayesian task-level transfer learning for non-linear regression," in Proc. Int. Conf. Comput. Sci. Software Eng., 2008, pp. 62-65.
17. L. Borzemski and G. Starczewski, "Application of transfer regression to TCP throughput prediction," in Proc. 1st Asian Conf. Intelligent Inform. Database Syst., 2009, pp. 28-33.
18. W. Mao, G. Yan, J. Bai, and H. Li, "Regression transfer learning based on principal curve," in Lecture Notes on Computer Science, vol. 6063. Berlin Heidelberg: Springer, 2010, pp. 365-372.
19. J. Liu, Y. Chen, and Y. Zhang, "Transfer regression model for indoor 3-D location estimation," in Lecture Note on Computer Science, vol. 5916. Berlin Heidelberg: Springer, 2010, pp. 603-613.
20. D. Pardoe and P. Stone, "Boosting for regression transfer," in Proc. Int. Conf. Mach. Learning, 2010, pp. 863-870.
21. W. Dai, Q. Yang, G. Xue, and Y. Yu, "Self-taught clustering," in Proc. 25th Int. Conf. Mach. Learning, Jul. 2008, pp. 200-207.
22. Z. Wang, Y. Song, and C. Zhang, "Transferred dimensionality reduction," in Proc. ECML/PKDD'08, Sep. 2008, pp. 550-565.
23. W. H. Jiang and F. L. Chung, "Transfer spectral clustering," in Machine Learning and Knowledge Discovery in Databases. Berlin, Germany: Springer, 2012, pp. 789-803.
24. Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang, "Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation," IEEE Trans. Fuzzy Syst., vol. 19, no. 2, pp. 210-226, Apr. 2011.
25. W. Dai, Q. Yang, G. Xue, and Y. Yu, "Boosting for transfer learning," in Proc. 24th Int. Conf. Mach. Learning, Jun. 2007, pp. 193-200.
26. J. Jiang and C. Zhai, "Instance weighting for domain adaptation in NLP," in Proc. 45th Ann. Meeting Assoc. Comput. Linguistics, Jun. 2007, pp. 264-271.
27. P. Wu and T. G. Dietterich, "Improving SVM accuracy by training on auxiliary data sources," in Proc. 21st Int. Conf. Mach. Learning, Jul. 2004.
28. A. Argyriou, T. Evgeniou, and M. Pontil, "Multitask feature learning," in Proc. 19th Ann. Conf. Neural Inform. Process. Syst., Dec. 2007, pp. 41-48.
29. E. Bonilla, K. M. Chai, and C. Williams, "Multi-Task Gaussian Process Prediction," in Proc. 20th Ann. Conf. Neural Inform. Process. Syst., 2008, pp. 153-160.
30. T. Evgeniou and M. Pontil, "Regularized multi-task learning," in Proc. 10th ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining, Aug. 2004, pp. 109-117.
31. M. Richardson and P. Domingos, "Markov logic networks," Mach. Learning, vol. 62, nos. 1-2, pp. 107-136, 2006.
32. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, no. 1, pp. 55-67, 1970.
33. J. Li. (2008, Sep. 4). Linear, Ridge Regression, and Principal Component Analysis [Online]. Available: http://sites.stat.psu.edu/?jiali/course/stat597e/notes2/lreg.pdf
34. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," in Proc. 5th Int. Conf. Mach. Learning, Jul. 1998, pp. 515-521.
35. J. S. R. Jang, C. T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft-Computing. Upper Saddle River, NJ, USA: Prentice-Hall, 1997.
36. Z. H. Deng, Y. Z. Jiang, F. L. Chung, and S. T. Wang, "Knowledge-leverage based fuzzy system and its modeling," IEEE Trans. Fuzzy Systems, vol. 21, no. 4, pp. 597-609, Aug. 2013.
37. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006.
38. G.-B. Huang and L. Chen, "Convex incremental extreme learning machine," Neurocomputing, vol. 70, nos. 16-18, pp. 3056-3062, Oct. 2007.
39. T. Takagi and M. Sugeno, "Fuzzy identification of systems and its application to modeling and control," IEEE Trans. Syst. Man Cybern., vol. 15, no. 1, pp. 116-132, Jan.-Feb. 1985.
40. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
41. Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang, "Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation," IEEE Trans. Fuzzy Syst., vol. 19, no. 2, pp. 210-226, Apr. 2011.
42. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, Jun. 1999.
43. R. Duda, P. Hart, and D. Stork, Pattern Classification. New York, NY, USA: Wiley, 2000.
44. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learning, vol. 20, no. 3, pp. 273-297, 1995.
45. P. Yang, Q. Tan, and Y. Ding, "Bayesian task-level transfer learning for non-linear regression," in Proc. Int. Conf. Comput. Sci. Software Eng., 2008, pp. 62-65.
46. S. Bickel, "ECML-PKDD discovery challenge 2006 overview," in Proc. ECML/PKDD Discovery Challenge Workshop, 2006, pp. 1-9.
47. Z. H. Deng, Y. Z. Jiang, K. S. Choi, F. L. Chung, and S. T. Wang, "Knowledge-leverage based TSK fuzzy system modeling," IEEE Trans. Neural Netw. Learning Syst., vol. 24, no. 8, pp. 1200-1212, Aug. 2013.
48. J. W. Tao, K. F. L. Chung, and S. T. Wang, "On minimum distribution discrepancy support vector machine for domain adaptation," Pattern Recognit., vol. 45, no. 11, pp. 3962-3984, 2012.
49. B. Quanz and J. Huan, "Large margin transductive transfer learning," in Proc. 18th ACM Conf. Inform. Knowledge Manag., 2009, pp. 1327-1336.