Multiple task learning with flexible structure regularization

Neurocomputing

Volumn 177, Issue , 2016, Pages 242-256

  Pu, Jian ^a   Wang, Jun ^b,c   Jiang, Yu Gang ^d   Xue, Xiangyang ^d  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b EAST CHINA NORMAL UNIVERSITY   (China)

^c ALIBABA GROUP   (United States)

^d FUDAN UNIVERSITY   (China)

Author keywords

Accelerated proximal gradient; Flexible structure regularization; Generic multiple task learning; Iteratively reweighted least square; Joint 11 21 norm regularization

Indexed keywords

ALGORITHMS; FLEXIBLE STRUCTURES; LEAST SQUARES APPROXIMATIONS; LINEARIZATION; STATISTICS;

ITERATIVELY REWEIGHTED LEAST SQUARES; MULTIPLE TASKS; PERFORMANCE GUARANTEES; REGULARIZATION TERMS; STATE-OF-THE-ART ALGORITHMS; STRUCTURAL CHARACTERISTICS; STRUCTURAL ELEMENTS; SYNTHETIC AND REAL DATA;

ITERATIVE METHODS;

ACCELERATED PROXIMAL GRADIENT ALGORITHM; ALGORITHM; ALZHEIMER DISEASE; ARTICLE; DATA PROCESSING; FLEXIBLE STRUCTURE REGULARIZATION; HUMAN; ITERATIVELY REWEIGHTED LEAST SQUARE ALGORITHM; MACHINE LEARNING; MATHEMATICAL COMPUTING; MILD COGNITIVE IMPAIRMENT; MULTIPLE TASK LEARNING; PRIORITY JOURNAL; ROBOTICS; SCHOOL;

EID: 84959482110     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2015.11.029     Document Type: Article

References (64)

1. Ando R.K., Zhang T. A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res. 2005, 6:1817-1853.
2. Argyriou A., Evgeniou T., Pontil M. Convex multi-task feature learning. Mach. Learn. 2008, 73(3):243-272.
3. J. Attenberg, K. Weinberger, A. Dasgupta, A. Smola, M. Zinkevich, Collaborative email-spam filtering with the hashing trick, in: The Sixth Conference on Email and Anti-Spam, 2009.
4. Bakker B., Heskes T. Task clustering and gating for Bayesian multitask learning. J. Mach. Learn. Res. 2003, 4:83-99.
5. Beck A., Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2009, 2(1):183-202.
6. Beck A., Teboulle M. Gradient-based algorithms with applications to signal recovery problems. Convex Optimization in Signal Processing and Communications 2010, 42-88. Cambridge University Press, New York, NY. Y. Eldar, D. Palomar (Eds.).
7. Beck A., Teboulle M. Smoothing and first order methods. a unified framework. SIAM J. Optim. 2012, 22(2):557-580.
8. S. Bickel, J. Bogojeska, T. Lengauer, T. Scheffer, Multi-task learning for HIV therapy screening, in: ICML, 2008.
9. D. Calandriello, A. Lazaric, M. Restelli, Sparse multi-task reinforcement learning, in: NIPS, 2014.
10. Caruana R. Multitask learning. Mach. Learn. 1997, 28(1):41-75.
11. G.H.-G. Chen, Forward-backward splitting techniques: theory and applications (Ph.D. thesis), University of Washington, 1994.
12. J. Chen, L. Tang, J. Liu, J. Ye, A convex formulation for learning shared structures from multiple tasks, in: ICML, 2009.
13. J. Chen, J. Zhou, J. Ye, Integrating low-rank and group-sparse structures for robust multi-task learning, in: SIGKDD, 2011.
14. X. Chen, Q. Lin, S. Kim, J.G. Carbonell, E.P. Xing, Smoothing proximal gradient method for general structured sparse learning, in: UAI, 2011.
15. I. Daubechies, R. Devore, M. Fornasier, C.S. Gntrk, Iteratively reweighted least squares minimization for sparse recovery, Commun. Pure Appl. Math. 63, 2010, 1-38.
16. J. Eckstein, Splitting methods for monotone operators with applications to parallel optimization (Ph.D. thesis), Massachusetts Institute of Technology, 1989.
17. Evgeniou T., Micchelli C.A., Pontil M. Learning multiple tasks with kernel methods. J. Mach. Learn. Res. 2005, 6:615-637.
18. Fei H., Huan J. Structured feature selection and task relationship inference for multi-task learning. Knowl. Inf. Syst. 2013, 2:345-364.
19. J. Ghosn, Y. Bengio, Multi-task learning for stock selection, in: NIPS, 1996.
20. P. Gong, J. Ye, C. Zhang, Robust multi-task feature learning, in: SIGKDD, 2012.
21. L. Jacob, F. Bach, J.-P. Vert, Clustered multi-task learning: a convex formulation, in: NIPS, 2008.
22. A. Jalali, P.D. Ravikumar, S. Sanghavi, C. Ruan, A dirty model for multi-task learning, in: NIPS, 2010.
23. Z. Kang, K. Grauman, F. Sha, Learning with whom to share in multi-task feature learning, in: ICML, 2011.
24. S. Kim, E.P. Xing, Tree-guided group lasso for multi-task regression with structured sparsity, in: ICML, 2010.
25. A. Kumar, H. Daumé III, Learning task grouping and overlap in multi-task learning, in: ICML, 2012.
26. N.D. Lawrence, J.C. Platt, Learning to learn with the informative vector machine, in: ICML, 2004.
27. S. Lee, J. Zhu, E.P. Xing, Adaptive multi-task lasso: with application to eQTL detection, in: NIPS, 2010.
28. Li Z., Liu J., Tang J., Lu H. Robust structured subspace learning for data representation. Trans. Pattern Anal. Mach. Intell. 2015, 37:2085-2098.
29. Li Z., Liu J., Yang Y., Zhou X., Lu H. Clustering-guided sparse structural learning for unsupervised feature selection. Trans. Knowl. Data Eng. 2014, 26:2138-2150.
30. Lions P.L., Mercier B. Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 1979, 16(6):964-979.
31. J. Liu, S. Ji, J. Ye, Multi-task feature learning via efficient ℓ2,1-norm minimization, in: UAI, 2009.
32. K. Lounici, M. Pontil, A.B. Tsybakov, S.A. van de Geer, Taking advantage of sparsity in multi-task learning, in: COLT, 2009.
33. Moreau J.J. Fonctions convexes duales et points proximaux dans un espace hilbertien. C. R. de l'Acad. Sci. (Paris), Sér. A 1962, 255:2897-2899.
34. S. Negahban, M.J. Wainwright, Joint support recovery under high-dimensional scaling: benefits and perils of ℓ1,∞-regularization, in: NIPS, 2008.
35. S. Negahban, M.J. Wainwright, Estimation of (near) low-rank matrices with noise and high-dimensional scaling, in: ICML, 2010.
36. Nesterov Y. Introductory Lectures on Convex Optimization: A Basic Course 2003, Springer, Boston, Dordrecht, London.
37. Nesterov Y. Smooth minimization of non-smooth functions. Math. Program. 2005, 103(1):127-152.
38. Pan S.J., Yang Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 2010, 22(10):1345-1359.
39. S. Parameswaran, K. Weinberger, Large margin multi-task metric learning, in: NIPS, 2010.
40. Pong T.K., Tseng P., Ji S., Ye J. Trace norm regularization. reformulations, algorithms, and multi-task learning. SIAM J. Optim. 2010, 20(6):3465-3489.
41. J. Pu, Y.-G. Jiang, J. Wang, X. Xue, Multiple task learning using iteratively reweighted least square, in: IJCAI, 2013.
42. J. Pu, Y.-G. Jiang, J. Wang, X. Xue, Which looks like which: exploring inter-class relationships in fine-grained visual categorization, in: ECCV, 2014.
43. B. Rakitsch, C. Lippert, K. Borgwardt, O. Stegle, It is all in the noise: efficient multi-task Gaussian process inference with structured residuals, in: NIPS, 2013.
44. Rao B.D., Engan K., Cotter S.F., Palmer J., Kreutz-delgado K. Subset selection in noise based on diversity measure minimization. IEEE Trans. Signal Process. 2003, 760-770.
45. Rao B.D., Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Trans. Image Process. 1999, 47(1):187-200.
46. N. Rao, C. Cox, R. Nowak, T.T. Rogers, Sparse overlapping sets lasso for multitask learning and its application to fMRI analysis, in: NIPS, 2013.
47. M.T. Rosenstein, Z. Marx, L.P. Kaelbling, T.G. Dietterich, To transfer or not to transfer, in: In NIPS05 Workshop, Inductive Transfer: 10 Years Later, 2005.
48. Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd edition, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2003.
49. A. Schwaighofer, V. Tresp, K. Yu, Learning Gaussian process kernels via hierarchical Bayes, in: NIPS, 2004.
50. K. Swersky, J. Snoek, R.P. Adams, Multi-task Bayesian optimization, in: NIPS, 2013.
51. S. Thrun, Is learning the n-th thing any easier than learning the first?, in: NIPS, 1996.
52. S. Thrun, J.O. Sullivan, Discovering structure in multiple learning tasks: the TC algorithm, in: ICML, 1996.
53. H. Wang, F. Nie, H. Huang, S.L. Risacher, C.H.Q. Ding, A. J. Saykin, L. Shen, Adni, Sparse multi-task regression and feature selection to identify brain imaging predictors for memory performance, in: ICCV, 2011.
54. X. Wang, C. Zhang, Z. Zhang, Boosted multi-task learning for face verification with applications to web image and video search, in: CVPR, 2009.
55. Xue Y., Liao X., Carin L., Krishnapuram B. Multi-task learning for classification with Dirichlet process priors. J. Mach. Learn. Res. 2007, 8:35-63.
56. X. Yang, S. Kim, E.P. Xing, Heterogeneous multitask learning with joint sparsity constraints, in: NIPS, 2009.
57. K. Yu, V. Tresp, A. Schwaighofer, Learning Gaussian processes from multiple tasks, in: ICML, 2005.
58. J. Zhang, Z. Ghahramani, Y. Yang, Learning multiple related tasks using latent independent component analysis, in: NIPS, 2005.
59. Y. Zhang, D.-Y. Yeung, Transfer metric learning by learning task relationships, in: SIGKDD, 2010.
60. Y. Zhang, D.-Y. Yeung, Learning high-order task relationships in multi-task learning, in: IJCAI, 2013.
61. Y. Zhang, D.-Y. Yeung, Q. Xu, Probabilistic multi-task feature selection, in: NIPS, 2010.
62. Zhou J., Liu J., Narayan V.A., Ye J. Modeling disease progression via multi-task learning. NeuroImage 2013, 78(0):233-248.
63. J. Zhou, L. Yuan, J. Liu, J. Ye, A multi-task learning formulation for predicting disease progression, in: SIGKDD, 2011.
64. Zou H., Hastie T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 2003, 67(2):301-320.