Learning incoherent sparse and low-rank patterns from multiple tasks

ACM Transactions on Knowledge Discovery from Data

Volumn 5, Issue 4, 2012, Pages

  Chen, Jianhui ^a   Liu, Ji ^a   Ye, Jieping ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

Author keywords

Low rank and sparse patterns; Multitask learning; Trace norm

Indexed keywords

CARDINALITIES; CONSTRAINED OPTIMIZATION PROBLEMS; EUCLIDEAN; GLOBAL CONVERGENCE; LEAST SQUARE; LOW-RANK AND SPARSE PATTERNS; MULTIPLE TASKS; MULTITASK LEARNING; NON-DIFFERENTIABLE; NONCONVEX; OBJECTIVE FUNCTIONS; OPTIMAL SOLUTIONS; OPTIMIZATION FORMULATIONS; PROJECTED GRADIENT; RATES OF CONVERGENCE; REAL WORLD DATA; SEMI-DEFINITE PROGRAMMING; TRACE-NORMS; UNCONSTRAINED OPTIMIZATION;

CONSTRAINED OPTIMIZATION; MATHEMATICAL PROGRAMMING; VIRTUAL REALITY;

ALGORITHMS;

EID: 84863232691     PISSN: 15564681     EISSN: 1556472X     Source Type: Journal    
DOI: 10.1145/2086737.2086742     Document Type: Conference Paper

References (59)

1. ABERNETHY, J., BACH, F., EVGENIOU, T., AND VERT, J.-P. 2009. A new approach to collaborative filtering: Operator estimation with spectral regularization. J. Mach. Learn. Res. 10, 803-826.
2. ANDO, R. K. 2007. BioCreative II gene mention tagging system at IBM Watson. In Proceedings of the 2nd BioCreative Challenge Evaluation Workshop.
3. ANDO, R. K. AND ZHANG, T. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res. 6, 1817-1853.
4. ARGYRIOU, A., EVGENIOU, T., AND PONTIL, M. 2008. Convex multi-task feature learning. Mach. Learn. 73, 3, 243-272.
5. BACH, F., JENATTON, R., MAIRAL, J., AND OBOZINSKI, G. 2011. Convex optimization with sparsity-inducing norms. In Optimization for Machine Learning. S. Sra, S. Nowozin, and S. J. Wright, Eds., MIT Press.
6. BAKKER, B. AND HESKES, T. 2003. Task clustering and gating for Bayesian multitask learning. J. Mach. Learn. R. 4, 83-99.
7. BECK, A. AND TEBOULLE, M. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 183-202.
8. BERTSEKAS, D. P. 1999. Nonlinear Programming. Athena Scientific.
9. BERTSEKAS, D. P., NEDIC, A., AND OZDAGLAR, A. E. 2003. Convex Analysis and Optimization. Athena Scientific.
10. BI, J., XIONG, T., YU, S., DUNDAR, M., AND RAO, R. B. 2008. An improved multi-task learning approach with applications in medical diagnosis. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD).
11. BICKEL, S., BOGOJESKA, J., LENGAUER, T., AND SCHEFFER, T. 2008. Multi-task learning for HIV therapy screening. In Proceedings of the International Conference on Machine Learning (ICML).
12. BOYD, S. AND VANDENBERGHE, L. 2004. Convex Optimization. Cambridge University Press.
13. CANDÈ S, E. J., LI, X., MA, Y., AND WRIGHT, J. 2009. Robust principal component analysis? J. ACM 58, 1, 1-37.
14. CARUANA, R. 1997. Multitask learning. Mach. Learn. 28, 1, 41-75. (Pubitemid 127507169)
15. CHANDRASEKARAN, V., SANGHAVI, S., PARRILO, P. A., AND WILLSKY, A. S. 2009. Sparse and low-rank matrix decompositions. In Proceedings of the15th IFAC Symposium on System Identification (SYSID).
16. CHANG, C.-C. AND LIN, C.-J. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27, 1-27.
17. CHEN, J., TANG, L., LIU, J., AND YE, J. 2009a. A convex formulation for learning shared structures from multiple tasks. In Proceedings of the International Conference on Machine Learning (ICML).
18. CHEN, X., PAN, W., KWOK, J. T., AND CARBONELL, J. G. 2009b. Accelerated gradient method for multi-task sparse learning problem. In Proceedings of the IEEE International Conference on Data Mining (ICDM).
19. CHEN, X., KIM, S., LIN, Q., CARBONELL, J. G., AND XING, E. P. 2010a. Graph-structured multi-task regression and an efficient optimization method for general fused lasso. Computing Research Repository (CoRR), abs/1005.3579.
20. CHEN, X., LIN, Q., KIM, S., CARBONELL, J. G., AND XING, E. P. 2010b. An efficient proximal-gradient method for single and multi-task regression with structured sparsity. Computing Research Repository (CoRR), abs/1005.4717v3.
21. EVGENIOU, T., MICCHELLI, C. A., AND PONTIL, M. 2005. Learning multiple tasks with kernel methods. J. Mach. Learn. Res. 6, 615-637.
22. FAZEL, M., HINDI, H., AND BOYD, S. 2001. A rank minimization heuristic with application to minimum order system approximation. In Proceedings of the American Control Conference (ACC).
23. FOWLKES, C. C., HENDRIKS, C. L. L., KERÄNEN, S. V., WEBER, G. H., RÜBEL, O., HUANG, M.-Y., CHATOOR, S., DEPACE, A. H., SIMIRENKO, L., HENRIQUEZ, C., BEATON, A., WEISZMANN, R., CELNIKER, S., HAMANN, B., KNOWLES, D. W., BIGGIN, M. D., EISEN, M. B., AND MALIK, J. 2008. A quantitative spatiotemporal atlas of gene expression in the drosophila blastoderm. Cell 133, 2, 364-374. (Pubitemid 351508300)
24. GOLUB, G. H. AND VAN LOAN, C. F. 1996. Matrix Computations. Johns Hopkins University Press.
25. GONZALEZ, R. C. AND WOODS, R. E. 2002. Digital Image Processing. Prentice Hall.
26. JACOB, L., BACH, F., AND VERT, J.-P. 2008. Clustered multi-task learning: A convex formulation. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
27. JALALI, A., RAVIKUMAR, P., SANGHAVI, S., AND RUAN, C. 2010. A dirty model for multi-task learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
28. JI, S. AND YE, J. 2009. An accelerated gradient method for trace norm minimization. In Proceedings of the International Conference on Machine Learning (ICML).
29. JI, S., YUAN, L., LI, Y.-X., ZHOU, Z.-H., KUMAR, S., AND YE, J. 2009. Drosophila gene expression pattern annotation using sparse features and term-term interactions. In Proceedings of the International SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).
30. KIM, S. AND XING, E. P. 2010. Tree-guided group lasso for multi-task regression with structured sparsity. In Proceedings of the International Conference on Machine Learning (ICML).
31. LAWRENCE, N. D. AND PLATT, J. C. 2004. Learning to learn with the informative vector machine. In Proceedings of the International Conference on Machine Learning (ICML).
32. LÉCUYER, E., YOSHIDA, H., PARTHASARATHY, N., ALM, C., BABAK, T., CEROVINA, T., HUGHES, T. R., TOMANCAK, P., AND KRAUSE, H. M. 2007. Global analysis of mRNA localization reveals a prominent role in organizing cellular architecture and function. Cell 131, 1, 174-187. (Pubitemid 47498530)
33. LIU, J. AND YE, J. 2009. Efficient Euclidean projections in linear time. In Proceedings of the International Conference on Machine Learning (ICML).
34. 2-1 Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI).
35. LIU, J., JI, S., AND YE, J. 2009b. SLEP: Sparse Learning with Efficient Projections. Arizona State University.
36. LOWE, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2, 91-110.
37. MARTINEZ, A. AND BENAVENTE, R. 1998. The AR face database. Tech. rep. 24, Computer Vision Center (CVC).
38. NEMIROVSKI, A. 1995. Efficient methods in convex programming. Lecture Notes, Technion.
39. NESTEROV, Y. 1998. Introductory lectures on convex programming. Lecture Notes.
40. OBOZINSKI, G., B.TASKAR, AND JORDAN, M. 2010. Joint covariate selection and joint subspace selection for multiple classification problems. Statist. Comput. 20, 2, 231-252.
41. PONG, T. K., TSENG, P., JI, S., AND YE, J. 2010. Trace norm regularization: Reformulations, algorithms, and multi-task learning. SIAM J. Optim. 20, 6.
42. SCHÖ LKOPF, B. AND SMOLA, A. J. 2002. Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press.
43. SCHWAIGHOFER, A., TRESP, V., AND YU, K. 2004. Learning Gaussian process kernels via hierarchical Bayes. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
44. SHAPIRO, A. 1982. Weighted minimum trace factor analysis. Psychometrika 47, 3, 243-264.
45. SI, S., TAO, D., AND GENG, B. 2010. Bregman divergence-based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22, 7, 929-942.
46. STURM, J. F. 2001. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11-12, 653-625.
47. TIBSHIRANI, R. 1996. Regression shrinkage and selection via the lasso. J. Royal Statist. Soc., Series B 58, 267-288.
48. UEDA, N. AND SAITO, K. 2002. Single-shot detection of multiple categories of text using parametric mixture models. In Proceedings of the International SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).
49. VANDENBERGHE, L. AND BOYD, S. 1996. Semidefinite programming. SIAM Rev. 38, 1, 49-95. (Pubitemid 126610912)
50. WATSON, G. A. 1992. Characterization of the subdifferential of some matrix norms. Linear Algebra Appl. 170, 33-45.
51. WRIGHT, J., PENG, Y., MA, Y., GANESH, A., AND RAO, S. 2009. Robust principal component analysis: Exact recovery of corrupted low-rank matrices by convex optimization. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
52. XUE, Y., LIAO, X., CARIN, L., AND KRISHNAPURAM, B. 2007. Multi-task learning for classification with Dirichlet process priors. J. Mach. Learn. Res. 8, 35-63. (Pubitemid 46155123)
53. YANG, Y. AND PEDERSEN, J. O. 1997. A comparative study on feature selection in text categorization. In Proceedings of the International Conference on Machine Learning (ICML).
54. YU, K., TRESP, V., AND SCHWAIGHOFER, A. 2005. Learning Gaussian processes from multiple tasks. In Proceedings of the International Conference on Machine Learning (ICML).
55. ZHANG, J., GHAHRAMANI, Z., AND YANG, Y. 2005. Learning multiple related tasks using latent independent component analysis. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
56. ZHANG, X., SAHA, A., AND VISHWANATHAN, S. V. N. 2010. Regularized risk minimization by Nesterov's accelerated gradient methods: Algorithmic extensions and empirical studies. Computing Research Repository (CoRR). abs/1011.0472.
57. ZHANG, Y. AND SCHNEIDER, J. 2010. Learning multiple tasks with a sparse matrix-normal penalty. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).
58. ZHANG, Y. AND YEUNG, D.-Y. 2010. A convex formulation for learning task relationship in multi-task learning. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI).
59. ZHOU, J., CHEN, J., AND YE, J. 2011. Clustered multi-task learning via alternating structure optimization. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).