Efficient online learning for multitask feature selection

ACM Transactions on Knowledge Discovery from Data

Volumn 7, Issue 2, 2013, Pages

  Yang, Haiqin ^a   Lyu, Michael R ^a   King, Irwin ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Dual averaging method; Feature selection; Multitask learning; Online learning; Supervised learning

Indexed keywords

BIOINFORMATICS; FEATURE EXTRACTION; ITERATIVE METHODS; LEARNING ALGORITHMS; PROBLEM SOLVING; SUPERVISED LEARNING;

CLOSED FORM SOLUTIONS; CONVERGENCE RATES; DUAL AVERAGING; FEATURE DIMENSIONS; MULTITASK LEARNING; ON-LINE ALGORITHMS; ONLINE LEARNING; ONLINE LEARNING ALGORITHMS;

E-LEARNING;

EID: 84893288034     PISSN: 15564681     EISSN: 1556472X     Source Type: Journal    
DOI: 10.1145/2499907.2499909     Document Type: Article

References (56)

1. Aaker, D. A., Kumar, V., And Day, G. S. 2006. Marketing Research 9Th Ed. Wiley.
2. Ando, R. K. And Zhang, T. 2005. A Framework For Learning Predictive Structures From Multiple Tasks And Unlabeled Data. J. Mach. Learn. Res. 6, 1817-1853.
3. Argyriou, A., Evgeniou, T., And Pontil, M. 2006. Multi-Task Feature Learning. Adv. Neural Inf. Process. Syst. 19, 41-48.
4. Argyriou, A., Evgeniou, T., And Pontil, M. 2008. Convex Multi-Task Feature Learning. Mach. Learn. 73, 3, 243-272.
5. Bai, J., Zhou, K., Xue, G.-R., Zha, H., Sun, G., Tseng, B. L., Zheng, Z., And Chang, Y. 2009. Multi-Task Learning For Learning To Rank In Web Search. In Proceedings Of The 18Th Acm Conference On Information And Knowledge Management (Cikm'09). 1549-1552.
6. Bakker, B. And Heskes, T. 2003. Task Clustering And Gating For Bayesian Multitask Learning. J. Mach. Learn. Res. 4, 83-99.
7. Balakrishnan, S. And Madigan, D. 2008. Algorithms For Sparse Linear Classifiers In The Massive Data Setting. J. Mach. Learn. Res. 9, 313-337. (Pubitemid 351469019)
8. Ben-David, S. And Borbely, R. S. 2008. A Notion Of Task Relatedness Yielding Provable Multiple-Task Learning Guarantees. Mach. Learn. 73, 3, 273-287.
9. Ben-David, S. And Schuller, R. 2003. Exploiting Task Relatedness For Mulitple Task Learning. In Proceedings Of The 16Th Annual Conference On Learning Theory (Colt'03). 567-580. (Pubitemid 37053230)
10. Boyd, S. And Vandenberghe, L. 2004. Convex Optimization. Cambridge University Press.
11. Caruana, R. 1997. Multitask Learning. Mach. Learn. 28, 1, 41-75. (Pubitemid 127507169)
12. Chen, J., Liu, J., And Ye, J. 2010. Learning Incoherent Sparse And Low-Rank Patterns From Multiple Tasks. In Proceedings Of The 16Th Acm Sigkdd International Conference On Knowledge Discovery And Data Mining (Kdd'10). 1179-1188.
13. Chen, J., Liu, J., And Ye, J. 2012. Learning Incoherent Sparse And Low-Rank Patterns From Multiple Tasks. Acm Trans. Knowl. Discov. Data 5, 4.
14. Chen, J.,Tang, L.,Liu, J., And Ye, J. 2009A. A Convex Formulation For Learning Shared Structures From Multiple Tasks. In Proceedings Of The 26Th Annual International Conference On Machine Learning (Icml'09). 18.
15. Chen, X., Pan, W.,Kwok, J. T., And Carbonell, J. G. 2009B. Accelerated Gradient Method For Multi-Task Sparse Learning Problem. In Proceedings Of The 9Th Ieee International Conference On Data Mining (Icdm'09). 746-751.
16. Dekel, O., Long, P. M., And Singer, Y. 2006. Online Multitask Learning. In Proceedings Of The 19Th Annual Conference On Learning Theory (Colt'06). 453-467. (Pubitemid 44072212)
17. Dhillon, P. S., Foster, D. P., And Ungar, L. H. 2011. Minimum Description Length Penalization For Group And Multi-Task Sparse Learning. J. Mach. Learn. Res. 12, 525-564.
18. Dhillon, P. S., Tomasik, B., Foster, D. P., And Ungar, L. H. 2009. Multi-Task Feature Selection Using The Multiple Inclusion Criterion (Mic). In Proceedings Of The European Conference On Machine Learning And Knowledge Discovery In Databases (Ecml/Pkdd'09). 276-289.
19. Duchi, J. And Singer, Y. 2009. Efficient Learning Using Forward-Backward Splitting. J. Mach. Learn. Res. 10, 2873-2898.
20. Evgeniou, T.,Micchelli, C. A., And Pontil, M. 2005. Learning Multiple Tasks With Kernel Methods. J. Mach. Learn. Res. 6, 615-637.
21. Evgeniou, T. And Pontil, M. 2004. Regularized Multi-Task Learning. In Proceedings Of The 10Th Acm Sigkdd International Conference On Knowledge Discovery And Data Mining (Kdd'04). 109-117. (Pubitemid 40114921)
22. Friedman, J., Hastie, T., And Tibshirani, R. 2010. A Note On The Group Lasso And A Sparse Group Lasso. Http://Arxiv.Org/Pdf/1001.0736.Pdf.
23. Han, J. And Kamber, M. 2006. Data Mining: Concepts And Techniques 2Nd Ed. Morgan Kaufmann, San Fransisco.
24. Han, Y., Wu, F., Jia, J., Zhuang, Y., And Yu, B. 2010. Multi-Task Sparse Discriminant Analysis (Mtsda) With Overlapping Categories. In Proceedings Of The 24Th Aaai Conference On Artificial Intelligence.
25. Hazan, E., Agarwal, A., And Kale, S. 2007. Logarithmic Regret Algorithms For Online Convex Optimization. Mach. Learn. 69, 2-3, 169-192. (Pubitemid 47574314)
26. Hu, C., Kwok, J., And Pan, W. 2009. Accelerated Gradient Methods For Stochastic Optimization And Online Learning. Adv. Neural Inf. Process. Syst. 22, 781-789.
27. Jebara, T. 2004. Multi-Task Feature And Kernel Selection For Svms. In Proceedings Of The 21St International Conference On Machine Learning (Icml'04).
28. Jebara, T. 2011. Multitask Sparsity Via Maximum Entropy Discrimination. J. Mach. Learn. Res. 12, 75-110.
29. Langford, J., Li, L., And Zhang, T. 2009. Sparse Online Learning Via Truncated Gradient. J. Mach. Learn. Res. 10, 777-801.
30. Lenk, P. J.,Desarbo,W. S., Green, P. E., And Young, M. R. 1996. Hierarchical Bayes Conjoint Analysis: Recovery Of Partworth Heterogeneity From Reduced Experimental Designs. Market. Sci. 15, 2, 173-191.
31. Ling, G., Yang, H., King, I., And Lyu, M. R. 2012. Online Learning For Collaborative Filtering. In Proceedings Of The Ieee World Congress On Computational Intelligence (Wcci'12). 1-8.
32. Liu, H., Palatucci, M., And Zhang, J. 2009A. Blockwise Coordinate Descent Procedures For The Multi-Task Lasso, With Applications To Neural Semantic Basis Discovery. In Proceedings Of The 26Th Annual International Conference On Machine Learning (Icml'09). 649-656.
33. Liu, J., Chen, J., And Ye, J. 2009B. Large-Scale Sparse Logistic Regression. In Proceedings Of The 15Th Acm Sigkdd International Conference On Knowledge Discovery And Data Mining (Kdd'09). 547-556.
34. Liu, J., Ji, S., And Ye, J. 2009C. Multi-Task Feature Learning Via Efficient L2; 1 Norm Minimization. In Proceedings Of The 25Th Conference On Uncertainty In Artificial Intelligence (Uai'09).
35. Liu, J., Ji, S., And Ye., J. 2009D. Slep: Sparse Learning With Efficient Projections. Http://Www.Public.Asu.Edu/Ye02/Software/Slep.
36. Nesterov, Y. 2009. Primal-Dual Subgradient Methods For Convex Problems. Math. Program. 120, 1, 221-259.
37. Obozinski, G., Taskar, B., And Jordan, M. I. 2009. Joint Covariate Selection And Joint Subspace Selection For Multiple Classification Problems. Statist. Comput. 20, 2, 231-252.
38. Pong, T. K., Tseng, P., Ji, S., And Ye, J. 2010. Trace Norm Regularization: Reformulations, Algorithms, And Multi-Task Learning. Siam J. Optim. 20, 6, 3465-3489.
39. Quattoni, A., Carreras, X., Collins, M., And Darrell, T. 2009. An Efficient Projection For L1,8 Regularization. In Proceedings Of The 26Th International Conference On Machine Learning (Icml'09), 857-864.
40. Shalev-Shwartz, S. And Singer, Y. 2007. A Primal-Dual Perspective Of Online Learning Algorithms. Mach. Learn. 69, 2-3, 115-142. (Pubitemid 47574312)
41. Shalev-Shwartz, S., Singer, Y., And Srebro, N. 2007. Pegasos: Primal Estimated Sub-Gradient Solver For Svm. In Proceedings Of The 24Th International Conference On Machine Learning (Icml'07). 807-814.
42. Tibshirani, R. 1996. Regression Shrinkage And Selection Via The Lasso. J. Roy. Statist. Soc. B58, 1, 267-288.
43. Vapnik, V. 1999. The Nature Of Statistical Learning Theory 2Nd Ed. Springer, New York.
44. Xiao, L. 2010. Dual Averaging Method For Regularized Stochastic Learning And Online Optimization. J. Mach. Learn. Res. 11, 2543-2596.
45. Xu, Z., Jin, R., Yang, H.,King, I., And Lyu, M. R. 2010. Simple And Efficient Multiple Kernel Learning By Group Lasso. In Proceedings Of The 27Th International Conference On Machine Learning (Icml'10). 1175-1182.
46. Yang, H.,King, I., And Lyu, M. R. 2010A. Multi-Task Learning For One-Class Classification. In Proceedings Of The International Joint Conference On Neural Networks (Ijcnn'10). 1-8.
47. Yang, H., King, I., And Lyu, M. R. 2010B. Online Learning For Multi-Task Feature Selection. In Proceedings Of The 19Th Acm Conference On Information And Knowledge Management (Cikm'10). 1693-1696.
48. Yang, H., King, I., And Lyu, M. R. 2011A. Sparse Learning Under Regularization Framework 1St Ed. Lambert Academic Publishing.
49. Yang, H., Xu, Z., King, I., And Lyu, M. R. 2010C. Online Learning For Group Lasso. In Proceedings Of The 27Th International Conference On Machine Learning (Icml'10). 1191-1198.
50. Yang, H., Xu, Z., Ye, J., King, I., And Lyu, M. R. 2011B. Efficient Sparse Generalized Multiple Kernel Learning. Ieee Trans. Neural Netw. 22, 3, 433-446.
51. Zhang, Y. 2010. Multi-Task Active Learning With Output Constraints. In Proceedings Of The 24Th Aaai Conference On Artificial Intelligence (Aaai'10).
52. Zhang, Y., Yeung, D.-Y., And Xu, Q. 2010. Probabilistic Multi-Task Feature Selection. Adv. Neural Inf. Process. Syst. 23, 2559-2567.
53. Zhao, P., Hoi, S. C. H., And Jin, R. 2011A. Double Updating Online Learning. J. Mach. Learn. Res. 12, 1587-1615.
54. Zhao, P., Hoi, S. C. H., Jin, R., And Yang, T. 2011B. Online Auc Maximization. In Proceedings Of The 28Th International Conference On Machine Learning (Icml'11). 233-240.
55. Zhou, Y., Jin, R., And Hoi, S. C. 2010. Exclusive Lasso For Multi-Task Feature Selection. In Proceeding Of The 14Th International Conference On Artificial Intelligence And Statistics (Aistats'10).
56. Zou, H. And Hastie, T. 2005. Regularization And Variable Selection Via The Elastic Net. J. Royal Statist. Soc. B67, 301-320. (Pubitemid 40465877)