Linear algorithms for online multitask classification

Journal of Machine Learning Research

Volumn 11, Issue , 2010, Pages 2901-2934

  Cavallanti, Giovanni ^a   Cesa Bianchi, Nicolò ^a   Gentile, Claudio ^b  

^a UNIVERSITY OF MILAN   (Italy)

^b UNIVERSITY OF INSUBRIA   (Italy)

Author keywords

Mistake bounds; Multitask learning; Perceptron algorithm; Spectral regularization

Indexed keywords

BINARY CLASSIFICATION PROBLEMS; LINEAR ALGORITHMS; MATRIX; MISTAKE BOUNDS; MULTI-TASK KERNELS; MULTITASK LEARNING; ON-LINE SETTING; PERCEPTRON; PERCEPTRON ALGORITHMS; REAL WORLD DATA; REFERENCE VECTORS; REGULARITY CONDITION; SPECTRAL REGULARIZATION; TASK RELATEDNESS;

LEARNING ALGORITHMS;

EID: 78649437219     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (38)

1. J. Abernethy, P. L. Bartlett, and A. Rakhlin. Multitask learning with expert advice. In Proceedings of the 20th Annual Conference on Learning Theory, pages 484-198. Springer, 2007.
2. A. Agarwal, A. Rakhlin, and P. Bartlett. Matrix regularization techniques for online multitask learning. Technical Report UCB/EECS-2008-138, EECS Department, University of California, Berkeley, 2008.
3. R. K. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817-1853, 2005.
4. A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In Advances in Neural Information Processing Systems 19, pages 41-48. MIT Press, 2007.
5. A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying. A spectral regularization framework for multitask structure learning. In Advances in Neural Information Processing Systems 20, pages 25-32. Curran Associates, 2008.
6. A. Beck and M. Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operation Research Letters, 31:167-175, 2003.
7. U. Brefeld, T. Gaertner, T. Scheffer, and S. Wrobel. Efficient co-regularised least squares regression. In Proceedings of the 23rd International Conference on Machine Learning. Omnipress, 2006.
8. N. Cesa-Bianchi and G. Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
9. N. Cesa-Bianchi, A. Conconi, and C. Gentile. A second-order Perceptron algorithm. SIAM Journal on Computing, 43(3):640-668, 2005.
10. O. Dekel, P. M. Long, and Y. Singer. Online learning of multiple tasks with a shared loss. Journal of Machine Learning Research, 8:2233-2264, 2007.
11. ECML/PKDD Discovery Challenge, 2006. URL: www.ecmlpkdd2006.org/challenge. html.
12. T. Evgeniou, C. Micchelli, and M. Pontil. Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6:614-637, 2005.
13. Y. Freund and R. E. Schapire. Large margin classification using the perceptron algorithm. Machine Learning, 37(3):277-296, 1999.
14. C. Gentile. The robustness of the p-norm algorithms. Machine Learning, 53(3):265-299, 2003.
15. A. J. Grove, N. Littlestone, and D. Schuurmans. General convergence results for linear discriminant updates. Machine Learning, 43(3):173-210, 2001.
16. M. Herbster and G. Lever. Predicting the labelling of a graph via minimum p-seminorm interpolation. In Proceedings of the 22nd Annual Conference on Learning Theory. Omnipress, 2009.
17. M. Herbster, M. Pontil, and L. Wainer. Online learning over graphs. In Proceedings of the 22nd International Conference on Machine Learning, pages 305-312. Omnipress, 2005.
18. L. Hogben. Handbook of Linear Algebra. CRC Press, 2006.
19. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, 1985.
20. R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, 1991.
21. L. Jacob, F. Bach, and J. P. Vert. Clustered multi-task learning. In Advances in Neural Information Processing Systems 21, pages 745-752. Curran Associates, 2009.
22. A. Juditsky and A. Nemirovski. Large deviations of vector-valued martingales in 2-smooth normed spaces. Manuscript, 2009.
23. S. Kakade and D. Foster. Multi-view regression via canonical correlation analysis. In Proceedings of the 20th Annual Conference on Learning Theory, pages 82-96. Springer, 2007.
24. S. Kakade, S. Shalev-Shwartz, and A. Tewari. On the duality of strong convexity and strong smoothness: Learning applications and matrix regularization. Manuscript, 2009.
25. J. Kivinen and M. Warmuth. Relative loss bounds for multidimensional regression problems. Machine Learning, 45:301-329, 2001.
26. A. S. Lewis. The convex analysis of unitarily invariant matrix functions. Journal of Convex Analsys, 2(1):173-183, 1995.
27. N. Littlestone. Mistake Bounds and Logarithmic Linear-threshold Learning Algorithms. PhD thesis, University of California at Santa Cruz, 1989.
28. G. Lugosi, O. Papaspiliopoulos, and G. Stoltz. Online multi-task learning with hard constraints. In Proceedings of the 22nd Annual Conference on Learning Theory. Omnipress, 2009.
29. J. R. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley, 1999.
30. A. Maurer. Bounds for linear multi-task learning. Journal of Machine Learning Research, 7:117-139, 2006.
31. A. Nemirovski and D. Yudin. Problem Complexity and Method Efficiency in Optimization. Nauka Publishers, 1978.
32. NIST, 2004. URL: trec.nist.gov/data/reuters/reuters.html.
33. V. Sindhwani D. R. Rosenberg. An RKHS for multi-view learning and manifold co-regularization. In Proceedings of the 26th International Conference on Machine Learning, pages 976-983. Omnipress, 2008.
34. V. Sindhwani, P. Niyogi, and M. Belkin. A co-regularized approach to semi-supervised learning. In Proceedings of the 22nd International Conference on Machine Learning Workshop on Learning with Multiple Views, 2005.
35. K. Tsuda, G. Raetsch, and M. K. Warmuth. Matrix exponentiated gradient updates for on-line learning and Bregman projection. Journal of Machine Learning Research, 6:995-1018, 2005.
36. M. K. Warmuth. Winnowing subspaces. In Proceedings of the 24th International Conference on Machine Learning, pages 999-1006. Omnipress, 2007.
37. M. K. Warmuth. Kernelization of matrix updates. Manuscript, 2009.
38. M. K. Warmuth and D. Kuzmin. Online variance minimization. In Proceedings of the 19th Annual Conference on Learning Theory, pages 514-528. Springer, 2006.