Large scale online kernel learning

Journal of Machine Learning Research

Volumn 17, Issue , 2016, Pages

  Lu, Jing ^a   Hoi, Steven C H ^a   Wang, Jialei ^b   Zhao, Peilin ^c   Liu, Zhi Yong ^d  

^a SINGAPORE MANAGEMENT UNIVERSITY   (Singapore)

^b THE UNIVERSITY OF CHICAGO   (United States)

^c INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^d INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

Kernel approximation; Large scale machine learning; Online learning

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BUDGET CONTROL; E-LEARNING; LEARNING SYSTEMS;

EFFECTIVENESS AND EFFICIENCIES; FUNCTIONAL APPROXIMATION TECHNIQUE; KERNEL APPROXIMATION; KERNEL MACHINE LEARNING; LARGE-SCALE MACHINE LEARNING; MAINTENANCE STRATEGIES; MULTI-CLASS CLASSIFICATION; ONLINE LEARNING;

LEARNING ALGORITHMS;

EID: 84979917125     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (46)

1. Giovanni Cavallanti, Nicolò Cesa-Bianchi, and Claudio Gentile. Tracking the best hyperplane with a simple budget perceptron. Machine Learning, 69(2-3):143-167, 2007.
2. Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.
3. Radha Chitta, Rong Jin, Timothy C Havens, and Anil K Jain. Approximate kernel k-means: Solution to large scale kernel clustering. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 895-903. ACM, 2011.
4. Radha Chitta, Rong Jin, and Anil Jain. Efficient kernel clustering using random fourier features. In IEEE International Conference on Data Mining, 2012.
5. Corinna Cortes, Mehryar Mohri, and Ameet Talwalkar. On the impact of kernel approximation on learning accuracy. In International Conference on Artificial Intelligence and Statistics, 2010.
6. Koby Crammer and Yoram Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265-292, 2001.
7. Koby Crammer and Yoram Singer. On the learnability and design of output codes for multiclass problems. Machine Learning, 47(2-3):201-233, 2002.
8. Koby Crammer, Jaz S Kandola, and Yoram Singer. Online classification on a budget. In Neural Information Processing Systems, volume 2, page 5, 2003.
9. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. Online passive-aggressive algorithms. Journal of Machine Learning Research, 7:551-585, 2006.
10. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008.
11. Ofer Dekel, Shai Shalev-Shwartz, and Yoram Singer. The forgetron: A kernel-based perceptron on a fixed budget. In Neural Information Processing Systems, 2005.
12. Mark Dredze, Koby Crammer, and Fernando Pereira. Confidence-weighted linear classification. In Proceedings of the International Conference on Machine Learning, pages 264-271, 2008.
13. Yoav Freund and Robert E. Schapire. Large margin classification using the perceptron algorithm. Maching Learning., 37(3):277-296, 1999.
14. Steven C. H. Hoi, Rong Jin, and Michael R. Lyu. Learning nonparametric kernel matrices from pairwise constraints. In Proceedings of the International Conference on Machine Learning, pages 361-368, Corvalis, Oregon, 2007. ISBN 978-1-59593-793-3.
15. Steven C. H. Hoi, Rong Jin, Peilin Zhao, and Tianbao Yang. Online multiple kernel classification. Machine Learning, 90(2):289-316, 2013.
16. Steven CH Hoi, Michael R Lyu, and Edward Y Chang. Learning the unified kernel machines for classification. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 187-196. ACM, 2006.
17. Steven C.H. Hoi, Jialei Wang, and Peilin Zhao. Libol: A library for online learning algorithms. Journal of Machine Learning Research, 15:495-499, 2014. URL http://jmlr.org/papers/v15/hoi14a.html.
18. Chih-Wei Hsu and Chih-Jen Lin. A comparison of methods for multiclass support vector machines. Neural Networks, IEEE Transactions on, 13(2):415-425, 2002.
19. Jyrki Kivinen, Alex J. Smola, and Robert C. Williamson. Online learning with kernels. In Neural Information Processing Systems, pages 785-792, 2001.
20. Bin Li, Peilin Zhao, Steven CH Hoi, and Vivekanand Gopalkrishnan. Pamr: Passive aggressive mean reversion strategy for portfolio selection. Machine learning, 87(2):221-258, 2012.
21. Yucheng Low, Danny Bickson, Joseph Gonzalez, Carlos Guestrin, Aapo Kyrola, and Joseph M Hellerstein. Distributed graphlab: a framework for machine learning and data mining in the cloud. Proceedings of the VLDB Endowment, 5(8):716-727, 2012.
22. Justin Ma, Lawrence K Saul, Stefan Savage, and Geoffrey M Voelker. Identifying suspicious urls: an application of large-scale online learning. In Proceedings of the International Conference on Machine Learning, pages 681-688. ACM, 2009.
23. Francesco Orabona, Joseph Keshet, and Barbara Caputo. The projectron: a bounded kernel-based perceptron. In Proceedings of the International Conference on Machine Learning, pages 720-727, 2008.
24. Francesco Orabona, Joseph Keshet, and Barbara Caputo. Bounded kernel-based online learning. Journal of Machine Learning Research, 10:2643-2666, 2009.
25. Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Neural Information Processing Systems, 2007.
26. Frank Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65:386-407, 1958.
27. W. Rudin. Fourier Analysis on Groups. Wiley-Interscience, 1990.
28. Bernhard Schölkopf, Ralf Herbrich, and Alex J. Smola. A generalized representer theorem. In Conference on Learning Theory, pages 416-426, 2001.
29. Shai Shalev-Shwartz, Yoram Singer, Nathan Srebro, and Andrew Cotter. Pegasos: primal estimated sub-gradient solver for svm. Math. Program., 127(1):3-30, 2011.
30. John Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Cambridge university press, 2004.
31. Ameet Talwalkar, Sanjiv Kumar, and Henry A. Rowley. Large-scale manifold learning. In Computer Vision and Pattern Recognition, 2008.
32. Vladimir N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag New York, Inc., New York, NY, USA, 1995. ISBN 0-387-94559-8.
33. Vladimir N Vapnik. Statistical Learning Theory. Wiley, 1998.
34. Jialei Wang, Peilin Zhao, and Steven C. H. Hoi. Cost-sensitive online classification. In IEEE International Conference on Data Mining, pages 1140-1145, 2012a.
35. Jialei Wang, Steven CH Hoi, Peilin Zhao, Jinfeng Zhuang, and Zhi-yong Liu. Large scale online kernel classification. In Proceedings of International Joint Conference on Artificial Intelligence, pages 1750-1756. AAAI Press, 2013.
36. Zhuang Wang and Slobodan Vucetic. Twin vector machines for online learning on a budget. In International Conference on Data Mining, pages 906-917. SIAM, 2009.
37. Zhuang Wang and Slobodan Vucetic. Online passive-aggressive algorithms on a budget. In International Conference on Artificial Intelligence and Statistics, pages 908-915, 2010.
38. Zhuang Wang, Koby Crammer, and Slobodan Vucetic. Breaking the curse of kernelization: Budgeted stochastic gradient descent for large-scale svm training. Journal of Machine Learning Research, 13:3103-3131, 2012b.
39. Christopher K. I. Williams and Matthias Seeger. Using the nyström method to speed up kernel machines. In Neural Information Processing Systems, pages 682-688, 2000.
40. Hao Xia, Pengcheng Wu, and Steven C. H. Hoi. Online multi-modal distance learning for scalable multimedia retrieval. In ACM International Conference on Web Search and Data Mining, pages 455-464, 2013.
41. Tianbao Yang, Yufeng Li, Mehrdad Mahdavi, Rong Jin, and Zhi hua Zhou. Nystrom method vs random fourier features: A theoretical and empirical comparison. In Neural Information Processing Systems, 2012.
42. Kai Zhang and James T. Kwok. Density-weighted nyström method for computing large kernel eigensystems. Neural Computation, 21(1):121-146, 2009.
43. Kai Zhang, Liang Lan, Zhuang Wang, and Fabian Moerchen. Scaling up kernel svm on limited resources: A low-rank linearization approach. In International Conference on Artificial Intelligence and Statistics, pages 1425-1434, 2012.
44. Peilin Zhao and Steven CH Hoi. Otl: A framework of online transfer learning. In Proceedings of the International Conference on Machine Learning, 2010.
45. Peilin Zhao, Steven C. H. Hoi, and Rong Jin. Double updating online learning. Journal of Machine Learning Research, 12:1587-1615, 2011.
46. Peilin Zhao, Jialei Wang, Pengcheng Wu, Rong Jin, and Steven C. H. Hoi. Fast bounded online gradient descent algorithms for scalable kernel-based online learning. In Proceedings of the International Conference on Machine Learning, 2012.