Sparse Online Learning via Truncated Gradient

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 777-801

  Langford, John ^a   Li, Lihong ^b   Zhang, Tong ^c  

^a YAHOO RESEARCH   (United States)

^b RUTGERS UNIVERSITY   (United States)

^c RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA SETS; GENERAL METHODS; LOSS FUNCTIONS; ON-LINE LEARNING; PARAMETER CONTROLS; REGULARIZATION METHODS; SPARSIFICATION; STOCHASTIC GRADIENT DESCENTS;

INTERNET;

E-LEARNING;

EID: 64149115569     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (19)

1. Arthur Asuncion and David J. Newman. UCI machine learning repository, 2007. University of California, Irvine, School of Information and Computer Sciences, http://www.ics.uci.edu/~mlearn/MLRepository.html.
2. Suhrid Balakrishnan and David Madigan. Algorithms for sparse linear classifiers in the massive data setting. Journal of Machine Learning Research, 9:313-337, 2008.
3. Bob Carpenter. Lazy sparse stochastic gradient descent for regularized multinomial logistic regression. Technical report, April 2008.
4. Nicolò Cesa-Bianchi, Philip M. Long, and Manfred Warmuth. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Transactions on Neural Networks, 7(3):604-619, 1996.
5. Cheng-Tao Chu, Sang Kyun Kim, Yi-An Lin, Yuan Yuan Yu, Gary Bradski, Andrew Y. Ng, and Kunle Olukotun. Map-reduce for machine learning on multicore. In Advances in Neural Information Processing Systems 20 (NIPS-07), 2008.
6. Ofer Dekel, Shai Shalev-Schwartz, and Yoram Singer. The Forgetron: A kernel-based perceptron on a fixed budget. In Advances in Neural Information Processing Systems 18 (NIPS-05), pages 259-266, 2006.
7. John Duchi and Yoram Singer. Online and batch learning using forward looking subgradients. Unpublished manuscript, September 2008.
8. 1-ball for learning in high dimensions. In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), pages 272-279, 2008.
9. Jyrki Kivinen and Manfred K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1): 1-63, 1997.
10. John Langford, Lihong Li, and Alexander L. Strehl. Vowpal Wabbit (fast online learning), 2007. http://hunch.net/~vw/.
11. Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Y. Ng. Efficient sparse coding algorithms. In Advances in Neural Information Processing Systems 19 (NIPS-06), pages 801-808, 2007.
12. David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
13. Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithms. Machine Learning, 2(4):285-318, 1988.
14. Nick Littlestone, Philip M. Long, and Manfred K. Warmuth. On-line learning of linear functions. Computational Complexity, 5(2): 1-23, 1995.
15. Shai Shalev-Shwartz, Yoram Singer, and Nathan Srebro. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In Proceedings of the Twenty-Fourth International Conference on Machine Learning (ICML-07), 2007.
16. Karl Sjöstrand. Matlab implementation of LASSO, LARS, the elastic net and SPCA, June 2005. Version 2.0, http://www2.imm.dtu.dk/pubdb/p.php?3897.
17. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, B., 58(1):267-288, 1996.
18. Tong Zhang. Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the Twenty-First International Conference on Machine Learning (ICML-04), pages 919-926, 2004.
19. Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the Twentieth International Conference on Machine Learning (ICML-03), pages 928-936, 2003.