Least squares revisited: Scalable approaches for multi-class prediction

31st International Conference on Machine Learning, ICML 2014

Volumn 3, Issue , 2014, Pages 1937-1945

  Agarwal, Alekh ^a   Kakade, Sham M ^b   Karampatziakis, Nikos ^c   Song, Le ^d   Valiant, Gregory ^e  

^a MICROSOFT RESEARCH   (United States)

^b MICROSOFT RESEARCH   (United Kingdom)

^c MICROSOFT   (United States)

^d Georgia Institute of Technology   (United States)

^e Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ITERATIVE METHODS; LEARNING SYSTEMS;

CLASS PREDICTION; DATA DIMENSIONS; FIRST-ORDER METHODS; HIGH DIMENSIONAL DATASETS; ITERATIVE LEAST SQUARES; OPTIMIZATION PACKAGES; SCALABLE APPROACH; SIMPLE ALGORITHM;

ALGORITHMS;

EID: 84919949624     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Agarwal, A. Selective sampling algorithms for cost-sensitive multiclass prediction. In ICML, 2013.
2. Agarwal, A., Kakade, S. M., Karampatziakis, N., Song, L., and Valiant, G. Least squares revisited: Scalable approaches for multi-class prediction. arXiv preprint arXiv.1310.1949, 2013.
3. Bordes, A., Bottou, L., and Gallinari, P. Sgd-qn: Careful quasi- newton stochastic gradient descent. Journal of Machine Learning Research, 10:1737-1754, July 2009.
4. Bottou, L. and Bousquet, O. The tradeoffs of large scale learning. In NIPS. 2008.
5. Bronshtein, E.M. e-entropy of convex sets and functions. Siberian Mathematical Journal, 17(3):393-398, 1976.
6. Byrd, R. H., Chin, G. M., Neveitt, W., and Nocedal, J. On the use of stochastic hessian information in optimization methods for machine learning. SIAM Journal on Optimization, 21(3): 977-995, 2011.
7. Chapelle, O. Training a support vector machine in the primal. Neural Comput., 19(5): 1155-1178, 2007.
8. Coates, A., Ng, A. Y., and Lee, H. An analysis of single-layer networks in unsupervised feature learning. Journal of Machine Learning Research - Proceedings Track, 15:215-223, 2011.
9. Collins, M. and Koo, T. Discriminative reranking for natural language parsing. In ICML, 2000.
10. 1-ball for learning in high dimensions. In ICML, 2008.
11. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., and Lin, C.- J. Liblinear: A library for large linear classification. Journal of Machine Learning Research, 9:1871-1874, 2008.
12. Friedman, J. H. Greedy function approximation: A gradient boosting machine.(english summary). Ann. Statist, 29(5): 1189- 1232, 2001.
13. Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A. C., and Bengio, Y. Maxout networks. CoRR, 2013.
14. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
15. Jebara, T. and Choromanska, A. Majorization for crfs and latent likelihoods. In NIPS, 2012.
16. Kakade, S. M., Kalai, A., Kanade, V., and Shamir, O. Efficient learning of generalized linear and single index models with isotonic regression. In NIPS, 2011.
17. Kalai, A. T. and Sastry, R. The isotron algorithm: High- dimensional isotonic regression. In COLT '09, 2009.
18. Lewis, D. D., Yang, Y., Rose, T. G., and Li, F. Rcvl: A new benchmark collection for text categorization research. The Journal of Machine Learning Research, 5:361-397, 2004.
19. Matsushima, S., Vishwanathan, S. V. N., and Smola, A. J. Linear support vector machines via dual cached loops. In KDD, 2012.
20. Nemirovsky, A. S. and Yudin, D. B. Problem Complexity and Method Efficiency in Optimization. New York, 1983.
21. Nesterov, Y. Introductory Lectures on Convex Optimization. New York, 2004.
22. Nesterov, Y. Efficiency of coordinate descent methods on huge- scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, 2012.
23. Piatt, J. C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Adavances in large margin classifiers, pp. 61-74. MIT Press, 1999.
24. Rahimi, A. and Recht, B. Random features for large-scale kernel machines. Advances in neural information processing systems, 20:1177-1184, 2007.
25. Recht, B., Re, C., Wright, S. J., and Niu, F. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In NIPS, pp. 693-701, 2011.
26. Richtarik, P. and Takac, M. Parallel coordinate descent methods for big data optimization. 2012. URL http://arxiv.org/abs/1212.0873.
27. Rockafellar, R.T. Characterization of the subdifferentials of convex functions. Pac. J. Math., 17:497-510, 1966.
28. Roux, N. L., Schmidt, M., and Bach, F. A stochastic gradient method with an exponential convergence rate for finite training sets. In NIPS, pp. 2672-2680. 2012.
29. Shalev-Shwartz, S. Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2), 2012.
30. Shalev-Shwartz, S. and Zhang, T. Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization. Journal of Machine Learning Reearch, 14:567-599, 2013.
31. Yu, H.-F., Hsieh, C.-J., Chang, K.-W., and Lin, C.-J. Large linear classification when data cannot fit in memory. TKDD, 5(4), 2012.