Robust large-scale machine learning in the cloud

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn 13-17-August-2016, Issue , 2016, Pages 1125-1134

  Rendle, Steffen ^a   Fetterly, Dennis ^a   Shekita, Eugene J ^a   Su, Bor Yiing ^a  

^a GOOGLE INC   (United States)

Author keywords

Coordinate descent; Distributed machine learning; Linear regression

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COST EFFECTIVENESS; COSTS; DATA MINING; LINEAR REGRESSION;

COMPRESSED DATUM; COMPUTING ENVIRONMENTS; CONVERGENCE BEHAVIORS; COORDINATE DESCENT; DISTRIBUTED MACHINE LEARNING; GENERALIZED LINEAR MODEL; LARGE-SCALE MACHINE LEARNING; MASSIVE DATA SETS;

LEARNING SYSTEMS;

EID: 84984950376     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2939672.2939790     Document Type: Conference Paper

References (27)

1. https://cloud. google. com/.
2. https://aws. amazon. com/.
3. https://www. csie. ntu. edu. tw/cjlin/liblinear/.
4. https://cran. r-project.org/web/packages/glmnet/.
5. J. Bennet and S. Lanning. The Netix prize. In KDD Cup and Workshop, 2007.
6. R. D. Blumofe and C. E. Leiserson. Scheduling multithreaded computations by work stealing. J. ACM, 46(5):720-748, Sept. 1999.
7. L. Bottou. Stochastic learning. In Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3176, pages 146-168. Springer Verlag, 2004.
8. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin. Parallel coordinate descent for l1-regularized loss minimization. In Proceedings of the International Conference on Machine Learning (ICML 2011), June 2011.
9. J. Canny and H. Zhao. Big data analytics with small footprint: Squaring the cloud. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '13, pages 95-103, 2013.
10. T. Chandra. Sibyl: A system for large scale machine learning at Google. Keynote talk at the 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, 2014.
11. K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. Coordinate descent method for large-scale l2-loss linear support vector machines. J. Mach. Learn. Res. 9:1369-1398, June 2008.
12. O. Chapelle, E. Manavoglu, and R. Rosales. Simple and scalable response prediction for display advertising. ACM Trans. Intell. Syst. Technol. 5(4):61:1-61:34, Dec. 2014.
13. Y. Chen, R. Grit, D. Zats, and R. H. Katz. Understanding TCP incast and its implications for big data workloads. Technical report, DTIC Document, 2012.
14. M. Collins, R. E. Schapire, and Y. Singer. Logistic regression, Adaboost and Bregman distances. Mach. Learn. 48(1-3):253-285, Sept. 2002.
15. J. Dean and L. A. Barroso. The tail at scale. Commun. ACM, 56(2):74-80, Feb. 2013.
16. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, M. A. Ranzato, A. Senior, P. Tucker, K. Yang, Q. V. Le, and A. Y. Ng. Large scale distributed deep networks. In Advances in Neural Information Processing Systems 25, pages 1223-1231. 2012.
17. J. Dean and S. Ghemawat. Mapreduce: Simpliffed data processing on large clusters. Commun. ACM, 51(1):107-113, Jan. 2008.
18. J. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2010.
19. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google le system. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP '03, pages 29-43, 2003.
20. T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich. Web-scale Bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. In Proceedings of the 27th International Conference on Machine Learning, pages 13-20, 2010.
21. X. He, J. Pan, O. Jin, T. Xu, B. Liu, T. Xu, Y. Shi, A. Atallah, R. Herbrich, S. Bowers, and J. Q. n. Candela. Practical lessons from predicting clicks on ads at Facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, ADKDD'14, pages 5:1-5:9, 2014.
22. M. Li, D. G. Andersen, J. W. Park, A. J. Smola, A. Ahmed, V. Josifovski, J. Long, E. J. Shekita, and B.-Y. Su. Scaling distributed machine learning with the parameter server. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, OSDI'14, pages 583-598, 2014.
23. D. Mahajan, S. S. Keerthi, and S. Sundararajan. A distributed block coordinate descent method for training l1 regularized linear classiffers. CoRR, abs/1405. 4544, 2014.
24. H. B. McMahan, G. Holt, D. Sculley, M. Young, D. Ebner, J. Grady, L. Nie, T. Phillips, E. Davydov, D. Golovin, S. Chikkerur, D. Liu, M. Wattenberg, A. M. Hrafnkelsson, T. Boulos, and J. Kubica. Ad click prediction: A view from the trenches. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '13, pages 1222-1230, 2013.
25. B. Recht, C. Re, S. Wright, and F. Niu. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in Neural Information Processing Systems 24, pages 693-701. 2011.
26. S. Shalev-Shwartz and A. Tewari. Stochastic methods for l1-regularized loss minimization. J. Mach. Learn. Res. 12:1865-1892, July 2011.
27. A. Verma, L. Pedrosa, M. R. Korupolu, D. Oppenheimer, E. Tune, and J. Wilkes. Large-scale cluster management at Google with Borg. In Proceedings of the European Conference on Computer Systems (EuroSys), 2015.