Unexpected challenges in large scale machine learning

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2012, Pages 1-6

  Parker, Charles ^a  

^a BigML Inc   (United States)

Author keywords

Big data; Concept drift; Non stationary analysis; Supervised learning

Indexed keywords

BIG DATUM; CONCEPT DRIFTS; NONSTATIONARY; SUPERVISED CLASSIFICATION; TRIVIAL OPERATIONS;

ALGORITHMS; LEARNING SYSTEMS; SUPERVISED LEARNING;

COMPUTER SYSTEMS PROGRAMMING;

EID: 84866628225     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2351316.2351317     Document Type: Conference Paper

References (31)

1. N. T. Andrea Gesmundo. HadoopPerceptron: a toolkit for distributed perceptron training and prediction with MapReduce. In Proceedings of the 2012 Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France, 2012.
2. P. L. Bartlett. Optimal online prediction in adversarial environments. In Proceedings of the 13th international conference on Discovery science, DS'10, pages 371-371, 2010.
3. R. Bekkerman, M. Bilenko, and J. Langford. Scaling up machine learning: parallel and distributed approaches. In Proceedings of the 17th ACM SIGKDD International Conference Tutorials, KDD '11 Tutorials, 2011.
4. Y. Ben-Haim and E. Tom-Tov. A streaming parallel decision tree algorithm. Journal of Machine Learning Research, 11:849-872, Mar. 2010.
5. A. Blum. On-line algorithms in machine learning. In Proceedings of the Workshop on On-Line Algorithms, pages 306-325, 1996.
6. R. Bradley. Basic properties of strong mixing conditions, pages 165-192. 1986.
7. C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. Bradski, A. Y. Ng, and K. Olukotun. Map-reduce for machine learning on multicore. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 281-288. MIT Press, Cambridge, MA, 2007.
8. P. Domingos and G. Hulten. A general method for scaling up machine learning algorithms and its application to clustering. In In Proceedings of the Eighteenth International Conference on Machine Learning, pages 106-113. Morgan Kaufmann, 2001.
9. A. Dries and U. Rückert. Adaptive concept drift detection. Statistical Analysis and Data Mining, 2(5âǍŘ6):311-327, Dec. 2009.
10. J. Duchi, M. Wainwright, and P. Bartlett. Randomized smoothing for (parallel) stochastic optimization. In NIPS 2011 Workshop on Parallel and Large-Scale Machine Learning (BigLearn), December 2011.
11. J. H. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2 2010.
12. R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismanis. Large-scale matrix factorization with distributed stochastic gradient descent. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '11, pages 69-77, 2011.
13. A. Ghoting, P. Kambadur, E. Pednault, and R. Kannan. NIMBLE: a toolkit for the implementation of parallel data mining and machine learning algorithms on mapreduce. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '11, pages 334-342, 2011.
14. J. E. Gonzalez, Y. Low, and C. Guestrin. Residual splash for optimally parallelizing belief propagation. In In In Artificial Intelligence and Statistics (AISTATS, pages 177-184, 2009.
15. S. Guha, N. Koudas, and K. Shim. Approximation and streaming algorithms for histogram construction problems. ACM Transactions on Database Systems, 31(1):396-438, 2006.
16. D. Hsu, N. Karampatziakis, J. Langford, and A. J. Smola. Parallel online learning. CoRR, abs/1103.4204, 2011.
17. N. S. L. P. Kumar, S. Satoor, and I. Buck. Fast parallel expectation maximization for gaussian mixture models on GPUs using CUDA. In HPCC, pages 103-109, 2009.
18. P. Li, A. Shrivastava, and A. C. König. Training logistic regression and svm on 200gb data using b-bit minwise hashing and comparisons with vowpal wabbit (VW). CoRR, abs/1108.3072, 2011.
19. N. Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn., 2(4):285-318, Apr. 1988.
20. N. Littlestone and M. K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212-261, Feb. 1994.
21. R. Meir. Nonparametric time series prediction through adaptive model selection. Machine Learning, 39(1):5-34, Apr. 2000.
22. H. Miller, P. Haller, and M. Odersky. Tools and frameworks for big learning in scala: Leveraging the language for high productivity and performance. In NIPS 2011 Workshop on Parallel and Large-Scale Machine Learning (BigLearn), 2011.
23. M. Mohri and A. Rostamizadeh. Stability bounds for stationary φ-mixing and β-mixing processes. Journal of Machine Learning Research (JMLR), 11:798-814, 2010.
24. N. Reyhani and P. Bickel. Nonparametric ICA for nonstationary instantaneous mixtures. In ECML 2009 Workshop on Learning from non-IID data: Theory, Algorithms and Practice, September 2009.
25. S. Shalev-Shwartz, Y. Singer, and N. Srebro. PEGASOS: Primal Estimated sub-GrAdient SOlver for SVM. In Proceedings of the 24th International Conference on Machine Learning, ICML '07, pages 807-814, 2007.
26. D. Steinkrau, P. Y. Simard, and I. Buck. Using gpus for machine learning algorithms. In Proceedings of the Eighth International Conference on Document Analysis and Recognition, ICDAR '05, pages 1115-1119, 2005.
27. A. Subramanya and J. Bilmes. Large-scale graph-based transductive inference. In NIPS Workshop on Large-Scale Machine Learning: Parallelism and Massive Datasets, December 2009.
28. H. Wang, W. Fan, P. S. Yu, and J. Han. Mining concept-drifting data streams using ensemble classifiers. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '03, pages 226-235, 2003.
29. M. Weimer, T. Condie, and R. Ramakrishnan. Machine learning in ScalOps, a higher order cloud computing language. In NIPS 2011 Workshop on Parallel and Large-Scale Machine Learning (BigLearn), December 2011.
30. G. Widmer and M. Kubat. Learning in the presence of concept drift and hidden contexts. Machine Learning, 23(1):69-101, Apr. 1996.
31. M. Zinkevich. Theoretical analysis of a warm start technique. In NIPS 2011 Workshop on Parallel and Large-Scale Machine Learning (BigLearn), December 2011.