Keystone ML: Optimizing pipelines for large-scale advanced analytics

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2017, Pages 535-546

  Sparks, Evan R ^a   Venkataraman, Shivaram ^a   Kaftan, Tomer ^a   Franklin, Michael J ^a   Recht, Benjamin ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

DISTRIBUTED ENVIRONMENTS; EXISTING SYSTEMS; LARGE-SCALE LEARNING; LARGE-SCALE MACHINE LEARNING; MACHINE LEARNING TECHNIQUES; REAL-WORLD DATASETS; RESOURCE REQUIREMENTS; STATISTICAL ACCURACY;

EDUCATION;

EID: 85021234082     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2017.109     Document Type: Conference Paper

References (60)

1. C. Manning and D. Klein, "Optimization, maxent models, and conditional estimation without magic, " in HLT-NAACL, Tutorial Vol 5, 2003.
2. P.-S. Huang, H. Avron, T. N. Sainath et al., "Kernel methods match deep neural networks on timit, " in ICASSP. IEEE, 2014, pp. 205-209.
3. J. Śanchez, F. Perronnin, T. Mensink, and J. Verbeek, "Image classification with the fisher vector: Theory and practice, " International journal of computer vision, vol. 105, no. 3, pp. 222-245, 2013.
4. D. Povey, A. Ghoshal, G. Boulianne, L. Burget et al., "The Kaldi speech recognition toolkit, " 2011.
5. G. Bradski and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library. O'Reilly Media, Inc., 2008.
6. J. Langford, L. Li, and A. Strehl, "Vowpal wabbit online learning project, " 2007.
7. X. Meng, J. K. Bradley, B. Yavuz, E. R. Sparks et al., "MLlib: Machine Learning in Apache Spark, " CoRR, vol. abs/1505.06807, 2015.
8. D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, and M. Young, "Machine learning: The high interest credit card of technical debt, " in SE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop), 2014.
9. A. Halevy, P. Norvig, and F. Pereira, "The unreasonable effectiveness of data, " Intelligent Systems, IEEE, vol. 24, no. 2, pp. 8-12, 2009.
10. J. M. Hellerstein, C. Ŕe, F. Schoppmann, D. Z. Wang et al., "The MADlib analytics library: or MAD skills, the SQL, " PVLDB, vol. 5, no. 12, pp. 1700-1711, 2012.
11. A. Ghoting, R. Krishnamurthy, E. Pednault, B. Reinwald et al., "SystemML: Declarative machine learning on MapReduce, " in ICDE, 2011.
12. F. Pedregosa, G. Varoquaux, A. Gramfort et al., "Scikit-learn: Machine learning in Python, " JMLR, vol. 12, pp. 2825-2830, 2011.
13. M. Abadi, A. Agarwal, P. Barham, E. Brevdo et al., "TensorFlow: Largescale machine learning on heterogeneous systems, " 2015, software available from tensorflow.org. [Online]. Available: http://tensorflow.org/.
14. C. Zhang and C. Ŕe, "Dimmwitted: A study of main-memory statistical analytics, " PVLDB, vol. 7, no. 12, pp. 1283-1294, 2014.
15. B. Recht, C. Re, S. Wright, and F. Niu, "Hogwild: A lock-free approach to parallelizing stochastic gradient descent, " in NIPS, 2011, pp. 693-701.
16. A. Crotty, A. Galakatos, and T. Kraska, "Tupleware: Distributed machine learning on small clusters, " IEEE Data Eng. Bull, vol. 37, no. 3, 2014.
17. T. Kraska, A. Talwalkar, J. C. Duchi, R. Griffith et al., "MLbase: A Distributed Machine-learning System, " CIDR, 2013.
18. K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman, "The devil is in the details: An evaluation of recent feature encoding methods, " in British Machine Vision Conference, 2011.
19. E. Jonas, V. Shankar, M. Bobra, and B. Recht, "Flare prediction using photospheric and coronal image data, " AGU Fall Meeting, 2016.
20. "ENCODE-DREAM in-vivo Transcription Factor Binding Site Prediction Challenge, " https://www.synapse.org/#Synapse:syn6131484, 2016.
21. X. Feng, A. Kumar, B. Recht, and C. Ŕe, "Towards a unified architecture for in-rdbms analytics, " in SIGMOD, 2012.
22. M. Stonebraker, J. Becla, D. J. DeWitt, K.-T. Lim, D. Maier, O. Ratzesberger, and S. B. Zdonik, "Requirements for science data bases and scidb." in CIDR, vol. 7, 2009, pp. 173-184.
23. S. Williams, A. Waterman, and D. Patterson, "Roofline: An Insightful Visual Performance Model for Multicore Architectures, " CACM, 2009.
24. G. M. Ballard, "Avoiding communication in dense linear algebra, " Ph.D. dissertation, University of California, Berkeley, 2013.
25. J. Demmel, L. Grigori, M. Hoemmen, and J. Langou, "Communicationoptimal parallel and sequential QR and LU factorizations, " SIAM Journal on Scientific Computing, vol. 34, no. 1, pp. A206-A239, 2012.
26. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and distributed computation: numerical methods. Prentice-Hall, Inc., 1989.
27. W. Chen, Z. Wang, and J. Zhou, "Large-scale l-bfgs using mapreduce, " in NIPS, 2014, pp. 1332-1340.
28. N. Halko, P. G. Martinsson, and J. A. Tropp, "Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions, " SIAM Review, 2011.
29. F. Abuzaid, S. Hadjis, C. Zhang, and C. Ŕe, "Caffe con Troll: Shallow Ideas to Speed Up Deep Learning." CoRR abs/1504.04343, 2015.
30. M. Mathieu, M. Henaff, and Y. LeCun, "Fast Training of Convolutional Networks through FFTs, " ICLR, 2014.
31. R. Chirkova and J. Yang, "Materialized Views, " Foundations and Trends in Databases, 2012.
32. D. C. Zilio, C. Zuzarte, G. M. Lohman, H. Pirahesh et al., "Recommending Materialized Views and Indexes with IBM DB2 Design Advisor, " in ICAC 2004, May 2004.
33. C. Zhang, A. Kumar, and C. Ŕe, "Materialization optimizations for feature selection workloads, " in SIGMOD, 2014.
34. L. A. Belady, "A study of replacement algorithms for a virtual-storage computer, " IBM Systems journal, vol. 5, no. 2, pp. 78-101, 1966.
35. P. Raghavan and M. Snir, "Memory versus randomization in on-line algorithms, " in ICALP. Springer, 1989, pp. 687-703.
36. A. Agarwal, O. Chapelle, and J. Langford, "A reliable effective terascale linear learning system." Journal of Machine Learning Research, vol. 15, no. 1, pp. 1111-1133, 2014.
37. C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel et al., "The Stanford CoreNLP natural language processing toolkit, " in ACL, 2014.
38. J. McAuley, R. Pandey, and J. Leskovec, "Inferring networks of substitutable and complementary products, " in KDD, 2015, pp. 785-794.
39. C.-W. Hsu, C.-C. Chang, C.-J. Lin et al., "A practical guide to support vector classification, " https://goo.gl/m68USr, 2003.
40. A. Rahimi and B. Recht, "Random features for large-scale kernel machines, " in NIPS, 2007, pp. 1177-1184.
41. V. Sindhwani and H. Avron, "High-performance kernel machines with implicit distributed optimization and randomization, " CoRR, vol. abs/1409.0940, 2014.
42. D. G. Lowe, "Object recognition from local scale-invariant features, " in ICCV, vol. 2. IEEE, 1999, pp. 1150-1157.
43. A. Krizhevsky and G. Hinton, "Convolutional Deep Belief Networks on CIFAR-10, " Unpublished manuscript, 2010.
44. J. Sanchez, F. Perronnin, and T. Mensink, "Improved fisher vector for large scale image classification, " http://image-net.org/challenges/LSVRC/2010/ILSVRC2010XRCE.pdf.
45. "TensorFlow CIFAR-10 Performance as reported in TensorFlow Source Code, " https://git.io/v2b4J.
46. J. Chen, R. Monga, S. Bengio, and R. Jozefowicz, "Revisiting distributed synchronous sgd, " arXiv preprint arxiv:1604.00981, 2016.
47. A. Coates and A. Y. Ng, "Learning Feature Representations with KMeans, " in Neural Networks: Tricks of the Trade, 2012.
48. C. Szegedy, V. Vanhoucke et al., "Rethinking the inception architecture for computer vision, " arXiv preprint arXiv:1512.00567, 2015.
49. S. Abu-El-Haija, N. Kothari et al., "YouTube-8M: A Large-Scale Video Classification Benchmark, " arXiv preprint arXiv: 1609.08675, 2016.
50. M. R. Berthold, N. Cebron, F. Dill, T. R. Gabriel et al., "KNIME: The Konstanz information miner, " in Data analysis, machine learning and applications. Springer, 2008, pp. 319-326.
51. Z. Cai, Z. J. Gao, S. Luo, L. L. Perez et al., "A comparison of platforms for implementing and running very large scale machine learning algorithms, " in SIGMOD 2014, 2014, pp. 1371-1382.
52. Y. Low, D. Bickson, J. Gonzalez, C. Guestrin et al., "Distributed graphlab: A framework for machine learning and data mining in the cloud, " PVLDB, vol. 5, no. 8, pp. 716-727, 2012.
53. Y. Zhang, H. Herodotou, and J. Yang, "RIOT: I/O-Efficient Numerical Computing without SQL, " in CIDR, 2009.
54. M. Li, D. G. Andersen, J. W. Park, A. J. Smola et al., "Scaling Distributed Machine Learning with the Parameter Server." OSDI, 2014.
55. C. Qin and F. Rusu, "Scalable I/O-bound Parallel Incremental Gradient Descent for Big Data Analytics in GLADE, " in DanaC, 2013.
56. X. Meng, J. Bradley, E. Sparks, and S. Venkataraman, "ML Pipelines: A New High-Level API for MLlib, " https://goo.gl/pluhq0 2015.
57. S. Chaudhuri and V. R. Narasayya, "AutoAdmin 'What-if' Index Analysis Utility." SIGMOD, 1998.
58. I. Elghandour and A. Aboulnaga, "ReStore: Reusing results of MapReduce jobs, " in PVLDB, 2012.
59. L. Perez and C. Jermaine, "History-Aware query optimization with materialized intermediate views, " in ICDE, March 2014, pp. 520-531.
60. E. R. Sparks, A. Talwalkar, D. Haas, M. J. Franklin et al., "Automating model search for large scale machine learning, " in SoCC '15, 2015.