On optimizing machine learning workloads via kernel fusion

Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

Volumn 2015-January, Issue , 2015, Pages 173-182

  Ashari, Arash ^a   Tatikonda, Shirish ^b   Boehm, Matthias ^b   Reinwald, Berthold ^b   Campbell, Keith ^c   Keenleyside, John ^c   Sadayappan, P ^a  

^a The Ohio State University   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

^c IBM CANADA LTD   (Canada)

Author keywords

Dense; Fused kernel; GPU; Machine learning; Sparse

Indexed keywords

ALGEBRA; ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; LIBRARIES; MATRIX ALGEBRA; PARALLEL ARCHITECTURES; PARALLEL PROGRAMMING; PROGRAM PROCESSORS;

DENSE; END-TO-END PERFORMANCE; FUSED KERNEL; GPU; HIERARCHICAL AGGREGATION; MATRIX VECTOR MULTIPLICATION; SCALABLE MACHINE LEARNING; SPARSE;

ALGORITHMS;

EID: 84939166673     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2688500.2688521     Document Type: Conference Paper

References (35)

1. E. Agullo, J. Demmel, J. Dongarra, B. Hadri, J. Kurzak, J. Langou, H. Ltaief, P. Luszczek, and S. Tomov. Numerical Linear Algebra on Emerging Architectures: The PLASMA and MAGMA Projects. Journal of Physics: Conference Series, 180(1):012037, 2009.
2. P. Baldi, P. Sadowski, and D. Whiteson. Searching for Exotic Particles in High-Energy Physics with Deep Learning. Nature communications, 5, 2014.
3. N. Bell and M. Garland. Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, page 18. ACM, 2009.
4. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: a CPU and GPU Math Expression Compiler. In Proceedings of the Python for scientific computing conference (SciPy), Volume 4, page 3, 2010.
5. M. Boehm, S. Tatikonda, B. Reinwald, P. Sen, Y. Tian, D. Burdick, and S. Vaithyanathan. Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML. Proceedings of the VLDB Endowment, 7(7):553-564, 2014.
6. J. Canny and H. Zhao. Big Data Analytics with Small Footprint: Squaring the Cloud. In Proceedings of the 19th international conference on Knowledge discovery and data mining, pages 95-103, 2013.
7. J. Canny and H. Zhao. BIDMach: Large-Scale Learning with Zero Memory Allocation. In BigLearning, NIPS Workshop, 2013.
8. B. Catanzaro, N. Sundaram, and K. Keutzer. Fast Support Vector Machine Training and Classification on Graphics Processors. In Proceedings of the 25th international conference on Machine learning, pages 104-111. ACM, 2008.
9. O. Chapelle. Training a Support Vector Machine in the Primal. Neural Computation, 19(5):1155-1178, 2007.
10. A. Coates, B. Huval, T. Wang, D. Wu, B. Catanzaro, and N. Andrew. Deep Learning with COTS HPC Systems. In Proceedings of the 30th International Conference on Machine Learning, pages 1337-1345, 2013.
11. R. Collobert, K. Kavukcuoglu, and C. Farabet. Torch7: A Matlab-like Environment for Machine Learning. In BigLearning, NIPS Workshop, 2011.
12. cuBLAS. The NVIDIA CUDA Basic Linear Algebra Subroutines Library. URL https://developer.nvidia.com/cublas.
13. CUDA. A Parallel Computing Platform and Programming Model Invented by NVIDIA. URL http://www.nvidia.com/object/cuda-home-new.html.
14. cuDNN. The NVIDIA CUDA Library of Primitives for Deep Neural Networks. URL https://developer.nvidia.com/cuDNN.
15. cuSPARSE. The NVIDIA CUDA Sparse Matrix Library. URL https://developer.nvidia.com/cusparse.
16. R. Farivar, D. Rebolledo, E. Chan, and R. H. Campbell. A Parallel Implementation of K-Means Clustering on GPUs. In PDPTA, pages 340-345, 2008.
17. A. Ghoting, R. Krishnamurthy, E. Pednault, B. Reinwald, V. Sindhwani, S. Tatikonda, Y. Tian, and S. Vaithyanathan. SystemML: Declarative Machine Learning on MapReduce. In IEEE 27th International Conference on Data Engineering, pages 231-242. IEEE, 2011.
18. HiPLAR. High Performance Linear Algebra in R. URL http://hiplar.org.
19. C.-H. Ho and C.-J. Lin. Large-Scale Linear Support Vector Regression. The Journal of Machine Learning Research, 13(1):3323-3348, 2012.
20. Intel. Math Kernel Library. URL https://software.intel.com/en-us/intel-mkl.
21. Khronos OpenCL Working Group. The OpenCL Specification, version 1.0.29, December 2008.
22. D. B. Kirk and W. H. Wenmei. Programming Massively Parallel Processors: a Hands-on Approach. Newnes, 2012.
23. J. M. Kleinberg. Authoritative Sources in a Hyperlinked Environment. Journal of the ACM (JACM), 46(5):604-632, 1999.
24. C.-J. Lin, R. C. Weng, and S. S. Keerthi. Trust Region Newton Method for Logistic Regression. Journal of Machine Learning Research, 9: 627-650, 2008.
25. N. Lopes and B. Ribeiro. GPUMLib: An Efficient Open-Source GPU Machine Learning Library. International Journal of Computer Information Systems and Industrial Management Applications, 3:355-362, 2011.
26. N. Lopes, B. Ribeiro, and R. Quintas. GPUMLib: a New Library to Combine Machine Learning Algorithms with Graphics Processing Units. In Hybrid Intelligent Systems (HIS), 2010 10th International Conference on, pages 229-232. IEEE, 2010.
27. MAGMA. Matrix Algebra on GPU and Multicore Architectures. URL http://icl.cs.utk.edu/magma.
28. P. McCullagh. Generalized Linear Models. European Journal of Operational Research, 16(3):285-292, 1984.
29. J. Nickolls, I. Buck, M. Garland, and K. Skadron. Scalable Parallel Programming with CUDA. Queue, 6(2):40-53, 2008.
30. NVIDIA. CUDA GPU Occupancy Calculator. URL http://developer.download.nvidia.com/compute/cuda/CUDA-Occupancy-calculator.xls.
31. NVVP. NVIDIA Visual Profiler. URL https://developer.nvidia.com/nvidia-visual-profiler.
32. R. Raina, A. Madhavan, and A. Y. Ng. Large-Scale Deep Unsupervised Learning Using Graphics Processors. In International Conference on Machine Learning, Volume 9, pages 873-880, 2009.
33. T. Sharp. Implementing Decision Trees and Forests on a GPU. In Computer Vision-ECCV 2008, pages 595-608. Springer, 2008.
34. J. Stamper, A. Niculescu-Mizil, S. Ritter, G. Gordon, and K. Koedinger. Algebra I 2008-2009. Challenge Data Set from KDD Cup 2010 Educational Data Mining Challenge, 2013. URL http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp.
35. C. Zhang, A. Kumar, and C. Ré. Materialization Optimizations for Feature Selection Workloads. In SIGMOD, 2014.