Performance analysis and optimization for SpMV on GPU using probabilistic modeling

IEEE Transactions on Parallel and Distributed Systems

Volumn 26, Issue 1, 2015, Pages 196-205

  Li, Kenli ^a,b   Yang, Wangdong ^a,b   Li, Keqin ^a,b,c  

^a HUNAN UNIVERSITY   (China)

^b National Supercomputing Center in Changsha   (China)

^c STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

GPU; performance modeling; probability mass function; sparse matrix vector multiplication

Indexed keywords

COMPUTER HARDWARE; FUNCTIONS; GRAPHICS PROCESSING UNIT;

COMPRESSION EFFICIENCY; DISTRIBUTION PATTERNS; PERFORMANCE ANALYSIS AND OPTIMIZATIONS; PERFORMANCE MODEL; PERFORMANCE MODELING AND ANALYSIS; PROBABILISTIC MODELING; PROBABILITY MASS FUNCTION; SPARSE MATRIX-VECTOR MULTIPLICATION;

MATRIX ALGEBRA;

EID: 84919470072     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2014.2308221     Document Type: Article

References (36)

1. N. Bell and M. Garland, "Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors," Proc. Conf. High Performance Computing Networking, Storage and Analysis, pp. 1-11, 2009.
2. The NVIDIA CUDA Sparse Matrix Library (cuSPARSE), second ed., NVIDIA, http://docs.nvidia.com/cuda/cusparse/index.html, 2012.
3. W. Yang, K. Li, Y. Liu, L. Shi, and C. Wang, Optimization of Quasi Diagonal Matrix-Vector Multiplication on GPU, Int'l J. High Performance Computing Applications, first published on September 2, 2013, doi:10.1177/1094342013501126, http://hpc.sagepub.com/content/early/2013/09/02/1094342013501126.full.pdf
4. J. Bolz, I. Farmer, E. Grinspun, and P. Schroder, "Sparse Matrix Solvers on the GPU: Conjugate Gradients and Multigrid," ACM Trans. Graphics, vol. 22, no. 3, pp. 917-924, July 2003.
5. NVIDIA CUDA C Programming Guide, Version 5.0, May 2012.
6. F. Vazquez, G. Ortega, J.J. Fernandez, and E.M. Garzon, "Improving the Performance of the Sparse Matrix Vector Product with GPUs," Proc. IEEE 10th Int'l Conf. Computer and Information Technology (CIT '10), pp. 1146-1151, 2010.
7. D. Grewe and A. Lokhmotov, "Automatically Generating and Tuning GPU Code for Sparse Matrix-Vector Multiplication from a High-Level Representation," Proc. Fourth Workshop General Purpose Processing on Graphics Processing Units (GPGPU-4), article 12, 2011.
8. J.C. Pichel, F.F. Rivera, M. Fernandez, and A. Rodriguez, "Optimization of Sparse Matrix-Vector Multiplication Using Reordering Techniques on GPUs," Microprocessors and Microsystems, vol. 36, no. 2, pp. 65-77, 2012.
9. T. Oberhuber, A. Suzuki, and J. Vacata, "New Row-Grouped CSR Format for Storing the Sparse Matrices on GPU with Implementation in CUDA," arXiv preprint arXiv:1012.2270, 2010.
10. A.-J.N. Yzelman and D. Roose, "High-Level Strategies for Parallel Shared-Memory Sparse Matrix-Vector Multiplication," IEEE Trans. Parallel and Distributed Systems, vol. 25, no. 1, pp. 116-125, http://doi.ieeecomputersociety.org/10.1109/TPDS.2013.31, Jan. 2014.
11. J.W. Choi, A. Singh, and R.W. Vuduc, "Model-Driven Autotuning of Sparse Matrix-Vector Multiply on GPUs," Proc. 15th ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP '10), pp. 115-126, 2010.
12. V. Karakasis, T. Gkountouvas, K. Kourtis, G. Goumas, and N. Koziris, "An Extended Compression Format for the Optimization of Sparse Matrix-Vector Multiplication," IEEE Trans. Parallel and Distributed Systems, vol. 24, no. 10, pp. 1930-1940, Sept. 2013.
13. J. Kurzak, W. Alvaro, and J. Dongarra, "Optimizing Matrix Multiplication for a Short-Vector SIMD Architecture-Cell Processor," Parallel Computing, vol. 35, no. 3, pp. 138-150, 2009.
14. E.-J. Im, K. Yelick, and R. Vuduc, "Sparsity: Optimization Framework for Sparse Matrix Kernels," Int'l J. High Performance Computing Applications, vol. 18, no. 1, pp. 135-158, 2004.
15. M.M. Baskaran and R. Bordawekar, "Optimizing Sparse Matrix-Vector Multiplication on GPUs,"Technical Report RC24704 IBM TJ Watson Research Center, Dec. 2008. 204.
16. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 26, NO. 1, JANUARY 2015
17. J. Demmel, J. Dongarra, V. Eijkhout, E. Fuentes, A. Petitet, R. C.W.R. Vuduc, and K. Yelick, "Self-Adapting Linear Algebra Algorithms and Software," Proc. IEEE, vol. 93, no. 2, pp. 293-312, Feb. 2005.
18. A. Monakov, A. Lokhmotov, and A. Avetisyan, "Automatically Tuning Sparse Matrix-Vector Multiplication for GPU Architectures," Proc. Fifth Int'l Conf. High Performance Embedded Architectures and Compilers, pp. 111-125, 2010.
19. Z. Wang, X. Xu, W. Zhao, Y. Zhang, and S. He, "Optimizing Sparse Matrix-Vector Multiplication on CUDA," Proc. Second Int'l Conf. Education Technology and Computer (ICETC), vol. 4, pp. 54109-54113, June 2010.
20. X. Yang, S. Parthasarathy, and P. Sadayappan, "Fast Sparse Matrix-Vector Multiplication on GPUs: Implications for Graph Mining," Proc. VLDB Endowment, vol. 4, no. 4, pp. 231-242, Jan. 2011.
21. A.H.El Zein and A.P. Rendell, "Generating Optimal CUDA Sparse Matrix-Vector Product Implementations for Evolving GPU Hardware," Concurrency and Computation: Practice and Experience, vol. 24, no. 1, pp. 3-13, 2012.
22. Y. Kubota and D. Takahashi, "Optimization of Sparse Matrix-Vector Multiplication by Auto Selecting Storage Schemes on GPU," Proc. Int'l Conf. Computational Science and Its Applications (ICCSA '11), pp. 547-561, 2011.
23. S. Ryoo, C.I. Rodrigues, S.S. Stone, S.S. Baghsorkhi, S.-Z. Ueng, J. A. Stratton, and W.-M.W. Hwu, "Program Optimization Space Pruning for a Multithreaded GPU," Proc. Sixth Ann. IEEE/ACM Int'l Symp. Code Generation and Optimization (CGO '08), pp. 195-204, 2008.
24. Y. Zhang and J. Owens, "A Quantitative Performance Analysis Model for GPU Architectures," Proc. IEEE 17th Int'l Symp. High Performance Computer Architecture (HPCA '11), pp. 382-393, Feb. 2011.
25. H. Pabst, B. Bachmayer, and M. Klemm, "Performance of a Structure-Detecting SpMV Using the CSR Matrix Representation," Proc. IEEE 11th Int'l Symp. Parallel and Distributed Computing (ISPDC), pp. 3-10, 2012.
26. M.M. Dehnavi, D. Fernandez, and J.L. Gaudiot, "Parallel Sparse Approximate Inverse Preconditioning on Graphic Processing Units," IEEE Trans. Parallel and Distributed Systems, vol. 24, no. 9, pp. 1852-1862, Sept. 2013.
27. S.S. Baghsorkhi, M. Delahaye, S.J. Patel, W.D. Gropp, and W.-m. W. Hwu, "An Adaptive Performance Modeling Tool for GPU Architectures," Proc. 15th ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP '10), pp. 105-114, 2010.
28. S. Hong and H. Kim, "An Analytical Model for a GPU Architecture with Memory-Level and Thread-Level Parallelism Awareness," Proc. 36th Ann. Int'l Symp. Computer Architecture (ISCA '09), pp. 152-163, 2009.
29. K. Kothapalli, R. Mukherjee, M. Rehman, S. Patidar, P. Narayanan, and K. Srinathan, "A Performance Prediction Model for the CUDA GPGPU Platform," Proc. Int'l Conf. High Performance Computing (HiPC), pp. 463-472, Dec. 2009.
30. J. Li, G. Tan, M. Chen, and N. Sun, "SMAT: An Input Adaptive Auto-Tuner for Sparse Matrix-Vector Multiplication," Proc. 34th ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 117-126, 2013.
31. D. Schaa and D. Kaeli, "Exploring the Multiple-GPU Design Space," Proc. IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '09), pp. 1-12, May 2009.
32. P. Guo, L. Wang, and P. Chen, "A Performance Modeling and Optimization Analysis Tool for Sparse Matrix-Vector Multiplication on GPUs," IEEE Trans. Parallel and Distributed Systems, vol. 25, no. 5, pp. 1112-1123, 2014.
33. J.C. Pichel and F.F. Rivera, "Sparse Matrix Vector Multiplication on the Single-Chip Cloud Computer Many-Core Processor," J. Parallel and Distributed Computing, vol. 73, no. 12, pp. 1539-1550, 2013.
34. S.B. Indarapu, M. Maramreddy, and K. Kothapalli, "Architectureand Workload-Aware Heterogeneous Algorithms for Sparse Matrix Vector Multiplication," Proc. Int'l Conf. Parallel and Distributed Systems (ICPADS), Dec. 2013, http://cstar.iiit.ac.in/kkishore/spmv2.pdf.
35. N.L. Johnson, S. Kotz, and A. Kemp, Univariate Discrete Distributions, second ed., p. 36, John Wiley & Sons, 1993.
36. T.A. Davis and Y. Hu, University of Florida Sparse Matrix Collection, 2009.