Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs

IEEE Transactions on Computers

Volumn 64, Issue 9, 2015, Pages 2623-2636

  Yang, Wangdong ^a,b   Li, Kenli ^a,b   Mo, Zeyao ^c   Li, Keqin ^a,b,d  

^a HUNAN UNIVERSITY   (China)

^b National Supercomputing Center in Changsha   (China)

^c INSTITUTE OF APPLIED PHYSICS AND COMPUTATIONAL MATHEMATICS   (China)

^d STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

GPU; matrix partition; multicore CPU; probability distribution; sparse matrix vector multiplication

Indexed keywords

PROBABILITY; PROBABILITY DISTRIBUTIONS; PROGRAM PROCESSORS;

GPU; HETEROGENEOUS PROCESSORS; MATRIX PARTITIONS; MULTI-CORE CPUS; PARALLEL COMPUTING MODELS; PERFORMANCE OPTIMIZATIONS; PROBABILITY MASS FUNCTION; SPARSE MATRIX-VECTOR MULTIPLICATION;

MATRIX ALGEBRA;

EID: 84939230567     PISSN: 00189340     EISSN: None     Source Type: Journal    
DOI: 10.1109/TC.2014.2366731     Document Type: Article

References (36)

1. D. P. Scarpazza, O. Villa, and F. Petrini, "Efficient breadth-first search on the cell/BE processor," IEEE Trans. Parallel Distrib. Syst., vol. 19, no. 10, pp. 1381-1395, Oct. 10, 2008.
2. Y. Shan, W. Tianji, Y. Wang, B. Wang, Z. Wang, N. Xu, and H. Yang, "FPGA and GPU implementation of large scale SpMV," in Proc. 8th Symp. Appl. Sp., 2010, pp. 64-70.
3. V. W. Lee, C. Kim, J. Chhugani, M. Deisher, D. Kim, A. D. Nguyen, N. Satish, M. Smelyanskiy, S. Chennupaty, P. Hammarlund, R. Singhal, and P. Dubey, "Debunking the 100X GPU versus CPU myth: An evaluation of throughput computing on CPU and GPU," in Proc. Int. Symp. Comput. Arch., vol. 38, no. 3, 2010, pp. 451-460.
4. A. Pedram, R. A. van de Geijn, and A. Gerstlauer, "Codesign tradeoffs for high-performance, low-power linear algebra architectures," IEEE Trans. Comput., vol. 61, no. 12, pp. 1724-1736, Oct. 2012.
5. NVIDIA. (2012). The NVIDIA CUDA Sparse Matrix library (cuSPARSE), 2nd ed. [Online]. Available: http://docs.nvidia.com/cuda/cusparse/index.html
6. J. Gustafson and B. Greer, "Clearspeed whitepaper: Accelerating the intel math kernel library," Tech. Rep., 2007. [Online]. Available: http://www.clearspeed.com/docs/resources/ClearSpeedIntelWhitepaperFeb07.pdf
7. W. Yang, K. Li, Y. Liu, L. Shi, and L. Wan, "Optimization of Quasi diagonal matrix vector multiplication on GPU," Int. J. High Performance Comput. Appl., vol. 28, no. 2, pp. 183-195, 2014.
8. J. Bolz, I. Farmer, E. Grinspun, and P. Schroder, "Sparse matrix solvers on the GPU: Conjugate gradients and multigrid," ACM Trans. Graph., vol. 22, no. 3, pp. 917-924, 2003.
9. NVIDIA CUDA C Programming Guide, Version 5.0, May 2012.
10. Z. Fu, T. J. Lewis, R. M. Kirby, and R. T. Whitaker, "Architecting the finite element method pipeline for the GPU," J. Comput. Appl. Math., vol. 257, pp. 195-211, Feb. 2014.
11. N. Bell and M. Garland, "Implementing sparse matrix-vector multiplication on throughput-oriented processors," in Proc. Conf. High Performance Comput. Netw., Storage Anal., 2009, p. 18.
12. W. T. Tang, W. J. Tan, R. Ray, Y. W. Wong, W. Chan, S. H. Kuo, R. S. M. Goh, S. J. Turner, and W. F. Wong, "Accelerating sparse matrix-vector multiplication on GPUs using bit-representation-optimized schemes," in Proc. Int. Conf. High Performance Comput., Netw., Storage Anal., 2013.
13. F. Vazquez, G. Ortega, J. J. Fernandez, and E. M. Garzon, "Improving the performance of the sparse matrix vector product with GPUs," in Proc. 10th IEEE Int. Conf. Comput. Inform. Technol., ser. CIT, 2010, pp. 1146-1151.
14. S. Williams, J. Carter, L. Oliker, J. Shalf, and K. Yelick, "Optimization of a lattice boltzmann computation on state-of-the-art multicore platforms," J. Parallel Distrib. Comput., vol. 69, no. 9, pp. 762-777, 2009.
15. B. Boyer, J. G. Dumas, and P. Giorgi, "Exact sparse matrix vector multiplication on GPU's and multicore architectures," in Proc. 4th Int. Workshop Parallel Symbolic Comput., Jul. 2010, pp. 80-88.
16. A. Buluc, S. Williams, L. Oliker, and J. Demmel, "Reduced-bandwidth multithreaded algorithms for sparse matrix-vector multiplication," in Proc. IEEE Int. Parallel Distrib. Process. Symp., pp. 721-733, May 2009.
17. S. Sun, M. Monga, P. H. Jones, and J. Zambreno, "An I/O bandwidth-sensitive sparse matrix vector multiplication engine on FPGAs," IEEE Trans. Circuits Syst. I: Reg. Papers, vol. 59, no. 1, pp. 113-123, Jan. 2012.
18. J. C. Pichel and F. F. Rivera, "Sparse matrix-vector multiplication on the single-chip cloud computer many-core processor," J. Parallel Distrib. Comput., vol. 73, no. 12, pp. 1539-1550, 2013.
19. L. Buatois, G. Caumon, and B. Levy, "Concurrent number cruncher: A GPU implementation of a general sparse linear solver," Int. J. Parallel Emerg. Distrib. Syst., vol. 24, no. 3, pp. 205-223, 2009.
20. T. Oberhuber, A. Suzuki, and J. Vacata, "New row-grouped CSR format for storing the sparse matrices on GPU with implementation in CUDA," arXiv preprint arXiv:1012.2270, 2010.
21. M. Belgin, G. Back, and C. J. Ribbens, "Pattern-based sparse matrix representation for memory-efficient SMVM kernels," in Proc. 23rd Int. Conf. Supercomput., Jun. 2009, pp. 100-109.
22. S. Williams, L. Oliker, R. Vuduc, J. Shalf, K. Yelick, and J. Demmel, "Optimization of sparse matrix vector multiplication on emerging multicore platforms," Parallel Comput., vol. 35, no. 3, pp. 178-194, 2009.
23. A. Monakov, A. Lokhmotov, and A. Avetisyan, "Automatically tuning sparse matrix vector multiplication for GPU architectures," High Performance Embedded Architectures and Compilers. Berlin, Germany: Springer, 2010, pp. 111-125.
24. J. W. Choi, A. Singh, and R. W. Vuduc, "Model-driven autotuning of sparse matrix vector multiply on GPUs," in Proc. 15th ACM SIGPLAN Symp. Principles Practice Parallel Programming, 2010, pp. 115-126.
25. X. Feng, H. Jin, R. Zheng, K. Hu, J. Zeng, and Z. Shao, "Optimization of sparse matrix vector multiplication with variant CSR on GPUs," in Proc. IEEE 17th Int. Conf. Parallel Distrib. Syst., 2011, pp. 165-172.
26. M. Kreutzer, G. Hager, G. Wellein, H. Fehske, A. Basermann, and A. R. Bishop, "Sparse matrix vector multiplication on GPGPU clusters: A new storage format and a scalable implementation," in Proc. IEEE 26th Int. Parallel Distrib. Process. Symp. Workshops PhD Forum, May 2012, pp. 1696-1702.
27. A. N. Yzelman and R. H. Bisseling, "Two-dimensional cacheoblivious sparse matrix vector multiplication," Parallel Comput., vol. 37, no. 12, pp. 806-819, 2011.
28. V. Karakasis, T. Gkountouvas, K. Kourtis, G. Goumas, and N. Koziris, "An extended compression format for the optimization of sparse matrix vector multiplication," IEEE Trans. Parallel Distrib. Syst. Sep. vol. 24, no. 10, pp. 1930-1940, Oct. 2013.
29. P. Chen, "A performance modeling and optimization analysis tool for sparse matrix vector multiplication on GPUs," IEEE Trans. Parallel Distrib. Syst., vol. 25, no. 5, pp. 1112-1123, May 2014.
30. B. Schmidt, H. Aribowo, and H.-V. Dang, "Iterative sparse matrix vector multiplication for accelerating the block Wiedemann algorithm over GF(2) on multi-graphics processing unit systems," Concurrency Comput.: Practice Experience, vol. 25, no. 4, pp. 586-603, 2013.
31. M. Cenk, C. Negre, and M. A. Hasan, "Improved three-way split formulas for binary polynomial and Toeplitz matrix vector products," IEEE Trans. Comput., vol. 62, no. 7, pp. 1345-1361, Jul. 2013.
32. M. A. Hasan and C. Negre, "Multiway splitting method for toeplitz matrix vector product," IEEE Trans. Comput., vol. 62, no. 7, pp. 1467-1471, May 2013.
33. A.-J. N. Yzelman and D. Roose, "High-level strategies for parallel shared-memory sparse matrix vector multiplication," IEEE Trans. Parallel Distrib. Syst., vol. 25, no. 1, pp. 116-125, Jan. 2014.
34. K. Li, W. Yang, and K. Li, "Performance analysis and optimization for SpMV on GPU using probabilistic modeling," IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 1, pp. 196-205, Jan. 2015.
35. N. L. Johnson, S. Kotz, and A. Kemp, Univariate Discrete Distributions. 2nd Ed., New York, NY, USA: Wiley, ISBN 0-471-54897-9, 1993 p. 36.
36. T. A. Davis and Y. Hu, University of Florida sparse matrix collection[J], 2009.