High performance and scalable radix sorting: A case study of implementing dynamic parallelism for GPU computing

Parallel Processing Letters

Volumn 21, Issue 2, 2011, Pages 245-272

  Merrill, Duane ^a   Grimshaw, Andrew ^a  

^a University of Virginia   (United States)

Author keywords

GPU; kernel fusion; Parallel sorting; prefix scan; prefix sum; radix sorting

Indexed keywords

GPU; KERNEL FUSION; PARALLEL SORTING; PREFIX SCAN; PREFIX SUM; RADIX SORTING;

ALGORITHMS; COMPUTER ARCHITECTURE; PARALLEL PROCESSING SYSTEMS; PROGRAM PROCESSORS;

SORTING;

EID: 79959718248     PISSN: 01296264     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129626411000187     Document Type: Article

References (47)

1. J D Owens et al., "GPU Computing," Proceedings of the IEEE, vol. 96, no. 5, pp. 879-899, May 2008.
2. Victor W Lee et al., "Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU," in Proceedings of the 37th annual international symposium on Computer architecture, Saint-Malo, France, 2010, pp. 451-460.
3. Richard Vuduc, Aparna Chandramowlishwaran, Jee Choi, Murat Guney, and Aashay Shringarpure, "On the limits of GPU acceleration," in Proceedings of the 2nd USENIX conference on Hot topics in parallelism (HotPar'10), Berkeley, CA, 2010, pp. 13-13.
4. Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software. Reading, MA, USA: Addisson-Wesley, 1995.
5. Nadathur Satish, Mark Harris, and Michael Garland, "Designing efficient sorting algorithms for manycore GPUs," in IPDPS '09: Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, 2009, pp. 1-10.
6. GPGPU.org. [Online]. http://gpgpu.org/developer/cudpp
7. Nadathur Satish et al., "Fast sort on CPUs and GPUs: a case for bandwidth oblivious SIMD sort," , 2010, pp. 351-362.
8. Jatin Chhugani et al., "Efficient implementation of sorting on multi-core SIMD CPU architecture," Proc. VLDB Endow., pp. 1313-1324, 2008.
9. Nadathur Satish et al., "Fast Sort on CPUs, GPUs and Intel MIC Architectures," techreport 2010.
10. Donald Knuth, The Art of Computer Programming. Reading, MA, USA: Addison-Wesley, 1973, vol. III: Sorting and Searching.
11. Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein, Introduction to Algorithms, 2nd ed.: McGraw-Hill, 2001.
12. Guojing Cong, George Almasi, and Vijay Saraswat, "Fast PGAS Implementation of Distributed Graph Algorithms," in Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC'10), 2010, pp. 1-11.
13. Kun Zhou, Minmin Gong, Xin Huang, and Baining Guo, "Data-Parallel Octrees for Surface Reconstruction," EEE Transactions on Visualization & Computer Graphics, p. to appear, 2010.
14. Kun Zhou, Qiming Hou, Rui Wang, and Baining Guo, "Real-time KD-tree construction on graphics hardware," in SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers, Singapore, 2008, pp. 1-11.
15. J Pantaleoni and D Luebke, "HLBVH: hierarchical LBVH construction for real-time ray tracing of dynamic geometry," in Proceedings of the Conference on High Performance Graphics (HPG '10), Saarbrucken, Germany, 2010, pp. 87-95.
16. Erik Sintorn and Ulf Assarsson, "Real-time approximate sorting for self shadowing and transparency in hair rendering," in I3D '08: Proceedings of the 2008 symposium on Interactive 3D graphics and games, Redwood City, California, 2008, pp. 157-162.
17. Kun Zhou et al., "RenderAnts: interactive Reyes rendering on GPUs," in SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers, Yokohama, Japan, 2009, pp. 1-11.
18. Bernhard Kainz et al., "Ray casting of multiple volumetric datasets with polyhedral boundaries on manycore GPUs," in SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers, Yokohama, Japan, 2009, pp. 1-9.
19. Peter Kipfer, Mark Segal, and Rüdiger Westermann, "UberFlow: a GPU-based particle engine," in HWWS '04: Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, Grenoble, France, 2004, pp. 115-122.
20. Jonathan M Cohen, Sarah Tariq, and Simon Green, "Interactive fluid-particle simulation using translating Eulerian grids," in I3D '10: Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, Washington, D.C., 2010, pp. 15-22.
21. Charles Loop and Kirill Garanzha, "Fast Ray Sorting and Breadth-First Packet Traversal for GPU Ray Tracing," in Eurographics, 2010.
22. Ignacio Castaño. (2007, February) High Quality DXT Compression Using CUDA. [Online]. http://developer.download.nvidia.com/compute/cuda/sdk/ website/projects/dxtc/doc/cuda-dxtc. pdf
23. Dan A Alcantara et al., "Real-time parallel hashing on the GPU," in SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers, Yokohama, Japan, 2009, pp. 1-9.
24. Bingsheng He et al., "Relational joins on graphics processors," in SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, Vancouver, Canada, 2008, pp. 511-524.
25. Naga Govindaraju, Jim Gray, Ritesh Kumar, and Dinesh Manocha, "GPUTeraSort: high performance graphics co-processor sorting for large database management," in SIGMOD '06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, Chicago, IL, 2006, pp. 325-336. (Pubitemid 46950863)
26. Naga Govindaraju, Nikunj Raghuvanshi, and Dinesh Manocha, "Fast and approximate stream mining of quantiles and frequencies using graphics processors," in SIGMOD '05: Proceedings of the 2005 ACM SIGMOD international conference on Management of data, Baltimore, MD, 2005, pp. 611-622. (Pubitemid 43038962)
27. Jeremy Shopf, Joshua Barczak, Christopher Oat, and Natalya Tatarchuk, "March of the Froblins: simulation and rendering massive crowds of intelligent and detailed creatures on GPU," in SIGGRAPH '08: ACM SIGGRAPH 2008 classes, Los Angeles, CA, 2008, pp. 52- 101.
28. A Hartstein, V Srinivasan, T R Puzak, and P G Emma, "Cache miss behavior: is it q2?;' in CF '06: Proceedings of the 3rd conference on Computing frontiers, Ischia, Italy, 2006, pp. 313-320.
29. R P Brent and H T Kung, "A Regular Layout for Parallel Adders," IEEE Trans. Comput., vol. 31, no. 3, pp. 260-264, March 1982.
30. Siddhartha Chatterjee, Guy Blelloch, and Marco Zagha, "Scan primitives for vector computers," in Supercomputing '90: Proceedings of the 1990 ACM/IEEE conference on Supercomputing, New York, New York, 1990, pp. 666-675. (Pubitemid 21675225)
31. Andrea C Dusseau, David E Culler, Klaus E Schauser, and Richard P Martin, "Fast Parallel Sorting Under LogP: Experience with the CM-5," IEEE Trans. Parallel Distrib. Syst., vol. 7, pp. 791-805, 1996. (Pubitemid 126784485)
32. Marco Zagha and Guy Blelloch, "Radix sort for vector multiprocessors," in Supercomputing '91: Proceedings of the 1991 ACM/IEEE conference on Supercomputing, Albuquerque, NM, 1991, pp. 712-721.
33. Shubhabrata Sengupta, Mark Harris, Yao Zhang, and John D Owens, "Scan Primitives for GPU Computing," in GH '07: Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware, San Diego, CA, 2007, pp. 97-106.
34. Bingsheng He, Naga K Govindaraju, Qiong Luo, and Burton Smith, "Efficient gather and scatter operations on graphics processors," in SC '07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing, Reno, NV, 2007, pp. 1-12.
35. Shubhabrata Sengupta, Mark Harris, and Michael Garland, "Efficient Parallel Scan Algorithms for GPUs," NVIDIA, Technical Report NVR-2008-003, 2008.
36. Yuri Dotsenko, Naga K Govindaraju, Peter-Pike Sloan, Charles Boyd, and John Manferdelli, "Fast scan algorithms on graphics processors," in ICS '08: Proceedings of the 22nd annual international conference on Supercomputing, Island of Kos, Greece, 2008, pp. 205-213.
37. Duane Merrill and Andrew Grimshaw, "Parallel Scan for Stream Architectures," University of Virginia, Department of Computer Science, Charlottesville, VA, USA, Technical Report CS2009-14, 2009.
38. Guy E Blelloch, Siddhartha Chatterjee, and Marco Zagha, "Solving Linear Recurrences with Loop Raking," in Proceedings of the 6th International Parallel Processing Symposium, 1992, pp. 416-424. (Pubitemid 23612403)
39. Mark Harris. (2007) Optimizing parallel reduction in CUDA. [Online]. http://developer.download.nvidia.com/compute/cuda/1-1/Website/projects/ reduction/doc/reduc tion.pdf
40. Peter M Kogge and Harold S Stone, "A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equations," IEEE Trans. Comput., vol. 22, pp. 786-793, 1973.
41. Duane Merrill and Andrew Grimshaw, "Revisiting Sorting for GPGPU Stream Architectures," University of Virginia, Charlottesville, VA, Technical Report CS2010-03, 2010.
42. K Thearling and S Smith, "An improved supercomputer sorting benchmark," in Proceedings of the 1992 ACM/IEEE conference on Supercomputing (SC '92), Minneapolis, Minnesota, 1992, pp. 14-19.
43. Daniel Cederman and Philippas Tsigas, "GPU-Quicksort: A practical Quicksort algorithm for graphics processors," J. Exp. Algorithmics, vol. 14, pp. 1.4-1.24, 2009.
44. Nikolaj Leischner, Vitaly Osipov, and Peter Sanders, GPU sample sort, 2009.
45. Frank Dehne and Hamidreza Zaboli, Deterministic Sample Sort For GPUs, 2010.
46. Guy Blelloch, "Prefix Sums and Their Applications," School of Computer Science, Carnegie Mellon University, Technical Report CMU-CS-90-190, 1990.
47. Thrust. [Online]. http://code.google.com/p/thrust/.