Slabpose columnsort: A new oblivious algorithm for out-of-core sorting on distributed-memory clusters

Algorithmica (New York)

Volumn 45, Issue 3, 2006, Pages 483-508

  Chaudhry, Geeta ^a   Cormen, Thomas H ^a  

^a Dartmouth Department of Computer Science_Dartmouth College   (United States)

Author keywords

Columnsort; Distributed memory cluster; Oblivious algorithms; Out of core; Parallel sorting

Indexed keywords

COMMUNICATION SYSTEMS; COMPUTATION THEORY; COSTS; DATA STORAGE EQUIPMENT; INPUT OUTPUT PROGRAMS; MULTIPROCESSING SYSTEMS;

COLUMNSORT; DISTRIBUTED-MEMORY CLUSTERS; OBLIVIOUS ALGORITHMS; OUT-OF-CORE; PARALLEL SORTING;

ALGORITHMS;

EID: 33745025294     PISSN: 01784617     EISSN: 14320541     Source Type: Journal    
DOI: 10.1007/s00453-006-1222-1     Document Type: Article

References (40)

1. Alok Aggarwal and C. Greg Plaxton. Optimal parallel sorting in multi-level storage. In Proceedings of the 5th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 659-668, January 1994.
2. Alok Aggarwal and Jeffrey Scott Vitter. The input/output complexity of sorting and related problems. Communications of the ACM, 31(9):1116-1127, September 1988.
3. Thomas E. Anderson, David E. Culler, David E. Patterson, and the NOW team. A case for NOW (Networks of Workstations). IEEE Micro, 15(1):54-64, February 1995.
4. Lars Arge. External-memory algorithms with applications in geographic information systems. In M. van Kreveld, J. Nievergelt, T. Roos, and P. Widmayer, editors, Algorithmic Foundations of GIS, pages 213-254. Volume 1340 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 1997.
5. Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, David E. Culler, Joseph M. Hellerstein, and David A. Patterson. High-performance sorting on networks of workstations. In Proceedings of SIGMOD '97, May 1997.
6. David A. Bader, David R. Helman, and Joseph Jájá. Practical parallel algorithms for personalized communication and integer sorting. Journal of Experimental Algorithmics, 1(3):1-42, 1996.
7. Rakesh D. Barve, Edward F. Grove, and Jeffrey Scott Vitter. Simple randomized mergesort on parallel disks. Parallel Computing, 23(4):601-631, June 1997.
8. Rakesh D. Barve and Jeffrey Scott Vitter. A simple and efficient parallel disk mergesort. Theory of Computing Systems, 35(2):189-215, April 2002.
9. Geeta Chaudhry and Thomas H. Cormen. Getting more from out-of-core columnsort. In Proceedings of the 4th Workshop on Algorithm Engineering and Experiments (ALENEX 02), pages 143-154, January 2002.
10. Geeta Chaudhry and Thomas H. Cormen. Oblivious vs. distribution-based sorting: an experimental evaluation. In 13th Annual European Symposium on Algorithms (ESA 2005), volume 3669 of Lecture Notes in Computer Science, pages 317-328. Springer-Verlag, October 2005.
11. Geeta Chaudhry. Parallel Out-of-Core Sorting: the Third Way. PhD thesis, Dartmouth College, 2004.
12. Geeta Chaudhry, Thomas H. Cormen, and Leonard F. Wisniewski. Columnsort lives! An efficient out-of-core sorting program. In Proceedings of the Thirteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 169-178, July 2001.
13. Geeta Chaudhry, Elizabeth A. Hamon, and Thomas H. Cormen. Relaxing the problem-size bound for out-of-core columnsort. In Proceedings of the Fifteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 250-251, June 2003.
14. Thomas H. Cormen and Melissa Hirschl. Early experiences in evaluating the Parallel Disk Model with the ViC* implementation. Parallel Computing, 23(4-5):571-600, June 1997.
15. Roman Dementiev and Peter Sanders. Asynchronous parallel disk sorting. In Proceedings of the Fifteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 138-148, June 2003. Also Max-Planck-Institut für Informatik Technical Report MPI-I-2003-1-001.
16. David J. DeWitt, Jeffrey F. Naughton, and Donovan A. Schneider. Parallel sorting on a shared-nothing architecture using probabilistic splitting. In Proceedings of the First International Conference on Parallel and Distributed Information Systems, pages 280-291. IEEE Computer Society Press, Los Alamitos, CA, 1991.
17. Goetz Graefe. Parallel external sorting in Volcano. Technical Report CU-CS-459-90, Department of Computer Science, University of Colorado at Boulder, March 1990.
18. Jim Gray. Sort benchmark home page. http://research.microsoft.com/barc/ SortBenchmark/.
19. William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, William Saphir, and Marc Snir. MPI-The Complete Reference, Volume 2, The MPI Extensions. The MIT Press, Cambridge, MA, 1998.
20. David R. Helman, David A. Bader, and Joseph JáJá. A randomized parallel sorting algorithm with an experimental study. Journal of Parallel and Distributed Computing, 52(1):1-23, July 1998.
21. David R. Helman and Joseph Jájá. Sorting on clusters of SMPs. Technical Report CS-TR-3833, Department of Computer Science, University of Maryland, College Park, 1997.
22. David R. Helman, Joseph JáJá, and David A. Bader. A new deterministic parallel sorting algorithm with an experimental evaluation. Journal of Experimental Algorithmics, 3(4):1-24, 1998.
23. David A. Hutchinson, Peter Sanders, and Jeffrey Scott Vitter. Duality between prefetching and queued writing with applications to external sorting. In Proceedings of the European Symposium on Algorithms, pages 62-73. Volume 2161 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, August 2001.
24. David A. Hutchinson, Peter Sanders, and Jeffrey Scott Vitter. The power of duality for prefetching and sorting with parallel disks. In Proceedings of the Thirteenth Annual ACM Symposium on Parallel Algorithms and Architectures, pages 334-335, July 2001.
25. Donald E. Knuth. The Art of Computer Programming, Volume 3: Sorting and Searching, Addison-Wesley, Reading, MA, 1998.
26. F. Thomson Leighton. Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes. Morgan Kaufmann, San Mateo, CA, 1992.
27. Tom Leighton. Tight bounds on the complexity of parallel sorting. IEEE Transactions on Computers, C-34(4):344-354, April 1985.
28. John M. Marberg and Eli Gafni. Soiting in constant number of row and column phases on a mesh. Algorithmica, 3:561-572, 1988.
29. Mark H. Nodine and Jeffrey Scott Vitter. Large-scale sorting in parallel memories. In Proceedings of the 3rd Annual ACM Symposium on Parallel Algorithms and Architectures, pages 29-39, July 1991.
30. Mark H. Nodine and Jeffrey Scott Vitter. Deterministic distribution sort in shared and distributed memory multiprocessors. In Proceedings of the 5th Annual ACM Symposium on Parallel Algorithms and Architectures, pages 120-129, June 1993.
31. Mark H. Nodine and Jeffrey Scott Vitter. Greed sort: optimal deterministic sorting on parallel disks. Journal of the ACM, 42(4):919-933, July 1995.
32. Markus Pawlowski and Rudolf Bayer. Parallel sorting of large data volumes on distributed memory multiprocessors. In Arndt Bode and Mario Dal Cin, editors, Parallel Computer Architectures: Theory, Hardware, Software, Applications, pages 246-264. Springer-Verlag, New York, 1993.
33. Matthew D. Pearson. Fast out-of-core sorting on parallel disk systems. Technical Report PCS-TR99-351, Department of Computer Science, Dartmouth College, 1999.
34. Sanguthevar Rajasekaran. A framework for simple sorting algorithms on parallel disk systems. Theory of Computing Systems, 34(2):101-114, 2001.
35. Sanguthevar Rajasekaran and Xiaoming Jin. A practical realization of parallel disks. In Proceedings of the International Workshop on Parallel Processing, pages 337-344, August 2000.
36. Marc Snir, Steve W. Otto, Steven Huss-Lederman, David W. Walker, and Jack Dongarra. MPI-The Complete Reference, Volume 1, The MPI Core. The MIT Press, Cambridge, MA, 1998.
37. Jeffrey Scott Vitter. Efficient memory access in large-scale computation. In Proceedings of the 1991 Symposium on Theoretical Aspects of Computer Science (STACS '91), pages 26-41. Volume 480 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 1991.
38. Jeffrey Scott Vitter. External memory algorithms and data structures: dealing with MASSIVE DATA. ACM Computing Surveys, 33(2):209-271, June 2001.
39. Jeffrey Scott Vitter and David A. Hutchinson. Distribution sort with randomized cycling. In Proceedings of the 12th Annual SIAM/ACM Symposium on Discrete Algorithms, pages 77-86, January 2001.
40. Jeffrey Scott Vitter and Elizabeth A. M. Shriver. Algorithms for parallel memory, I: Two-level memories. Algorithmica, 12(2/3):110-147, August and September 1994.