Reducing fragmentation on torus-connected supercomputers

Proceedings - 25th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011

Volumn , Issue , 2011, Pages 828-839

  Tang, Wei ^a   Lan, Zhiling ^a   Desai, Narayan ^b   Buettner, Daniel ^b   Yu, Yongen ^a  

^a Illinois Institute of Technology   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALLOCATION DECISION; ARGONNE NATIONAL LABORATORY; BLUE GENE; EVENT-DRIVEN SIMULATIONS; JOB ALLOCATION; NETWORK COSTS; PETASCALE;

DISTRIBUTED PARAMETER NETWORKS; PARALLEL PROCESSING SYSTEMS; SUPERCOMPUTERS;

NETWORK ARCHITECTURE;

EID: 80053269654     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.82     Document Type: Conference Paper

References (33)

1. Y. Aridor, T. Domany, O. Goldshmidt, J. E. Moreira, and E. Shmueli, "Resource allocation and utilization in the Blue Gene/L supercomputer," IBM Journal of Research and Development, 2005.
2. Blue Gene Team, "Overview of the IBM Blue Gene/P project,"IBM Journal of Research and Development, 2008.
3. Cobalt resoure manager. http://trac.mcs.anl.gov/projects/cobalt
4. Cray XT. http://www.cray.com/Products/XT.aspx
5. W. Cirne and F. Berman, "A comprehensive model of the super-computer workload," in Proc. of IEEE International Workshop on Workload Characterization, 2001.
6. M.-S. Chen and K. Shin, "Subcube allocation and task migration in hypercube multiprocessors," IEEE Transactions on Computers, 1990.
7. DOE INCITE program. http://www.er.doe.gov/ascr/incite
8. N. Desai, D. Buntinas, D. Buettner, P. Balaji, and A. Chan, "Improving resource availability by relaxing network allocation constraints on the Blue Gene/P," In Proceedings of ICPP'09, 2009.
9. N. Desai, R. Bradshaw, C. Lueninghoener, A. Cheery, S. Coghlan, and W. Scullin, "Petascale system management experience,"in Proceedings of USENIX Large Installation System Administration Conference (LISA'08), 2008.
10. O. Diessel and H. ElGindy, "Ordered partial task compaction on mesh connected computers," Technical report TR96-11, Department of Computer Science and Software Engineering, The University of Newcastle, 1996.
11. Y. Etsion and D. Tsafrir, "A short survey of commercial cluster batch schedulers," Technical report 2005-13, the Hebrew University of Jerusalem, 2005.
12. A. Gara, M.A. Blumrich, D. Chen, G. L.-T. Chiu, et al, "Overview of the Blue Gene/L system architecture," IBM Journal of Research and Development, 2005.
13. L. K. Goh and B. Veeravalli, "Design and performance evaluation of combined first-fit task allocation and migration strategies in mesh multiprocessor systems," Parallel Computing, 34(9), 508-520, 2008.
14. J. Jacob and S.-Y. Lee, "Task spreading and shrinking on multiprocessor systems and networks of workstations," IEEE Transaction on Parallel and Distributed Systems, 10(10), 1082-1101, 1999. (Pubitemid 30529923)
15. C. Lee, Y. Schwartzman, J. Hardy, and A. Snavely, "Are user runtime estimates inherently inaccurate?" in Proc. of Job Scheduling Strategies for Parallel Processing (JSSPP), 2004.
16. K. Li and K.-H. Cheng, "A two-dimentional buddy system for dynamic resource allocation in partitionable mesh connected system," Journal of Parallel and Distributed Computing (JPDC), 12(1), 79-83, 1991.
17. A. Mu'alem and D. Feitelson, "Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling," IEEE Transactions on Parallel and Distributed Systems 12(6), 529-543, 2001. (Pubitemid 32611060)
18. Parallel workloads archive (maintained by D. Feitelson). http://www.cs.hji.ac.il/labs/parallel/workload
19. W. Smith, "Prediction services for distributed computing," in Proc. of IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS), 2007.
20. D. D. Sharma and D. K. Pradhan, "A fast and efficient strategy for submesh allocation in mesh-connected parallel computers,"in IEEE Symp. on Parallel and Ditributed Processing, 1993.
21. E. Shmueli and D. Feitelson, "Backfilling with lookahead to optimize the packing of parallel jobs," in Journal of Parallel and Distributed Systems, 2005.
22. W. Smith, V. Taylor, and I. Fostor, "Using run-time predictions to estimate queue wait times and improve scheduler performance," in Proc. of Job Scheduling Strategies for Parallel Processing (JSSPP), 1999.
23. W. Tang, N. Desai, D. Buettner, and Z. Lan, "Analyzing and adjusting user runtime estimates to improve job scheduling on the Blue Gene/P," in Proc. of IEEE Int'l Parallel & Distributed Processing Symp. (IPDPS), 2010.
24. W. Tang, Z. Lan, N. Desai, and D. Buettner, "Fault-aware, utility-based job scheduling on Blue Gene/P systems," in Proc. of IEEE Int'l Conf. on Cluster Computing (Cluster), 2009.
25. D. Tsafrir, Y. Etsion, and D. Feitelson, "Backfilling using system-generated predictions rather than user runtime estimates,"IEEE Transactions on Parallel and Distributed Systems 18(6), 789-803, 2007. (Pubitemid 46767742)
26. TOP500 Supercomputing web site, http://www.top500.org
27. N.-C. Wang and T.-S. Chen. Task migration in all-port wormhole-routed 2D mesh multicomputers. In Proc. 7th Int'l Symp. Parallel Architectures, Algorithms and Networks, 2004.
28. F. Wu, C.-C. Hsu, and L.-P. Chou, "Processor allocation in the mesh multiprocessors using the leapfrog method," IEEE Transactions on Parallel and Distributed Systems, 14(3), 276-289, 2003.
29. W. Ward, C. Mahood, and J. West, "Scheduling jobs on parallel systems using a relaxed backfill strategy," in Proc. of Job Scheduling Strategies for Parallel Processing, 2002.
30. B. S. Yoo and C. R. Das, "A fast and efficient processor allocation scheme for mesh-connected multicomputers," IEEE Transactions on Computers, 60(5), 616-638, 2002.
31. S.-M. Yoo, H. Choo, H.-Y. Young, C. Yu, and Y. Lee, "On task relocation in two-dimensional meshes," Journal of Parallel and Distributed Computing, 2000.
32. Y. Zhang, H. Franke, J. Moreira, and A. Sivasubramaniam, "Improving parallel job scheduling by combining gang scheduling and backfilling techniques," in Proc. of IEEE International Parallel and Distributed Processing Symposium, 2000.
33. Y. Zhu, "Efficient processor allocation strategies for meshconnected parallel computers," Journal of Parallel and Distributed Computing, 16, 328-337,1992.