RDMA-based job migration framework for MPI over InfiniBand

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2010, Pages 116-125

  Ouyang, Xiangyong ^a   Marcarelli, Sonya ^a   Rajachandrasekar, Raghunath ^a   Panda, Dhabaleswar K ^a  

^a Ohio State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHECK POINTING; CHECKPOINT/RESTART; COORDINATED CHECKPOINTS; HIGH PERFORMANCE COMMUNICATION; IMAGE TRANSMISSION; INFINIBAND; JOB MIGRATION; NODE FAILURE; OPEN-SOURCE; PROCESS APPLICATIONS; QUEUING DELAY; STABLE STORAGE; STORAGE AREA; STORAGE SUBSYSTEMS;

CLUSTER COMPUTING; FAULT TOLERANCE; QUEUEING NETWORKS;

QUALITY ASSURANCE;

EID: 78649483996     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTER.2010.20     Document Type: Conference Paper

References (35)

1. "MPI over InfiniBand, 10GigE/iWARP and RDMAoE," in http://mvapich.cse.ohio-state.edu/.
2. "MPI 3.0 Standardization Effort," http://meetings.mpi-forum. org/MPI-3.0-main-page.php.
3. H. Song, C. b. Leangsuksun, and R. Nassar, "Availability Modeling and Analysis on High Performance Cluster Computing Systems," in ARES '06: Proceedings of the First International Conference on Availability, Reliability and Security. Washington, DC, USA: IEEE Computer Society, 2006, pp. 305-313.
4. S. Fu and C.-Z. Xu, "Exploring event correlation for failure prediction in coalitions of clusters," in SC '07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing. New York, NY, USA: ACM, 2007, pp. 1-12.
5. "Intelligent Platform Management Interface (IPMI)," http://www.intel.com/design/servers/ipmi/.
6. S. Chakravorty and L. V. Kale, "A fault tolerance protocol with fast fault recovery," in IPDPS 2003, 2003.
7. R. K. Sahoo, A. J. Oliner, I. Rish, M. Gupta, J. E. Moreira, S. Ma, R. Vilalta, and A. Sivasubramaniam, "Critical event prediction for proactive management in large-scale computer clusters," in KDD '03, 2003, pp. 426-435.
8. InfiniBand Trade Association, "The InfiniBand Architecture," http://www.infinibandta.org.
9. Chao Wang and Frank Mueller and Christian Engelmann and Stephen L. Scott, "Proactive process-level live migration in HPC environments," in SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, 2008.
10. C. Wang, F. Mueller, C. Engelmann, and S. L. Scott, "A Job Pause Service under LAM/MPI+BLCR for Transparent Fault Tolerance," in IPDPS, 2007, pp. 1-10.
11. A. B. Nagarajan, F. Mueller, C. Engelmann, and S. L. Scott, "Proactive fault tolerance for HPC with Xen virtualization," in ICS '07: Proceedings of the 21st annual international conference on Supercomputing, 2007, pp. 23-32.
12. W. Huang, Q. Gao, J. Liu, and D. K. Panda, "High performance virtual machine migration with rdma over modern interconnects," in CLUSTER '07: Proceedings of the 2007 IEEE International Conference on Cluster Computing, 2007.
13. Q. Gao, W. Huang, M. J. Koop, and D. K. Panda, "Groupbased Coordinated Checkpointing for MPI: A Case Study on InfiniBand," in ICPP '07: Proceedings of the 2007 International Conference on Parallel Processing. Washington, DC, USA: IEEE Computer Society, 2007, p. 47.
14. Q. Gao, W. Yu, W. Huang, and D. K. Panda, "Application- Transparent Checkpoint/Restart for MPI Programs over InfiniBand," in ICPP '06: Proceedings of the 2006 International Conference on Parallel Processing. Washington, DC, USA: IEEE Computer Society, 2006, pp. 471-478.
15. X. Ouyang, K. Gopalakrishnan, and D. K. Panda, "Accelerating Checkpoint Operation by Node-Level Write Aggregation on Multicore Systems," ICPP 2009, September 2009.
16. X. Ouyang, K. Gopalakrishnan, T. Gangadharappa, and D. K. Panda, "Fast Checkpointing by Write Aggregation with Dynamic Buffer and Interleaving on Multicore Architecture," HiPC 2009, December 2009.
17. Duell, J., Hargrove, P., and Roman, E., "The Design and Implementation of Berkeley Lab's Linux Checkpoint/Restart," Lawrence Berkeley National Laboratory, Berkeley, CA 94720, Tech. Rep. LBNL-54941, 2002. [Online]. Available: {https: //ftg.lbl.gov/CheckpointRestart/Pubs/LBNL-54941. pdf}
18. "CIFTS Web Page," http://www.mcs.anl.gov/research/cifts.
19. R. Gupta, P. Beckman, B. Park, E. Lusk, P.Hargrove, A. Geist, D. Panda, A.Lumsdaine, and J. Dongarra, "CIFTS: A Coordinated Infrastucture for Fault-Tolerant Systems." in In Intĺ Conference on Parallel Processing (ICPP ' 09), 2009.
20. "Top 500 Supercomputers," http://www.top500.org.
21. P. H. Hargrove and J. C. Duell, "Berkeley Lab Checkpoint/Restart (BLCR) for Linux Clusters," in SciDAC, 6 2006.
22. J. K. Sridhar, M. J. Koop, J. L. Perkins, and D. K. Panda, "ScELA: Scalable and Extensible Launching Architecture for Clusters," in HiPC, 2008, pp. 323-335.
23. J. Bent, G. Gibson, G. Grider, B. McClelland, P. Nowoczynski, J. Nunez, M. Polte, and M. Wingate, "PLFS: a checkpoint filesystem for parallel applications," in SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, 2009.
24. F. C. Wong and R. P. M. etc., "Architectural requirements and scalability of the NAS parallel benchmarks," in Supercomputing '99, 1999, p. 41.
25. "PVFS2," http://www.pvfs.org/.
26. S. Al-Kiswany, M. Ripeanu, S. Vazhkudai, and A. Gharaibeh, "stdchk: A Checkpoint Storage System for Desktop Grid Computing," June 2008.
27. S. Sankaran, J. M. Squyres, B. Barrett, A. Lumsdaine, J. Duell, P. Hargrove, and E. Roman, "The LAM/MPI checkpoint/restart framework: System-initiated checkpointing," International Journal of High Performance Computing Applications, vol. 19, no. 4, pp. 479-493, Winter 2005.
28. J. Hursey, J. Squyres, T. Mattox, and A. Lumsdaine, "The Design and Implementation of Checkpoint/Restart Process Fault Tolerance for Open MPI," in 12th IEEE Workshop on Dependable Parallel, Distributed and Network-Centric Systems, March 2007.
29. G. Stellner, "CoCheck: Checkpointing and Process Migration for MPI," in Proceedings of the 10th International Parallel Processing Symposium (IPPS '96), 1996.
30. A. Agbaria and R. Friedman, "Starfish: Fault-Tolerant Dynamic MPI Programs on Clusters of Workstations," High-Performance Distributed Computing, International Symposium on, vol. 0, p. 31, 1999.
31. Y. Chen, J. S. Plank, and K. Li, "CLIP: A Checkpointing Tool for Message-Passing Parallel Programs," in In SC97: High Performance Networking and Computing, 1997, pp. 1-11.
32. G. Bosilca, A. Bouteiller, S. Djilali, G. Fedak, C. Germain, T. Herault, V. Neri, and A. Selikhov, "MPICH-V: Toward a scalable fault tolerant MPI for volatile nodes," in In Supercomputing, 2002, pp. 1-18.
33. B. Randell, "System structure for software fault tolerance," in Proceedings of the international conference on Reliable software. New York, NY, USA: ACM, 1975, pp. 437-449.
34. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson, "A survey of rollback-recovery protocols in message-passing systems," ACM Comput. Surv., vol. 34, no. 3, pp. 375-408, 2002.
35. S. Chakravorty, C. Mendes, and L. Kale, " Proactive fault tolerance in MPI applications via task migration ," in HiPC, 2006.