Hybrid checkpointing for MPI jobs in HPC environments

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn , Issue , 2010, Pages 524-533

  Wang, Chao ^a   Mueller, Frank ^a   Engelmann, Christian ^b   Scott, Stephen L ^b  

^a North Carolina State University   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Checkpoint restart; Fault tolerance; High performance computing

Indexed keywords

CHECK POINTING; CHECKPOINT/RESTART; HIGH PERFORMANCE APPLICATIONS; HIGH PERFORMANCE COMPUTING SYSTEMS; HIGH-PERFORMANCE COMPUTING; OPTIMAL BALANCE; ORDER OF MAGNITUDE;

COST ACCOUNTING; FAULT TOLERANCE; FAULT TOLERANT COMPUTER SYSTEMS;

QUALITY ASSURANCE;

EID: 79951790076     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPADS.2010.48     Document Type: Conference Paper

References (28)

1. C.-H. Hsu and W.-C. Feng, "A power-aware run-time system for high-performance computing," in Supercomputing, 2005.
2. O. R. N. Laboratory, "National center for computational sciences," http://info.nccs.gov/resources/jaguar, Jun. 2007.
3. I. Philp, "Software failures and the road to a petaflop machine," in Workshop on High Performance Computing Reliability Issues, 2005.
4. J. T. Daly, L. A. Pritchett-Sheats, and S. E. Michalak, "Application MTTFE vs. platform MTTF: A fresh perspective on system reliability and application throughput for computations at scale," in Workshop on Resiliency in High Performance Computing, May 2008, pp. 19-22.
5. C. Wang, F. Mueller, C. Engelmann, and S. Scott, "A job pause service under LAM/MPI+BLCR for transparent fault tolerance," in International Parallel and Distributed Processing Symposium, Apr. 2007.
6. J. Ruscio, M. Heffner, and S. Varadarajan, "Dejavu: Transparent userlevel checkpointing, migration, and recovery for distributed systems," in International Parallel and Distributed Processing Symposium, 2007.
7. J. M. Squyres and A. Lumsdaine, "A Component Architecture for LAM/MPI," in European PVM/MPI Users' Group Meeting, ser. Lecture Notes in Computer Science, no. 2840, Sep. 2003, pp. 379-387.
8. J. Duell, "The design and implementation of berkeley lab's linux checkpoint/restart," Lawrence Berkeley National Laboratory, TR, 2000.
9. S. Sankaran, J. M. Squyres, B. Barrett, A. Lumsdaine, J. Duell, P. Hargrove, and E. Roman, "The LAM/MPI checkpoint/restart framework: System-initiated checkpointing," in LACSI Symposium, Oct. 2003.
10. J. Hursey, J. M. Squyres, T. I. Mattox, and A. Lumsdaine, "The design and implementation of checkpoint/restart process fault tolerance for Open MPI," in Workshop on Dependable Parallel, Distributed and Network-Centric Systems, 03 2007.
11. M. Rosenblum and J. K. Ousterhout, "The design and implementation of a log-structured file system," in ACM Trans. on Computer Systems, Vol. 10, No. 1, Feb. 1992.
12. C. Wang, F. Mueller, C. Engelmann, and S. Scott, "Proactive processlevel live migration in hpc environments," in Supercomputing, 2008.
13. F. Wong, R. Martin, R. Arpaci-Dusseau, and D. Culler, "Architectural requirements and scalability of the NAS parallel benchmarks," in Supercomputing, 1999.
14. A. Darling, L. Carey, and W. Feng, "The design, implementation, and evaluation of mpiBLAST," in ClusterWorld Conference and Expo, 2003.
15. N. Naksinehaboon, Y. Liu, C. B. Leangsuksun, R. Nassar, M. Paun, and S. Scott, "Reliability-aware approach: An incremental checkpoint/restart model in hpc environments," in Symposium on Cluster Computing and the Grid, 2008, pp. 783-788.
16. G. Stellner, "CoCheck: checkpointing and process migration for MPI," in Proceedings of IPPS'96. The 10th International Parallel Processing Symposium: Honolulu, HI, USA, 15-19 April 1996, IEEE, Ed. 1109 Spring Street, Suite 300, Silver Spring, MD 20910, USA: IEEE Computer Society Press, 1996, pp. 526-531.
17. G. Bosilca, A. Boutellier, and F. Cappello, "MPICH-V: Toward a scalable fault tolerant MPI for volatile nodes," in Supercomputing, Nov. 2002.
18. B. Bouteiller, F. Cappello, T. Herault, K. Krawezik, P. Lemarinier, and M. Magniette, "MPICH-V2: a fault tolerant MPI for volatile nodes based on pessimistic sender based message logging," in Supercomputing, 2003.
19. B. Barrett, J. M. Squyres, A. Lumsdaine, R. L. Graham, and G. Bosilca, "Analysis of the component architecture overhead in Open MPI," in European PVM/MPI Users' Group Meeting, September 2005.
20. R. Gioiosa, J. C. S., S. Jiang, and F. Petrini, "Transparent, incremental checkpointing at kernel level: a foundation for fault tolerance for parallel computers," in Supercomputing, 2005.
21. J. Heo, S. Yi, Y. Cho, J. Hong, and S. Y. Shin, "Space-efficient page-level incremental checkpointing," in ACM Symposium on Applied computing, 2005, pp. 1558-1562.
22. S.-T. Hsu and R.-C. Chang, "Continuous checkpointing: joining the checkpointing with virtual memory paging," Softw. Pract. Exper., vol. 27, no. 9, pp. 1103-1120, 1997. (Pubitemid 127582120)
23. S. Yi, J. Heo, Y. Cho, and J. Hong, "Adaptive page-level incremental checkpointing based on expected recovery time," in ACM Symposium on Applied computing, 2006, pp. 1472-1476. (Pubitemid 44759028)
24. S. Agarwal, R. Garg, M. S. Gupta, and J. E. Moreira, "Adaptive incremental checkpointing for massively parallel systems," in International Conference on Supercomputing. New York, NY, USA: ACM, 2004, pp. 277-286.
25. J. Mehnert-Spahn, E. Feller, and M. Schoettner, "Incremental checkpointing for grids," in Linux Symposium, Jul. 2009.
26. J. W. Young, "A first order approximation to the optimum checkpoint interval," Commun. ACM, vol. 17, no. 9, pp. 530-531, 1974.
27. J. T. Daly, "A higher order estimate of the optimum checkpoint interval for restart dumps," Future Gener. Comput. Syst., vol. 22, no. 3, pp. 303-312, 2006. (Pubitemid 41689812)
28. Y. Liu, R. Nassar, C. B. Leangsuksun, N. Naksinehaboon, M. Paun, and S. Scott, "An optimal checkpoint/restart model for a large scale high performance computing system," in International Parallel and Distributed Processing Symposium, Apr. 2008.