HydEE: Failure containment without event logging for large scale send-deterministic MPI applications

Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012

Volumn , Issue , 2012, Pages 1216-1227

  Guermouche, Amina ^a   Ropars, Thomas ^b   Snir, Marc ^c   Cappello, Franck ^a,c  

^a INRIA SACLAY   (France)

^b EPFL   (Switzerland)

^c UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

failure containment; fault tolerance; High performance computing; MPI; send determinism

Indexed keywords

CHECK POINTING; CONCURRENT FAILURES; DIFFERENT PROTOCOLS; EVENT LOGGING; EXECUTION MODEL; EXPERIMENTAL EVALUATION; FAULT TOLERANT PROTOCOLS; HIGH PERFORMANCE COMPUTING; HYBRID PROTOCOLS; MEAN TIME BETWEEN FAILURES; MESSAGE LOGGING; MPI; MPI APPLICATIONS; ROLLBACK RECOVERY; SEND-DETERMINISM;

DISTRIBUTED PARAMETER NETWORKS; FAULT TOLERANCE;

MESSAGE PASSING;

EID: 84866852589     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2012.111     Document Type: Conference Paper

References (32)

1. L. Alvisi and K. Marzullo. Message Logging: Pessimistic, Optimistic, Causal, and Optimal. IEEE Transactions on Software Engineering, 24(2):149-159, 1998.
2. L. Alvisi, S. Rao, S. A. Husain, A. de Mel, and E. Elnozahy. An Analysis of Communication-Induced Checkpointing. In Proceedings of the Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing, FTCS '99, pages 242-, Washington, DC, USA, 1999. IEEE Computer Society.
3. D. Bailey, T. Harris, W. Saphir, R. van der Wilngaart, A. Woo, and M. Yarrow. The NAS Parallel Benchmarks 2.0. Technical Report NAS-95-020, NASA Ames Research Center, 1995.
4. R. Baldoni, F. Quaglia, and B. Ciciani. A VP-Accordant Checkpointing Protocol Preventing Useless Checkpoints. In Proceedings of the The 17th IEEE Symposium on Reliable Distributed Systems, SRDS '98, pages 61-, Washington, DC, USA, 1998. IEEE Computer Society.
5. B. Bhargava and L. Shu-Renn. Independent Checkpointing and Concurrent Rollback for Recovery in Distributed Systems-an Optimistic Approach. In Seventh Symposium on Reliable Distributed Systems, pages 3-12, Columbus, OH , USA, 1988.
6. G. Bosilca, A. Bouteiller, T. Herault, P. Lemarinier, and J. J. Dongarra. Dodging the Cost of Unavoidable Memory Copies in Message Logging Protocols. In Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface, EuroMPI'10, pages 189-197, Berlin, Heidelberg, 2010. Springer-Verlag.
7. A. Bouteiller, G. Bosilca, and J. Dongarra. Redesigning the Message Logging Model for High Performance. Concurrency and Computation : Practice and Experience, 22:2196-2211, November 2010.
8. A. Bouteiller, T. Herault, G. Bosilca, and J. Dongarra. Correlated Set Coordination in Fault Tolerant Message Logging Protocols. In Euro-Par 2011, volume 6853 of Lecture Notes in Computer Science, pages 51-64. Springer Berlin / Heidelberg, 2011.
9. A. Bouteiller, T. Ropars, G. Bosilca, C. Morin, and J. Dongarra. Reasons for a Pessimistic or Optimistic Message Logging Protocol in MPI Uncoordinated Failure Recovery. In IEEE International Conference on Cluster Computing (Cluster 2009), New Orleans, USA, 2009.
10. F. Cappello, A. Guermouche, and M. Snir. On Communication Determinism in Parallel HPC Applications. In 19th International Conference on Computer Communications and Networks (ICCCN 2010), 2010.
11. K. Chandy and L. Lamport. Distributed Snapshots: Determining Global States of Distributed Systems. ACM Transactions on Computer Systems, 3(1):63-75, 1985.
12. J. Dongarra, P. Beckman, T. Moore, et al. The international exascale software project roadmap. International Journal of High Performance Computing Applications, 25:3-60, February 2011.
13. E. N. Elnozahy et al. System Resilience at Extreme Scale. Technical report, DARPA, 2008.
14. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson. A Survey of Rollback-Recovery Protocols in Message-Passing Systems. ACM Computing Surveys, 34(3):375-408, 2002.
15. L. B. Gomez, N. Maruyama, D. Komatitsch, S. Tsuboi, F. Cappello, S. Matsuoka, and T. Nakamura. FTI: high performance Fault Tolerance Interface for hybrid systems. In IEEE/ACM SuperComputing 2011, Seatle, USA, November 2011.
16. A. Guermouche, T. Ropars, E. Brunet, M. Snir, and F. Cappello. Uncoordinated Checkpointing Without Domino Effect for Send-Deterministic Message Passing Applications. In 25th IEEE International Parallel & Distributed Processing Symposium (IPDPS2011), Anchorage, USA, 2011. to appear.
17. B. Gupta, R. Nikolaev, and R. Chirra. A Recovery Scheme for Cluster Federations Using Sender-based Message Logging. volume 19, pages 127-139, 2011.
18. J. C. Y. Ho, C.-L. Wang, and F. C. M. Lau. Scalable Group-Based Checkpoint/Restart for Large-Scale Message-Passing Systems. In 22nd IEEE International Parallel and Distributed Processing Symposium (IPDPS'08), Miami, USA, 2008.
19. D. B. Johnson and W. Zwaenepoel. Sender-Based Message Logging. In Digest of Papers: The 17th Annual International Symposium on Fault-Tolerant Computing, pages 14-19, 1987.
20. P. Kogge et al. Exascale Computing Study: Technology Challenges in Achieving Exascale Systems. Technical report, DARPA, 2008.
21. L. Lamport. Time, Clocks, and the Ordering of Events in a Distributed System. Communications of the ACM, 21(7):558-565, 1978.
22. E. Meneses, C. L. Mendes, and L. V. Kale. Team-based Message Logging: Preliminary Results. In 3rd Workshop on Resiliency in High Performance Computing (Resilience) in Clusters, Clouds, and Grids (CCGRID 2010)., May 2010.
23. Message Passing Interface Forum. MPI: A Message-Passing Interface Standard. www.mpi-forum.org, 1995.
24. S. Monnet, C. Morin, and R. Badrinath. Hybrid Checkpointing for Parallel Applications in Cluster Federations. In Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid (CCGRID'04), pages 773-782, Washington, DC, USA, 2004. IEEE Computer Society.
25. R. A. Oldfield, S. Arunagiri, P. J. Teller, S. Seelam, M. R. Varela, R. Riesen, and P. C. Roth. Modeling the Impact of Checkpoints on Next-Generation Systems. In MSST '07: Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies, pages 30-46, Washington, DC, USA, 2007. IEEE Computer Society.
26. S. Rao, L. Alvisi, and H. M. Vin. The Cost of Recovery in Message Logging Protocols. In Symposium on Reliable Distributed Systems, pages 10-18, 1998.
27. R. Riesen, K. Ferreira, and J. Stearley. See Applications Run and Throughput Jump: The Case for Redundant Computing in HPC. In Proceedings of the 2010 International Conference on Dependable Systems and Networks Workshops (DSN-W), DSNW '10, pages 29-34, Washington, DC, USA, 2010. IEEE Computer Society.
28. T. Ropars, A. Guermouche, B. Uçar, E. Meneses, L. V. Kalé, and F. Cappello. On the Use of Cluster-Based Partial Message Logging to Improve Fault Tolerance for MPI HPC Applications. In Euro-Par 2011, pages 567-578, 2011.
29. T. Ropars and C. Morin. Active optimistic and distributed message logging for message-passing applications. Concurrency and Computation: Practice and Experience, 23(17):2167-2178, 2011.
30. Q. O. Snell, A. R. Mikler, and J. L. Gustafson. NetPIPE: A Network Protocol Independent Performance Evaluator. In IASTED International Conference on Intelligent Information Management and Systems, June 1996.
31. N. Vaidya. Distributed recovery units: An approach for hybrid and adaptive distributed recovery. Technical Report 93-052, Departement of Computer Science Texas, A&M, University, 1993.
32. J.-M. Yang, K. F. Li, W.-W. Li, and D.-F. Zhang. Trading Off Logging Overhead and Coordinating Overhead to Achieve Efficient Rollback Recovery. Concurrency and Computation : Practice and Experience, 21:819-853, April 2009.