BlobCR: Efficient checkpoint-restart for HPC applications on iaas clouds using virtual disk image snapshots

Proceedings of 2011 SC - International Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2011, Pages

  Nicolae, Bogdan ^a   Cappello, Franck ^a,b  

^a INRIA SACLAY   (France)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Capture application state; Checkpoint restart; Cloud computing; Disk snapshots; Fault tolerance; IaaS; MPI applications; Science clouds; Scientific computing

Indexed keywords

CAPTURE APPLICATION STATE; CHECKPOINT-RESTART; DISK SNAPSHOTS; IAAS; MPI APPLICATIONS;

CLOUDS; COMPUTER SOFTWARE SELECTION AND EVALUATION; FAULT TOLERANCE; MESSAGE PASSING; NATURAL SCIENCES COMPUTING;

CLOUD COMPUTING;

EID: 83155184555     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2063384.2063429     Document Type: Conference Paper

References (34)

1. Amazon Elastic Block Storage (EBS). http://aws.amazon.com/ebs/.
2. File System in UserspacE (FUSE). http://fuse.sourceforge.net.
3. Nasa nebula. http://nebula.nasa.gov.
4. Nimbus. http://www.nimbusproject.org/.
5. Opennebula. http://www.opennebula.org/.
6. Amazon Elastic Compute Cloud (EC2). http://aws.amazon.com/ec2/.
7. Amazon Simple Storage Service (S3). http://aws.amazon.com/s3/.
8. M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. A view of cloud computing. Commun. ACM, 53:50-58, April 2010.
9. J. Bent, G. Gibson, G. Grider, B. McClelland, P. Nowoczynski, J. Nunez, M. Polte, and M. Wingate. PLFS: A checkpoint filesystem for parallel applications. In SC'09: Proceedings of the 22nd Conference on High Performance Computing Networking, Storage and Analysis, pages 1-12, Portland, USA, 2009.
10. R. Bolze, F. Cappello, E. Caron, M. Daydé, F. Desprez, E. Jeannot, Y. Jégou, S. Lanteri, J. Leduc, N. Melab, G. Mornet, R. Namyst, P. Primet, B. Quetier, O. Richard, E.-G. Talbi, and I. Touche. Grid'5000: A large scale and highly reconfigurable experimental grid testbed. Int. J. High Perform. Comput. Appl., 20:481-494, November 2006. (Pubitemid 44605333)
11. R. Brightwell, K. Ferreira, and R. Riesen. Transparent redundant computing with mpi. In EuroMPI'10: Proceedings of the 17th European MPI user's group meeting conference on recent advances in the message passing interface, pages 208-218, Stuttgart, Germany, 2010.
12. G. H. Bryan and R. Rotunno. The maximum intensity of tropical cyclones in axisymmetric numerical model simulations. Journal of the American Meteorological Society, 137:1770-1789, 2009.
13. P. H. Carns, W. B. Ligon, R. B. Ross, and R. Thakur. PVFS: A parallel file system for Linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, USA, 2000.
14. J. Duell, P. Hargrove, and E. Roman. The Design and Implementation of Berkeley Lab's Linux Checkpoint/Restart. Technical Report LBNL-54941, Future Technologies Group, 2002.
15. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson. A survey of rollback-recovery protocols in message-passing systems. ACM Comput. Surv., 34:375-408, September 2002.
16. M. Gagné. Cooking with Linux-still searching for the ultimate Linux distro? Linux J., 2007(161):9, 2007.
17. W. Gropp, E. Lusk, and A. Skjellum. Using MPI (2nd ed.): portable parallel programming with the message-passing interface. MIT Press, Cambridge, MA, USA, 1999.
18. J. G. Hansen and E. Jul. Scalable virtual machine storage using local disks. SIGOPS Oper. Syst. Rev., 44:71-79, December 2010.
19. Q. He, S. Zhou, B. Kobler, D. Duffy, and T. McGlynn. Case study for running hpc applications in public clouds. In HPDC'10: Proceedings of the 19th International Symposium on High Performance Parallel and Distributed Computing, pages 395-401, Chicago, USA, 2010.
20. A. Lakshman and P. Malik. Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev., 44:35-40, April 2010.
21. X. Liu, J. Huai, Q. Li, and T. Wo. Network state consistency of virtual machine in live migration. In SAC'10: Proceedings of the 2010 ACM Symposium on Applied Computing, pages 727-728, Sierre, Switzerland, 2010.
22. B. Nicolae. BlobSeer: Towards Efficient Data Storage Management for Large-Scale, Distributed Systems. PhD thesis, University of Rennes 1, November 2010.
23. B. Nicolae, G. Antoniu, L. Bougé, D. Moise, and A. Carpen-Amarie. BlobSeer: Next-generation data management for large scale infrastructures. J. Parallel Distrib. Comput., 71:169-184, February 2011.
24. B. Nicolae, J. Bresnahan, K. Keahey, and G. Antoniu. Going Back and Forth: Efficient Multi-Deployment and Multi-Snapshotting on Clouds. In HPDC'11: The 20th International ACM Symposium on High-Performance Parallel and Distributed Computing, pages 147-158, San Jose, USA, 2011.
25. B. Nicolae, F. Cappello, and G. Antoniu. Going Back and Forth: Efficient Multi-Deployment and Multi-Snapshotting on Clouds. In Euro-Par'11: Proceedings of the 17th International Euro-Par Conference on Parallel Processing, pages 503-513, Bordeaux, France, 2011.
26. L. Ramakrishnan, P. T. Zbiegel, S. Campbell, R. Bradshaw, R. S. Canon, S. Coghlan, I. Sakrejda, N. Desai, T. Declerck, and A. Liu. Magellan: experiences from a science cloud. In Proceedings of the 2nd international workshop on Scientific cloud computing, pages 49-58, San Jose, USA, 2011.
27. D. Reimer, A. Thomas, G. Ammons, T. Mummert, B. Alpern, and V. Bala. Opening black boxes: Using semantic information to combat virtual machine image sprawl. In VEE'08: Proceedings of the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pages 111-120, Seattle, USA, 2008.
28. M. Rosenblum and J. K. Ousterhout. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst., 10(1):26-52, 1992. (Pubitemid 23598979)
29. J. F. Ruscio, M. A. Heffner, and S. Varadarajan. Dejavu: Transparent user-level checkpointing, migration, and recovery for distributed systems. In IPDPS'07: Proceedings of the 21th IEEE International Parallel and Distributed Processing Symposium, pages 1-10, Long Beach, USA, 2007.
30. F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In FAST'02: Proceedings of the 1st USENIX Conference on File and Storage Technologies, Monterey, USA, 2002.
31. G. Vallée, T. Naughton, H. Ong, and S. Scott. Checkpoint/restart of virtual machines based on Xen. In HAPCW'06: Proceedings of the High Availability and Performance Workshop, Santa Fe, USA, 2006.
32. O. Villa, S. Krishnamoorthy, J. Nieplocha, and D. M. Brown, Jr. Scalable transparent checkpoint-restart of global address space applications on virtual machines over Infiniband. In CF'09: Proceedings of the 6th ACM Conference on Computing Frontiers, pages 197-206, Ischia, Italy, 2009.
33. K. V. Vishwanath and N. Nagappan. Characterizing cloud computing hardware reliability. In SoCC'10: Proceedings of the 1st ACM symposium on Cloud computing, pages 193-204, Indianapolis, USA, 2010.
34. R. Wartel, T. Cass, B. Moreira, E. Roche, M. Guijarro, S. Goasguen, and U. Schwickerath. Image distribution mechanisms in large scale cloud providers. In CloudCom'10: Proceedings 2nd IEEE International Conference on Cloud Computing Technology and Science, Indianapolis, USA, 2010.