CATCH: A cloud-based adaptive data transfer service for HPC

Proceedings - 25th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011

Volumn , Issue , 2011, Pages 1242-1253

  Monti, Henry M ^a   Butt, Ali R ^a   Vazhkudai, Sudharshan S ^b  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Cloud for HPC; data staging; end user data delivery; HPC center serviceability; HPC data management

Indexed keywords

CHANGING DYNAMICS; DATA PATHS; DATA SERVICES; DATA TRANSPORT; DATA-STAGING; DELIVERY DEADLINES; DISTRIBUTED DATA; DISTRIBUTED SOURCES; END USERS; FAULT-TOLERANT; HPC CENTER SERVICEABILITY; INPUT DATAS; LARGE AMOUNTS OF DATA; MODERN HIGH PERFORMANCE; RESEARCH CHALLENGES; RESOURCE USAGE; STORAGE RESOURCES; TRANSFER TIME; WEB REPOSITORIES;

COMPUTER SOFTWARE SELECTION AND EVALUATION; DATA TRANSFER; DISTRIBUTED PARAMETER NETWORKS; INFORMATION MANAGEMENT; USER INTERFACES;

CLOUD COMPUTING;

EID: 80053223878     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.118     Document Type: Conference Paper

References (33)

1. Spallation Neutron Source. http://www.sns.gov/, 2008.
2. Conseil Européen pour la Recherche Nucléaire (CERN). LHC- the large hadron collider, July 2007. http://lhc.web.cern.ch/lhc/.
3. Earth system grid. http://www.earthsystemgrid.org, 2006.
4. Nsf teragrid. http://www.teragrid.org, 2009.
5. National Center for Computational Sciences. http://www.nccs.gov/, 2009.
6. National Institute of Computational Sciences. http://www.nics.tennessee. edu/computing-resources/kraken, 2008.
7. J. Bester, I. Foster, C. Kesselman, J. Tedesco, and S. Tuecke. GASS: A data movement and access service for wide area computing systems. In Proc. IOPADS, 1999.
8. FUSE: Filesystem in Userspace. http://fuse.sourceforge.net/, 2010.
9. Albeaus Bayucan, Robert L. Henderson, Casimir Lesiak, Bhroam Mann, Tom Proett, and Dave Tweten. Portable Batch System: External reference specification. http://www-unix.mcs.anl.gov/openpbs/docs/v2-2-ers.pdf, 1999.
10. Rich Wolski, Neil Spring, and Jim Hayes. The Network Weather Service: A distributed resource performance forecasting service for metacomputing. Future Generation Computing Systems, 15(5):757-768, 1999.
11. Henry Monti, Ali R. Butt, and Sudharshan S. Vazhkudai. Timely offloading of result-data in hpc centers. In Proc. ICS, 2008.
12. Henry M. Monti, Ali R. Butt, and Sudharshan S. Vazhkudai. /scratch as a cache: rethinking hpc center scratch storage. In Proc. ICS, 2009.
13. Henry Monti, Ali R. Butt, and Sudharshan S. Vazhkudai. Reconciling Scratch Space Consumption, Exposure, and Volatility to Achieve Timely Staging of Job Input Data. In Proc. IPDPS, 2010.
14. Microsoft. Windows Azure Platform. http://www.microsoft.com/windowsazure/ , 2010.
15. S3FS. http://code.google.com/p/s3fs/, 2010.
16. S.A. Kiswany, M. Ripeanu, S. S. Vazhkudai, and A. Gharaibeh. stdchk: A Checkpoint Storage System for Desktop Grid Computing. In Proc. ICDCS, 2008.
17. M. Li, S.S. Vazhkudai, A.R. Butt, F. Meng, X. Ma, Y. Kim, C. Engelmann, and G. Shipman. Functional partitioning to optimize end-to-end performance on many-core architectures. In Proc. Supercomputing, 2010.
18. Microsoft. Windows Azure Pricing. http://www.microsoft.com/windowsazure/ pricing/, 2010.
19. Batch Queue Prediction. http://nws.cs.ucsb.edu/ewiki/nws.php?id= Batch+Queue+Prediction, 2008.
20. Amazon ec2 homepage. http://aws.amazon.com/ec2/, 2010.
21. Planetlab: An open platform for developing, deploying and accessing planetary-scale services. http://www.planet-lab.org, 2005.
22. J. Plank, M. Beck, W. Elwasif, T. Moore, M. Swany, and R. Wolski. The Internet Backplane Protocol: Storage in the network. In Proc. NETSTORE, 1999.
23. Bbcp. http://www.slac.stanford.edu/%7Eabh/bbcp/, 2010.
24. Nccs user support - data transfer. http://www.nccs.gov/user-support/ general-support/data-transfer/, 2010.
25. D. Thain, S. Son J. Basney, and M. Livny. The kangaroo approach to data movement on the grid. In Proc. HPDC, 2001.
26. P. Rizk, C. Kiddle, and R. Simmonds. A gridftp overlay network service. In Proc. International Conference on Grid Computing, 2007.
27. G. Khanna, U. Catalyurek, T. Kurc, R. Kettimuthu, P. Sadayappan, I. Foster, and J. Saltz. Using overlays for efficient data transfer over shared wide-area networks. In Proc. Supercomputing, 2008.
28. T. Kosar and M. Livny. Stork: Making data placement a first class citizen in the grid. In Proc. ICDCS, 2004.
29. D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat. Bullet: High bandwidth data dissemination using an overlay mesh. In Proc. SOSP, 2003.
30. Dejan Kostic, Adolfo Rodriguez, Jeannie Albrecht, Abhijeet Bhirud, and Amin M. Vahdat. Using random subsets to build scalable network services. In Proc. USITS, 2003.
31. S. Annapureddy, M. J. Freedman, and D. Mazires. Shark: Scaling file servers via cooperative caching. In Proc. NSDI, 2005.
32. L. Wang, K. Park, R. Pang, V. Pai, and L. Peterson. Reliability and security in the CoDeeN content distribution network. In Proc. USENIX ATC, 2004.
33. KyoungSoo Park and Vivek S. Pai. Scale and performance in the CoBlitz large-file distribution service. In Proc. NSDI, 2006.