Using overlays for efficient data transfer over shared wide-area networks

2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008

Volumn , Issue , 2008, Pages

  Khanna, Gaurav ^a   Catalyurek, Umit ^a,b   Kurc, Tahsin ^b   Kettimuthu, Rajkumar ^c   Sadayappan, P ^a   Foster, Ian ^c   Saltz, Joel ^b  

^a Ohio State University   (United States)

^b OHIO STATE UNIVERSITY   (United States)

^c ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION FILES; DATA REDISTRIBUTION; DATA-INTENSIVE APPLICATION; FILE TRANSFER PROTOCOLS; FILE TRANSFERS; GRIDFTP; INTERMEDIATE NODE; LARGE AMOUNTS OF DATA; MULTI-HOP PATH; NETWORK BANDWIDTH; PERFORMANCE IMPROVEMENTS; RESOURCE AVAILABILITY; ROUTING DATA; WIDE AREA;

BANDWIDTH; COMPUTER SCIENCE; DATA TRANSFER; INFORMATION SERVICES; NETWORK ARCHITECTURE; NETWORK PROTOCOLS; OVERLAY NETWORKS; SCHEDULING ALGORITHMS; SERVICE ORIENTED ARCHITECTURE (SOA); WIRELESS TELECOMMUNICATION SYSTEMS;

WIDE AREA NETWORKS;

EID: 70350757789     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2008.5213292     Document Type: Conference Paper

References (27)

1. GT C WS core. http://www.globus.org/toolkit/docs/4.0/common/cwscore/.
2. The Large Haldron Collider (LHC) . http://lhc.web.cern.ch/lhc/.
3. Globus FTP Client API, 2002. http://www.globus.org/api/c/globus ftp client/html/index.html.
4. Predicting the performance of wide area data transfers. In IPDPS '02: Proceedings of the 16th International Symposium on Parallel and Distributed Processing, page 34, Washington, DC, USA, 2002. IEEE Computer Society.
5. W. Allcock. Gridftp: Protocol extensions to ftp for the grid. In Global Grid ForumGFD-R-P.020, 2003.
6. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster. The globus striped gridftp framework and server. In SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing, Washington, DC, USA, 2005. IEEE Computer Society.
7. A. Bakre and B. R. Badrinath. I-tcp: indirect tcp for mobile hosts. In ICDCS '95: Proceedings of the 15th International Conference on Distributed Computing Systems, page 136, Washington, DC, USA, 1995. IEEE Computer Society.
8. M. Beck, T. Moore, J. S. Plank, and M. Swany. Logistical networking: Sharing more than the wires. In S. Hariri, C. A. Lee, and C. S. Raghavendra, editors, Active Middleware Services, Norwell, MA, 2000. Kluwer Academic.
9. K. Brown and S. Singh. M-tcp: Tcp for mobile cellular networks. SIGCOMM Comput. Commun. Rev., 27(5):19-43, 1997.
10. Z. Cai, V. Kumar, and K. Schwan. Iq-paths: Selfregulating data streams across network overlays. In Proceedings of the The 15th IEEE International Symposium on High Performance Distributed Computing (HPDC'06), 2006.
11. A. Giersch, Y. Robert, and F. Vivien. Scheduling tasks sharing files from distributed repositories. In Euro-Par 2004: Parallel Processing: 10th International Euro-Par Conference, volume 3149 of LNCS, pages 246-253, Sept. 2004.
12. K. Holtman. Cms data grid system overview and requirements. In Computing in High Energy and Nuclear Physics (CHEP), 2001.
13. O. Ibarra and C. Kim. Heuristic algorithms for scheduling independent tasks on nonindentical processors. Journal of the ACM, 24(2):280-289, Apr 1977.
14. G. Khanna, U. Catalyurek, T. Kurc, P. Sadayappan, and J. Saltz. Scheduling file transfers for data-intensive jobs on heterogeneous clusters. In A.-M. Kermarrec, L. Bougé, and T. Priol, editors, Euro-Par, volume 4641 of Lecture Notes in Computer Science, pages 214-223. Springer, 2007.
15. G. Khanna, T. Kurc, U. Catalyurek, R. Kettimuthu, P. Sadayappan, and J. Saltz. A dynamic scheduling approach for coordinated wide-area data transfers using gridftp. In Proc. of 22th International Parallel and Distributed Processing Symposium (IPDPS), Miami, Florida, 2008.
16. G. Khanna, N. Vydyanathan, T. Kurc, U. Catalyurek, P. Wyckoff, J. Saltz, and P. Sadayappan. A hypergraph partitioning based approach for scheduling of tasks with batch-shared i/o. In CCGRID '05: Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2, pages 792-799, Cardiff, UK, 2005. IEEE Computer Society.
17. M. S. Kim, T. Kim, Y. Shin, S. S. Lam, and E. J. Powers. A wavelet-based approach to detect shared congestion. SIGCOMM Comput. Commun. Rev., 34(4):293-306, 2004.
18. T. Kosar and M. Livny. Stork: Making data placement a first class citizen in the grid. In ICDCS '04: Proc.of the 24th International Conference on Distributed Computing Systems (ICDCS'04), pages 342-349, Washington, DC, USA, 2004. IEEE Computer Society.
19. H. Pucha and Y. C. Hu. Overlay tcp: ending end-to-end transport for higher throughput. In Poster in ACM SIGCOMM, Philadelphia, PA, 2005.
20. P. Rizk, C. Kiddle, and R. Simmonds. A gridftp overlay network service. In Proceedings of the 7th IEEE/ACM International Conference on Grid Computing, Baercelona, Spain, 2007.
21. D. Rubenstein, J. Kurose, and D. Towsley. Detecting shared congestion of flows via end-to-end measurement. In SIGMETRICS '00: Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, pages 145-155, New York, NY, USA, 2000. ACM.
22. M. Swany. Improving throughput for grid applications with network logistics. In SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing, page 23, Washington, DC, USA, 2004. IEEE Computer Society.
23. TeraGrid. http://www.teragrid.org.
24. B. Tierney, J. Lee, L. T. Chen, H. Herzog, G. Hoo, G. Jin, and W. E. Johnston. Distributed parallel data storage systems: a scalable approach to high speed image servers. In MULTIMEDIA '94: Proceedings of the second ACMinternational conference on Multimedia, pages 399-405, New York, NY, USA, 1994. ACM Press.
25. B. L. Tierney, J. Lee, B. Crowley, M. Holding, J. Hylton, and F. L. D. Jr. A network-aware distributed storage cache for data intensive environments. In HPDC '99: Proceedings of the The Eighth IEEE International Symposium on High Performance Distributed Computing, page 33,Washington, DC, USA, 1999. IEEE Computer Society.
26. R. Wolski. Dynamically forecasting network performance using the network weather service. Cluster Computing, 1(1):119-132, 1998.
27. L. Yang, J. M. Schopf, and I. Foster. Improving parallel data transfer times using predicted variances in shared networks. In CCGRID '05: Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2, pages 734-742, Washington, DC, USA, 2005. IEEE Computer Society.