High performance data transfer in Grid environment using GridFTP over InfiniBand

CCGrid 2010 - 10th IEEE/ACM International Conference on Cluster, Cloud, and Grid Computing

Volumn , Issue , 2010, Pages 557-564

  Subramoni, Hari ^a   Lai, Ping ^a   Kettimuthu, Raj ^a   Panda, Dhabaleswar K ^a  

^a The Ohio State University   (United States)

Author keywords

Cluster of clusters; Globus XIO; GridFTP; InfiniBand; InfiniBand WAN and iWARP; Obsidian longbow XR; RDMA; Zero copy

Indexed keywords

COMMUNICATION BANDWIDTH; COMMUNICATION EFFICIENCY; COMMUNICATION FEATURES; COMMUNICATION PERFORMANCE; DISK-BASED; EXPERIMENTAL EVALUATION; GRID ENVIRONMENTS; GRIDFTP; IN-DEPTH ANALYSIS; INFINIBAND; MICROBENCHMARKS; PERFORMANCE DATA; PERFORMANCE IMPROVEMENTS; REAL-WORLD APPLICATION; WAN ENVIRONMENT; ZERO-COPY;

CLUSTER COMPUTING; COMMUNICATION; DESIGN; GRID COMPUTING; NETWORK PROTOCOLS; QUEUEING NETWORKS; TELECOMMUNICATION SYSTEMS; TRANSMISSION CONTROL PROTOCOL; WIDE AREA NETWORKS;

DATA TRANSFER;

EID: 77954933100     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2010.115     Document Type: Conference Paper

References (50)

1. J. Postel and J. Reynolds, "File Transfer Protocol. RFC 959.".
2. W. Allcock, "GridFTP: Protocol Extensions to FTP for the Grid. Global Grid ForumGFD-R-P.020, 2003.".
3. M. Allman and S. Ostermann, "Multiple Data Connection FTP Extensions," Ohio University, Tech. Rep., 1996.
4. D. Bush, "UFTP," http://www.tcnj.edu/bush/uftp.html.
5. K. R. Sollins, "The Trivial File Transfer Protocol - TFTP 2 RFC 1350. July, 1992.".
6. S. Ladha and P. D. Amer, "Improving Multiple File Transfers Using SCTP Multistreaming," in Int'l on Performance, Computing, and Communications Conference, April 2004.
7. M. Allman, S. Ostermann, and C. Metz, "FTP Extensions for IPv6 and NATs. RFC 2428. Sep. 1998.".
8. "InfiniBand Trade Association," http://www.infinibandta.org.
9. "Architectural Specifications for RDMA over TCP/IP," http://www.rdmaconsortium.org/.
10. "Obsidian Research Corp." http://www.obsidianresearch.com/.
11. Net NX 5010 High Speed Exchange, http://www.net.com/products/products-nx. shtml.
12. P. Lai, H. Subramoni, S. Narravula, A. Mamidala, and D. K. Panda, "Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand," in (ICPP 09), Sep. 2009.
13. W. Allcock, J. Bresnahan, R. Kettimuthu, and J. Link, "The Globus eXtensible Input/Output System (XIO): A Protocol Independent IO System for the Grid," Parallel and Distributed Processing Symposium, International, vol. 5, p. 179a, 2005.
14. Community Climate System Model, http://www.ccsm.ncar.edu.
15. Institute for Ultra Scale Visualization, "Ultravis: Ultra Scale Visualization," http://ultravis.ucdavis.edu/.
16. Earth System Grid, http://www.earthsystemgrid.org.
17. SciDAC Computational Astrophysics Consortium, http://www.supersci.org.
18. "Energy Sciences Network (ESnet)," http://www.es.net.
19. W. Allcock, J. Bresnahan, R. Kettimuthu, and M. Link, "The Globus Striped GridFTP Framework and Server," in SC '05: Proceedings of the 2005 ACM/IEEE conference on Supercomputing. Washington, D.C.: IEEE Computer Society, 2005, p. 54.
20. D. X. Wei, C. Jin, S. H. Low, and S. Hegde, "FAST TCP: Motivation, Architecture, Algorithms, Performance," IEEE/ACM Trans. Netw., vol. 14, no. 6, pp. 1246-1259, 2006.
21. T. Kelly, "Scalable TCP: Improving Performance in High-speed Wide Area Networks," SIGCOMM Computer Communication Review, vol. 33, no. 2, pp. 83-91, 2003.
22. D. J. Leith and R. Shorten, "H-TCP protocol for high-speed long distance networks," in Second International Workshop on Protocols for Fast Long-Distace Networks, Argonne, IL,, 16-17 February 2004.
23. S. Ha, I. Rhee, and L. Xu, "CUBIC: A New TCP-Friendly High-Speed TCP Variant," SIGOPS Oper. Syst. Rev., vol. 42, no. 5, pp. 64-74, 2008.
24. Y. Gu and R. L. Grossman, "UDT: UDP-based Data Transfer for High-Speed Wide Area Networks," Comput. Netw., vol. 51, no. 7, pp. 1777-1799, 2007.
25. E. He, J. Leigh, O. Yu, and T. A. DeFanti, "Reliable Blast UDP : Predictable High Performance Bulk Data Transfer," Cluster Computing, IEEE International Conference on, vol. 0, p. 317, 2002.
26. J. Bresnahan, M. Link, R. Kettimuthu, and I. Foster, "UDT as an Alternative Transport Protocol for GridFTP," in International Workshop on Protocols for Future, Large-Scale and Diverse Network Transports (PFLDNeT), 21-22 May 2009. [Online]. Available: http://www.mcs.anl.gov/kettimut/PFLDNeT09. pdf.
27. "Open Fabrics Enterprise Distribution," http://www.openfabrics. org/.
28. Argonne National Laboratory, "Advanced Photon Source," http://www.aps.anl.gov/.
29. "LHC: Large Hadron Collider," http://lhc.web.cern.ch/lhc/.
30. Mellanox OFED Stack for Linux Users Manual, http://www.mellanox.com/pdf/ products/software/Mellanox-OFED-Linux-User-Manual-1-20.pdf.
31. D. A. Patterson, G. Gibson, and R. H. Katz, "A Case for Redundant Arrays of Inexpensive Disks (RAID)," in SIGMOD '88: Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data. New York, NY, USA: ACM, 1988, pp. 109-116.
32. "Grid Computing," http://en.wikipedia.org/wiki/Gridcomputing.
33. I. Foster, C. Kesselman, and S. Tuecke, "The anatomy of the grid: Enabling scalable virtual organizations," Int. J. High Perform. Comput. Appl., vol. 15, no. 3, pp. 200-222, 2001. (Pubitemid 33099013)
34. Office of Science, DOE, http://fasterdata.es.net/TCPtuning/linux.html.
35. N. S. V. Rao, W. Yu, W. R. Wing, S. W. Poole, and J. S. Vetter, "Wide-Area Performance Profiling of 10GigE and InfiniBand Technologies," in Proceedings of the 2008 ACM/IEEE conference on Supercomputing, 2008.
36. F. Anklesaria et.al, "The Internet Gopher Protocol. RFC 1436. Network Working Group," March 1993.
37. M. Allman, V. Paxson, and W. Stevens, "TCP Congestion Control - RFC 2581," United States, 1999.
38. L. Xu, K. Harfoush, and I. Rhee, "Binary Increase Congestion Control (BIC) for Fast Long-Distance Networks," in IEEE Infocom. IEEE, 2004. [Online]. Available: http://www.ieee-infocom.org/2004/Papers/524.PDF.
39. "HighSpeed TCP," http://www.icir.org/floyd/hstcp.html.
40. D. Katabi, M. Handley, and C. Rohrs, "Congestion Control for High Bandwidth-Delay Product Networks," in SIGCOMM '02: Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications. New York, NY, USA: ACM, 2002, pp. 89-102.
41. W. T. Strayer, M. J. Lewis, and R. E. Cline, Jr., "XTP as a Transport Protocol for Distributed Parallel Processing," in HSNS'94: Proceedings of the High-Speed Networking Symposium on USENIX 1994 High-Speed Networking Symposium. Berkeley, CA, USA: USENIX Association, 1994, pp. 6-6.
42. Tsunami Network Protocol Implementation, http://www.indiana.edu/uits/cpo/ tsunami, 2002.
43. R. L. Grossman, M. Mazzucco, H. Sivakumar, Y. Pan, and Q. Zhang, "Simple Available Bandwidth Utilization Library for High-Speed Wide Area Networks," J. Supercomput., vol. 34, no. 3, pp. 231-242, 2005.
44. R. Noronha, L. Chai, T. Talpey, and D. K. Panda, "Designing NFS with RDMA for Security, Performance and Scalability," in In The 2007 International Conference on Parallel Processing (ICPP-07), Aug 2007.
45. S. Carter, M. Minich, and N. S. V. Rao, "Experimental evaluation of infiniband transport over local- and wide-area networks," in High Performance Computing Symposium, 2007.
46. W. Yu, N. S. Rao, P. Wyckoff, and J. S. Vetter, "Performance of RDMA-capable Storage Protocols on Wide-Area Network," in Peta-byte Storage Workshop, in conjunction with SC08, 2008.
47. S. Narravula, H. Subramoni, P. Lai, R. Noronha, and D. K. Panda, "Performance of HPC Middleware over InfiniBand WAN," in ICPP'08, Sep. 2008.
48. "IOzone Filesystem Benchmark," http://www.iozone.org/.
49. "Bonnie - IO Throughput Benchmark," http://www.garloff.de/kurt/ linux/bonnie/.
50. "FIO - Linux IO Benchmarking Tool," http://freshmeat.net/ projects/fio/.