Network-aware end-to-end data throughput optimization

NDM'11 - Proceedings of the 2011 International Workshop on Network-Aware Data Management, Co-located with SC'11

Volumn , Issue , 2011, Pages 21-30

  Yildirim, Esma ^a   Kosar, Tevfik ^a  

^a the State University of New York   (United States)

Author keywords

Data parallelism; GridFTP; Optimization; Parallel TCP streams; Prediction; Striping

Indexed keywords

AVAILABLE BANDWIDTH; DATA CENTERS; DATA PARALLELISM; DATA THROUGHPUT; DISK I/O; END-SYSTEMS; END-TO-END THROUGHPUT; END-USERS; GRIDFTP; HIGH BANDWIDTH; OPTICAL NETWORKING; OPTIMIZATION FRAMEWORK; PARALLEL DISK SYSTEMS; PARALLEL STREAMS; PARALLEL TCP; PARAMETER COMBINATION; PROCESSOR SPEED; STRIPING; TRANSPORT PROTOCOLS;

FORECASTING; INFORMATION MANAGEMENT; TRANSMISSION CONTROL PROTOCOL;

OPTIMIZATION;

EID: 84857951633     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2110217.2110221     Document Type: Conference Paper

References (23)

1. Arra/ani testbed. https://sites.google.com/a/lbl.gov/ani-100g-network.
2. R. Ahuja, T. Magnanti, and J. Orlin. Network Flows. Prentice Hall, 1993.
3. W. Allcock, J. Bresnahan, R. Kettimuth, M. Link, C. Dumirescu, I. Raicu, and I. Foster. The globus striped gridftp framework and server. In Proceedings of the ACM/IEEE Conference on Supercomputing, page 54, 2005.
4. E. Altman, D. Barman, B. Tuffin, and M. Vojnovic. Parallel tcp sockets: Simple model, throughput and validation. In Proc. IEEE Conference on Computer Communications (INFOCOM'06), pages 1-12, Apr. 2006.
5. J. Crowcroft and P. Oechslin. Differentiated end-to-end internet services using a weighted proportional fair sharing tcp. ACM SIGCOMM Computer Communication Review, 28(3):53-69, July 1998.
6. S. Floyd. Rfc3649: Highspeed tcp for large congestion windows.
7. W. Gropp, E. Lusk, and R. Thakur. Using MPI-2: Advanced Features of the Message-Passing Interface. The MIT Press, 1999.
8. T. J. Hacker, B. D. Noble, and B. D. Atley. The end-to-end performance effects of parallel tcp sockets on a lossy wide area network. In Proc. IEEE International Symposium on Parallel and Distributed Processing(IPDPS'02), pages 434-443, 2002.
9. G. Hasegawa, T. Terai, T. Okamoto, and M. M. Scalable socket buffer tuning for high-performance web servers. In International Conference on Network Protocols(ICNP01), page 281, 2001.
10. The lustre file system. http://wiki.lustre.org.
11. M. Jain, R. S. Prasad, and C. Davrolis. The tcp bandwidth-delay product revisited: network buffering, cross traffic, and socket buffer auto-sizing. Technical report, Georgia Institute of Technology, 2003.
12. C. Jin, D. X. Wei, S. H. Low, G. Buhrmaster, J. Bunn, D. H. Choe, R. L. A. Cottrell, J. C. Doyle, W. Feng, O. Martin, H. Newman, F. Paganini, S. Ravot, and S. Singh. Fast tcp: from theory to experiments. IEEE Network, 19(1):4-11, Feb. 2005. (Pubitemid 40195955)
13. G. Kola, T. Kosar, and M. Livny. Run-time adaptation of grid data-placement jobs. Scalable Computing: Practice and Experience, 6(3):33-43, 2005.
14. LBNL. The distributed parallel storage system. http://www-didc.lbl.gov/ DPSS.
15. D. Lu, Y. Qiao, P. A. Dinda, and F. E. Bustamante. Modeling and taming parallel tcp on the wide area network. In Proc. IEEE International Symposium on Parallel and Distributed Processing (IPDPS'05), page 68b, Apr. 2005.
16. R. S. Prasad, M. Jain, and C. Davrolis. Socket buffer auto-sizing for high-performance data transfers. Journal of Grid Computing, 1(4):361-376, Aug. 2004.
17. J. Semke, J. Madhavi, and M. Mathis. Automatic tcp buffer tuning. ACM SIGCOMM'98, 28(4):315-323, Oct. 1998. (Pubitemid 128566468)
18. N. Stone, B. Gill, J. Kochmar, R. Light, P. Nowoczynski, J. R. Scott, J. Sommerfield, and C. Vizino. Dmover: Parallel data migration for mainstream users. Technical report, Pittsburgh Supercomputing Center, 2010.
19. E. Weigle and W. Feng. Dynamic right-sizing: A simulation study. In Proc. IEEE International Conference on Computer Communications and Networks (ICCCN'01), 2001.
20. W. Liu, B. Tieman, R. Kettimuthu, and I. Foster. A data transfer framework for large-scale science experiments. In Proc. 19th ACM International Symposium on High-Performance Distributed Computing (HPDC'10), 2010.
21. Louisiana optical network initiative. http://www.loni.org.
22. The teragrid. http://www.teragrid.org.
23. E. Yildirim, D. Yin, and T. Kosar. Prediction of optimal parallelism level in wide area data transfers. IEEE Transactions on Parallel and Distributed Systems, 22(12), 2011.