A highly-accurate and low-overhead prediction model for transfer throughput optimization

Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012

Volumn , Issue , 2012, Pages 787-795

  Kim, Jangyoung ^a   Yildirim, Esma ^b   Kosar, Tevfik ^a  

^a the State University of New York   (United States)

^b FATIH UNIVERSITY   (Turkey)

Author keywords

big data; high accuracy; low overhead; parallel streams; prediction; throughput optimization

Indexed keywords

BIG-DATA; HIGH-ACCURACY; LOW-OVERHEAD; PARALLEL STREAMS; THROUGHPUT OPTIMIZATION;

DATA FLOW ANALYSIS; DIGITAL STORAGE; FORECASTING; MATHEMATICAL MODELS; OPTIMIZATION; STATISTICAL TESTS; THROUGHPUT;

C (PROGRAMMING LANGUAGE);

EID: 84876579251     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.Companion.2012.109     Document Type: Conference Paper

References (31)

1. "XSEDE," http://www.xsede.org/.
2. "Energy sciences network (ESNet)," http://www.es.net/.
3. "Internet2," http://www.internet2.edu/.
4. "Arra/ani testbed," https://sites.google.com/a/lbl.gov/ani- 100g-network.
5. S. Garfienkel, "An evaluation of Amazons Grid computing services: EC2, S3 and SQS," Tech. Rep. TR-08-07, Aug 2007.
6. B. Cho and I. Gupta, "Budget-constrained bulk data transfer via internet and shipping networks," in The 8th International Conference on Autonomic Computing (ICAC), 2011.
7. H. Sivakumar, S. Bailey, and R. L. Grossman, "Psockets: The case for application-level network striping for data intensive applications using high speed wide area networks," in Proc. of Supercomputing, 2000.
8. J. Lee, D. Gunter, B. Tierney, B. Allcock, J. Bester, J. Bresnahan, and S. Tuecke, "Applied techniques for high bandwidth data transfers across wide area networks," in Proc. international Conference on Computing in High Energy and Nuclear Physics (CHEP01), 2001.
9. H. Balakrishman, V. N. Padmanabhan, S. Seshan, and R. H. K. M. Stemm, "Tcp behavior of a busy internet server: Analysis and improvements," in Proc. of INFOCOM, 1998.
10. T. J. Hacker, B. D. Noble, and B. D. Atley, "The endto-end performance effects of parallel tcp sockets on a lossy wide area network," in Proc. of IPDPS, 2002.
11. L. Eggert, J. Heideman, and J. Tough, "Effects of ensemble tcp," ACM Computer Communication Review, vol. 30(1), pp. 15-29, 2000.
12. G. Kola, T. Kosar, and M. Livny, "Run-time adaptation of grid data-placement jobs," Scalable Computing: Practice and Experience, vol. 6(3), pp. 33-43, 2005.
13. R. P. Karrer, J. Park, and J. Kim, "Adaptive data block scheduling for parallel streams," Tech. Report, vol. 17(2), 2006.
14. E. Yildirim, I. H. Suslu, and T. Kosar, "Which network measurement tool is right for you? a multidimensional comparison study," in Proc. of the 2008 9th IEEE/ACM International Conference on Grid Computing( GRID'08), sep 2008.
15. D. Lu, Y. Qiao, and P. A. Dinda, "Characterizing and predicting tcp throughput on the wide area network," in Proc. IEEE International Conference on Distributed Computing Systems (ICDCS05), 2005.
16. E. Yildirim, D. Yin, and T. Kosar, "Prediction of optimal parallelism level in wide area data transfers," IEEE Transactions on Parallel and Distributed Systems (TPDS), vol. 22(12), 2011.
17. D. Yin, E. Yildirim, and T. Kosar, "A data throughput prediction and optimization service for widely distributed many-task computing," IEEE Transactions on Parallel and Distributed Systems, vol. 22(6), 2011.
18. W. Allcock, Gridftp protocol specification, GGF, 2003.
19. E. Yildirim and T. Kosar, "Network-aware end-toend data throughput optimization," in Proc. of the Network-Aware Data Management Workshop (NDM 2011), 2012.
20. Emulab-network emulation testbed. [Online]. Available: http://www.emulab.net/
21. Cron-10gbps high-speed network emulation testbed. [Online]. Available: http://cron.loni.org
22. "Louisiana optical network initiative (LONI)," http://www.loni.org/.
23. D. Lu, Y. Qiao, P. A. Dinda, and F. E. Bustamante, "Modeling and taming parallel tcp on the wide area network," in Proc. of IPDPS, 2005.
24. E. Altman, D. Barman, B. Tuffin, and M. Vojnovic, "Parallel tcp sockets: Simple model, throughput and validation," in Proc. IEEE Conference on Computer Communications (INFOCOM06), 2006.
25. J. Crowcroft and P. Oechslin, "Differentiated end-to-end internet services using a weighted proportional fair sharing tcp," ACM SIGCOMM Computer Communication Review, vol. 28(3), pp. 53-69, Jul. 1998.
26. G. Kola and M. K. Vernon, "Target bandwidth sharing using endhost measures," Performance Evaluation, vol. 64(9-12), pp. 948-964, 2007.
27. E. Yildirim, J. Kim, and T. Kosar, "Optimizing the sample size for a cloud-hosted data scheduling service," in Proc. 2nd International Workshop on Cloud Computing and Scientific Applications (CCSA in conjunction with CCGRID'12), 2012.
28. M. Mathis, J. Heffner, and R. Reddy, "Web100:extended tcp instrumentation for research, education and diagnosis," ACM Computer Communications Review, vol. 33(3), 2003.
29. T. Kosar and M. Livny, "Stork: Making data placement a first class citizen in the grid," in Proceedings of ICDCS'04, March 2004, pp. 342-349.
30. T. Kosar and M. Balman, "A new paradigm: Dataaware scheduling in grid computing," Future Generation Computing Systems, vol. 25, no. 4, pp. 406-413, 2009.
31. T. Kosar, M. Balman, E. Yildirim, S. Kulasekaran, and B. Ross, "Stork data scheduler: Mitigating the data bottleneck in e-science," The Phil. Transactions of the Royal Society A, vol. 369(3254-3267), 2011.