Improving parallel data transfer times using predicted variances in shared networks

2005 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2005

Volumn 2, Issue , 2005, Pages 734-742

  Yang, Lingyun ^a   Schopf, Jennifer M ^b   Foster, Ian ^a,b  

^a THE UNIVERSITY OF CHICAGO   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA SETS; GLOBUS TOOLKIT; NETWORK PERFORMANCE;

COMPUTER NETWORKS; DATA STRUCTURES; PARALLEL PROCESSING SYSTEMS; PROBLEM SOLVING;

DATA TRANSFER;

EID: 33751083499     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2005.1558636     Document Type: Conference Paper

References (23)

1. Thet-Test: http://trochim.human.cornell.edu/kb/stat_t.htm.
2. Allcock, B., Bester, J., Bresnahan, J., et al., Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing. Proceedings of the Mass Storage Conference, 2001.
3. Beck, M. and Moore, T., The I-2 DSI project: An Architecture for Internet Content Channels, Computer Networking and ISDN Systems, 30 (1998) 2141-2148.
4. Berman, F., Chien, A., Cooper, K., et al., The GrADs Project: Software Support for High-level Grid Application Development, The International Journal of High Performance Computing Applications, 15 (2001) 327-344.
5. Czajkowski, K., Foster, I., Kesselman, C., et al., SNAP: A Protocol for Negotiating Service Level Agreements and Coordinating Resource management in Distributed Systems. 8th Workshop On Job Scheduling Strategies for Parallel Processing, Edinburgh, Scotland, 2002.
6. Dinda, P.A., Online Prediction of the Running Time of Tasks, Cluster Computing, 5 (2002).
7. Feng, J. and Humphrey, M., Eliminating Replica Selection -Using Multiple Replicas to Accelerate Data Transfer on Grid. Proceedings of the Tenth International Conference on Parallel and Distributed Systems (ICPADS 2004), Newport Beach, CA, 2004.
8. Foster, I. and Kesselman, C., The Globus Project: A Status Report. Proc. IPPS/SPDP '98 Heterogeneous Computing Workshop, 1998.
9. Freitas, C.J., Coffin, D.B. and Murphy, R.L., The Characterization of a Wide Area Network Computation, Parallel Computing, 29 (2003) 879-894.
10. Hafeez, M., Samar, A. and Stockinger, H., A Data Grid Prototype for Distributed Data Production in CMS. Proceedings of the 7th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT2000), 2000.
11. Izal, M., Urvoy-Keller, G., Biersack, E.W., et al., Dissecting BitTorrent: Five Months in a Torrent's Lifetime. Proceedings of the 5th Passive & Active Measurement Workshop, 2004.
12. Leland, W.E., Taqqu, M.S., Willinger, W., et al., On the Self-Similar Nature of Ethernet Traffic (Extended Version), IEEE/ACM Transactions on Networking, 2 (1994).
13. Malon, D., May, E., Resconi, S., et al., Grid-enabled Data Access in the ATLAS Athena Framework. Proceedings of Computing and High Energy Physics 2001 (CHEP'01) Conference, 2001.
14. Schopf, J.M. and Berman, F., Using Stochastic Information to Predict Application Behavior on Contended Resources, International Journal of Foundations of Computer Science, Special Issue on Parallel Distributed Computing (2001).
15. Shen, X. and Choudhary, A., A Distributed Multi-Storage Resource Architecture and I/O Performance Prediction for Scientific Computing. Proceedings of the 9th IEEE Symposium on High Performance Distributed Computing (HPDC 9), 2000, pp. 21-30.
16. Smith, W., Foster, I. and Taylor, V., Predicting Application Run Times Using Historical Information. Proceedings of the IPPS/SPDP'98 Workshop on Job Scheduling Strategies for Parallel Processing, 1998.
17. Sun, X. and Wu, M., Grid Harvest Service: A System for Long-term, Application-level Task Scheduling. International Parallel and Distributed Processing Symposium (IPDPS'03), Nice, France, 2003.
18. Tierney, B., Johnston, W.E., Herzog, H., et al., Distributed Parallel Data Storage Systems: A Scalable Approach to High Speed Image Servers. Proceedings of the ACM Multimedia, 1994.
19. Vazhkudai, S. and Schopf, J.M., Using Regression Techniques to Predict Large Data Transfers, Journal of High Performance Computing Applications-Special Issue on Grid Computing: Infrastructure and Applications (2003).
20. Wolski, R., Spring, N. and Hayes, J., The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing, Journal of Future Generation Computing Systems (1998) 757-768.
21. Yang, L., Foster, I. and Schopf, J.M., Homeostatic and Tendency-based CPU Load Predictions. Proceedings of International Parallel and Distributed Processing Symposium (IPDPS2003), 2003.
22. Yang, L., Schopf, J.M. and Foster, I., Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments. Proceedings of SuperComputing 2003, 2003.
23. Allcock, B., Bresnahan, J., Kettimuthu, R., Link, M., Dumitrescu, C., Raicu, I. and Foster, I. The Globus Striped GridFTP Framework and Server. www.globus.org, 2005.