Rethinking FTP: Aggressive block reordering for large file transfers

ACM Transactions on Storage

Volumn 4, Issue 4, 2009, Pages

  Anastasiadis, Stergios V ^a   Wickremesinghe, Rajiv G ^b   Chase, Jeffrey S ^c,d  

^a UNIVERSITY OF IOANNINA   (Greece)

^b ORACLE CORPORATION   (United States)

^c DUKE UNIVERSITY   (United States)

^d Duke University   (United States)

Author keywords

Disk access; File transfer protocols; Scheduling

Indexed keywords

ANALYTICAL MODELS; ARM MOVEMENTS; BLOCK SIZES; BUFFERING REQUIREMENTS; CONTENT SERVERS; DISK ACCESS; DISK TRAFFICS; DISK TRANSFER RATES; FILE ACCESS; FILE TRANSFER PROTOCOLS; FILE TRANSFER SYSTEMS; FILE TRANSFERS; INTERNET CONTENTS; NETWORK BANDWIDTHS; POTENTIAL IMPACTS; RAPID GROWTHS;

INTERNET PROTOCOLS; SCHEDULING; SERVERS;

DISKS (STRUCTURAL COMPONENTS);

EID: 60449092042     PISSN: 15533077     EISSN: 15533093     Source Type: Journal    
DOI: 10.1145/1480439.1480442     Document Type: Article

References (39)

1. ACHARYA, S., FRANKLIN, M., AND ZDONIK, S. 1997. Balancing push and pull for data broadcast. In Proceedings of the ACM SIGMOD, 183-194.
2. ALLCOCK, B., BESTER, J., BRESNAHAN, J., CHERVENAK, A. L., FOSTER, I., KESSELMAN, C., MEDER, S., NEFEDOVA, V., QUESNAL, D., AND TUECKE, S. 2002. Data management and transfer in high performance computational grid environments. Parallel Comput. J. 28, 5, 749-771.
3. ALMEIDA, J. M., KRUEGER, J., EAGER, D. L., AND VERNON, M. K. 2001. Analysis of educational media server workloads. In Proceedings of the International Workshop onNetworh and Operating System Support for Digital Audio and Video, 21-30.
4. ANASTASIADIS, S. V., SEVCIK, K. C., AND STUMM, M. 2001. Modular and efficient resource management in the exedra media server. In Proceedings of the USENIX Symposium on Internet Technologies and Systems, 25-36.
5. ANASTASIADIS, S. V., WICKREMESINGHE, R. G., AND CHASE, J. S. 2004. Circus: Opportunistic block reordering for scalable content servers. In Proceedings of the USENLX Conference on File and Storage Technologies, 201-212.
6. ARLITT, M. P. AND WILLIAMSON, C. L. 1996. Web server workload characterization: The search for invariants. In Proceedings of the ACM SIGMETRICS, 126-137.
7. BAKER, M. G., HARTMAN, J. H., KUPFER, M. D., SHIRRIFF, K. W., AND OUSTERHOUT, J. K. 1991. Measurements of a distributed file system. In Proceedings of the ACM Symposium on Operating Systems Principles, 198-212.
8. BARFORD, P. AND CROVELLA, M. 1998. Generating representative Web workloads for network and server performance evaluation. In Proceedings of the ACM SIGMETRICS, 151-160.
9. BROWN, A. D., MOWRY, T. C., AND KRIEGER, O. 2001. Compiler-Based I/O prefetching for out-of-core applications. ACM Trans. Comput. Syst. 19, 2, 111-170.
10. BYERS, J., CONSIDINE, J., MITZENMACHER, M., AND ROST, S. 2002. Informed content delivery across adaptive overlay networks. In Proceedings of the ACM SIGCOMM, 47-60.
11. BYERS, J. W., LUBY, M., MITZENMACHER, M., AND REGE, A. 1998. A digital fountain approach to reliable distribution of bulk data. In Proceedings of the ACM SIGCOMM, 57-67.
12. CAO, P., FELTEN, E. W., KARLIN, A., AND LI, K. 1995. A study of integrated prefetching and caching strategies. In Proceedings of the SIGMETRICS/Peformance'95.
13. CHESIRE, M., WOLMAN, A., VOELKER, G. M., AND LEVY, H. M. 2001. Measurement and analysis of a streaming-media workload. In Proceedings of the USENIX Symposium, on Internet Technologies and Systems, 1-12.
14. CLARK, D. D. AND TENNENHOUSE, D. L. 1990. Architectural considerations for a new generation of protocols. In Proceedings of the ACM SIGCOMM, 200-208.
15. COFFMAN, K. AND ODLYZKO, A. M. 2002. Internet growth: Is there a "moore's law" for data traffic? In Proceedings of the Handbook of Massive Data, Sets. Kluwer Academic, 47-93.
16. COHEN, B. 2003. Incentives build robustness in bittorrent. bitconjurer.org.
17. DIOT, C. AND GAGNON, F. 1999. Impact of out-of-sequence processing on the performance of data transmission. Comput. Netw. 31, 475-492.
18. DOYLE, R. P., CHASE, J. S., GADDE, S., AND VAHDAT, A. M. 2001. The trickle-down effect: Web caching and server request distribut ion. In Proceedings of the International Workshop on Web Caching and Content Delivery.
19. EAGER, D., VERNON, M., AND ZAHORJAN, J. 2001. Minimizing bandwidth requirements for on-demand data delivery. IEEE Trans. Knowl. Data Eng. 13, 5, 742-757.
20. GAREY, M. R. AND JOHNSON, D. S. 1979. Computers and Intractability. Freeman, New York.
21. JIN, S. AND BESTAVROS, A. 2002. Scalability of multicast delivery for non-sequential streaming access. In of the ACM SIGMETRICS, 97-107.
22. LUBY, M. 2002. Lt codes. In Proceedings of the IEEE Symposium on Foundations of Computer Science, 271-282.
23. MEGIDDO, N. AND MODHA, D. S. 2003. Arc: A self-tuning, low overhead replacement cache. In Proceedings of the 2nd USENLX Conference on File and Storage Technologies (FAST'03).
24. PADHYE, J., FIROIU, V., TOWSLEY, D. F., AND KUROSE, J. F. 2000. Modeling TCP Reno performance: A simple model and its empirical validation. IEEE/ACM Trans. Netw. 8, 2, 133-145.
25. PAI, V. S., ARON, M., BANGA, G., SVENDSEN, M., DRUSCHEL, P., ZWAENEPOEL, W., AND NAHUM, E. 1998. Locality-Aware request distribution in cluster-based network servers. In Proceedings of the ACM ASPLOS, 205-216.
26. PARK, K. AND PAI, V. S. 2006. Scale and performance in the Coblitz large-file distribution service. In Proceedings of the USENIX Symposium on Networked Systems Design & Implementation, 29-44.
27. PATTERSON, R. H., GIBSON, G. A., GINTING, E., STODOLSKY, D., AND ZELENKA, J. 1995. Informed prefetching and caching. In Proceedings of the ACM Symposium on Operating Systems Principles, 79-95.
28. POSTEL, J. AND REYNOLDS, J. 1985. File transfer protocol (ftp). USC/ISI, Network Working Group RFC 959.
29. RAMAN, S., BALAKRISHNAN, H., AND SRINIVASAN, M. 2000. An image transport protocol for the internet. In Proceedings of the International Conference on Network Protocols, 209-219.
30. RIZZO, L. 1997. Dummynet: A simple approach to the evaluation of network protocol. ACM Commun. Rev. 47, 1, 31-41.
31. ROST, S., BYERS, J., AND BESTAVROS, A. 2001. The cyclone server architecture: Streamlining delivery of popular content. In Proceedings of the International Workshop on Web Caching and Content Distribution. Boston, MA.
32. SAROIU, S., GUMMADI, P. K., DUNN, R. J., GRIBBLE, S. D., AND LEVY, H. M. 2002. An analysis of internet content delivery systems. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, 315-328.
33. SAROIU, S., GUMMADI, P. K., AND GRIBBLE, S. D. 2002. A measurement study of peer-to-peer file sharing systems. In Proceedings of the SPIEIACM Multimedia Computing and Networking Conference.
34. STEERE, D. C. 1997. Exploitingthe non-determinism and asynchrony of set iterators to reduce aggregate file I/O latency. In Proceedings of the ACM Symposium, on Operating Systems Principles, 252-263.
35. TRIVEDI, K. S. 1982. Probability and Statistics with Reliability, Queuing and Computer Science Applications. Prentice-Hall, Englewood Cliffs, NJ.
36. VITTER, J. S. AND KRISHNAN, P. 1996. Optimal prefetching via data compression. J. ACM 43, 5, 771-793.
37. VOGELS, W. 1999. File system usage in windows nt 4.0. In Proceedings of the ACM Symposium on Operating Systems Principles, 93-109.
38. WANG, L., PAI, V. S., AND PETERSON, L. L. 2002. The effectiveness of request redirection on CDN robustness. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, 345-360.
39. ZHANG, Y., BRESLAU, L., PAXSON, V., AND SHENKER, S. 2002. On the characteristics and origins of internet flow rates. In Proceedings of the ACM SIGCOMM.