Pipeline and batch sharing in grid workloads

Proceedings of the IEEE International Symposium on High Performance Distributed Computing

Volumn 2003-January, Issue , 2003, Pages 152-161

  Thain, Douglas ^a   Bent, John ^a   Arpaci Dusseau, Andrea C ^a   Arpaci Dusseau, Remzi H ^a   Livny, Miron ^a  

^a University of Wisconsin   (United States)

Author keywords

Application software; Biology computing; Grid computing; Physics computing; Pipelines; Production; Random access memory; Scalability; Telecommunication traffic; Throughput

Indexed keywords

APPLICATION PROGRAMS; DISTRIBUTED COMPUTER SYSTEMS; PIPELINES; PRODUCTION; RANDOM ACCESS STORAGE; SCALABILITY; TELECOMMUNICATION TRAFFIC; THROUGHPUT;

BIOLOGY COMPUTING; COMPUTATIONAL GRIDS; FILE STORAGE; INDIVIDUAL COMPONENTS; MEASUREMENTS OF; RANDOM ACCESS MEMORY; SCIENTIFIC WORKLOADS; SEQUENTIAL PROCESS;

GRID COMPUTING;

EID: 38049143367     PISSN: 10828907     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HPDC.2003.1210025     Document Type: Conference Paper

References (35)

1. A. Acharya, M. Uysal, R. Bennett, A. Mendelson, M. Beynon, J. K. Hollingsworth, J. Saltz, and A. Sussman. Tuning the performance of I/O intensive parallel applications. In Proceedings of the Fourth Workshop on Input/Output in Parallel and Distributed Systems (IOPADS), pages 15-27, Philadelphia, Pennsylvania, 1996.
2. S. F. Altschul, T. L. Madden, A. A. Schaffer, J. Zhang, Z. Zhang, W. Miller, , and D. J. Lipman. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. In Nucleic Acids Research, pages 3389-3402, 1997.
3. G. Amdahl. Storage and I/O Parameters and System Potential. In IEEE Computer Group Conference, pages 371-72, June 1970.
4. M. G. Baker, J. H. Hartmann, M. D. Kupfer, K. W. Shirriff, and J. K. Ousterhout . Measurements of a distributed file system. In Proceedings of the 13th ACM Symposium on Operating Systems Principles (SOSP), July 1991.
5. L. A. Barroso, K. Gharachorloo, and E. Bugnion. Memory system characterization of commercial workloads. In Proceedings of the International Symposium on Computer Architecture (ISCA), pages 3-14, 1998.
6. J. F. Cantin and M. D. Hill. Cache Performance for Selected SPEC CPU2000 Benchmarks. Computer Architecture News (CAN), September 2001.
7. P. E. Crandall, R. A. Aydt, A. A. Chien, and D. A. Reed . Input/output characteristics of scalable parallel applications. In Proceedings of the IEEE/ACM Conference on Supercomputing, San Diego, California, 1995.
8. R. Cypher, A. Ho, S. Konstantinidou, and P. Messina. Architectural Requirements of Parallel Scientific Applications with Explicit Communication. In Proceedings of the 20th Annual International Symposium on Computer Architecture, pages 2-13, San Diego, California, May 17-19, 1993.
9. ACM SIGARCH and IEEE Computer Society TCCA. Computer Architecture News, 21(2), May 1993.
10. J. Foley. An integrated biosphere model of land surface processes, terrestrial carbon balance, and vegetation dynamics. Global Biogeochemical Cycles, 10(4):603-628, 1996.
11. I. Foster, J. Voeckler, M. Wilde, and Y. Zhou. Chimera: A virtual data system for representing, querying, and automating data derivation. In Proceedings of the 14th Conference on Scientific and Statistical Database Management, Edinburgh, Scotland, July 2002.
12. J. Gray and P. Shenoy. Rules of thumb in data engineering. In Proceedings of the Sixteenth IEEE International Conference on Data Engineering (ICDE), pages 3-12, 2000.
13. K. Holtman. CMS data grid system overview and requirements. CMS Note 2001/037, CERN, July 2001.
14. P. Hulith. The AMANDA experiment. In Proceedings of the XVII International Conference on Neutrino Physics and Astrophysics, Helsinki, Finland, June 1996.
15. S. Kuo, M. Winslett, Y. Cho, J. Lee, and Y.Chen. Efficient input and output for scientific simulations. In Proceedings of I/O in Parallel and Distributed Systems (IOPADS), pages 33-44, 1999.
16. D. C. Lee, P. J. Crowley, J.-L. Bear, T. E. Anderson, and B. N. Bershad . Execution characteristics of desktop applications on Windows NT. In Proceedings of the 25th International Symposium on Computer Architecture (ISCA), pages 27-38, 1998.
17. M. Litzkow, M. Livny, and M. Mutka. Condor - a hunter of idle workstations. In Proceedings of the 8th International Conference of Distributed Computing Systems, June 1988.
18. E. L. Miller and R. H. Katz. Input/output behavior of super-computing applications. In Proceedings of the ACM/ IEEE conference on Supercomputing, pages 567-576, 1991.
19. J. K. Ousterhout, H. D. Costa, D. Harrison, J. A. Kunze, M. Kupfer, and J. G. Thompson . A trace-driven analysis of the UNIX 4.2 BSD file system. In Procedings of the 10th ACM Symposium on Operating Systems Principles (SOSP), pages 15-24, December 1985.
20. B. K. Pasquale and G. C. Polyzos . A static analysis of I/O characteristics of scientific applications in a production workload. In Proceedings of the ACM/ IEEE conference on Supercomputing, pages 388-397, November 1993.
21. A. Rajasekar, M. Wan, and R. Moore. MySRB and SRB - components of a data grid. In In Proceedings of the Eleventh IEEE Symposium on High Performance Distributed Computing (HPDC), Edinburgh, Scotland, July 2002.
22. K. Ranganathan and I. Foster. Decoupling computation and data scheduling in distributed data-intensive applications. In Proceedings of the Eleventh IEEE Symposium on High Performance Distributed Computing (HPDC), Edinburgh, Scotland, July 2002.
23. D. Roselli, J. Lorch, and T. Anderson. A comparison of file system workloads. In USENIX Annual Technical Conference, 2000.
24. E. Rosti, G. Serazzi, E. Smirni, and M. S. Squillante . The impact of I/O on program behavior and parallel scheduling. In Proceedings of the Joint International Conference on Measurement and Modelling of Computer Systems (SIG-METRICS), pages 56-65, 1998.
25. A. Samar and H. Stockinger. Grid Data Management Pilot. In Proceedings of IASTED International Conference on Applied Informatics (AI2001), February 2001.
26. M. Satyanarayanan. A study of file sizes and functional life-times. In Proceedings of the 8th Symposium on Operating Systems Principles (SOSP), pages 96-108, 1981.
27. W. T. Sullivan, D. Werthimer, S. Bowyer, J. Cobb, D. Gedye, and D. Anderson. A new major SETI project based on Project Serendip data and 100,000 personal computers. In Proceedings of the 5th International Conference on Bioastronomy, 1997.
28. A. K. Sum and J. J. de Pablo. Nautilus: Molecular Simulations code. Technical report, University of Wisconsin - Madison, Dept. of Chemical Engineering, 2002.
29. D. Thain, J. Bent, R. Arpaci-Dusseau, A. Arpaci-Dusseau, and M. Livny. The architectural implications of pipeline and batch sharing in scientific workloads. Technical Report CS-TR-1463, University of Wisconsin, Computer Sciences Department, January 2003.
30. D. Thain and M. Livny. Multiple bypass: Interposition agents for distributed computing. Journal of Cluster Computing, 4:39-47, 2001.
31. A. Vahdat and T. Anderson. Transparent result caching. Technical Report CSD-97-974, Computer Science Division, University of California-Berkeley, 1997.
32. S. Vazhkudai, S. Tuecke, and I. Foster. Replica selection in the globus data grid. IEEE International Symposium on Cluster Computing and the Grid (CCGrid), May 2001.
33. F. Wong, R. P. Martin, R. H. Arpaci-Dusseau, D. Wu, and D. E. Culler . Architectural Requirements and Scalability of the NAS Parallel Benchmarks. In Supercomputing ' 99, Portland, Oregon, Nov. 1999.
34. S. C. Woo, M. Ohara, E. Torrie, J. P. Shingh, and A. Gupta . The SPLASH-2 Programs: Characterization and Methodological Considerations. In Proceedings of the 22nd Annual International Symposium on Computer Architecture, pages 24-36, Santa Margherita Ligure, Italy, June 22-24, 1995.
35. ACM SIGARCH and IEEE Computer Society TCCA. Computer Architecture News, 23(2), May 1994.