Explicit control in a batch-aware distributed file system

1st Symposium on Networked Systems Design and Implementation, NSDI 2004

Volumn , Issue , 2004, Pages

  Bent, John ^a   Thain, Douglas ^a   Arpaci Dusseau, Andrea C ^a   Arpaci Dusseau, Remzi H ^a   Livny, Miron ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FAULT TOLERANCE; SCHEDULING; SYSTEMS ANALYSIS;

CACHE CONSISTENCY; DISTRIBUTED FILE SYSTEMS; END-TO-END PERFORMANCE; MICRO-BENCHMARKS; NOVEL COMPONENT; PERFORMANCE BENEFITS; REMOTE COMPUTING; SPACE MANAGEMENT;

FILE ORGANIZATION;

EID: 4544343377     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (57)

1. A. Adya, W. J. Bolosky, M. Castro, G. Cermak, R. Chaiken, J. R. Douceur, J. Howell, J. R. Lorch, M. Theimer, and R. P. Wattenhofer. FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02), Boston, MA, Dec 2002.
2. R. Agrawal, T. Imielinski, and A. Swami. Database Mining: A Performance Perspective. IEEE Transactions on Knowledge and Data Engineering, 5(6):914-925, Dec 1993.
3. S. F. Altschul, T. L. Madden, A. A. Schaffer, J. Zhang, Z. Zhang, W. Miller,, and D. J. Lipman. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. In Nucleic Acids Research, pages 3389-3402, 1997.
4. T. Anderson, M. Dahlin, J. Neefe, D. Patterson, and R. Wang. Serverless Network File Systems. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP'95), pages 109-26, Copper Mountain, CO, Dec 1995.
5. A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, N. C. Burnett, T. E. Denehy, T. J. Engle, H. S. Gunawi, J. Nugent, and F. I. Popovici. Transforming Policies into Mechanisms with Infokernel. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP'03), Bolton Landing (Lake George), NY, Oct 2003.
6. Avery, P. et al. CMS Virtual Data Requirements. kholtman.home.cern.ch/kholtman/tmp/cmsreqsv6.ps, 2001.
7. M. Baker, J. Hartman, M. Kupfer, K. Shirriff, and J. Ousterhout. Measurements of a Distributed File System. In Proceedings of the 13th ACM Symposium on Operating Systems Principles (SOSP'91), pages 198-212, Pacific Grove, CA, Oct 1991.
8. J. Bent, V. Venkataramani, N. Leroy, A. Roy, J. Stanley, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and M. Livny. Flexibility, Manageability, and Performance in a Grid Storage Appliance. In Proceedings of High-Performance Distributed Computing (HPDC-11), pages 3-12, Edinburgh, Scotland, Jul 2002.
9. B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. E. Fiuczynski, D. Becker, C. Chambers, and S. Eggers. Extensibility, Safety and Performance in the SPIN Operating System. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP'95), pages 267-284, Copper Mountain, CO, Dec 1995.
10. Y. Breitbart, A. Deacon, H.-J. Schek, A. P. Sheth, and G. Weikum. Merging Application-centric and Data-centric Approaches to Support Transaction-oriented Multi-system Workflows. SIGMOD Record, 22(3):23-30, 1993.
11. J. F. Cantin and M. D. Hill. Cache Performance for Selected SPEC CPU2000 Benchmarks. Computer Architecture News (CAN), Sep 2001.
12. S. Chandra, M. Dahlin, B. Richards, R. Y. Wang, T. E. Anderson, and J. R. Larus. Experience with a Language for Writing Coherence Protocols. In Proceedings of the USENIX Conference on Domain-Specific Languages, Santa Barbara, CA, Oct 1997.
13. F. W. Chang and G. A. Gibson. Automatic I/O Hint Generation Through Speculative Execution. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation (OSDI'99), pages 1-14, New Orleans, Louisiana, Feb 1999.
14. J. S. Chase, L. E. Grit, D. E. Irwin, J. D. Moore, and S. Sprenkle. Dynamic Virtual Clusters in a Grid Site Manager. In Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC 12), Seattle, WA, June 2003.
15. F. Dabek, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Wide-Area Cooperative Storage with CFS. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP'01), Banff, Canada, Oct 2001.
16. M. D. Dahlin, R. Y. Wang, T. E. Anderson, and D. A. Patterson. Cooperative Caching: Using Remote Client Memory to Improve File System Performance. In Proceedings of the 1st Symposium on Operating Systems Design and Implementation (OSDI'94), Monterey, CA, Nov 1994.
17. EDA Industry Working Group. The EDA Resource. http://www.eda.org/, 2003.
18. D. A. Edwards and M. S. McKendry. Exploiting Read-Mostly Workloads in The FileNet File System. In Proceedings of the 12th ACM Symposium on Operating Systems Principles (SOSP'89), pages 58-70, Litchfield Park, Arizona, Dec 1989.
19. D. R. Engler, M. F. Kaashoek, and J. W. O'Toole. Exokernel: An Operating System Architecture for Application-Level Resource Management. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP'95), pages 251-266, Copper Mountain, CO, Dec 1995.
20. M. J. Feeley, W. E. Morgan, F. H. Pighin, A. R. Karlin, and H. M. Levy. Implementing Global Memory Management in a Workstation Cluster. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP'95), pages 201-212, Copper Mountain, CO, Dec 1995.
21. J. Flinn, S. Sinnamohideen, N. Tolia, and M. Satyanarayanan. Data Staging on Untrusted Surrogates. In Proceedings of the 2nd USENIX Symposium on File and Storage Technologies (FAST'03), San Francisco, CA, Apr 2003.
22. B. Ford, M. Hibler, J. Lepreau, P. Tullman, G. Back, and S. Claw-son. Microkernels Meet Recursive Virtual Machines. In Proceedings of the 2nd Symposium on Operating Systems Design and Implementation (OSDI'96), Seattle, WA, Oct 1996.
23. I. Foster and P. Avery. Petascale Virtual Data Grids for Data Intensive Science. GriPhyn White Paper, 2001.
24. I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke. A Security Architecture for Computational Grids. In Proceedings of the 5th ACM Conference on Computer and Communications Security Conference, pages 83-92, 1998.
25. I. Foster, C. Kesselman, and S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal of Supercomputer Applications, 15(3), 2001.
26. J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tuecke. Condor-G: A Computation Management Agent for Multi- Institutional Grids. In Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC 10), San Francisco, CA, Aug 2001.
27. E. Gelenbe. On the Optimal Checkpoint Interval. Journal of the ACM, 26(2):259-270, Apr 1979.
28. D. Georgakopoulos, M. F. Hornick, and A. P. Sheth. An Overview of Workflow Management: From Process Modeling to Workflow Automation Infrastructure. Distributed and Parallel Databases, 3(2):119-153, 1995.
29. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP'03), Bolton Landing (Lake George), NY, Oct 2003.
30. I. Goldberg, D. Wagner, R. Thomas, and E. Brewer. A Secure Environment for Untrusted Helper Applications. In Proceedings of the Sixth USENIX Security Symposium, July 1996.
31. S. D. Gribble, E. A. Brewer, J. M. Hellerstein, and D. Culler. Scalable, Distributed Data Structures for Internet Service Construction. In Proceedings of the 4th Symposium on Operating Systems Design and Implementation (OSDI'00), San Diego, CA, Oct 2000.
32. A. Gupta and I. S. Mumick. Maintenance of Materialized Views: Problems, Techniques and Applications. IEEE Quarterly Bulletin on Data Engineering; Special Issue on Materialized Views and Data Warehousing, 18(2):3-18, 1995.
33. M. B. Jones. Interposition Agents: Transparently Interposing User Code at the System Interface. In Proceedings of the 14th ACM Symposium on Operating Systems Principles (SOSP'93), pages 80-93, Asheville, North Carolina, Dec 1993.
34. J. Kistler and M. Satyanarayanan. Disconnected Operation in the Coda File System. ACM Transactions on Computer Systems, 10(1), Feb 1992.
35. J. Kubiatowicz, D. Bindel, P. Eaton, Y. Chen, D. Geels, R. Gummadi, S. Rhea, W. Weimer, C. Wells, H. Weatherspoon, and B. Zhao. OceanStore: An Architecture for Global-Scale Persistent Storage. In Proceedings of the 9th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS IX), pages 190-201, Cambridge, MA, Nov 2000.
36. T. L. Lancaster. The Renderman Web Site. http://www.renderman.org/, 2002.
37. W. Litwin, M.-A. Neimat, and D. Schneider. RP*: A Family of Order Preserving Scalable Distributed Data Structures. In Proceedings of the 20th International Conference on Very Large Databases (VLDB 20), pages 342-353, Santiago, Chile, Sep 1994.
38. M. J. Litzkow, M. Livny, and M. W. Mutka. Condor - A Hunter of Idle Workstations. In Proceedings of ACM Computer Network Performance Symposium, pages 104-111, June 1988.
39. A. Muthitacharoen, R. Morris, T. M. Gil, and B. Chen. Ivy: A Read/Write Peer-to-Peer File System. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02), Boston, MA, Dec 2002.
40. S. Osman, D. Subhraveti, G. Su, and J. Nieh. The Design and Implementation of Zap: A System for Migrating Computing Environments. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02), Boston, MA, Dec 2002.
41. J. K. Ousterhout, H. D. Costa, D. Harrison, J. A. Kunze, M. Kupfer, and J. G. Thompson. A Trace-Driven Analysis of the UNIX 4.2 BSD File System. In Proceedings of the 10th ACM Symposium on Operating System Principles (SOSP'85), pages 15-24, Orcas Island, WA, Dec 1985.
42. Platform Computing. Improving Business Capacity with Distributed Computing. www.platform.com/industry/financial/, 2003.
43. R. Raman. Matchmaking Frameworks for Distributed Resource Management. PhD thesis, University of Wisconsin-Madison, Oct 2000.
44. L. Rizzo. Dummynet: A Simple Approach to the Evaluation of Network Protocols. ACM Computer Communication Review, 27(1):31-41, 1997.
45. D. Roselli, J. R. Lorch, and T. E. Anderson. A Comparison of File System Workloads. In Proceedings of the USENIX Annual Technical Conference (USENIX'00), pages 41-54, San Diego, CA, June 2000.
46. A. Rowstron and P. Druschel. Storage Management and Caching in PAST, A Large-scale, Persistent Peer-to-peer Storage Utility. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP'01), Banff, Canada, Oct 2001.
47. M. Rusinkiewicz and A. P. Sheth. Specification and Execution of Transactional Workflows. In Modern Database Systems: The Object Model, Interoperability, and Beyond., pages 592-620. 1995.
48. Y. Saito, C. Karamanolis, M. Karlsson, and M. Mahalingam. Taming Aggressive Replication in the Pangaea Wide-area File System. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02), Boston, MA, Dec 2002.
49. C. P. Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M. S. Lam, and M. Rosenblum. Optimizing the Migration of Virtual Computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02), Boston, MA, Dec 2002.
50. M. Satyanarayanan. A Study of File Sizes and Functional Lifetimes. In Proceedings of the 8th ACM Symposium on Operating Systems Principles (SOSP'81), pages 96-108, Pacific Grove, CA, Dec 1981.
51. M. I. Seltzer, Y. Endo, C. Small, and K. A. Smith. Dealing With Disaster: Surviving Misbehaved Kernel Extensions. In Proceedings of the 2nd Symposium on Operating Systems Design and Implementation (OSDI'96), pages 213-228, Seattle, WA, Oct 1996.
52. S. Soderbergh. Mac, Lies, and Videotape. www.apple.com/hotnews/articles/2002/04/fullfrontal/, 2002.
53. W. T. Sullivan, D. Werthimer, S. Bowyer, J. Cobb, D. Gedye,, and D. Anderson. A New Major SETI Project based on Project Serendip Data and 100,000 Personal Computers. In Proceedings of the 5th International Conference on Bioastronomy, 1997.
54. D. Thain, J. Bent, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and M. Livny. Pipeline and Batch Sharing in Grid Workloads. In Proceedings of High-Performance Distributed Computing (HPDC-12), pages 152-161, Seattle, WA, June 2003.
55. D. Thain and M. Livny. Parrot: Transparent User-Level Middleware for Data-Intensive Computing. In Workshop on Adaptive Grid Middleware, New Orleans, Louisiana, Sep 2003.
56. A. Vahdat and T. E. Anderson. Transparent Result Caching. In Proceedings of the USENIX Annual Technical Conference (USENIX'98), New Orleans, Louisiana, June 1998.
57. W. Vogels. File System Usage in Windows NT 4.0. In Proceedings of the 17th ACM Symposium on Operating Systems Principles (SOSP'99), pages 93-109, Kiawah Island Resort, South Carolina, Dec 1999.