Efficient access to many small files in a filesystem for grid computing

Proceedings - IEEE/ACM International Workshop on Grid Computing

Volumn , Issue , 2007, Pages 243-250

  Thain, Douglas ^a   Moretti, Christopher ^a  

^a UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DIRECTORY TREES; END USERS; FILE SYSTEMS; GRID COMPUTING SYSTEMS; INTERNATIONAL CONFERENCES; ORDER-OF MAGNITUDES; ORDERS-OF-MAGNITUDE; PERFORMANCE IMPROVEMENTS; POTENTIAL USERS; STORAGE STRUCTURES; SYSTEM CALLS; TO MANY;

ARSENIC COMPOUNDS; COMPUTER SYSTEMS;

GRID COMPUTING;

EID: 47249140783     PISSN: 15505510     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/GRID.2007.4354139     Document Type: Conference Paper

References (33)

1. W. Allcock, A. Chervenak, I. Foster, C. Kesselman, and S. Tuecke, "Protocols and services for distributed data-intensive science," in Proceedings of Advanced Computing and Analysis Techniques in Physics Research, 2000, pp. 161-163.
2. C. Baru, R. Moore, A. Rajasekar, and M. Wan, "The SDSC storage resource broker," in Proceedings of CASCON, Toronto, Canada, 1998.
3. M. Beynon, R. Ferreira, T. Kure, A. Sussman, and J. Saltz, "Middleware for filtering very large scientific datasets on archival storage systems," in IEEE Symposium on Mass Storage Systems, 2000.
4. S. Vazhkudai, X. Ma, V. Freeh, J. Strickland, N. Tammineedi, and S. Scott, "Freeloader:scavenging desktop storage resources for scientific data," in International Conference for High Performance Computing and Communications (Supercomputing), Seattle, Washington, November 2005.
5. M. Ernst, P. Fuhrmann, M. Gasthuber, T. Mkrtchyan, and C. Waldman, "dCache, a distributed storage data caching system," in Proceedings of Computing in High Energy Physics, Beijing, China, 2001.
6. O. Barring, J. Baud, and J. Durand, "CASTOR project status," in Proceedings of Computing in High Energy Physics, Padua, Italy, 2000.
7. D. A. Patterson, G. Gibson, and R. Katz, "A case for redundant arrays of inexpensive disks (RAID)," in ACM SIGMOD international conference on management of data, June 1988, pp. 109-116.
8. P. H. Carns, W. B. Ligon, R. B. Ross, and R. Thakur, "PVFS: A parallel file system for linux clusters," in Annual Linux Showcase and Conference, 2000.
9. Cluster File Systems, "Lustre: A scalable, high performance file system," white paper, November 2002.
10. E. Lee and C. Thekkath, "Petal: Distributed virtual disks," in Architectural Support for Programming Languages and Operating Systems, 1996.
11. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, "The globus striped GridFTP framework and server," in IEEE/ACM Supercomputing, November 2005.
12. T. Hacker, B. Noble, and B. Athey, "Improving throughput and maintaining fairness using parallel tcp," in Proceedings of INFOCOM, 2004.
13. R. Brightwell and L. Fisk, "Scalable parallel application launch on cplant," in IEEE/ACM Supercomputing, 2001.
14. S. Altschul, W. Gish, W. Miller, E. Myers, and D. Lipman, "Basic local alignment search tool," Journal of Molecular Biology, vol. 3, no. 215, pp. 403-410, Oct 1990.
15. J. Postel, "FTP: File transfer protocol specification," Internet Engineering Task Force Request for Comments (RFC) 765, June 1980.
16. R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon, "Design and implementation of the Sun network filesystem," in USENIX Summer Technical Conference, 1985, pp. 119-130.
17. M. Silberstein, M. Factor, and D. Lorenz, "DYNAMO: DirectorY, Net Archiver and MOver," in Third International Grid Computing Workshop, 2002.
18. J. Bresnahan, M. Link, R. Kettimuthu, D. Fraser, and I. Foster, "GridFTP Pipelining," in Teragrid Conference, 2007.
19. D. Thain and M. Livny, "Parrot: Transparent user-level middleware for data-intensive computing," in Proceedings of the Workshop on Adaptive Grid Middleware, New Orleans, September 2003.
20. D. Thain, S. Klous, J. Wozniak, P. Brenner, A. Striegel, and J. Izaguirre, "Separating abstractions from resources in a tactical storage system," in IEEE/ACM Supercomputing, November 2005.
21. D. Thain, "Operating system support for space allocation in grid storage," in IEEE Conference on Grid Computing, September 2006.
22. I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke, "A Security Architecture for Computational Grids," in Proceedings of the 5th ACM Conference on Computer and Communications Security, San Francisco, CA, November 1998, pp. 83-92.
23. J. Steiner, C. Neuman, and J. I. Schiller, "Kerberos: An authentication service for open network systems," in Proceedings of the USENIX Winter Technical Conference, 1988, pp. 191-200.
24. "Filesystem in user space," http://sourceforge.net/projects/ fuse.
25. D. Mazieres, "A toolkit for user-level file systems," in USENIX Annual Technical Conference, June 2001.
26. D. Gifford, P. Jouvelot, M. Sheldon, and J. O'Toole, "Semantic file systems," in ACM Symposium on Operating Systems Principles, October 1991.
27. A. Batsakis and R. Burns, "Cluster delegation: High-performance faulttolerant data sharing in nfs," in High Performance Distributed Computing, 2004.
28. D. Mazieres, M. Kaminsky, M. F. Kaashoek, and E. Witchel, "Separating key management from file system security," in Symposium on Operating Systems Principles, 1999, pp. 124-139.
29. S. V. Anastasiadis, R. G. Wickremesinghe, and J. S. Chase, "Lerna: An active storage framework for flexible data access and management," in High Performance Distributed Computing, 2005.
30. P. Honeyman, W. A. Adamson, and S. McKee, "GridNFS: Global storage for global collaboration," in Local to Global Data Interoperability, 2005.
31. M. Mesnier, G. Ganger, and E. Riedel, "Object based storage," IEEE Communications, vol. 41, no. 8, August 2003.
32. G. A. Gibson, D. Nagle, K. Amiri, F. W. Chang, E. M. Feinberg, H. Gobioff, C. Lee, B. Ozceri, E. Riedel, D. Rochberg, and J. Zelenka, "File server scaling with network-attached secure disks," in Measurement and Modeling of Computer Systems (SIGMETRICS), 1997.
33. E. Riedel, G. A. Gibson, and C. Faloutsos, "Active storage for large scale data mining and multimedia," in Very Large Databases (VLDB), 1998.