In search of an API for scalable file systems: Under the table or above it?

Workshop on Hot Topics in Cloud Computing, HotCloud 2009

Volumn , Issue , 2009, Pages

  Patil, Swapnil ^a   Gibson, Garth A ^a   Ganger, Gregory R ^a   Lopez, Julio ^a   Polte, Milo ^a   Tantisiroj, Wittawat ^a   Xiao, Lin ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD COMPUTING;

COMMON DENOMINATORS; DATA INTENSIVE; FILE SYSTEMS; FLEXIBLE QUERIES; LAYOUT INFORMATION; SCALABLE COMPUTING; SEARCH FUNCTIONALITY; STRUCTURED DATA;

FILE ORGANIZATION;

EID: 84884469245     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. Private Communication with Dave Presotto, Google Inc.
2. AMAZON-S3. Amazon Simple Storage Service (Amazon S3). http://www.amazon.com/s3.
3. BENT, J., GIBSON, G., GRIDER, G., MCCLELLAND, B., NOWOCZYNSKI, P., NUNEZ, J., POLTE, M., AND WINGATE, M. PLFS: A Checkpoint Filesystem for Parallel Applications. Tech. Rep. LA-UR 09-02117, LANL.
4. BURROWS, M. The Chubby lock service for loosely-coupled distributed systems. In 7th Symposium on Operating Systems Design and Implementation (OSDI'06).
5. CAFARELLA, M., CHANG, E., FIKES, A., HALEVY, A., HSIEH, W., LERNER, A., MADHAVAN, J., AND MUTHUKRISHNAN, S. Data Management Projects at Google. SIGMOD Record 37, 1 (Mar. 2008).
6. CHANG, F., DEAN, J., GHEMAWAT, S., HSIEH, W. C., WALLACH, D. A., BURROWS, M., CHANDRA, T., FIKES, A., AND GRUBER, R. Bigtable: A Distributed Storage System for Structured Data. In 7th Symposium on Operating Systems Design and Implementation (OSDI'06).
7. CIFS. Common Internet File Systems. http://www.samba.org/cifs/.
8. CUSTER, H. Inside the Windows NT File System. Microsoft Press, 1994.
9. DEAN, J., AND GHEMAWAT, S. MapReduce: Simplified Data Processing on Large Clusters. In 6th Symposium on Operating Systems Design and Implementation (OSDI'04).
10. DECANDIA, G., HASTORUN, D., JAMPANI, M., KAKULAPATI, G., LAKSHMAN, A., PILCHIN, A., SIVASUBRAMANIAN, S., VOSSHALL, P., AND VOGELS, W. Dynamo: Amazon's Highly Available Key-Value Store. In 21st ACM Symposium on Operating Systems Principles (SOSP'07).
11. DMAPI. Data Storage Management (XDSM) API Specification. http://www.opengroup.org/onlinepubs/9657099/.
12. GHEMAWAT, S., GOBIOFF, H., AND LUENG, S.-T. Google File System. In 19th ACM Symposium on Operating Systems Principles (SOSP'03).
13. HADOOP. Apache Hadoop Project. http://hadoop.apache.org/.
14. HAMMERBACHER, J. Global Information Platforms: Evolving the Data Warehouse. PDL SDI talk at CMU on April 9, 2009 in Pittsburgh PA.
15. HDFS. The Hadoop Distributed File System: Architecture and Design. http://hadoop.apache.org/core/docs/r0.16.4/hdfsdesign.html.
16. HECEWG. High End Computing Extensions Working Group - man readdirplus(). http://www.opengroup.org/platform/hecewg/.
17. IETF. NFS v4.1 specifications. http://tools.ietf.org/wg/nfsv4/.
18. ISARD, M., BUDIU, M., YU, Y., BIRRELL, A., AND FETTERLY, D. Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. In 2007 EuroSys Conference.
19. KICZALES, G. Towards a New Model of Abstraction in the Engineering of Software. In the IMSA Workshop on Reflection and Meta-level Architectures (1992).
20. LEUNG, A., SHAO, M., BISSON, T., PASUPATHY, S., AND MILLER, E. L. Spyglass: Fast, Scalable Metadata Search for Large-Scale Storage Systems. In FAST'09 Conference on File and Storage Technologies.
21. LUSTRE. Lustre File System. http://www.lustre.org.
22. MATHUR, A., CAO, M., BHATTACHARYA, S., DILGER, A., TOMAS, A., AND VIVIER, L. The new ext4 filesystem: current status and future plans. In Ottawa Linux Symposium 2007.
23. MCKUSICK, M. K., JOY, W. N., LEFFLER, S. J., AND FABRY, R. S. A Fast File System for UNIX. ACM Transactions on Computer Systems 2, 3 (Aug. 1984).
24. NATURE. Big Data. http://www.nature.com/news/specials/bigdata/.
25. PATIL, S. V., AND GIBSON, G. GIGA+: Scalable Directories for Shared File Systems. Tech. Rep. CMU-PDL-08-108, Carnegie Mellon University, Oct. 2008.
26. PVFS2. Parallel Virtual File System, Version 2. http://www.pvfs2.org.
27. SANDBERG, R., GOLDBERG, D., KLEIMAN, S., WALSH, D., AND LYON, B. Design and Implementation of the Sun Network Filesystem. In Summer USENIX Conference'96.
28. SCHMUCK, F., AND HASKIN, R. GPFS: A Shared-Disk File System for Large Computing Clusters. In FAST'02 Conference on File and Storage Technologies.
29. STONEBRAKER, M., BECLA, J., DEWITT, D., LIM, K.-T., MAIER, D., RATZESBERGER, O., AND ZDONIK, S. Requirements for Science Data Bases and SciDB. In 4th Biennial Conference on Innovative Data Systems Research (CIDR'09).
30. STONEBRAKER, M., AND ÇETINTEMEL, U. One Size Fits All": An Idea Whose Time Has Come and Gone (Abstract). In 21st International Conference on Data Engineering (ICDE'05).
31. SWEENEY, A., DOUCETTE, D., HU, W., ANDERSON, C., NISHIMOTO, M., AND PECK, G. Scalability in the XFS File System. In USENIX Conference'96.
32. TANTISIROJ, W., PATIL, S. V., AND GIBSON, G. Data-intensive file systems for Internet services: A rose by any other name.... Tech. Rep. CMU-PDL-08-114, Carnegie Mellon University, Oct. 2008.
33. VOGELS, W. Eventually Consistent. In ACM Queue, 6(6) (Oct. 2008).
34. WELCH, B., UNANGST, M., ABBASI, Z., GIBSON, G., MUELLER, B., SMALL, J., ZELENKA, J., AND ZHOU, B. Scalable Performance of the Panasas Parallel File System. In FAST'08 Conference on File and Storage Technologies.
35. WINFS. Microsoft WinFS. http://www.microsoft.com.