The Hadoop distributed file system

2010 IEEE 26th Symposium on Mass Storage Systems and Technologies, MSST2010

Volumn , Issue , 2010, Pages

  Shvachko, Konstantin ^a   Kuang, Hairong ^a   Radia, Sanjay ^a   Chansler, Robert ^a  

^a YAHOO INC   (United States)

Author keywords

Distributed file system; Hadoop; HDFS

Indexed keywords

DIGITAL STORAGE; INFORMATION MANAGEMENT;

APPLICATION TASKS; DISTRIBUTED FILE SYSTEMS; ENTERPRISE DATA; HADOOP; HADOOP DISTRIBUTED FILE SYSTEM (HDFS); HADOOP DISTRIBUTED FILE SYSTEMS; HDFS; VERY LARGE DATUM;

FILE ORGANIZATION;

EID: 77957838299     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MSST.2010.5496972     Document Type: Conference Paper

References (19)

1. Apache Hadoop. http://hadoop.apache.org/
2. P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur. "PVFS: A parallel file system for Linux clusters," in Proc. of 4th Annual Linux Showcase and Conference, 2000, pp. 317-327.
3. J. Dean, S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," In Proc. of the 6th Symposium on Operating Systems Design and Implementation, San Francisco CA, Dec. 2004.
4. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam, C. Olston, B. Reed, S. Srinivasan, U. Srivastava. "Building a High-Level Dataflow System on top of MapReduce: The Pig Experience," In Proc. of Very Large Data Bases, vol 2 no. 2, 2009, pp. 1414-1425
5. S. Ghemawat, H. Gobioff, S. Leung. "The Google file system," In Proc. of ACM Symposium on Operating Systems Principles, Lake George, NY, Oct 2003, pp 29-43.
6. F. P. Junqueira, B. C. Reed. "The life and times of a zookeeper," In Proc. of the 28th ACM Symposium on Principles of Distributed Computing, Calgary, AB, Canada, August 10-12, 2009.
7. Lustre File System. http://www.lustre.org
8. M. K. McKusick, S. Quinlan. "GFS: Evolution on Fast-forward," ACM Queue, vol. 7, no. 7, New York, NY. August 2009.
9. O. O'Malley, A. C. Murthy. Hadoop Sorts a Petabyte in 16.25 Hours and a Terabyte in 62 Seconds. May 2009. http://developer.yahoo.net/blogs/hadoop/2009/ 05/hadoop-sorts-a-petabyte-in-162.html
10. R. Pike, D. Presotto, K. Thompson, H. Trickey, P. Winterbottom, "Use of Name Spaces in Plan9," Operating Systems Review, 27(2), April 1993, pages 72-76.
11. S. Radia, "Naming Policies in the spring system," In Proc. of 1st IEEE Workshop on Services in Distributed and Networked Environments, June 1994, pp. 164-171.
12. S. Radia, J. Pachl, "The Per-Process View of Naming and Remote Execution," IEEE Parallel and Distributed Technology, vol. 1, no. 3, August 1993, pp. 71-80.
13. K. V. Shvachko, "HDFS Scalability: The limits to growth," ;login:. April 2010, pp. 6-16.
14. W. Tantisiriroj, S. Patil, G. Gibson. "Data-intensive file systems for Internet services: A rose by any other name ..." Technical Report CMU-PDL-08-114, Parallel Data Laboratory, Carnegie Mellon University, Pittsburgh, PA, October 2008.
15. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, R. Murthy, "Hive - A Warehousing Solution Over a Map-Reduce Framework," In Proc. of Very Large Data Bases, vol. 2 no. 2, August 2009, pp. 1626-1629.
16. J. Venner, Pro Hadoop. Apress, June 22, 2009.
17. th Symposium on Operating Systems Design and Implementation, Seattle, WA, November 2006.
18. th USENIX Conference on File and Storage Technologies, San Jose, CA, February 2008
19. T. White, Hadoop: The Definitive Guide. O'Reilly Media, Yahoo! Press, June 5, 2009.