Correlation based file prefetching approach for Hadoop

Proceedings - 2nd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2010

Volumn , Issue , 2010, Pages 41-48

  Dong, Bo ^a   Zhong, Xiao ^b   Zheng, Qinghua ^a   Jian, Lirong ^b   Liu, Jian ^a   Qiu, Jie ^b   Li, Ying ^b  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b IBM CHINA RESEARCH LAB   (China)

Author keywords

Cloud storage; File correlation; Hadoop distributed file system; Prefetching

Indexed keywords

ACCESS LATENCY; ACCESS MECHANISM; COURSEWARE; DISTRIBUTED FILE SYSTEMS; DYNAMIC REPLICA; E-LEARNING RESOURCES; FILE CORRELATION; FILE PREFETCHING; HADOOP DISTRIBUTED FILE SYSTEM; INTERNET APPLICATION; LEVEL CORRELATION; PREFETCHING; SELECTION ALGORITHM; STORAGE CAPABILITY;

CLOUD COMPUTING; COMPUTER SYSTEMS; CORRELATION METHODS; E-LEARNING; FILE ORGANIZATION; INTERNET; TELECOMMUNICATION NETWORKS;

ELECTRONIC DOCUMENT EXCHANGE;

EID: 79952379409     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CloudCom.2010.60     Document Type: Conference Paper

References (21)

1. Wittawat Tantisiriroj, Swapnil Patil, and Garth Gibson, "Data-intensive file systems for internet services: A rose by any other name....", Tech. Report CMU-PDL-08-114, Carnegie Mellon University, October 2008.
2. Bo Dong, Qinghua Zheng, Mu Qiao, Jian Shu, and Jie Yang, "BlueSky Cloud Framework: an e-Learning Framework Embracing Cloud Computing", Proc. of the 1st Conf. on Cloud Computing (CloudCom 09), Lecture Notes in Computer Science 5931, Spring Press, Beijing, China, Dec. 2009, pp. 577-582.
3. Gokul Soundararajan, Madalin Mihailescu, and Cristiana Amza, "Context-aware prefetching at the storage server", Proc. of the 2008 USENIX Annual Tech. Conf. (USENIX 08), USENIX Association Press, Berkeley, CA, USA, June 2008, pp. 377-390.
4. James Hendricks, Raja R. Sambasivan, Shafeeq Sinnamohideenand, and Gregory R. Ganger, "Improving small file performance in object-based storage", Tech. Report CMU-PDL-06-104, Carnegie Mellon University, May 2006.
5. E. Shriver, C. Small, and K. A. Smith, "Why Does File System Prefetching Work?", Proc. of the 1999 USENIX Annual Tech. Conf. (USENIX 99), USENIX Association Press, Monterey, CA, USA, June 1999, pp. 71-84.
6. Hadoop Rack Awareness, http://hadoop.apache.org/common/docs/r0.19.1/ cluster-setup.html#Hadoop+Rack+Awareness.
7. HDFS Architecture, http://hadoop.apache.org/common/docs/r0.20.2/hdfs- design.html.
8. Zhe Zhang, Amit Kulkarni, Xiaosong Ma, and Yuanyuan Zhou, "Memory resource allocation for file system prefetching: from a supply chain management perspective", Proc. of the 4th ACM European conf. on Computer systems (EuroSys 09), ACM Press, Nuremberg, Germany, April 2009, pp. 75-88.
9. Tom White, Hadoop: The Definitive Guide, O'Reilly Media, Inc., June 2009.
10. Tom White, "The Small Files Problem", http://www.cloudera.com/ blog/2009/02/02/the-small-files-problem/.
11. Mingju Li, Elizabeth Varki, Swapnil Bhatia, and Arif Merchant, "TaP: Table-based Prefetching for Storage Caches", Proc. of the 6th USENIX Conf. on File and Storage Technologies (FAST 08), USENIX Association Press, San Jose, USA, Feb. 2008, pp. 81-96.
12. Hadoop jira, http://issues.apache.org/jira/browse/HADOOP-1687.
13. Peng Xia, Dan Feng, Hong Jiang, Lei Tian, and Fang Wang, "FARMER: a novel approach to file access correlation mining and evaluation reference model for optimizing peta-scale file system performance", Proc. of the 17th ACM Symp. on High performance distributed computing (HPDC 08), ACM Press, Boston, MA, USA, June 2008, pp.185-196.
14. Frank Schmuck and Roger Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters", Proc. of the 1st USENIX Conf. on File and Storage Technologies (FAST 02), USENIX Association Press, Monterey, CA, USA, Jan. 2002, pp. 231-244.
15. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, "The Google File System", Proc. of the 19th ACM Symp. on Operating Systems Principles (SOSP 03), ACM Press, Lake George, New York, USA, Oct. 2003, pp. 29-43.
16. Sage Weil, Scott Brandt, Ethan Miller, Darrell Long, and Carlos Maltzahn, "Ceph: A Scalable, High-Performance Distributed File System", Proc. of the 7th Conf. on Operating Systems Design and Implementation (OSDI 06), USENIX Association Press, Seattle, Washington, USA, Nov. 2006, pp. 307-320.
17. Binny S. Gill and Dharmendra S. Modha, "SARC: Sequential prefetching in adaptive replacement cache", Proc. of the 2005 USENIX Annual Tech. Conf. (USENIX 2005), USENIX Association Press, Anaheim, CA, USA, April 2005, pp. 293-308.
18. Binny S. Gill, and Luis Angel D. Bathen, "AMP: Adaptive Multistream Prefetching in a Shared Cache", Proc. of the 5th USENIX Conf. on File and Storage Technologies (FAST 07), USENIX Association Press, San Jose, CA, USA, Feb. 2007, pp. 185-198.
19. STEERE, D. C, "Using dynamic sets to reduce the aggregate latency of data access", Ph.D. thesis, CMU-CS-96-194, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, 1997.
20. J. Griffioen and R. Appleton, "Reducing file system latency using a predictive approach", Proc. of the 1994 Summer USENIX Tech. Conf. (USENIX 94), USENIX Association Press, Boston, Ma, USA. June 1994, pp. 185-198.
21. D. K. Gifford, P. Jouvelot, M. A. Sheldon, and J. W. O'Toole, "Semantic file systems", Proc. of the 13th ACM Symp. on Operating Systems Principles (SOSP 91), ACM Press, Pacific Grove, CA, USA, Oct. 1991, pp. 16-25.