A Dynamic Data Placement Strategy for Hadoop in Heterogeneous Environments

Big Data Research

Volumn 1, Issue , 2014, Pages 14-22

  Lee, Chia Wei ^a   Hsieh, Kuang Yu ^a   Hsieh, Sun Yuan ^a   Hsiao, Hung Chang ^a  

^a NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

Data placement; Hadoop; Heterogeneous; MapReduce

Indexed keywords

CLOUD COMPUTING; CLUSTER COMPUTING; DATA MINING; DATA TRANSFER; OPEN SOURCE SOFTWARE;

CLOUD-COMPUTING; COMPUTING CAPACITY; DATA PLACEMENT; DYNAMIC DATA; HADOOP; HETEROGENEOUS; HETEROGENEOUS ENVIRONMENTS; MAP-REDUCE; PERFORMANCE; PLACEMENT STRATEGY;

MAPREDUCE;

EID: 84918796928     PISSN: None     EISSN: 22145796     Source Type: Journal    
DOI: 10.1016/j.bdr.2014.07.002     Document Type: Article

References (15)

1. Amazon Elastic MapReduce, . http://aws.amazon.com/elasticmapreduce/.
2. Apache, . http://httpd.apache.org/.
3. Hadoop, . http://hadoop.apache.org/.
4. Hadoop Distributed File System, . http://hadoop.apache.org/docs/stable/hdfs_design.html.
5. Hadoop MapReduce, . http://hadoop.apache.org/docs/stable/mapred_tutorial.html.
6. Hadoop Yahoo, . http://www.ithome.com.tw/itadm/article.php?c=49410&s=4.
7. Borthakur D., Muthukkaruppan K., Ranganathan K., Rash S., Sarma J.-S., Spiegelberg N., Molkov D., Schmidt R., Gray J., Kuang H., Menon A., Aiyer A. Apache Hadoop goes realtime at Facebook. SIGMOD '11 2011.
8. Chang F., Dean J., Ghemawat S., Hsieh W.-C., Wallach D.A., Burrows M., Chandra T., Fiker A., Gruber R.E. BigTable: a distributed storage system for structured data. 7th USENIX Symposium on Operating Systems Design and Implementation 2006, 205-218.
9. Chen Q., Zhang D., Guo M., Deng Q., Guo S. SAMR: a self-adaptive MapReduce scheduling algorithm in heterogeneous environment. 2010 IEEE 10th International Conference on Computer and Information Technology (CIT) 2010, 2736-2743. IEEE.
10. Dean J., Ghemawat S. MapReduce: simplified data processing on large clusters. OSDI '04 Dec. 2004, 137-150.
11. Ghemawat S., Gobioff H., Leung S.-T. The Google file system. Proc. SOSP 2003 2003, 29-43.
12. He B., Fang W., Luo Q., Govindaraju N., Wang T. Mars: a MapReduce framework on graphics processors. ACM 2008 2008, 260-269.
13. Lee G., Chun B.G., Katz R.H. Heterogeneity-aware resource allocation and scheduling in the cloud. Proceedings of the 3rd USENIX Workshop on Hot Topics in Cloud Computing, HotCloud, vol. 11 2011.
14. Tian C., Zhou H., He Y., Zha L. A dynamic MapReduce scheduler for heterogeneous workloads. Eighth International Conference on Grid and Cooperative Computing 2009, IEEE.
15. Zaharia M., Konwinski A., Joseph A.D., Katz R., Stoica I. Improving MapReduce performance in heterogeneous environments. Proc. OSDI, San Diego, CA December 2008, 29-42.