Dynamic data redistribution for MapReduce joins

Proceedings - 2011 3rd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2011

Volumn , Issue , 2011, Pages 717-723

  Lynden, Steven ^a   Tanimura, Yusuke ^a   Kojima, Isao ^a   Matono, Akiyoshi ^a  

^a NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMIC DATA; INTERMEDIATE RESULTS; LARGE-SCALE DATASETS; MAP-REDUCE; MULTI-JOIN; PERFORMANCE ANALYSIS; PROGRAMMING PARADIGMS;

DATA TRANSFER;

CLOUD COMPUTING;

EID: 84857178178     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CloudCom.2011.111     Document Type: Conference Paper

References (17)

1. Apache Hive. http://hive.apache.org/.
2. Apache Pig. http://pig.apache.org/.
3. Apache ZooKeeper. http://hadoop.apache.org/zookeeper/.
4. HBase. http://hadoop.apache.org/hbase/.
5. F. N. Afrati and J. D. Ullman. Optimizing joins in a map-reduce environment. In Proceedings of the 13th International Conference on Extending Database Technology, EDBT '10, pages 99-110, New York, NY, USA, 2010. ACM.
6. S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian. A comparison of join algorithms for log processing in mapreduce. In Proceedings of the 2010 international conference on Management of data, SIGMOD '10, pages 975-986, New York, NY, USA, 2010. ACM.
7. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: a distributed storage system for structured data. In Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7, pages 15-15, Berkeley, CA, USA, 2006. USENIX Association.
8. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. Commun. ACM, 51:107-113, January 2008.
9. A. Gounaris, N. W. Paton, A. A. A. Fernandes, and R. Sakellariou. Adaptive Query Processing: A Survey. In BNCOD 19: Proceedings of the 19th British National Conference on Databases, pages 11-25, London, UK, 2002. Springer-Verlag.
10. Q. Li, M. Sha, V. Markl, K. Beyer, L. Colby, and G. Lohman. Adaptively Reordering Joins during Query Execution. In Proc. ICDE, pages 26-35. IEEE Computer Society, 2007. (Pubitemid 47422006)
11. S. J. Lynden, I. Kojima, A. Matono, and Y. Tanimura. Adaptive integration of distributed semantic web data. In S. Kikuchi, S. Sachdeva, and S. Bhalla, editors, DNIS, volume 5999 of Lecture Notes in Computer Science, pages 174-193. Springer, 2010.
12. J. Myung, J. Yeon, and S. Lee. Sparql basic graph pattern processing with iterative mapreduce. In Proceedings of the 2010 Workshop on Massive Data Analytics on the Cloud, MDAC '10, pages 6:1-6:6, New York, NY, USA, 2010. ACM.
13. P. Ravindra, V. V. Deshpande, and K. Anyanwu. Towards scalable RDF graph analytics on MapReduce. In Proceedings of the 2010 Workshop on Massive Data Analytics on the Cloud (MDAC), 2010.
14. M. Stillger, G. M. Lohman, V. Markl, and M. Kandil. Leo - db2's learning optimizer. In Proceedings of the 27th International Conference on Very Large Data Bases, VLDB '01, pages 19-28, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc.
15. Y. Tanimura, A. Matono, S. J. Lynden, and I. Kojima. Extensions to the pig data processing platform for scalable rdf data processing using hadoop. In ICDE Workshops, pages 251-256. IEEE, 2010.
16. J. Urbani, S. Kotoulas, E. Oren, and F. van Harmelen. Scalable Distributed Reasoning Using MapReduce. In Proceedings of the International Semantic Web Conference (ISWC), 2009.
17. World Wide Web Consortium (W3C) Recommendation. Resource Description Framework (RDF). http://www.w3.org/RDF/, 2004.