Accelerating MapReduce with distributed memory cache

Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS

Volumn , Issue , 2009, Pages 472-478

  Zhang, Shubin ^a,b   Han, Jizhong ^a   Liu, Zhiyong ^a   Wang, Kai ^a,b   Feng, Shengzhong ^c  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

Author keywords

Cluster computing; Distributed memory cache; High speed access; MapReduce

Indexed keywords

DISTRIBUTED MEMORY; HIGH-SPEED ACCESS; MAP TASK; PARALLEL PROGRAM; PARALLEL PROGRAMMING MODEL; SMALL-SCALE CLUSTERS;

CLUSTER COMPUTING; COMPUTER SYSTEMS; PARALLEL ARCHITECTURES; PARALLEL PROGRAMMING;

CACHE MEMORY;

EID: 77949641022     PISSN: 15219097     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPADS.2009.88     Document Type: Conference Paper

References (18)

1. W. Gropp, E. Lusk, and A. Skjellum, Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press, Cambridge, MA, USA, 1999.
2. A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam, PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing. MIT Press, Cambridge, MA, USA, 1995.
3. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, 2008, pp. 107.
4. R. Lämmel, "Google's MapReduce Programming Model - Revisited," Science of Computer Programming, ELSEVIER, vol. 70, 2008, pp. 1-30.
5. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, "Improving mapreduce performance in heterogeneous environm-ents," OSDI, 2008, pp. 29-42.
6. R. Pike, S. Dorward, R. Griesemer, S. Quinlan. "Interpreting the Data: Parallel Analysis with Sawzall," Scientific Programming Journal, vol. 13, no. 4, Oct. 2005, pp. 227-298.
7. C. Olston, B. Reed, U. Srivastava, R. Kumar and A. Tomkins. "Pig Latin: A Not-So-Foreign Language for Data Processing," SIGMOD, June 2008.
8. Apache Software Foundation, Hive, http://hadoop.apache.org/ hive/.
9. Apache Applications powered by Hadoop, http://wiki.apache. org/hadoop/PoweredBy.
10. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, and K. Olukotun, "Map-Reduce for machine learning on multicore," NIPS, MIT Press, Cambridge, MA, 2007, pp. 281-288.
11. S. Zhang, J. Han, Z. Liu, K. Wang and Z. Xu, "SJMR: Parallelizing Spatial Join with MapReduce on Clusters," Cluster 09, New Orleans, Louisiana, USA, 2009.
12. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis, "Evaluating mapreduce for multi-core and multiprocessor systems," HPCA, 2007, pp. 13-24.
13. B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang, "Mars : a mapreduce framework on graphics processors," PACT, New York, NY, USA, 2008, pp. 260-269.
14. S. Ghemawat, H. Gobioff, and S.T. Leung, "The Google file system," ACM SIGOPS Operating Systems Review, vol. 37, no. 5, 2003, pp. 29-43.
15. A. Bialecki, M. Cafarella, D. Cutting, and O. O'Malley, "Hadoop: a framework for running applications on large clusters built of commodity hardware," Wiki at http://lucene .apache.org/hadoop.
16. S. Y. Ko, I. Hoque, B. Cho, and I. Gupta, "On Availability of Intermediate Data in Cloud Computations," Proc. 12th Usenix Workshop on Hot Topics in Operating Systems, 2009.
17. Danga Interactive, memcached, http://www.danga.com/mem cached/.
18. Alibaba, memcached client for java, http://code.google.com/p /memcache-client-forjava/.