HPMR: Prefetching and pre-shuffling in shared MapReduce computation environment

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2009, Pages

  Seo, Sangwon ^a   Jang, Ingook ^a   Woo, Kyungchang ^b   Kim, Inkyo ^c   Kim, Jin Soo ^d   Maeng, Seungryoul ^a  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b YAHOO INC   (United States)

^c Samsung Advanced Institute of Technology (SAIT)   (South Korea)

^d Sungkyunkwan University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

CLUSTER NODES; COMPUTING RESOURCE; DATA SETS; DATA-INTENSIVE APPLICATION; DISTRIBUTED AND PARALLEL PROCESSING; EXECUTION TIME; FACEBOOK; GRID PLATFORM; MACHINE-LEARNING; MULTIPLE USER; OPEN SOURCE IMPLEMENTATION; OPTIMIZATION SCHEME; PLUG-INS; PREFETCHING; PROGRAMMING MODELS; SCIENTIFIC SIMULATIONS; SEARCH LOGS; SINGLE USERS; SYSTEM UTILIZATION; TEST SETS;

CLUSTER COMPUTING; COMPUTER SCIENCE; SIMULATORS; TECHNICAL PRESENTATIONS;

INTERNET;

EID: 72049096376     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTR.2009.5289171     Document Type: Conference Paper

References (19)

1. J. Dean and S. Ghemawat. MapReduce:Simplified Data Processing on Large Clusters. In the Proceedings of the 6th Symposium on Operating Systems Design and Implementation, Dec. 2004.
2. S. Ghemawat, H. Gobioff, et al. The Google file system. In the Proceedings of 19th Symposium on Operating Systems Principles, Oct. 2003.
3. Hadoop: http://hadoop.apache.org/
4. Hadoop On Demand: http://hadoop.apache.org/core/docs/r0.18.3/hod.html
5. HAMA: http://incubator.apache.org/hama/
6. E. Nightingale, P. Chen, et al. Speculative execution in a distributed file system. ACM Transactions on Computer Systems, Nov. 2006.
7. C. Olston, B. Reed, et al. Pig Latin: A Not-So-Foreign Language for Data Processing. In the Proceedings of the ACM SIGMOD international conference on Management of data, June 2008.
8. M. Zaharia, A. Konwinski, et al. Improving MapReduce Performance in Heterogeneous Environments. In the Proceedings of the 8th Symposium on Operating Systems Design and Implementation, Dec. 2008.
9. M. Isard, M. Budiu, et al. Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, Mar. 2007.
10. R. Lammel. Googles MapReduce Programming Model. Revisited. Science of Computer Programming, July 2007.
11. H. Yang, A. Dasdan, et al. Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters. In the Proceedings of the ACM SIGMOD international conference on Management of data, June 2007.
12. M. Burrows, The Chubby lock service for loosely-coupled distributed systems. In the Proceedings of In the Proceedings of the 7th Symposium on Operating Systems Design and Implementation, Nov. 2006.
13. G. Blelloch. Programming parallel algorithms. In Communications of the ACM, Mar. 1996.
14. A. Fox, S. D. Gribble, et al. Cluster-based scalable network services. In ACM SIGOPS Operating Systems Review, Dec. 1997.
15. J. Larus and M. Parkes. Using cohort scheduling to enhance server performance. In the Proceedings of the USENIX Annual Technical Conference, June 2002.
16. V. Padmanabhan and J. Mogul. Using predictive prefetching to improve world wide web latency. In Proceedings of the ACM SIGCOMM Conference, July 1996.
17. J. Vitter and P. Krishnan. Optimal prefetching via data compression. Journal of the ACM, Sep. 1996.
18. T. Kroeger and D. Long. Design and Implementation of a Predictive File Prefetching Algorithm. In the Proceedings of the USENIX Annual Technical Conference. June 2001.
19. P. Cao, E. Felten, et al. A study of integrated prefetching and caching strategies. In the Proceedings of the SIGMETRICS Conference, May 1995.