MapReduce with communication overlap (MaRCO)

Journal of Parallel and Distributed Computing

Volumn 73, Issue 5, 2013, Pages 608-620

  Ahmad, Faraz ^a   Lee, Seyong ^b   Thottethodi, Mithuna ^a   Vijaykumar, T N ^a  

^a Purdue University   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Cloud computing; Distributed processing; Large scale data processing; MapReduce; Parallel computing; Performance optimization

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLOUD COMPUTING; CLUSTER COMPUTING; CLUSTERING ALGORITHMS; DATA HANDLING; DATA MINING; DISTRIBUTED COMPUTER SYSTEMS; FAULT TOLERANCE; INFORMATION MANAGEMENT; LEARNING SYSTEMS; MULTIPROCESSING SYSTEMS; PARALLEL PROCESSING SYSTEMS; WEB CRAWLER;

BISECTION BANDWIDTH; COMMUNICATION OVERLAP; DISTRIBUTED PROCESSING; LARGE AMOUNTS OF DATA; LARGE-SCALE DATA PROCESSING; MAP-REDUCE; PERFORMANCE OPTIMIZATIONS; PROGRAMMING MODELS;

SEARCH ENGINES;

EID: 84893670179     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2012.12.012     Document Type: Article

References (30)

1. J. Aczél, W. Ertel, A new formula for speedup and its characterization, Acta Informatica 34 (8) (1997) 637-652.
2. R. Agrawal, R. Srikant, Fast algorithms for mining association rules, in: Proc. of 20th VLDB, vol. 1215, 1994, pp. 487-499.
3. F. Ahmad, S. Chakradhar, A. Raghunathan, T.N. Vijaykumar, Tarazu: optimizing MapReduce on heterogeneous clusters, in: ASPLOS XVII: Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, March, 2012.
4. A. Alameldeen, D. Wood, Variability in architectural simulations of multithreaded workloads, in: Proc. of HPCA-9, 2003.
5. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, E. Harris, Reining in the outliers in MapReduce clusters using Mantri, in: Proceedings of the Usenix Symposium on Operating Systems Design and Implementation, OSDI, 2010.
6. A.C. Arpaci-Dusseau, R.H. Arpaci-Dusseau, D.E. Culler, J.M. Hellerstein, D.A. Patterson, High-performance sorting on networks of workstations, in: Proc. 1997 SIGMOD, Tucson, Arizona, May 1997, pp. 243-254.
7. L. Barroso, J. Dean, U. Holzle, Web search for a planet: the Google cluster architecture, in: Micro, IEEE, 2003.
8. J. Bent, D. Thain, A. Arpaci-Dusseau, R. Arpaci-Dusseau, Explicit control in a batch-aware distributed file system, in: Proc. First NSDI, March, 2004.
9. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, K. Olukotun, Map-reduce for machine learning on multicore, in: Proc. 20th NIPS, December 2006.
10. T. Condie, N. Conway, P. Alvaro, J.M. Hellerstein, K. Elmleegy, R. Sears, MapReduce online, in: NSDI, 2010.
11. J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, in: Proc. OSDI'04, 2004.
12. Facebook hive. http://hadoop.apache.org/hive.
13. S. Ghemawat, H. Gobioff, S. Leung, The Google file system, ACM SIGOPS Operating Systems Review 37 (5) (2003) 29-43.
14. Hadoop. http://lucene.apache.org/hadoop/.
15. J. Hartigan, Clustering Algorithms, Wiley, 1975.
16. B. He, W. Fang, Q. Luo, N.K. Govindaraju, T. Wang, Mars: a MapReduce framework on graphics processors, in: PACT'08: Proc. of the 17th PACT, 2008, pp. 260-269.
17. https://issues.apache.org/jira/browse/HADOOP-3226.
18. M. Isard, M. Budiu, Y. Yu, A. Birrell, D. Fetterly, Dryad: distributed data-parallel programs from sequential building blocks, in: Proc. EuroSys'07, 2007.
19. V. Kumar, A. Grama, A. Gupta, G. Karypis, Introduction to Parallel Computing: Design and Analysis of Algorithms, 1994.
20. M.D. Linderman, J.D. Collins, H. Wang, T.H. Meng, Merge: a programming model for heterogeneous multi-core systems, in: ASPLOS XIII: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, 2008.
21. Netflix movies data. http://www.netflixprize.com/download.
22. A. Pavlo, E. Paulson, A. Rasin, D. Abadi, D. DeWitt, S. Madden, M. Stonebraker, A comparison of approaches to large-scale data analysis, in: Proceedings of the 2009 SIGMOD International Conference on Management of Data, 2009.
23. R. Pike, Interpreting the data: parallel analysis with Sawzall, Scientific Programming 13 (4) (2005) 277-298.
24. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, C. Kozyrakis, Evaluating MapReduce for multi-core and multiprocessor systems, in: IEEE 13th International Symposium on High Performance Computer Architecture, 2007. HPCA 2007, 2007, pp. 13-24.
25. The Hadoop distributed file system: architecture and design. http://lucene.apache.org/hadoop/hdfs-design.html.
26. A. Verma, N. Zea, B. Cho, I. Gupta, R. Campbell, Breaking the MapReduce stage barrier, in: CLUSTER 2010, Heraklion, Crete, Greece, 2010, pp. 235-244.
27. D. Weld, Lecture notes on MapReduce (based on Jeff Dean's slides). http://rakaposhi.eas.asu.edu/cse494/notes/s07-map-reduce.ppt.
28. H.-C. Yang, A. Dasdan, R.-L. Hsiao, D.S. Parker, Map-reduce-merge: simplified relational data processing on large clusters, in: SIGMOD'07: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, 2007.
29. Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P.K. Gunda, J. Currey, Dryadlinq: a system for general-purpose distributed data-parallel computing using a high-level language, in: Proceedings of International Symposium on Operating System Design and Implementation, OSDI, 2008.
30. M. Zaharia, A. Konwinski, A. Joseph, R. Katz, I. Stoica, Improving MapReduce performance in heterogeneous environments, in: Proceedings of OSDI 2008.