Performance management of accelerated MapReduce workloads in heterogeneous clusters

Proceedings of the International Conference on Parallel Processing

Volumn , Issue , 2010, Pages 653-662

  Polo, Jordà ^a   Carrera, David ^a   Becerra, Yolanda ^a   Beltran, Vicenç ^a   Torres, Jordi ^a   Ayguadé, Eduard ^a  

^a BARCELONA SUPERCOMPUTING CENTER   (Spain)

Author keywords

Accelerators; Heterogeneity; MapReduce; Performance management; Scheduling

Indexed keywords

ACCELERATORS; CLUSTER PERFORMANCE; CO-LOCATED; COMPLETION TIME; DATA CENTERS; DISTRIBUTED SERVICE; DYNAMIC RESOURCE ALLOCATIONS; ENERGY CONSUMPTION; EXTREME CONDITIONS; HETEROGENEITY; HETEROGENEOUS CLUSTERS; MAP-REDUCE; PERFORMANCE MANAGEMENT; PROGRAMMING MODELS; SPECIFIC HARDWARE; WORKLOAD CONSOLIDATION;

COMPUTER RESOURCE MANAGEMENT; ELECTRIC GROUNDING; ENERGY UTILIZATION; HYBRID COMPUTERS; HYBRID SYSTEMS; MANAGEMENT;

SCHEDULING;

EID: 78649552787     PISSN: 01903918     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPP.2010.73     Document Type: Conference Paper

References (22)

1. NASA Nebula project. URL http://nebula.nasa.gov
2. J. Dean, S. Ghemawat, MapReduce: Simplified data processing on large clusters, in: OSDI '04: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, 2004, pp. 137-150.
3. J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguadé, M. Steinder, I. Whaley, Performance-driven task co-scheduling for mapreduce environments, in: NOMS '10: Proceedings of the 12th IEEE/IFIP Network Operations and Management Symposium, IEEE, Osaka, Japan, 2010.
4. Los Alamos National Laboratory, High-Performance Computing: Roadrunner. URL http://www.lanl.gov/roadrunner/
5. B.-G. Chun, G. Iannaccone, G. Iannaccone, R. Katz, G. Lee, L. Niccolini, An energy case for hybrid datacenters, in: Workshop on Power Aware Computing and Systems (HotPower'09), Big Sky, MT, USA, 2009, 2009.
6. M. Gschwind, P. Hofstee, B. Flachs, M. Hopkins, Y. Watanabe, T. Yamazaki, A novel simd architecture for the cell heterogeneous chipmultiprocessor, 2005.
7. Y. Becerra, V. Beltran, D. Carrera, M. Gonzalez, J. Torres, E. Ayguadé, Speeding up distributed mapreduce applications using hardware accelerators, in: ICPP '09: Proceedings of the 2009 International Conference on Parallel Processing, IEEE Computer Society, Washington, DC, USA, 2009, pp. 42-49.
8. K. Group, OpenCL (Open Computing Language) - The open standard for parallel programming of heterogeneous systems. URL http://www.khronos.org/ opencl/
9. Apache Software Foundation, Hadoop MapReduce Tutorial. URL http://hadoop.apache.org/mapreduce/
10. Apache Software Foundation, Hadoop Distributed File System (HDFS) Architecture. URL http://hadoop.apache.org/core/docs/current/hdfs-design.html
11. A. E. Eichenberger, K. O'Brien, K. O'Brien, P. Wu, T. Chen, P. H. Oden, D. A. Prener, J. C. Shepherd, B. So, Z. Sura, A. Wang, T. Zhang, P. Zhao, M. Gschwind, Optimizing compiler for the cell processor, in: PACT '05: Proceedings of the 14th International Conference on Parallel Architectures and Compilation Techniques, 2005, pp. 161-172.
12. Sun Microsystems, Inc, Java Native Interface. URL http://java.sun.com/ docs/books/jni
13. Adaptive Scheduler. Apache issue MAPREDUCE-1380. URL https://issues. apache.org/jira/browse/MAPREDUCE-1380
14. D. Carrera, M. Steinder, I. Whalley, J. Torres, E. Ayguadé, Enabling resource sharing between transactional and batch workloads using dynamic application placement, in: Middleware '08: Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware, 2008, pp. 203-222.
15. D. G. Feitelson, L. Rudolph, Parallel job scheduling: Issues and approaches, in: JSSPP, 1995, pp. 1-18.
16. P. R. Jelenkovic, X. Kang, J. Tan, Adaptive and scalable comparison scheduling, in: SIGMETRICS '07: Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, ACM, New York, NY, USA, 2007, pp. 215-226.
17. A. Wierman, M. Nuyens, Scheduling despite inexact job-size information, in: SIGMETRICS '08: Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, ACM, New York, NY, USA, 2008, pp. 25-36.
18. Apache Software Foundation, Hadoop on demand. URL http://hadoop.apache. org/core/docs/r0.20.0/hod-user-guide.html
19. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, I. Stoica, Job scheduling for multi-user MapReduce clusters, Tech. Rep. UCB/EECS-2009-55, EECS Department, University of California, Berkeley (Apr 2009).
20. T. Sandholm, K. Lai, MapReduce optimization using regulated dynamic prioritization, in: SIGMETRICS '09: Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems, ACM, New York, NY, USA, 2009, pp. 299-310.
21. N. Drosinos, N. Koziris, Performance comparison of pure mpi vs hybrid mpi-openmp parallelization models on smp clusters, in: Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS04), 2004.
22. C. Augonnet, S. Thibault, R. Namyst, P.-A. Wacrenier, StarPU: A Unified Platform for Task Scheduling on Heterogeneous Multicore Architectures, in: Proceedings of the 15th International Euro-Par Conference, Lecture Notes in Computer Science, Vol. 5704 of Lecture Notes in Computer Science, Springer, Delft, The Netherlands, 2009, pp. 863-874.