A Dispatching-rule-based task scheduling policy for mapreduce with multi-type jobs in heterogeneous environments

Proceedings - 7th ChinaGrid Annual Conference, ChinaGrid 2012

Volumn , Issue , 2012, Pages 17-24

  Gao, Xiang ^a   Chen, Qing ^a   Chen, Yurong ^a   Sun, Qingwei ^a   Liu, Yan ^b   Li, Mingzhu ^b  

^a CHINA MOBILE COMMUNICATIONS CORPORATION   (China)

^b SIEMENS LTD   (China)

Author keywords

Cloud computing; Distributed and parallel computing; Job shop scheduling problem; Mapreduce

Indexed keywords

COMPLETION TIME; COMPUTING NODES; DISTRIBUTED AND PARALLEL COMPUTING; EASE-OF-USE; ENTERPRISE DATA; ENTERPRISE IT; HETEROGENEOUS COMPUTING; HETEROGENEOUS ENVIRONMENTS; JOB SHOP SCHEDULING PROBLEMS; JOB-SHOP SCHEDULING; MAP-REDUCE; NUMERICAL RESULTS; ONLINE SCHEDULING; OPERATIONAL DATA; OPTIMIZATION PROBLEMS; PERFORMANCE GAIN; PROCESSING TIME; PROGRAMMING MODELS; SCHEDULING PROBLEM; TASK-SCHEDULING; TASK-SCHEDULING ALGORITHMS;

APPROXIMATION ALGORITHMS; CLOUD COMPUTING; DATA PROCESSING; INFORMATION MANAGEMENT; OPTIMIZATION;

SCHEDULING ALGORITHMS;

EID: 84870831749     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ChinaGrid.2012.27     Document Type: Conference Paper

References (16)

1. C. Pettey, Gartner Identifies the Top 10 Strategic Technologies for 2011, Gartner Symposium/ITxpo in Orlando, Oct. 2010.
2. F. Berman, G. Fox, A. J. G. Hey, Grid computing: making the global infrastructure a reality, Wiley Series in Communications Networking & Distributed Systems, 2003.
3. M. A. Rappa, The utility business model and the future of computing services, IBM Systems Journal, 53:1275-1280, Nov. 2004.
4. K. Asanovic, R. Bodik, B. C. Catanzaro, etc., The landscape of parallel computing research: a view from berkeley, Technical Report, UC Berkeley, Dec. 2006.
5. D. Thain, T. Tannenbaum, M. Livny, Distributed computing in practice: the Condor experience, Concurrency and Computation: Practice and Exprience, 17(2-4):323-358, Apr. 2005.
6. J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, Communications of the ACM, 51(1):107-113, Jan. 2008.
7. Apache Hadoop, http://hadoop.apache.org/.
8. M. Isard, M. Budiu, Y. Yu, A. Birrell, D. Fetterly, Dryad: distributed data-parallel programs from sequential building blocks, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, 41(3):59-72, Jun. 2007.
9. M. Zaharia, A. Konwinski, A. D.Joseph, R. Katz, I. Stoica, Improving MapReduce performance in heterogeneous environments, Proc. of 8th USENIX Symposium on Operating Systems Design and Implementation, 57(4):29-42, 2008.
10. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, EECS Department, University of California, Berkeley, Apr 2009.
11. C. Tian, H. Zhou, Y. He, L. Zha, A dynamic MapReduce scheduler for heterogeneous workloads, in Proc. of IEEE Eighth International Conference on Grid and Cooperative Computing, 2009.
12. J. Dhok, N. Maheshwari and V. Varma, Learning Based Opportunistic Admission Control Algorithm for MapReduce as a Service, Proc. of the 3rd India software engineering conference, 2010.
13. M. Pinedo, Scheduling - Theory, Agorithms and Systems, New Jersey : Prentice Hall, 1995.
14. M. Ghirardi and C. N. Potts, Makespan minimization for scheduling unrelated parallel machines: A recovering beam search approach, European Journal of Operational Research, 165(2):457-467, 2005.
15. L. F. Peyro and R. Ruiz, Iterated greedy local search methods for unrelated parallel machine scheduling, European Journal of Operational Research, vol. 207(1): 55-69, 2010.
16. A. Woclaw, Scheduling unrelated parallel machines. Algorithms, Complexity, and Performance. PhD thesis, Fakultat fur Elektrotechnik, Informatik und Mathematik der Universitat Paderborn, Deutschland, 2006.