Two sides of a coin: Optimizing the schedule of mapreduce jobs to minimize their makespan and improve cluster performance

Proceedings of the 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS 2012

Volumn , Issue , 2012, Pages 11-18

  Verma, Abhishek ^a   Cherkasova, Ludmila ^b   Campbell, Roy H ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

batch workloads; Hadoop; MapReduce; minimized makespan; optimized schedule

Indexed keywords

BATCH WORKLOADS; HADOOP; MAKESPAN; MAP-REDUCE; OPTIMIZED SCHEDULE;

OPTIMIZATION; SCHEDULING ALGORITHMS; TELECOMMUNICATION SYSTEMS;

COMPUTER SIMULATION;

EID: 84868221206     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MASCOTS.2012.12     Document Type: Conference Paper

References (17)

1. X. Wang, C. Olston, A. Sarma, and R. Burns, "CoScan: Cooperative Scan Sharing in the Cloud," in Proc. of SOCC, 2011.
2. S. Johnson, "Optimal Two- and Three-Stage Production Schedules with Setup Times Included," Naval Res. Log. Quart., 1954.
3. A. Verma, L. Cherkasova, and R. H. Campbell, "ARIA: Automatic Resource Inference and Allocation for MapReduce Environments," in Proc. of ICAC,, 2011.
4. -, "Play It Again, SimMR!" in Proc. of Intl. IEEE Cluster'2011.
5. M. Garey and D. Johnson, Computers and Intractability: A Guide to the Theory of NP-completeness. WH Freeman & Co., 1979.
6. "Capacity Scheduler Guide." [Online]. Available: http://hadoop.apache.org/common/docs/r0.20.1/capacity-scheduler.html
7. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan, "An Analysis of Traces from a Production MapReduce Cluster," in Proc of CCGrid'2010.
8. J. Blazewicz, Scheduling in computer and manufacturing systems. Springer-Verlag, New York, USA, 1996.
9. R. Blumofe and C. Leiserson, "Scheduling multithreaded computations by work stealing," Journal of the ACM (JACM), vol. 46, no. 5, pp. 720-748, 1999.
10. C. Chekuri and M. Bender, "An efficient approximation algorithm for minimizing makespan on uniformly related machines," Integer Programming and Combinatorial Optimization, pp. 383-393, 1998.
11. B. Lampson, "A scheduling philosophy for multiprocessing systems," Communications of the ACM, vol. 11, no. 5, 1968.
12. J. Leung, Handbook of scheduling: algorithms, models, and performance analysis. CRC Press, 2004.
13. L. Tan and Z. Tari, "Dynamic task assignment in server farms: Better performance by task grouping," in Proc. of the Intl. Symp. on Computers and Communications (ISCC), 2002.
14. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling," in Proc. of EuroSys. ACM, 2010, pp. 265-278.
15. J. Wolf and et al, "FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads," ACM/IFIP/USENIX Intl. Middleware Conference, 2010.
16. H. Herodotou and S. Babu, "Profiling, What-if Analysis, and Costbased Optimization of MapReduce Programs." in Proc. of the VLDB Endowment, Vol. 4, No. 11, 2011.
17. B. Moseley, A. Dasgupta, R. Kumar, and T. Sarlós, "On scheduling in map-reduce and flow-shops," in Proc. of SPAA, 2011.