Performance analysis of Coupling Scheduler for MapReduce/Hadoop

Proceedings - IEEE INFOCOM

Volumn , Issue , 2012, Pages 2586-2590

  Tan, Jian ^a   Meng, Xiaoqiao ^a   Zhang, Li ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

JOB PROCESSING TIME; MAP SERVICE; MAP TASK; MAP-REDUCE; PERFORMANCE ANALYSIS; PROCESSOR SHARING; PROCESSOR SHARING QUEUE; THEORETICAL RESULT;

DESIGN;

SCHEDULING;

EID: 84861624840     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFCOM.2012.6195658     Document Type: Conference Paper

References (21)

1. Hadoop, http://hadoop.apache.org.
2. Applications powered by Hadoop, http://wiki.hadoop.apache.org/hadoop/ PoweredBy.
3. Fair Scheduler, http://hadoop.apache.org/mapreduce/docs/r0.21.0/fairs- cheduler.html.
4. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Job scheduling for multi-user MapReduce clusters," EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-55, April 2009.
5. Capacity Scheduler, http://hadoop.apache.org/mapreduce/docs/r0.21.0/ capacity-scheduler.html.
6. J. Wolf, D. Rajan, K. Hildrum, R. Khandekar, V. Kumar, S. Parekh, K.-L. Wu, and A. Balmin, "FLEX: A slot allocation scheduling optimizer for MapReduce workloads," in Middleware 2010, ser. Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 2010, vol. 6452, pp. 1-20.
7. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg, "Quincy: fair scheduling for distributed computing clusters," in Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (SOSP '09). ACM, 2009, pp. 261-276.
8. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris, "Reining in the outliers in map-reduce clusters using mantri," in Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation (OSDI'10), 2010, pp. 1-16.
9. J. Polo, C. Castillo, D. Carrera, Y. Becerra, I. Whalley, M. Steinder, J. Torres, and E. Ayguadé, "Resource-aware adaptive scheduling for mapreduce clusters," in Middleware, ser. Lecture Notes in Computer Science, vol. 7049. Springer, 2011, pp. 187-207.
10. T. Sandholm and K. Lai, "Dynamic proportional share scheduling in hadoop," in Proceedings of the 15th international conference on job scheduling strategies for parallel processing, ser. JSSPP'10. Berlin, Heidelberg: Springer-Verlag, 2010, pp. 110-131.
11. B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. Katz, S. Shenker, and I. Stoica, "Mesos: a platform for fine-grained resource sharing in the data center," in Proceedings of the 8th USENIX conference on Networked systems design and implementation (NSDI'11), 2011, pp. 22-22.
12. A. Verma, L. Cherkasova, and R. H. Campbell, "ARIA: Automatic Resource Inference and Allocation for MapReduce Environments," in Proc. of Intl. Conference on Autonomic Computing (ICAC), 2011.
13. J. Tan, X. Meng, and L. Zhang, "Coupling task progress for mapreduce resource-aware scheduling," IBM T. J. Watson Research Center, Hawthorne, New York, Tech. Rep., July 2011.
14. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan, "An analysis of traces from a production mapreduce cluster," in Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, ser. CCGRID '10, Washington, DC, USA, 2010, pp. 94-103.
15. N. H. Bingham, C. M. Goldie, and J. L. Teugels, Regular Variation, G.-C. Rota, Ed. Cambridge University Press, 1987, vol. 27.
16. J. Tan, X. Meng, and L. Zhang, "Delay tails in mapreduce scheduling," IBM T. J. Watson Research Center, Hawthorne, New York, Tech. Rep., November 2011.
17. R. W. Wolff, Stochastic Modeling and Theory of Queues. Prentice Hall, 1989.
18. K. Sigman and R. W. Wolff, "A review of regenerative processes," SIAM Review, vol. 35, no. 2, pp. 269-288, June 1993.
19. P. R. Jelenkovic and P. Momcilovic, "Large deviation analysis of subexponential waiting times in a processor-sharing queue." Math. Oper. Res.
20. F. P. Kelly, Reversibility and Stochastic Networks. John Wiley and Sons Ltd, 1979.
21. P. R. Jelenkovic, X. Kang, and J. Tan, "Heavy-tailed limits for medium size jobs and comparison scheduling," Annals of Operations Research, vol. 170, no. 1, pp. 133-159, 2009.