ARIA: Automatic resource inference and allocation for mapreduce environments

Proceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops

Volumn , Issue , 2011, Pages 235-244

  Verma, Abhishek ^a   Cherkasova, Ludmila ^b   Campbell, Roy H ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

mapreduce; modeling; resource allocation; scheduling

Indexed keywords

DATA SETS; EXTENSIVE SIMULATIONS; JOB COMPLETION; JOB DEMANDS; JOB SCHEDULER; LARGE SCALE DATA; MAP-REDUCE; PERFORMANCE CHARACTERISTICS; PERFORMANCE MODEL; REALISTIC APPLICATIONS; SERVICE LEVEL OBJECTIVE; SOFT DEADLINES;

DATA HANDLING; RESOURCE ALLOCATION;

SCHEDULING;

EID: 79960196705     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1998582.1998637     Document Type: Conference Paper

References (22)

1. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the Outliers in Map-Reduce Clusters using Mantri. In Proc. of OSDI'2010.
2. Apache. Capacity Scheduler Guide, 2010. URL http://hadoop.apache.org/ common/docs/r0.20.1/capacity-scheduler.html.
3. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51 (1):107-113, 2008.
4. A. Ganapathi, Y. Chen, A. Fox, R. Katz, and D. Patterson. Statistics-driven workload modeling for the cloud. In Proc. of 5th Intl. Workshop on Self Managing Database Systems, 2010.
5. R.L. Graham. Bounds for certain multiprocessing anomalies. Bell System Tech. Journal, 45(9):1563-1581, 1966.
6. Intel. Optimizing Hadoop* Deployments, 2010. URL http://communities.intel.com/docs/DOC-4218.
7. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS OS Review, 41(3):72, 2007.
8. M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: fair scheduling for distributed computing clusters. In Proc. of SOSP'2009.
9. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In Proc. of the First Workshop on Hot Topics in Cloud Computing, 2009.
10. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan. An Analysis of Traces from a Production MapReduce Cluster. In Proc. of CCGrid'2010.
11. A. Konwinski, M. Zaharia, R. Katz, and I. Stoica. X-tracing Hadoop. Hadoop Summit, 2008.
12. H. Kwak, C. Lee, H. Park, and S. Moon. What is Twitter, a social network or a news media? In Proc. of WWW'2010.
13. O. O'Malley and A.C. Murthy. Winning a 60 second dash with a yellow elephant, 2009.
14. K. Morton, M. Balazinska, D. Grossman.ParaTimer: a progress indicator for MapReduce DAGs. In Proc. of SIGMOD'2010.
15. L. Phan, Z. Zhang, B. Loo, and I. Lee. Real-time MapReduce Scheduling. Tech. Report No. MS-CIS-10-32, UPenn, 2010.
16. J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguadé, M. Steinder, and I. Whalley. Performance-driven task co-scheduling for MapReduce environments. In 12th IEEE/IFIP Network Operations and Management Symposium. ACM, 2010.
17. T. Sandholm and K. Lai. Dynamic Proportional Share Scheduling in Hadoop. LNCS: Proc. of the 15th Workshop on Job Scheduling Strategies for Parallel Processing, 2010.
18. J. Tan, X. Pan, S. Kavulya, E. Marinelli, R. Gandhi, and P. Narasimhan. Kahuna: Problem Diagnosis for MapReduce-based Cloud Computing Environments. In 12th IEEE/IFIP NOMS, 2010.
19. G. Wang, A.R. Butt, P. Pandey, and K. Gupta. A simulation approach to evaluating design decisions in MapReduce setups. In Proc of MASCOTS'2009.
20. J. Wolf, et al. FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads. In Proc.of Middleware'2010.
21. M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In Proc. of EuroSys, pages 265-278. ACM, 2010.
22. M. Zaharia, A. Konwinski, A.D. Joseph, R. Katz, and I. Stoica. Improving MapReduce performance in heterogeneous environments. In OSDI, 2008.