AROMA: Automated resource allocation and configuration of mapreduce environment in the cloud

ICAC'12 - Proceedings of the 9th ACM International Conference on Autonomic Computing

Volumn , Issue , 2012, Pages 63-72

  Lama, Palden ^a   Zhou, Xiaobo ^a  

^a University of Colorado Colorado Springs   (United States)

Author keywords

Auto configuration; Mapreduce; Resource allocation

Indexed keywords

AUTO-CONFIGURATION; BLADE SERVERS; DISTRIBUTED DATA PROCESSING; END USERS; IN-DEPTH KNOWLEDGE; MAP-REDUCE; OPTIMIZATION FRAMEWORK; PARAMETER-TUNING; PERFORMANCE GUARANTEES; PERFORMANCE MODEL; RESOURCE PROVISIONING; RESOURCE RESERVATIONS; RESOURCE USAGE; RESOURCE UTILIZATIONS; TWO-PHASE MACHINE;

DATA PROCESSING; QUALITY OF SERVICE; RESOURCE ALLOCATION;

BENCHMARKING;

EID: 84867709937     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2371536.2371547     Document Type: Conference Paper

References (27)

1. A. Abouzid, K. Bajda-Pawlikowski, D. Abadi, A. Silberschatz, and A. Rasin. Hadoopdb: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads. In Proc. of the VLDB, 2009.
2. X. Bu, J. Rao, and C.-Z. Xu. A reinforcement learning approach to online web system auto-configuration. In Proc. IEEE Int'l Conference on Distributed Computing Systems (ICDCS), 2009.
3. C. Chang and C. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2, 2011.
4. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. MapReduce online. In Proc. USENIX NSDI, 2010.
5. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. Communications of the ACM, 51(1), 2008.
6. J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad. Hadoop++: making a yellow elephant run like a cheetah (without it even noticing). Proc. of the VLDB, 3:515-529, 2010.
7. Y. Geng, S. Chen, Y. Wu, R. Wu, G. Yang, and W. Zheng. Location-aware MapReduce in virtual cloud. In Proc. IEEE Int'l Conference on Parallel Processing (ICPP), 2011.
8. F. Goiri, K. Le, J. Guitart, J. Torres, and R. Bianchini. Intelligent placement of datacenters for internet services. In Proc. IEEE Int'l Conference on Distributed Computing Systems (ICDCS), 2011.
9. W. Guanying, A. Butt, P. Pandey, and K. Gupta. A simulation approach to evaluating design decisions in MapReduce setups. In Proc. IEEE Int'l Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 2009.
10. H. Herodotou and S. Babu. Profiling, what-if analysis, and cost-based optimization of MapReduce programs. In Proc. of the VLDB, 2011.
11. b. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In Proc. USENIX NSDI, 2011.
12. K. Kambatla, A. Pathak, and H. Pucha. Towards optimizing hadoop provisioning in the cloud. In HotCloud Workshop in conjunction with USENIX Annual Technical Conference, 2009.
13. P. Lama and X. Zhou. Autonomic provisioning with self-adaptive neural fuzzy control for end-to-end delay guarantee. In Proc. IEEE/ACM Int'l Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 2010.
14. P. Lama and X. Zhou. PERFUME: Power and performance guarantee with fuzzy mimo control in virtualized servers. In Proc. IEEE Int'l Workshop on Quality of Service (IWQoS), 2010.
15. G. Lee, B. Chun, and H. K. Randy. Heterogeneity-aware resource allocation and scheduling in the cloud. In HotCloud Workshop in conjunction with USENIX Annual Technical Conference, 2011.
16. R. Lee, T. Luo, F. Wang, Y. Huai, Y. He, and X. Zhang. Ysmart: Yet another SQL-to-MapReduce translator. In Proc. IEEE Int'l Conference on Distributed Computing Systems (ICDCS), 2011.
17. X. Meng, C. Isci, J. Kephart, L. Zhang, and E. Bouillet. Efficient resource provisioning in compute clouds via vm multiplexing. In Proc. Int'l Conference on Autonomic Computing (ICAC), 2010.
18. J. Polo, D. Carrera, Y. Becerra, J. Torres, E. Ayguade, M. Steinder, and I. Whalley. Performance-driven task co-scheduling for MapReduce environments. In Proc. of the IEEE/IFIP Network Operations and Management Symposium (NOMS), 2010.
19. J. Rao, X. Bu, C. Xu, L. Wang, and G. Yin. Vconf: A reinforcement learning approach to virtual machines auto-conguration. In Proc. IEEE Int'l Conference on Autonomic Computing Systems (ICAC), 2009.
20. J. Rao and C. Xu. CoSL: a coordinated statistical learning approach to measuring the capacity of multi-tier Websites. In Proc. IEEE Int'l Parallel and Distributed Processing Symposium (IPDPS), 2008.
21. L. Shi, X. Li, and K. L. Tan. S3: An efficient shared scan scheduler on MapReduce framework. In Proc. IEEE Int'l Conference on Parallel Processing (ICPP), 2011.
22. R. Singh, U. Sharma, E. Cecchet, and P. Shenoy. Autonomic mix-aware provisioning for non-stationary data center workloads. In Proc. IEEE Int'l Conference on Autonomic Computing (ICAC), 2010.
23. A. Verma, L. Cherkasova, and R. Campbell. ARIA: automatic resource inference and allocation for MapReduce environments. In Proc. IEEE/ACM Int'l Conference on Autonomic Computing (ICAC), 2011.
24. D. Warneke and O. Kao. Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans. on Parallel and Distributed Systems, 22(6), 2011.
25. P. Xiong, Z. Wang, S. Malkowski, D. Jayasinghe, Q. Wang, and C. Pu. Economical and robust provisioning of n-tier cloud workloads: A multi-level control approach. In Proc. IEEE Int'l Conference on Distributed Computing Systems (ICDCS), 2011.
26. J. Xu and J. Fortes. A multi-objective approach to virtual machine management in datacenters. In Proc. of IEEE/ACM Int'l Conference on Autonomic computing (ICAC), 2011.
27. M. Zaharia, A. Konwinshi, A. D. Josepj, R. Katz, and I. Stoica. Improving MapReduce performance in heterogeneous environments. In Proc. the USENIX OSDI, 2008.