Towards optimal resource provisioning for Running MapReduce programs in public clouds

Proceedings - 2011 IEEE 4th International Conference on Cloud Computing, CLOUD 2011

Volumn , Issue , 2011, Pages 155-162

  Tian, Fengguang ^a   Chen, Keke ^a  

^a Wright State University   (United States)

Author keywords

Cloud computing; MapReduce; Performance modeling; Resource provisioning

Indexed keywords

COST MODELS; DECISION PROBLEMS; FINANCIAL COSTS; INPUT DATAS; MAP-REDUCE; MODEL PARAMETERS; PERFORMANCE MODELING; RESOURCE PROVISIONING; SYSTEM RESOURCES; TEST RUNS; WHOLE PROCESS;

COSTS; OPTIMIZATION;

CLOUD COMPUTING;

EID: 80053154819     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLOUD.2011.14     Document Type: Conference Paper

References (21)

1. S. Babu, "Towards automatic optimization of mapreduce programs," in Proceedings of the 1st ACM symposium on Cloud computing. New York, NY, USA: ACM, 2010, pp. 137-142.
2. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 2004.
3. S. Brin and L. Page, "The anatomy of a large-scale hyper-textual web search engine," in International Conference on World Wide Web, 1998.
4. A. S. Das, M. Datar, A. Garg, and S. Rajaram, "Google news personalization: scalable online collaborative filtering," in International Conference on World Wide Web. New York, NY, USA: ACM, 2007, pp. 271-280. (Pubitemid 47582257)
5. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," in USENIX Symposium on Operating Systems Design and Implementation, 2004.
6. D. Jiang, B. C. Ooi, L. Shi, and S. Wu, "The performance of mapreduce: An in-depth study," in Proceedings of Very Large Databases Conference (VLDB), 2010.
7. T. Joachims, L. Granka, B. Pan, and G. Gay, "Accurately interpreting clickthrough data as implicit feedback," in Proceedings of ACM SIGIR Conference, 2005.
8. K. Kambatla, A. Pathak, and H. Pucha, "Towards optimizing hadoop provisioning in the cloud," in USENIX Workshop on Hot Topics in Cloud Computing (HotCloud09), 2009.
9. U. Kang, C. E. Tsourakakis, and C. Faloutsos, "Pegasus: Mining peta-scale graphs," Knowledge and Information Systems (KAIS), 2010.
10. H. Karloff, S. Suri, and S. Vassilvitskii, "A model of computation for mapreduce," in Symposium on Discrete Algorithms (SODA) (2010), 2010.
11. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan, "An analysis of traces from a production mapreduce cluster," in IEEE/ACM International Conference on Cluster Cloud and Grid Computing, 2010, pp. 94-103.
12. J. Lin and C. Dyer, Data-intensive text processing with MapReduce. Morgan & Claypool Publishers, 2010.
13. K. Morton, A. Friesen, M. Balazinska, and D. Grossman, "Estimating the progress of mapreduce pipelines," in Proceedings of IEEE International Conference on Data Engineering (ICDE), 2010.
14. B. Panda, J. S. Herbach, S. Basu, and R. J. Bayardo, "Planet: Massively parall learning of tree ensembles with mapreduce," in Proceedings of Very Large Databases Conference (VLDB), 2009.
15. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker, "A comparison of approaches to large-scale data analysis," in Proceedings of ACM SIGMOD Conference, 2009.
16. T. Sandholm and K. Lai, "Mapreduce optimization using regulated dynamic prioritization," in SIGMETRICS/Performance09, 2009.
17. A. Thusoo, Z. Shao, S. Anthony, D. Borthakur, N. Jain, J. Sen Sarma, R. Murthy, and H. Liu, "Data warehousing and analytics infrastructure at facebook," in Proceedings of ACM SIGMOD Conference. ACM, 2010, pp. 1013-1020.
18. G. Wang, A. Butt, P. Pandey, and K. Gupta, "A simulation approach to evaluating design decisions in mapreduce setups," in the IEEE/ACM Intl. Symposium on Modelling, Analysis and Simulation of Computer and Telecomm. Systems, 2009.
19. T. White, Hadoop: The Definitive Guide. O'Reilly Media, 2009.
20. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica, "Job scheduling for multi-user mapreduce clusters," University of California at Berkeley, Tech. Rep. UCB/EECS-2009-55, april 2009.
21. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, "Improving mapreduce performance in heterogeneous environments," in 8th USENIX Symposium on Operating Systems Design and Implementation(OSDI08), 2008.