On modelling and prediction of total CPU usage for applications in mapreduce environments

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7439 LNCS, Issue PART 1, 2012, Pages 414-427

  Rizvandi, Nikzad Babaii ^a,b   Taheri, Javid ^a   Moraveji, Reza ^a,b   Zomaya, Albert Y ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

^b CSIRO   (Australia)

Author keywords

Configuration parameters; Hadoop; input data scaling; MapReduce; Resource provisioning; total CPU usage in clock cycles

Indexed keywords

CLOCK CYCLES; CONFIGURATION PARAMETERS; HADOOP; INPUT DATAS; MAP-REDUCE; RESOURCE PROVISIONING;

ALGORITHMS; INPUT OUTPUT PROGRAMS;

CLOCKS;

EID: 84866662630     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-33078-0_30     Document Type: Conference Paper

References (30)

1. Hadoop example for Exim logs with Python, http://blog.gnucom.cc/2010/ hadoop-example-for-exim-logs-with-python/
2. Rizvandi, N.B., Boloori, A.J., Kamyabpour, N., Zomaya, A.: MapReduce Implementation of Prestack Kirchhoff Time Migration (PKTM) on Seismic Data. Presented at the The 12th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), Gwangju, Korea (2011)
3. Arumugam, K., Tan, Y.S., Lee, B.S., Kanagasabai, R.: "Cloud-enabling Sequence Alignment with Hadoop MapReduce: A Performance Analysis. Presented at the 2012 4th International Conference on Bioinformatics and Biomedical Technology (2012)
4. NCBI, http://www.ncbi.nlm.nih.gov/
5. Kavulya, S., Tan, J., Gandhi, R., Narasimhan, P.: An Analysis of Traces from a Production MapReduce Cluster. Presented at the Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing (2010)
6. Kambatla, K., Pathak, A., Pucha, H.: Towards Optimizing Hadoop Provisioning in the Cloud. Presented at the the 2009 Conference on Hot Topics in Cloud Computing, San Diego, California (2009)
7. Zaharia, M., Konwinski, A., Joseph, A.D., Katz, R., Stoica, I.: Improving MapReduce Performance in Heterogeneous Environments. In: 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2008), December 18, pp. 29-42 (2008)
8. Verma, A., Cherkasova, L., Campbell, R.H.: Resource Provisioning Framework for MapReduce Jobs with Performance Goals. In: Kon, F., Kermarrec, A.-M. (eds.) Middleware 2011. LNCS, vol. 7049, pp. 165-186. Springer, Heidelberg (2011)
9. Chen, Y., Ganapathi, A.S., Fox, A., Katz, R.H., Patterson, D.A.: Statistical Workloads for Energy Efficient MapReduce. University of California at Berkeley, Technical Report No. UCB/EECS-2010-6 (2010)
10. Karloff, H., Suri, S., Vassilvitskii, S.: A model of computation for MapReduce. Presented at the Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, Austin, Texas (2010)
11. Kambatla, K., Pathak, A., Pucha, H.: Towards optimizing hadoop provisioning in the cloud. Presented at the Proceedings of the 2009 Conference on Hot Topics in Cloud Computing, San Diego, California (2009)
12. Rizvandi, N.B., Taheri, J., Zomaya, A.Y., Moraveji, R.: A Study on Using Uncertain Time Series Matching Algorithms in Map-Reduce Applications. In: Concurrency and Computation: Practice and Experience (2012)
13. Wieder, A., Bhatotia, P., Post, A., Rodrigues, R.: Brief Announcement: Modelling MapReduce for Optimal Execution in the Cloud. Presented at the Proceeding of the 29th ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, Zurich, Switzerland (2010)
14. Wieder, A., Bhatotia, P., Post, A., Rodrigues, R.: Conductor: orchestrating the clouds. Presented at the 4th International Workshop on Large Scale Distributed Systems and Middleware, Zurich, Switzerland (2010)
15. Oprescu, A.-M., Kielmann, T.: Bag-of-Tasks Scheduling under Budget Constraints. Presented at the IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), Indianapolis, IN, U.S.A (2010)
16. Wood, T., Cherkasova, L., Ozonat, K., Shenoy, P.D.: Profiling and Modeling Resource Usage of Virtualized Applications. In: Issarny, V., Schantz, R. (eds.) Middleware 2008. LNCS, vol. 5346, pp. 366-387. Springer, Heidelberg (2008)
17. Rizvandi, N.B., Nabavi, A., Hessabi, S.: An Accurate Fir Approximation of Ideal Fractional Delay Filter with Complex Coefficients in Hilbert Space. Journal of Circuits, Systems, and Computers 14, 497-506 (2005)
18. Chisnall, D.: The definitive guide to the xen hypervisor, 1st edn. Prentice Hall Press (2007)
19. Hadoop-0.20.2, http://www.apache.org/dyn/closer.cgi/hadoop/core
20. Sysstat-9.1.6, http://perso.orange.fr/sebastien.godard/
21. Optimizing Hadoop Deployments, Intel Corporation (2009)
22. Mao, A., Morris, R., Kaashoek, M.F.: Optimizing MapReduce for Multicore Architectures. Massachusetts Institute of Technology (2010)
23. Babu, S.: Towards automatic optimization of MapReduce programs. Presented at the 1st ACM Symposium on Cloud Computing, Indianapolis, Indiana, USA (2010)
24. Sort Benchmark Home Page, http://sortbenchmark.org/
25. Wang, G., Butt, A.R., Pandey, P., Gupta, K.: A Simulation Approach to Evaluating Design Decisions in MapReduce Setups. Presented at the MASCOTS (2009)
26. Wang, G., Butt, A.R., Pandey, P., Gupta, K.: Using realistic simulation for performance analysis of mapreduce setups. Presented at the Proceedings of the 1st ACM Workshop on Large-Scale System and Application Performance, Garching, Germany (2009)
27. Moise, D., Trieu, T.-T.-L., Boug, L., #233, Antoniu, G.: Optimizing intermediate data management in MapReduce computations. Presented at the Proceedings of the First International Workshop on Cloud Computing Platforms, Salzburg, Austria (2011)
28. Gillick, D., Faria, A., DeNero, J.: MapReduce: Distributed Computing for Machine Learning, http://www.icsi.berkeley.edu/~arlo/publications/gillick- cs262a-proj.pdf2008
29. Rizvandi, N.B., Taheri, J., Zomaya, A.Y.: On using Pattern Matching Algorithms in MapReduce Applications. Presented at the The 9th IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA), Busan, South Korea (2011)
30. Islam, S., Keung, J., Lee, K., Liu, A.: Empirical prediction models for adaptive resource provisioning in the cloud. Future Generation Comp. Syst. 28, 155-162 (2012)