An approach for characterizing workloads in google cloud to derive realistic resource utilization models

Proceedings - 2013 IEEE 7th International Symposium on Service-Oriented System Engineering, SOSE 2013

Volumn , Issue , 2013, Pages 49-60

  Moreno, Ismael Solis ^a   Garraghan, Peter ^a   Townend, Paul ^a   Xu, Jie ^a  

^a UNIVERSITY OF LEEDS   (United Kingdom)

Author keywords

Cloud computing workload patterns; Mapreduce analysis; Resource usage patterns; Workload characterization

Indexed keywords

CLOUD COMPUTING ENVIRONMENTS; COMPUTING WORKLOADS; MAP-REDUCE; OPERATIONAL ENVIRONMENTS; RESOURCE OPTIMIZATION; RESOURCE USAGE PATTERNS; RESOURCE UTILIZATIONS; WORKLOAD CHARACTERIZATION;

CLOUD COMPUTING; COMPUTER SYSTEMS; SYSTEMS ENGINEERING;

COMPUTER SIMULATION;

EID: 84880901911     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SOSE.2013.24     Document Type: Conference Paper

References (32)

1. R. Buyya, et al., "Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility, " Future Gener. Comput. Syst., vol. 25, pp. 599-616, 2009.
2. Google. Google Cluster Data V1. Available: http://code.google.com/p/ googleclusterdata/wiki/TraceVersion1.
3. Yahoo. Yahoo! M45 Supercomputing Project. Available: http://research. yahoo.com/node/1884.
4. Qi Zhang, et al., "Characterizing Task Usage Shapes in Google Compute Clusters, " in In Proc. of The 5th International Workshop on Large Scale Distributed Systems and Middleware, 2011.
5. S. Kavulya, et al., "An Analysis of Traces from a Production MapReduce Cluster, " in Cluster, Cloud and Grid Computing (CCGrid), 2010 10th IEEE/ACM International Conference on, 2010, pp. 94-103.
6. A. K. Mishra, et al., "Towards characterizing cloud backend workloads: insights from Google compute clusters, " SIGMETRICS Perform. Eval. Rev., vol. 37, pp. 34-41, 2010.
7. S. Aggarwal, et al., "Characterization of Hadoop Jobs Using Unsupervised Learning, " in Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, 2010, pp. 748-753.
8. Google. Google Cluster Data V2. Available: http://code.google.com/p/ googleclusterdata/wiki/ClusterData2011-1.
9. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters, " Commun. ACM, vol. 51, pp. 107-113, 2008.
10. C. Reiss, et al., "Google Cluster-Usage Traces: Format + Schema, " Google Inc., White Paper, 2011.
11. A. Bahga and V. K. Madisetti, "Synthetic Workload Generation for Cloud Computing Applications, " Journal of Software Engineering and Applications, vol. 4, pp. 396-410, July 2011.
12. A. Beitch, et al., "Rain: A Workload Generation Toolkit for Cloud Computing Applications, " Electrical Engineering and Computer Sciences University of California at Berkeley, White paper UCB/EECS-2010-14, 2010.
13. Y. Chen, et al., "Analysis and Lessons from a Publicly Available Google Cluster Trace, " EECS Department, University of California, Berkeley UCB/EECS-2010-95, June 14 2010.
14. J. W. Smith and I. Sommerville, "Workload Classification & Software Energy Measurement for Efficient Scheduling on Private Cloud Platforms, " presented at the ACM SOCC, 2011.
15. G. Wang, et al., "Towards Synthesizing Realistic Workload Traces for Studying the Hadoop Ecosystem, " in Proceedings of the 19th Annual Meeting of the IEEE International Symposium on Modeling, Analysis and Smulation of Computer and Telecommunication Systems (MASCOTS), Singapore, 2011.
16. P. Garraghan, et al., "Real-Time Fault-Tolerance in Federated Cloud Environments, " in Object/Component/Service-Oriented Real-Time Distributed Computing Workshops (ISORCW), 2012 15th IEEE International Symposium on, 2012, pp. 118-123.
17. Standard Performance Evaluation Corporation. (2012, July 7). SPECpower-ssj2008 Results. Available: http://www.spec.org/power-ssj2008/results/ .
18. J. A. Quiane-Ruiz, et al., "RAFTing MapReduce: Fast recovery on the RAFT, " in Data Engineering (ICDE), 2011 IEEE 27th International Conference on, 2011, pp. 589-600.
19. X. Rui and D. Wunsch, II, "Survey of clustering algorithms, " Neural Networks, IEEE Transactions on, vol. 16, pp. 645-678, 2005.
20. D. T. Pham, et al., "Selection of K in K-means clustering, " Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, vol. 219, pp. 103-119, January 1, 2005.
21. W. Tu, "Zero-Inflated Data, " in Encyclopedia of Environmetrics, ed: John Wiley & Sons, Ltd, 2006.
22. Minitab, "Distribution Analysis, " in Minitab Users' Guide, ed, 2011.
23. Institute of Statistics and Methemathics. (2012, July 22). The R Project for Statistical Computing. Available: http://www.rproject.org/.
24. Minitab, "Quality Control, " in Minitab Users' Guide, ed, 2003.
25. C. Reiss, et al., "Towards understanding heterogeneous clouds at scale:Google trace analysis, " Intel Science & Technology Center for Cloud Computing, White Paper, ISTC-CC-TR-12-101, 2012.
26. R. Buyya, et al., "Modeling and simulation of scalable Cloud computing environments and the CloudSim toolkit: Challenges and opportunities, " in Proc. of the International Conference on High Performance Computing & Simulation., 2009, pp. 1-11.
27. R. N. Calheiros, et al., "CloudSim: A Toolkit for Modeling and Simulation of Cloud Computing Environments and Evaluation of Resource Provisioning Algorithms, " Software: Practice and Experience, 2010.
28. S. K. Garg and R. Buyya, "NetworkCloudSim: Modelling Parallel Applications in Cloud Simulations, " in Utility and Cloud Computing (UCC), 2011 Fourth IEEE International Conference on, 2011, pp. 105-113.
29. B. Wickremasinghe, et al., "CloudAnalyst: A CloudSim-Based Visual Modeller for Analysing Cloud Computing Environments and Applications, " in Advanced Information Networking and Applications (AINA), 2010 24th IEEE International Conference on, 2010, pp. 446-452.
30. S. Pal and P. Bhattacharyya, "Multipeak histogram analysis in region splitting: a regularisation problem, " Computers and Digital Techniques, IEE Proceedings E, vol. 138, pp. 285-288, 1991.
31. I. Solis Moreno and X. Jie, "Neural Network-Based Overallocation for Improved Energy-Efficiency in Real-Time Cloud Environments, " in Object/Component/Service-Oriented Real-Time Distributed Computing (ISORC), 2012 IEEE 15th International Symposium on, 2012, pp. 119-126.
32. A. Quiroz, et al., "Towards autonomic workload provisioning for enterprise Grids and clouds, " in Grid Computing, 2009 10th IEEE/ACM International Conference on, 2009, pp. 50-57.