Heterogeneity and dynamicity of clouds at scale: Google trace analysis

Proceedings of the 3rd ACM Symposium on Cloud Computing, SoCC 2012

Volumn , Issue , 2012, Pages

  Reiss, Charles ^a   Tumanov, Alexey ^b   Ganger, Gregory R ^b   Katz, Randy H ^a   Kozuch, Michael A ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c INTEL CORPORATION   (United States)

Author keywords

Measurement

Indexed keywords

MULTI-PURPOSE; RESOURCE ASSIGNMENT; RESOURCE UTILIZATIONS; SCHEDULING DECISIONS; SIMPLIFYING ASSUMPTIONS; TASK MIGRATION; TRACE DATA; WORKLOAD CHARACTERISTICS;

CLOUD COMPUTING; DIGITAL STORAGE; MEASUREMENTS; SCHEDULING;

TRACE ANALYSIS;

EID: 84870524514     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2391229.2391236     Document Type: Conference Paper

References (27)

1. ISTC-CC: Research Overview, 2012. http://www.istc-cc.cmu.edu/research.
2. B. Abrahao and A. Zhang. Characterizing application workloads on CPU utilization for utility computing. Technical Report HPL-2004-157, HP Labs, 2004.
3. P. Bodik, A. Fox, M. J. Franklin, M. I. Jordan, and D. A. Patterson. Characterizing, modeling, and generating workload spikes for stateful services. In Proc. of the 1st ACM Symposium on Cloud Computing, 2010.
4. Y. Chen, S. Alspaugh, and R. H. Katz. Design insights for mapreduce from diverse production workloads. Technical Report UCB/EECS-2012-17, EECS Dept, University of California, Berkeley, Jan 2012.
5. Y. Chen, A. Ganapathi, R. Griffith, and R. Katz. The case for evaluating MapReduce performance using workload suites. In Proc. of 19th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), pages 390-399, July 2011.
6. A. Clauset, C. R. Shalizi, and M. E. J. Newman. Power-law distributions in empirical data. SIAM Rev., 51(4):661-703, Nov. 2009.
7. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008.
8. S. Di, D. Kondo, and W. Cirne. Characterization and Comparison of Google Cloud Load versus Grids. http://hal.archives-ouvertes.fr/hal-00705858, 2012.
9. A. B. Downey and D. G. Feitelson. The elusive goal of workload characterization. SIGMETRICS Perform. Eval. Rev., 26:14-29, March 1999.
10. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In Proc. of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, 2007.
11. S. Kavulya, J. Tan, R. Gandhi, and P. Narasimhan. An analysis of traces from a production MapReduce cluster. In Proc. of the 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, May 2010.
12. M. Kozuch, M. Ryan, R. Gass, S. Schlosser, D. O'Hallaron, J. Cipar, E. Krevat, J. López, M. Stroucken, and G. Ganger. Tashi: location-aware cluster management. In Proc. of the 1st Workshop on Automated Control for Datacenters and Clouds, 2009.
13. Z. Liu and S. Cho. Characterizing machines and workloads on a Google cluster. In Eighth International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems, September 2012.
14. U. Lublin and D. Feitelson. The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing, 63(11):1105-1122, 2003.
15. A. Oliner and J. Stearley. What supercomputers say: a study of five system logs. In Proc. of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN '07), pages 575-584, June 2007.
16. N. Poggi, D. Carrera, R. Gavalda, J. Torres, and E. Ayguadé. Characterization of workload and resource consumption for an online travel and booking site. In Proc. of the IEEE International Symposium on Workload Characterization (IISWC), 2010.
17. C. Reiss, A. Tumanov, G. R. Ganger, R. H. Katz, and M. A. Kozuch. Towards understanding heterogeneous clouds at scale: Google trace analysis. Technical Report ISTC-CC-TR-12-101, Intel Science and Technology Center for Cloud Computing, Apr 2012.
18. C. Reiss, J. Wilkes, and J. L. Hellerstein. Google cluster-usage traces: format + schema, 2011. http://goo.gl/5uJri.
19. C. Reiss, J. Wilkes, and J. L. Hellerstein. Obfuscatory obscanturism: making workload traces of commercially-sensitive systems safe to release. In Proc. of IEEE/IFIP International Workshop on Cloud Management (CloudMan'12), April 2012.
20. M. Schwarzkopf, D. G. Murray, and S. Hand. The seven deadly sins of cloud computing research. In Proc. of the 4th USENIX Workshop on Hot Topics in Cloud Computing, 2012.
21. B. Sharma, V. Chudnovsky, J. L. Hellerstein, R. Rifaat, and C. R. Das. Modeling and synthesizing task placement constraints in Google compute clusters. In Proc. of the 2nd ACM Symposium on Cloud Computing, SOCC '11, 2011.
22. A. Tumanov, J. Cipar, M. A. Kozuch, and G. R. Ganger. alsched: algebraic scheduling of mixed workloads in heterogeneous clouds. In Proc. of the 3nd ACM Symposium on Cloud Computing, SOCC '12, 2012.
23. J. Wilkes. Omega: Cluster management at google. Video from 2011 Google Faculty Summit; https://youtu.be/0ZFMlO98Jkc, July 2011.
24. J. Wilkes and C. Reiss. Details of the ClusterData-2011-1 trace, 2011. https://code.google.com/p/googleclusterdata/wiki/ClusterData2011-1.
25. M. Zaharia, B. Hindman, A. Konwinski, A. Ghodsi, A. Joseph, R. Katz, S. Shenker, and I. Stoica. The datacenter needs an operating system. In HotCloud, June 2011.
26. Q. Zhang, M. F. Zhani, S. Zhang, Q. Zhu, R. Boutaba, and J. L. Hellerstein. Dynamic energy-aware capacity provisioning for cloud computing enviornments. In Proceedings of the 9th ACM International Conference on Autonomic Computing, 2012.
27. Z. Zheng, L. Yu, W. Tang, Z. Lan, R. Gupta, N. Desai, S. Coghlan, and D. Buettner. Co-analysis of RAS log and job log on Blue Gene/P. In Proc. of IEEE International Parallel Distributed Processing Symposium (IPDPS), May 2011.