WPress: An Application-Driven Performance Benchmark for Cloud-Based Virtual Machines

Proceedings . IEEE 18th international Enterprise Distributed object computing conference

Volumn 2014-December, Issue December, 2014, Pages 101-109

  Borhani, Amir Hossein ^a   Leitner, Philipp ^b   Lee, Bu Sung ^c   Li, Xiaorong ^a   Hung, Terence ^a  

^a INSTITUTE OF HIGH PERFORMANCE COMPUTING   (Singapore)

^b UNIVERSITY OF ZURICH   (Switzerland)

^c NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Amazon EC2; Benchmarking; Cloud Computing; CPU Micro Benchmark; Microsoft Azure; OLTP; Rackspace Cloud; Virtual Machine

Indexed keywords

APPLICATION PROGRAMS; CLOUD COMPUTING; INFRASTRUCTURE AS A SERVICE (IAAS); NETWORK SECURITY; OPEN SOURCE SOFTWARE; OPEN SYSTEMS; VIRTUAL MACHINE; WINDOWS OPERATING SYSTEM;

AMAZON EC2; COMPREHENSIVE PERFORMANCE; FUNDAMENTAL FEATURES; MICRO-BENCHMARK; MICROSOFT; OLTP; ONLINE TRANSACTION PROCESSING; WORKLOAD GENERATORS;

BENCHMARKING;

EID: 84937398988     PISSN: 15417719     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/EDOC.2014.23     Document Type: Conference Paper

References (39)

1. R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, "Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility, " Future Generation Computer Systems, vol. 25, no. 6, pp. 599-616, 2009.
2. R. Prodan and S. Ostermann, "A survey and taxonomy of infrastructure as a service and web hosting cloud providers, " in Grid Computing, 2009 10th IEEE/ACM International Conference on, Oct 2009, pp. 17-25.
3. K. Huppler, "Performance evaluation and benchmarking." Berlin, Heidelberg: Springer-Verlag, 2009, ch. The Art of Building a Good Benchmark, pp. 18-30.
4. D. Kossmann, T. Kraska, and S. Loesing, "An evaluation of alternative architectures for transaction processing in the cloud, " in Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD '10, 2010, pp. 579-590.
5. D. Kossmann and T. Kraska, "Data management in the cloud: Promises, state-of-the-art, and open questions, " Datenbank-Spektrum, vol. 10, no. 3, pp. 121-129, 2010.
6. Z. Hill, J. Li, M. Mao, A. Ruiz-Alvarez, and M. Humphrey, "Early observations on the performance of Windows Azure, " in Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC '10, 2010, pp. 367-376.
7. C. Binnig, D. Kossmann, T. Kraska, and S. Loesing, "How is the weather tomorrow: Towards a benchmark for the cloud, " in Proceedings of the Second International Workshop on Testing Database Systems, ser. DBTest '09, 2009, pp. 9:1-9:6.
8. K. Huppler, "Benchmarking with your head in the cloud, " in Proceedings of the Third TPC Technology Conference on Topics in Performance Evaluation, Measurement and Characterization, ser. TPCTC'11. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 97-110.
9. A. Floratou, J. M. Patel, W. Lang, and A. Halverson, "When free is not really free: What does it cost to run a database workload in the cloud" in Proceedings of the Third TPC Technology Conference on Topics in Performance Evaluation, Measurement and Characterization, ser. TPCTC'11. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 163-179.
10. E. Folkerts, A. Alexandrov, K. Sachs, A. Iosup, V. Markl, and C. Tosun, "Benchmarking in the cloud: What it should, can, and cannot be, " in Selected Topics in Performance Evaluation and Benchmarking, ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, vol. 7755, pp. 173-188.
11. WordPress-Blog Tool, Publishing Platform, and CMS. [Online]. Available: https://wordpress.org/
12. AWS-Amazon Elastic Compute Cloud (EC2)-Scalable Cloud Hosting. [Online]. Available: http://aws.amazon.com/ec2/
13. Azure: Microsoft's Cloud Platform-Cloud Hosting-Cloud Services. [Online]. Available: http://azure.microsoft.com/en-us/
14. Rackspace: The Leader in Hybrid Cloud. [Online]. Available: http://www.rackspace.com/
15. G. Wang and T. S. E. Ng, "The impact of virtualization on network performance of Amazon EC2 data center, " in Proceedings of the 29th Conference on Information Communications, ser. INFOCOM'10, 2010, pp. 1163-1171.
16. J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz, "Runtime measurements in the cloud: Observing, analyzing, and reducing variance, " Proc. VLDB Endow., vol. 3, no. 1-2, pp. 460-471, Sep. 2010.
17. A. Lenk, M. Menzel, J. Lipsky, S. Tai, and P. Offermann, "What are you paying for performance benchmarking for infrastructure-as-a-service offerings, " in Cloud Computing (CLOUD), 2011 IEEE International Conference on, July 2011, pp. 484-491.
18. L. Zhao, A. Liu, and J. Keung, "Evaluating cloud platform architecture with the care framework, " in Software Engineering Conference (APSEC), 2010 17th Asia Pacific, Nov 2010, pp. 60-69.
19. M. Klems, D. Bermbach, and R. Weinert, "A runtime quality measurement framework for cloud database service systems, " in Quality of Information and Communications Technology (QUATIC), 2012 Eighth International Conference on the, Sept 2012, pp. 38-46.
20. D. Bermbach and S. Tai, "Eventual consistency: How soon is eventual an evaluation of Amazon S3's consistency behavior, " in Proceedings of the 6th Workshop on Middleware for Service Oriented Computing, ser. MW4SOC '11, 2011, pp. 1:1-1:6.
21. D. Bermbach, L. Zhao, and S. Sakr, "Towards comprehensive measurement of consistency guarantees for cloud-hosted data storage services, " in Performance Characterization and Benchmarking, ser. Lecture Notes in Computer Science, 2014, vol. 8391, pp. 32-47.
22. C. A. Curino, D. E. Difallah, A. Pavlo, and P. Cudre-Mauroux, "Benchmarking OLTP/web databases in the cloud: The OLTP-bench framework, " in Proceedings of the Fourth International Workshop on Cloud Data Management, ser. CloudDB '12, 2012, pp. 17-20.
23. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, "Benchmarking cloud serving systems with YCSB, " in Proceedings of the 1st ACM Symposium on Cloud Computing, ser. SoCC '10, 2010, pp. 143-154.
24. TPC-Homepage. [Online]. Available: http://www.tpc.org/
25. RUBiS-Home Page. [Online]. Available: http://rubis.ow2.org/
26. T. Chen and R. Bahsoon, "Self-adaptive and sensitivity-aware QoS modeling for the cloud, " in Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2013 ICSE Workshop on, May 2013, pp. 43-52.
27. W. Dawoud, I. Takouna, and C. Meinel, "Dynamic scalability and contention prediction in public infrastructure using internet application profiling, " in Cloud Computing Technology and Science (CloudCom), 2012 IEEE 4th International Conference on, Dec 2012, pp. 208-216.
28. H. Wu, A. Tantawi, and T. Yu, "A self-optimizing workload management solution for cloud applications, " in Web Services (ICWS), 2013 IEEE 20th International Conference on, June 2013, pp. 483-490.
29. L. Zhao, S. Sakr, and A. Liu, "A framework for consumer-centric SLA management of cloud-hosted databases, " IEEE Transactions on Services Computing, vol. 99, no. PrePrints, p. 1, 2013.
30. A. Iosup, S. Ostermann, M. Yigitbasi, R. Prodan, T. Fahringer, and D. H. J. Epema, "Performance analysis of cloud computing services for many-tasks scientific computing, " Parallel and Distributed Systems, IEEE Transactions on, vol. 22, no. 6, pp. 931-945, June 2011.
31. Q. He, S. Zhou, B. Kobler, D. Duffy, and T. McGlynn, "Case study for running HPC applications in public clouds, " in Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC '10, 2010, pp. 395-401.
32. S. Akioka and Y. Muraoka, "HPC benchmarks on Amazon EC2, " in Advanced Information Networking and Applications Workshops (WAINA), 2010 IEEE 24th International Conference on, April 2010, pp. 1029-1034.
33. P. Mehrotra, J. Djomehri, S. Heistand, R. Hood, H. Jin, A. Lazanoff, S. Saini, and R. Biswas, "Performance evaluation of Amazon EC2 for NASA HPC applications, " in Proceedings of the 3rd Workshop on Scientific Cloud Computing Date, ser. ScienceCloud '12, 2012, pp. 41-50.
34. M. Silva, M. Hines, D. Gallo, Q. Liu, K. D. Ryu, and D. Da Silva, "CloudBench: Experiment automation for cloud environments, " in Cloud Engineering (IC2E), 2013 IEEE International Conference on, March 2013, pp. 302-311.
35. The 15 Best Blogging Platforms on the Web Today. [Online]. Available: http://thenextweb.com/apps/2013/08/16/best-blogging-services/
36. WordPress Use Cases. [Online]. Available: http://www.slideshare.net/teamphp/word-press-use-cases
37. Selenium-Web Browser Automation. [Online]. Available: http://docs.seleniumhq.org/
38. WPressClient Source Code. [Online]. Available: http://figshare.com/articles/WPressClient source code/978486
39. IaaS Providers List: 2014 Comparison And Guide-Tom's IT Pro. [Online]. Available: http://www.tomsitpro.com/articles/iaas-providers, 1-1560.html