Towards a taxonomy of performance evaluation of commercial cloud services

Proceedings - 2012 IEEE 5th International Conference on Cloud Computing, CLOUD 2012

Volumn , Issue , 2012, Pages 344-351

  Li, Zheng ^a   O'Brien, Liam ^b   Cai, Rainbow ^a   Zhang, He ^c  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b CSIRO   (Australia)

^c UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Cloud Computing; Commercial Cloud Services; Performance Evaluation; Systematic Literature Review; Taxonomy

Indexed keywords

CLOUD PROVIDERS; CLOUD SERVICES; COMPUTING PARADIGM; PERFORMANCE EVALUATION; SYSTEMATIC LITERATURE REVIEW; TWO-DIMENSION;

CLOUD COMPUTING; DISTRIBUTED DATABASE SYSTEMS; WEB SERVICES;

TAXONOMIES;

EID: 84866747597     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLOUD.2012.74     Document Type: Conference Paper

References (24)

1. R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, "Cloud Computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility," Future Gener. Comp. Sy., vol. 25, no. 6, Jun. 2009, pp. 599-616.
2. R. Prodan and S. Ostermann, "A survey and taxonomy of Infrastructure as a Service and Web hosting Cloud providers," Proc. 10th IEEE/ACM Int. Conf. Grid Computing (GRID 2009), IEEE Computer Society, Oct. 2009, pp. 17-25.
3. W. Sobel, S. Subramanyam, A. Sucharitakul, J. Nguyen, H. Wong, A. Klepchukov, S. Patil, A. Fox, and D. Patterson, "Cloudstone: Multi-platform, multi-language benchmark and measurement tools for web 2.0," Proc. 1st Workshop on Cloud Computing and Its Applications (CCA '08), Oct. 2008, pp. 1-6.
4. B. A. Price, R. M. Baecker, and I. S. Small, "A principled taxonomy of software visualization," J. Visual Lang. Comput., vol. 4, no. 3, Sept. 1993, pp. 211-266.
5. A. Li, X. Yang, S. Kandula, and M. Zhang, "CloudCmp: Comparing public Cloud providers," Proc. 10th Annu. Conf. Internet Measurement (IMC '10), ACM Press, Nov. 2010, pp. 1-14.
6. J. Dejun, G. Pierre, and C.-H. Chi, "EC2 performance analysis for resource provisioning of service-oriented applications," Proc. 2009 Int. Conf. Service-Oriented Computing (ICSOC/ServiceWave '09), Springer-Verlag, Nov. 2009, pp. 197-207.
7. J. Wang, P. Varman, and C. Xie, "Avoiding performance fluctuation in cloud storage," Proc. 17th Annu. IEEE Int. Conf. High Performance Computing (HiPC 2010), IEEE Computer Society, Dec. 2010, pp. 1-9.
8. Q. He, S. Zhou, B. Kobler, D. Duffy, and T. McGlynn, "Case study for running HPC applications in public clouds," Proc. 19th ACM Int. Symp. High Performance Distributed Computing (HPDC '10), ACM Press, Jun. 2010, pp.395-401.
9. S. Garfinkel, "An evaluation of Amazon's Grid Computing services: EC2, S3 and SQS," Sch. Eng. Appl. Sci., Harvard Univ., Canbridge, MA, Tech. Rep. TR-08-07, Jul. 2007.
10. P. Barringer and T. Bennett, "Availability is not equal to Reliability," Barringer & Associates, Inc., Jul. 21, 2003. [online]. Available: http://www.barringer1.com/ar.htm.
11. A. B. Bondi, "Characteristics of scalability and their impact on performance," Proc. 2nd Int. Workshop on Software and Performance (WOSP '00), ACM Press, Sept. 2000, pp. 195-203.
12. A. Iosup, N. Yigitbasi, and D. Epema, "On the performance variability of production Cloud services," Delft Univ. Technol., Netherlands, Tech. Rep. PDS-2010-002, Jan. 2010.
13. D. Chiu and G. Agrawal, "Evaluating caching and storage options on the Amazon Web Services Cloud," Proc. 11th ACM/IEEE Int. Conf. Grid Computing (Grid 2010), IEEE Computer Society, Oct. 2010, pp. 17-24.
14. CCSK Guide, "Amazon Elastic Cloud Computing (EC2)," CCSK Guide, Jan. 12, 2011. [online]. Available: http://ccskguide.org/2011/01/amazon-ec2/.
15. J. Li, M. Humphrey, D. Agarwal, K. Jackson, C. van Ingen, and Y. Ryu, "eScience in the Cloud: A MODIS satellite data reprojection and reduction pipeline in the Windows Azure platform," Proc. 23rd IEEE Int. Symp. Parallel and Distributed Processing (IPDPS 2010), IEEE Computer Society, Apr. 2010, pp. 1-10.
16. N. Yigitbasi, A. Iosup, D. Epema, and S. Ostermann, "C-Meter: A framework for performance analysis of computing Clouds," Proc. 9th IEEE/ACM Int. Symp. Cluster Computing and the Grid (CCGRID '09), IEEE Computer Society, May 2009, pp. 472-477.
17. Z. Hill and M. Humphrey, "A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster?," Proc. 10th IEEE / ACM Int. Conf. Grid Computing (Grid 2009), IEEE Computer Society, Oct. 2009, pp. 26-33.
18. W. A. Thompson, "On the foundations of reliability," Technometrics, vol. 23, no. 1, Feb. 1981, pp. 1-13.
19. Z. Hill, J. Li, M. Mao, A. Ruiz-Alvarez, and M. Humphrey, "Early observations on the performance of Windows Azure," Proc. 19th ACM Int. Symp. High Performance Distributed Computing (HPDC '10), ACM Press, Jun. 2010, pp.367-376.
20. T. Dybå, B. A. Kitchenham, and M. Jørgensen, "Evidence-based software engineering for practitioners," IEEE Softw., vol. 22, no. 1, Jan. 2005, pp. 58-65.
21. B. A. Kitchenham, "Guidelines for performance systematic literature reviews in software engineering," Keele Univ. and Univ. Durham, UK, EBSE Tech. Rep. EBSE-2007-01, Ver. 2.3, Jul. 2007.
22. Q. Zhang, L. Cheng, and R. Boutaba, "Cloud computing: State-of-the-art and research challenges," J. Internet Serv. Appl., vol. 1, no. 1, May 2010, pp. 7-18.
23. A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. H. J. Epema, "Performance Analysis of Cloud Computing services for many-tasks Scientific Computing," IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 6, Jun. 2011, pp. 931-945.
24. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, "A view of Cloud computing," Commun. ACM, vol. 53, no. 4, Apr. 2010, pp. 50-58.