Cloud Performance Modeling with Benchmark Evaluation of Elastic Scaling Strategies

IEEE Transactions on Parallel and Distributed Systems

Volumn 27, Issue 1, 2016, Pages 130-143

  Hwang, Kai ^a   Bai, Xiaoying ^b   Shi, Yue ^c   Li, Muyang ^b   Chen, Wen Guang ^b   Wu, Yongwei ^b  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b TSINGHUA UNIVERSITY   (China)

^c University of Southern California   (United States)

Author keywords

And resources scaling; Cloud benchmarks; Cloud computing; Performance evaluation

Indexed keywords

BENCHMARKING; CLOUD COMPUTING; COST EFFECTIVENESS; EFFICIENCY; ELASTICITY; PLATFORM AS A SERVICE (PAAS); PRODUCTIVITY; SCALABILITY; SOFTWARE AS A SERVICE (SAAS);

AND RESOURCES SCALING; BENCHMARK EVALUATION; BENCHMARK EXPERIMENTS; PERFORMANCE EVALUATION; PRODUCTION SERVICES; RESOURCE UTILIZATIONS; RESOURCES UTILIZATIONS; SCALE-UP EXPERIMENT;

INFRASTRUCTURE AS A SERVICE (IAAS);

EID: 84961757531     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPDS.2015.2398438     Document Type: Article

References (37)

1. R. Appuswamy, C. Gkantsidis, D. Narayanan, O. Hodson, and A. Rowstron, "Scale-up vs scale-out for hadoop: Time to rethink," presented at the ACM Symp. Cloud Comput., Santa Clara, CA, USA, Oct. 2013.
2. X. Bai, Y. Wang, G. Dai, W. T. Tsai, and Y. Chen, "A framework for collaborative verification and validation of web services," Component-Based Software Engineering, Springer, 2007.
3. C. Binnig, D. Kossmann, T. Kraska, and S. Loesing, "How is the weather tomorrow Towards a benchmark for the cloud," in Proc. ACM 2nd Int. Workshop Testing Database Syst., Jun. 2009.
4. Bitcurrent, Inc. (2010). Cloud computing performance report. [Online]. Available: http:// bitcurrent.com
5. A. Bondi, "Characteristics of scalability and their impact on performance," in Proc. 2nd Int. Workshop Softw. Performance, 2000, pp. 195-203.
6. R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, "Cloud computing and emerging IT platforms: Vision, Hype, and Reality for delivering computing as the 5th utility," Future Gen. Comput. Syst., vol. 25, no. 6, pp. 599-616, 2009.
7. J. Cao, K. Hwang, K. Li, and A. Zomaya, "Optimal multiserver configuration for profit maximization in cloud computing," IEEE Trans. Parallel, Distrib. Syst., vol. 24, no. 6, pp. 1087-1096, Jun. 2013.
8. G. Chen, X. Bai, X. Huang, M. Li, and L. Zhou, "Evaluating services on the cloud using ontology QoS model," in Proc. IEEE Int. Symp. Service Oriented Syst. Eng., 2011, pp. 312-317.
9. Y. Chen, A. Ganapathi, R. Griffith, and R. Katz, "The case for evaluating mapreduce performance using workload suites," in Proc. IEEE Int. Symp. Model., Anal. Simulation Comput. Telecom Syst., 2011 pp. 390-399.
10. CloudHarmony. (2014). Benchmark evaluation of 114 public clouds. [Online]. Available: http://cloudharmony.com/clouds
11. B. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, "Benchmarking cloud serving systems with YCSB," in Proc. 1st ACM Symp. Cloud Comput., 2010, pp. 143-154.
12. J. Dongarra, J. Martin, and Worlton, "Computer benchmarking: Paths and pitfalls," IEEE Spectrum, vol. 24, no. 7, pp. 38-43, Jul. 1986.
13. M. Farber and S. Kounev, "Existing cloud benchmark efforts and proposed next steps," Slide Presentation, Karlsruhe Inst.Technol. (KIT), Karlsruhe, Germany, Aug. 31, 2011.
14. M. Ferdman, A. Adileh, O. Kocberber, S. Volos, M. Alisafaee, D. Jevdjic, C. Kaynak, A. D. Popescu, A. Ailamaki, and B. Falsafi, "Clearing the clouds: A study of emerging scale-out workloads on modern hardware," presented at the ACM 17th Int. Conf. Architectural Support Program. Languages Operating Syst., London, United Kingdom, Mar. 2012.
15. E. Folkerts, A. Alexander, K. Sacks, A. Iosup, Markl, and C. Tosun, "Benchmarking in the cloud: What it should, can and cannot be," presented at the 4th TPC Technol. Conf. Performance Eval. Benchmarking, Istanbul, Turkey, Aug. 2012.
16. J. Gao, X. Bai, and W. T. Tsai, "Cloud-testing: Issues, challenges, needs and practice," Int. J. Softw. Eng., vol. 1, no. 1, pp. 9-23, 2011.
17. A. Gupta and V. Kumar, "Performance properties of large scale parallel systems," in Proc. 26th Hawaii Int. Conf. Syst. Sci., 1993, pp. 234-244.
18. N. Herbst, S. Kounev, and R. Reussner, "Elasticity in cloud computing: What it is, and what it is not," presented at the Int. Conf. Autonomic Comput., San Jose, CA, USA, Jun. 2013.
19. M. Hill "What is scalability," ACM SIGARCH Comput. Archit. News, 1990.
20. S. Huang, J. Huang, J. Dai, and T. Xie, and B. Hong, "The hi-bench benchmark suite: Characterization of the mapreduce-based data analysis," in Proc. Int. Conf. Data Eng. Workshops, Mar. 2010, pp. 41-51.
21. K. Hwang, G. Fox, and J. Dongarra, Distributed and Cloud Computing, Morgan Kaufmann Publisher, 2012.
22. K. Hwang and Z. Xu, Scalable Parallel Computing, Chapter 2 on Performance Benchmarking, McGraw-Hill, 1998.
23. K. Hwang, Y. Shi, and X. Bai, "Scale-out and scale-up techniques for cloud performance and productivity," presented at the IEEE Cloud Comput. Sci., Technol. Appl. (CloudCom), Singapore, Dec. 18, 2014.
24. A. Iosup, S. Ostermann, M. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema, "Performance Analysis of cloud computing services for many-tasks scientific computing," IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 6, pp. 931-945, 2011.
25. R. Krebs, C. Momm, and S. Knounev, "Metrics and techniques for quantifying performance isolation in cloud environments," presented at the ACM QoSA, Bertino, Italy, Jun. 2012.
26. A. Li, X. Yang, S. Kandula, and M. Zhang, "CloudCmp: Comparing public cloud providers," in Proc. 10th Annu. Conf. Internet Meas., Nov. 2010, pp. 1-14.
27. Z. Li, L. O'Brien, H. Zhang, and R. Cai, "On a catalogue of metrics for evaluating commercial cloud services," in Proc. ACM/IEEE 13th Int. Conf. Grid Comput., Sep. 2012, pp. 167-173.
28. P. Mell and T. Grance, "The NIST definition of cloud computing," NIST Special Pub., vol. 800, no. 145, Jul. 2011
29. M. Michael, J. Moreira, D. Shiloach, and R. Wisniewski, "Scale-up x scale-out : A case study using nutch/luene," in Proc. IEEE Int. Parallel Distrib. Proc. Symp., Mar. 26, 2007, pp. 1-8.
30. S. Ostermann, A. Iosup, N. Yigitbasi, R. Prodan, T. Fahringer, and, D. Epema, "A performance analysis of EC2 Cloud Computing services for Scientific computing," in Proceedings of the International Conference On Cloud Computing, Springer, 2010.
31. C. Pham "QoS for cloud computing," Slide presentation, Univ. Pau, Pau, France, May 10, 2011.
32. D. Plummer, D. M. Smith, T. J. Bittman, D. W. Cearley, D. J. Cappuccio, D. Scott, R. Kumar, and B. Robertson. (2009). Five refining attributes of public and private cloud computing. Gartner Research. [Online]. Available: http://gartner.com/DisplayDocumentdoc-cd167182
33. U Sharma, P. Shenoy, S. Sahu, and A. Shaikh, "A cost-aware elasticity provisioning system for the cloud," in Proc. IEEE Int. Conf. Distrib. Comput. Syst., Jun. 2011, pp. 559-570.
34. W. Smith, "TCP-W: Benchmarking: An E-commerce solution," Intel, 2005.
35. W. Sobel, S. Subramanyam, A. Sucharitakul, J. Nguyen, H. Wong, Patil, and D. Patterson, "Cloudstone: multi-platform, multi-language benchmark and measurement tools for web 2.0," in Proc. 1st Workshop Cloud Comput. Appl., Oct. 2008.
36. W. Tsai, Y. Huang, and Q. Shao, "Testing the scalability of SaaS applications," in Proc. IEEE Int. Conf. Service-Oriented Comput. Appl., 2011, pp. 1-4.
37. N. Yigitbasi, A. Iosup, D. Epema, and S. Ostermann, "C-Meter: A framework for performance analysis of computing clouds," in Proc. IEEE/ACM Proc. 9th Int. Symp. Cluster Comput. Grid, 2009, pp. 472-477.