DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool

Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016

Volumn , Issue , 2016, Pages 213-222

  Varghese, Blesson ^a   Subba, Lawan Thamsuhang ^b   Thai, Long ^c   Barker, Adam ^c  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^b AALBORG UNIVERSITY   (Denmark)

^c UNIVERSITY OF ST ANDREWS   (United Kingdom)

Author keywords

cloud benchmarking; containers; Docker; hybrid benchmark; lightweight benchmark

Indexed keywords

CLUSTER COMPUTING; CONTAINERS; GRID COMPUTING; VIRTUAL MACHINE;

ACCURATE PERFORMANCE; BENCHMARKING METHODS; BENCHMARKING TOOLS; CLOUD BENCHMARKING; DOCKER; DOCKER CONTAINERS; PARALLEL EXECUTIONS; REAL-TIME ANALYTICS;

BENCHMARKING;

EID: 84983391446     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2016.14     Document Type: Conference Paper

References (27)

1. A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer and D. Epema, "Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing," IEEE Transactions on Parallel and Distributed Systems, Vol. 22, Issue 6, 2011, pp. 931-945.
2. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan and R. Sears, "Benchmarking Cloud Serving Systems with YCSB," Proceedings of the 1st ACM Symposium on Cloud Computing, 2010, pp. 143-154.
3. B. Varghese, O. Akgun, I. Miguel, L. Thai and A. Barker, "Cloud Benchmarking for Performance," Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science, 2014, pp. 535-540.
4. E. Folkerts, A. Alexandrov, K. Sachs, A. Iosup, V. Markl and C. Tosun, "Benchmarking in the Cloud: What it Should, Can, and Cannot Be," Selected Topics in Performance Evaluation and Benchmarking (Editors: R. Nambiar and M. Poess), Lecture Notes in Computer Science, 2013.
5. A. Barker, B. Varghese and L. Thai, "Cloud Services Brokerage: A Survey and Research Roadmap," Proceedings of the 8th IEEE International Conference on Cloud Computing, 2015, pp. 1029-1032.
6. A. Barker, B. Varghese, J. S. Ward and I. Sommerville "Academic Cloud Computing Research: Five Pitfalls and Five Opportunities," 6th USENIX Workshop on Hot Topics in Cloud Computing, 2014.
7. C.,erson, "Docker [Software Engineering]," IEEE Software, Vol.32, No.3, 2015, pp. 102-105.
8. J. Bottomley and P. Emelyanov, "Containers," USENIX ;login:, Vol. 39, No. 5, 2014, pp. 6-10.
9. W. Felter, A. Ferreira, R. Rajamony and J. Rubio, "An Updated Performance Comparison of Virtual Machines and Linux Containers," IBM Research Report, RC25482 (AUS1407-001), 2014.
10. D. Bernstein, "Containers and Cloud: From LXC to Docker to Kubernetes," IEEE Cloud Computing, Vol. 1, Issue 3, 2014, pp. 81-84.
11. VMWare Blogs, "Docker Containers Performance in VMware vSphere," https://blogs.vmware.com/performance/2014/10/dockercontainers-performance-vmware-vsphere.html [Last accessed: 01 February 2016]
12. VMWare Blogs, "Running Transactional Workloads Using Docker Containers on vSphere 6.0," https://blogs.vmware.com/performance/2015/05/running-transactionalworkloads-using-docker-containers-vsphere-6-0.html [Last accessed: 01 February 2016]
13. C. Staelin, "lmbench-An Extensible Micro-benchmark Suite," Software-Practice and Experience, Vol. 35, Issue 11, 2005, pp. 1079-1105.
14. S. Ostermann, A. Iosup, R. Prodan, T. Fahringer and D. Epema, "A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing," Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, Vol. 34, 2010, pp 115-131.
15. K. J. Bowers, E. Chow, H. Xu, R. O. Dror, M. P. Eastwood, B. A. Gregersen, J. L. Klepeis, I. Kolossvary, M. A. Moraes, F. D. Sacerdoti, J. K. Salmon, Y. Shan and D. E. Shaw, "Scalable Algorithms for Molecular Dynamics Simulations on Commodity Clusters," Proceedings of the ACM/IEEE Conference on Supercomputing, Article No. 84, 2006.
16. A. K. Bahl, O. Baltzer, A. Rau-Chaplin, and B. Varghese, "Parallel Simulations for Analysing Portfolios of Catastrophic Event Risk," SC Companion in International Conference of High Performance Computing, Networking, Storage and Analysis, 2012, pp. 1176-1184.
17. D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, L. Dagum, R. A. Fatoohi, P. O. Frederickson, T. A. Lasinski, R. S. Schreiber, H. D. Simon, V. Venkatakrishnan, S. K. Weeratunga, "The NAS Parallel Benchmarks-Summary and Preliminary Results," Proceedings of the ACM/IEEE Conference on Supercomputing, 1991, pp. 158-165.
18. D. Kaeli and K. Sachs, "Computer Performance Evaluation and Benchmarking," Lecture Notes on Computer Science 5419, Springer, 2009.
19. J. J. Dongarra, P. Luszczek and A. Petitet, "The LINPACK Benchmark: Past, Present and Future," Concurrency and Computation: Practice and Experience, Vol. 15, 2013, pp. 803-820.
20. Z. Li, L. O'Brien, H. Zhang and R. Cai, "On a Catalogue of Metrics for Evaluating Commercial Cloud Services," Proceedings of the 13th International Conference on Grid Computing, 2012, pp.164-173.
21. Brendan Gregg's Blog, "Performance Tuning Linux Instances on EC2," http://www.brendangregg.com/blog/2015-03-03/performancetuning-linux-instances-on-ec2.html [Last accessed: 01 February 2016]
22. S. Soltesz, H. Potzl, M. E. Fiuczynski, A. Bavier and L. Peterson, "Container-based Operating System Virtualization: A Scalable, Highperformance Alternative to Hypervisors," Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, 2007, pp. 275-287.
23. M. G. Xavier, M. V. Neves, F. D. Rossi, T. C. Ferreto, T. Lange, C. A. F. De Rose, "Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments," Proceedings of the 21st Euromicro International Conference on Parallel, Distributed and Network-Based Processing, 2013, pp. 233-240.
24. W. Li, A. Kanso and A. Gherbi, "Leveraging Linux Containers to Achieve High Availability for Cloud Services," Proceedings of the IEEE International Conference on Cloud Engineering, 2015, pp. 76-83.
25. H. Yoon, M. Ravichandran and K. Schwan, "Distributed Cloud Storage Services with FleCS Containers," Open Cirrus Summit, 2011.
26. C. Boettiger, "An Introduction to Docker for Reproducible Research," ACM SIGOPS Operating Systems Review-Special Issue on Repeatability and Sharing of Experimental Artifacts, Vol. 49, Issue 1, 2015, pp. 71-79.
27. T. Bui, "Analysis of Docker Security," arXiv:1501.02967 [cs.CR], 2015.