Observing the clouds: a survey and taxonomy of cloud monitoring

Journal of Cloud Computing

Volumn 3, Issue 1, 2014, Pages

  Ward, Jonathan Stuart ^a   Barker, Adam ^a  

^a UNIVERSITY OF ST ANDREWS   (United Kingdom)

Author keywords

Cloud computing; Monitoring

Indexed keywords

CLOUD COMPUTING; CLUSTER COMPUTING; GRID COMPUTING; LARGE SCALE SYSTEMS; SURVEYS; TAXONOMIES;

CLOUD MONITORING; ENGINEERING CHALLENGES; ENTERPRISE COMPUTING; HIGH PERFORMANCE COMPUTING; MONITORING STRATEGY; MONITORING TOOLS; ON DEMANDS; SOCIO-TECHNICAL ASPECTS;

MONITORING;

EID: 84926427503     PISSN: None     EISSN: 2192113X     Source Type: Journal    
DOI: 10.1186/s13677-014-0024-2     Document Type: Article

References (98)

1. Tremante M (2013) Amazon Web Services’ growth unrelenting. http://news.netcraft.com/archives/2013/05/20/amazon-web-services-growth-unrelenting.html.
2. Mendoza D (2012) Amazon outage takes down Reddit, Foursquare, others - CNN.com. http://edition.cnn.com/2012/10/22/tech/web/reddit-goes-down/, 2012.
3. Mell P, Grance T (2011) The NIST Definition of Cloud Computing. Technical Report 800-145, National Institute of Standards and Technology (NIST), Gaithersburg, MD.
4. Armbrust M, Stoica I, Zaharia M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A (2010) A view of cloud computing. Commun ACM 53(4): 50.
5. Subramaniyan R, Raman P, George AD, Radlinski M (2006) GEMS: Gossip-Enabled Monitoring Service for Scalable Heterogeneous Distributed Systems. Cluster Computing 9(1): 101–120.
6. Povedano-Molina J, Lopez-Vega JM, Lopez-Soler JM, Corradi A, Foschini L (2013) DARGOS: A highly adaptable and scalable monitoring architecture for multi-tenant Clouds. Future Generation Comput Syst 29(8): 2041–2056.
7. Konig B, Alcaraz Calero JM, Kirschnick J (1306) Elastic monitoring framework for cloud infrastructures. IET Commun 6(10).
8. Van Renesse R, Birman KP, Vogels W (2003) Astrolabe. ACM Trans Comput Syst 21(2): 164–206.
9. Riemann. http://riemann.io/.
10. cloudinit.d. http://www.nimbusproject.org/doc/cloudinitd/1.2/.
11. El-Khamra Y, Kim H, Jha S, Parashar M (2010) Exploring the Performance Fluctuations of HPC Workloads on Clouds In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 383–387. IEEE.
12. Amazon Web Services FAQs. https://aws.amazon.com/ec2/faqs/.
13. Machine Types - Google Compute Engine - Google Developers. https://developers.google.com/compute/docs/machine-types.
14. Got Steal? | CopperEgg. http://copperegg.com/got-steal/.
15. Link D (2011) Netflix and Stolen Time. http://blog.sciencelogic.com/netflix-steals-time-in-the-cloud-and-from-users/03/2011.
16. Avresky DR, Diaz M, Bode A, Ciciani B, Dekel E (2010) A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing, volume 34 of Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering. Springer Berlin Heidelberg, Berlin, Heidelberg.
17. Top, 500.orgAmazon EC2 Cluster Compute Instances - Amazon EC2 Cluster, Xeon X5570 2.95 Ghz, 10G Ethernet | TOP500 Supercomputer Sites. http://www.top500.org/system/10661.
18. BuisnessWeekAnother Amazon Outage Exposes the Cloud’s Dark Lining - Businessweek. http://www.businessweek.com/articles/2013-08-26/another-amazon-outage-exposes-the-clouds-dark-lining.
19. ZDNetAmazon Web Services suffers outage, takes down Vine, Instagram, others with it, ZDNet. http://www.zdnet.com/amazon-web-services-suffers-outage-takes-down-vine-instagram-flipboard-with-it-7000019842/.
20. Ec, 2Global Infrastructure. https://aws.amazon.com/about-aws/globalinfrastructure/?nc1=h_l2_cc.
21. AzureMicrosoft Azure Trust Center - Privacy. http://azure.microsoft.com/en-us/support/trust-center/privacy/.
22. Ward JS, Barker A (2012) Semantic based data collection for large scale cloud systems In: Proceedings of the fifth international workshop on Data-Intensive Distributed Computing Date, 13–22. ACM, New York, NY, USA.
23. Foster I, Zhao Y, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared In: Grid Computing Environments Workshop, 2008. GCE’08, 1–10. IEEE.
24. Aceto G, Botta A (2013) Walter de Donato and Antonio Pescapè. Cloud monitoring: A survey. Comput Netw 57(9): 2093–2115.
25. Deelman E, Singh G, Livny M, Berriman B, Good J (2008) The cost of doing science on the cloud: The Montage example.1–12.
26. Greenberg A, Hamilton J, Maltz DA, Patel P (2008) The cost of a cloud. ACM SIGCOMM Comput Commun Rev 39(1): 68.
27. Sarathy V, Narayan P, Mikkilineni R (2010) Next Generation Cloud Computing Architecture: Enabling Real-Time Dynamism for Shared Distributed Physical Infrastructure In: 2010 19th IEEE International, Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises, 48–53. IEEE.
28. Ian Foster, Yong Zhao, Ioan Raicu, Shiyong L uCloud Computing and Grid Computing 360-Degree Compared In: 2008 Grid Computing Environments Workshop, 1–10. IEEE.
29. Kozuch MA, Ganger GR, Ryan MP, Gass R, Schlosser SW, O’Hallaron D, Cipar J, Krevat E, López J, Stroucken M (2009) Tashi In: Proceedings of the 1st workshop on, Automated control for datacenters and clouds - ACDC ’09, 43. ACM Press, New York, New York, USA.
30. Marshall P, Keahey K, Freeman T (2011) Improving Utilization of Infrastructure Clouds In: 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 205–214. IEEE.
31. Massie ML, Chun BN, Culler DE (2004) The ganglia distributed monitoring system: design, implementation, and experience. Parallel Comput 30(7): 817–840.
32. Tobias OetikerRRDtool. http://oss.oetiker.ch/rrdtool/.
33. Eisler MXDR: External Data Representation Standard. http://tools.ietf.org/html/rfc4506.
34. Daniel Pocockganglia-nagios-bridge. https://github.com/ganglia/ganglia-nagios-bridge.
35. NagiosNagios - The Industry Standard in IT Infrastructure Monitoring. http://www.nagios.org/.
36. Intellectual ReserveDistributed Nagios Executor (DNX). http://dnx.sourceforge.net/.
37. NagiosConfigMain Configuration File Options. http://nagios.sourceforge.net/docs/3_0/configmain.html.
38. Kowall JGot Nagios? Get rid of it. http://blogs.gartner.com/jonah-kowall/2013/02/22/got-nagios-get-rid-of-it/.
39. Collectdcollectd – The system statistics collection daemon. http://collectd.org/.
40. RightscaleCloud Portfolio Management by RightScale. http://www.rightscale.com/.
41. Logstash. http://logstash.net/.
42. statsd. https://github.com/etsy/statsd/.
43. Bucky. https://github.com/trbs/bucky.
44. Graphite. http://graphite.wikidot.com/.
45. drraw. http://web.taranis.org/drraw/.
46. Cabot. http://cabotapp.com/.
47. Google (2014) Google Protocol Buffers. https://developers.google.com/protocol-buffers/docs/overview.
48. sFlow. http://www.sflow.org/.
49. Host sFlow. http://host-sflow.sourceforge.net/.
50. PeakFlow. http://www.arbornetworks.com/products/peakflow.
51. Scrutinizer sFlow Analyzer. http://www.plixer.com/Scrutinizer-Netflow-Sflow/scrutinizer.html.
52. MonALISA. http://monalisa.caltech.edu/monalisa.htm.
53. Legrand I, Newman H, Voicu R, Cirstoiu C, Grigoras C, Dobre C, Muraru A, Costan A, Dediu M, Stratan C (2009) MonALISA: An agent based, dynamic service system to monitor, control and optimize distributed systems. Comput Phys Commun 180(12): 2472–2498.
54. Arnold K, Scheifler R, Waldo J, O’Sullivan B, Wollrath A (1999) Jini Specification. http://dl.acm.org/citation.cfm?id=554054.
55. Lee D, Dongarra JJ, Ramakrishna RS (2003) visperf: Monitoring tool for grid computing In: Sloot PMA, Abramson D, Bogdanov AV, Gorbachev YE, Dongarra JJ, Zomaya AY (eds) Computational Science – ICCS 2003 volume 2659 of Lecture Notes in Computer Science, 233–243. Springer Berlin Heidelberg.
56. Kermarrec A-M, Massoulie L, Ganesh AJ (2003) Probabilistic reliable dissemination in large-scale systems. IEEE Trans Parallel Distributed Syst 14(3): 248–258.
57. GEMS Java Implementation. https://github.com/nsivabalan/gems.
58. OmniTI Labs (2014) Reconnoiter. https://labs.omniti.com/labs/reconnoiter.
59. Icinga. https://www.icinga.org/.
60. Zenoss (2014) Zenoss. http://www.zenoss.com/.
61. The Cacti Group (2014) Cacti. http://www.cacti.net/.
62. GroundWork. http://www.gwos.com/.
63. Munin. http://munin-monitoring.org/.
64. OpenNMSThe OpenNMS Project. http://www.opennms.org/.
65. SpiceworksSpiceworks. http://www.spiceworks.com/.
66. ZabbixZabbix. http://www.zabbix.com/.
67. cloudinit.d monitoring README. https://github.com/nimbusproject/cloudinit.d/blob/master/docs/monitor.txt.
68. Sensu. http://sensuapp.org/.
69. RabbitMQ. https://www.rabbitmq.com/.
70. Bloom AHow fast is a Rabbit? Basic RabbitMQ Performance Benchmarks. https://blogs.vmware.com/vfabric/2013/04/how-fast-is-a-rabbit-basic-rabbitmq-performance-benchmarks.html.
71. An K, Pradhan S, Caglar F, Gokhale A (2012) A Publish/Subscribe Middleware for Dependable and Real-time Resource Monitoring in the Cloud In: Proceedings of the Workshop on Secure and Dependable Middleware for Cloud Monitoring and Management, 3:1–3:6. ACM, New York, NY, USA.
72. Goldsack P, Guijarro J, Loughran S, Coles A, Farrell A, Lain A, Murray P, Toft P (2009) The SmartFrog configuration management framework. ACM SIGOPS Operating Syst Rev 43(1): 16.
73. Dhingra M, Lakshmi J, Nandy SK (2012) Resource Usage Monitoring in Clouds In: Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing, 184–191. IEEE Computer Society, Washington, DC, USA.
74. Pardo-Castellote G (2003) OMG data-distribution service: architectural overview In: 23rd International Conference on Distributed Computing Systems Workshops, 2003. Proceedings, 200–206. IEEE.
75. Kung HT, Lin C-k, Vlah D (2011) CloudSense: Continuous Fine-Grain Cloud Monitoring With Compressive Sensing.
76. Candes EJ, Wakin MB (2008) An Introduction To Compressive Sampling. Signal Processing Magazine, IEEE: 21–30.
77. Montes J, Sánchez A, Memishi B, Pérez MS, Antoniu G (2013) GMonE: A complete approach to cloud monitoring. Future Generation Comput Syst 29(8): 2026–2040.
78. Lakshman A, Malik P (2010) Cassandra: A Decentralized Structured Storage System. SIGOPS Oper. Syst. Rev. 44(2): 35–40.
79. Clayman S, Galis A, Mamatas L (2010) Monitoring virtual networks with Lattice In: Network Operations and Management Symposium Workshops (NOMS Wksps), 2010 IEEE/IFIP, 239–246.
80. OpenNebula. http://opennebula.org/.
81. OpenNebulaKVM and Xen UDP-push Monitoring. http://docs.opennebula.org/4.4/administration/monitoring/imudppushg.html.
82. Melis JOpenNebula 4.4: New Massively Scalable Monitoring Driver. http://opennebula.org/opennebula-4-4-new-massively-scalable-monitoring-driver/.
83. De Chaves SA, Uriarte RB, Westphall CB (2011) Toward an architecture for monitoring private clouds. Communications Magazine, IEEE 49(12): 130–137.
84. PCMONS Download. https://github.com/pedrovitti/pcmons.
85. Eucalyptus: Open Source Private Cloud Software. https://www.eucalyptus.com/eucalyptus-cloud/iaas.
86. Ward JS, Barker A (2013) Varanus: In Situ Monitoring for Large Scale Cloud Systems In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science, 341–344. IEEE.
87. Amazon CloudWatch. http://aws.amazon.com/cloudwatch/.
88. HP Cloud Monitoring. http://www.hpcloud.com/products-services/monitoring.
89. HP Public Cloud. http://www.hpcloud.com/.
90. Monitoring System - RightScale. http://support.rightscale.com/12-Guides/RightScale_101/08-Management_Tools/Monitoring_System.
91. New Relic. http://newrelic.com/.
92. CopperEgg. http://copperegg.com/.
93. CloudSleuth. https://cloudsleuth.net/.
94. Network & IT Systems Monitoring | Monitis - Monitor Everything. http://www.monitis.com/.
95. Cloud Monitoring as a Service for AWS and Rackspace. https://www.stackdriver.com/.
96. Boundary: Unified Monitoring for Web-Scale IT and Modern IT Operations Monitoring. http://boundary.com/.
97. Multi cloud Optimization: AWS & Google Cloud Services | Cloud Analytics | Cloudyn | Actionable Recommendations. http://www.cloudyn.com/.
98. Netflix Chaos Monkey. http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html.