An online visualization system for streaming log data of computing clusters

Tsinghua Science and Technology

Volumn 18, Issue 2, 2013, Pages 196-205

  Xia, Jing ^a   Wu, Feiran ^a   Guo, Fangzhou ^a   Xie, Cong ^a   Liu, Zhen ^b   Chen, Wei ^a  

^a ZHEJIANG UNIVERSITY   (China)

^b HANGZHOU DIANZI UNIVERSITY   (China)

Author keywords

Computing cluster; Performance metrics monitoring; Streaming data; Visualization

Indexed keywords

CLUSTER COMPUTING; FLOW VISUALIZATION; VISUALIZATION;

COMPUTING CLUSTERS; INTEGRATED VISUALIZATION SYSTEMS; ONLINE VISUALIZATIONS; PARALLEL COORDINATES; PERFORMANCE METRICS; STREAMING DATA; SYSTEM FRAMEWORK; VISUALIZATION TECHNIQUE;

DATA VISUALIZATION;

EID: 84897584155     PISSN: 10070214     EISSN: None     Source Type: Journal    
DOI: 10.1109/tst.2013.6509102     Document Type: Article

References (26)

1. J. J. Thomas and K. A. Cook, Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Computer Society, 2005.
2. Big data research and development initiative, http://www. whitehouse.gov/sites/default/files/microsites/ostp/big data press release final 2.pdf, 2012, Mar. 29.
3. C. C. Aggarwal, Data Streams: Models and Algorithms (Advances in Database Systems), Springer, 2007.
4. R. Anand and L. Jure, Mining of Massive Datasets, Cambridge University Press, 2011.
5. A. Konwinski and M. Zaharia, Anomaly detection in the data center. Technical report. RAD lab, 2008.
6. M. Kutare, G. Eisenhauer, C.Wang, K. Schwan, V. Talwar, and M.Wolf, Monalytics: Online monitoring and analytics for managing large scale data centers, in Proceeding of the 7th International Conference on Autonomic Computing, 2010, pp. 141-150.
7. M. Raffael, Applied Security Visualization, Indianapolis: Addison Wesley Professional, 2008, pp.164.
8. Jogent homepage, http://joyent.com/, 2012.
9. Splunk homepage, http://www.splunk.com/, 2012.
10. Loggly homepage, http://www.loggly.com/, 2012.
11. Logstash homepage, http://logstash.net/, 2012.
12. Elasticsearch homepage, http://www.elasticsearch.org/, 2012.
13. Sumo logic homepage, http://www.sumologic.com/, 2012.
14. Flume Github page, https://github.com/cloudera/flume, 2012.
15. Brendan Greggs computing cluster visualization blog, http://dtrace.org/ blogs/brendan/2011/12/18/visualizingdevice-utilization/, 2011, Dec. 18.
16. B. Topol, J. T. Stasko, and V. Sunderam, Integrating visualization support into distributed computing systems, in Proceedings of the 15th International Conference on Distributed Computing Systems, 1994, p. 19.
17. C. D.Carothers, B. Topol, R. M. Fujimoto, J. T. Stasko, and V. Sunderam, Visualizing parallel simulations in network computing environments: A case study, in Proceedings of the 29th Conference on Winter Simulation, 1997, pp. 110-117.
18. B. Topol, J. T. Stasko, and V. Sunderam, PVaniM: A tool for visualization in network computing environments, Concurrency: Practice and Experience, vol. 10, no. 14, pp. 1197-1222, 1998. (Pubitemid 128447175)
19. R. Ross, T. Peterka, H.-W. Shen, Y. Hong, K.-L. Ma, H. Yu, and K. Moreland, Visualization and parallel I/O at extreme scale, Journal of Physics: Conference Series, 2008, doi: 10.1088/1742-6596/125/1/012099.
20. H. Yu and K.-L. Ma, A study of I/O techniques for parallel visualization, Journal of Parallel Computing, vol. 31, no. 2, pp. 167-183, 2005. (Pubitemid 40612563)
21. H. Yu, K.-L. Ma, J. Welling, A parallel visualization pipeline for terascale earthquake simulations, in Proceedings of ACM/IEEE Supercomputing, 2004.
22. C. Muelder, C. Sigovan, K.-L. Ma, J. Cope, S. Lang, K. Iskra, P. Beckman, R. Ross, Visual analysis of I/O system behavior for high-end computing, in Proceedings of the Third InternationalWorkshop on Large-Scale System and Application Performance, 2011, pp. 19-26.
23. D. Zeckzer, R. Kalcklosch, L. Schroder, H. Hagen, T. Klein, Analyzing the reliability of communication between software entities using a 3D visualization of clustered graphs, in Proceedings of the 4th ACM Symposium on Software Visualization, 2008, pp. 37-46.
24. P. McLachlan, T. Munzner, E. Koutsofios, and S. North, LiveRAC: Interactive visual exploration of system management time-series data, in Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, 2008, pp. 1483-1492.
25. I. Tomonori, K. Shinya, J. Masato, M. Kyohei, E. Masatoshi, H. Hiroaki, and O. Takeshi, FUJIN: A cloud computing environment visualization system for specifying points of failure, http:// iplab.naist.jp/publications/, 2010.
26. S. Few, Information Dashboard Design: The Effective Visual Communication of Data, O'Reilly Media, Inc., 2006.