Assessing time coalescence techniques for the analysis of supercomputer logs

Proceedings of the International Conference on Dependable Systems and Networks

Volumn , Issue , 2012, Pages

  Di Martino, Catello ^a   Cinque, Marcello ^b   Cotroneo, Domenico ^b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b UNIVERSITY OF NAPLES FEDERICO II   (Italy)

Author keywords

data coalescence; dependability assessment; Event Log Analysis; supercomputer dependability

Indexed keywords

AUTOMATICALLY GENERATED; DEPENDABILITY ASSESSMENT; EVENT LOG ANALYSIS; FAILURE PROCESS; GROUND TRUTH;

ERRORS; SUPERCOMPUTERS;

COALESCENCE;

EID: 84866712387     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DSN.2012.6263946     Document Type: Conference Paper

References (36)

1. Y. Liang, Y. Zhang, A. Sivasubramaniam, M. Jette, and R. Sahoo. Bluegene/l failure analysis and prediction models. DSN 2006: Int. Conference on Dependable Systems and Networks, pages 425-434, 2006.
2. Bianca Schroeder and Garth A. Gibson. Understanding disk failure rates: What does an mttf of 1,000,000 hours mean to you? IEEE Trans. Storage, 3(3), October 2007.
3. A. Oliner and J. Stearley. What supercomputers say: A study of five system logs. Dependable Systems and Networks, 2007. DSN '07. 37th Annual IEEE/IFIP Int. Conference on, pages 575-584, June 2007.
4. Michael M. Tsao and Dan P. Siewiorek. Trend analysis on system error files. In Thirteenth Annual International Symposium on Fault-Tolerant Computing, pages 116-119, Jul 1983.
5. J.P. Hansen and D.P. Siewiorek. Models for time coalescence in event logs. Fault-Tolerant Computing, 1992. FTCS-22. Digest of Papers., Twenty-Second Int. symp. on, pages 221-227, Jul 1992.
6. R. Lal and G. Choi. Error and failure analysis of a unix server. High-Assurance Systems Engineering symp., 1998. Proc. Third IEEE Int., pages 232-239, Nov 1998.
7. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In DSN '04: Proc. of the 2004 Int. Conference on Dependable Systems and Networks, page 772, Washington, DC, USA, 2004. IEEE Computer Society.
8. Y. Liang, A. Sivasubramaniam, J. Moreira, Y. Zhang, R.K. Sahoo, and M. Jette. Filtering failure logs for a bluegene/l prototype. In DSN '05: Proc. of the 2005 Int. Conference on Dependable Systems and Networks, pages 476-485, Washington, DC, USA, 2005. IEEE Computer Society.
9. A. Thakur and R.K. Iyer. Analyze-now - an environment for collection and analysis of failures in a network of workstations. Reliability, IEEE Transactions on, 45(4):561-570, Dec 1996.
10. M. Kalyanakrishnam, Z. Kalbarczyk, and R. Iyer. Failure data analysis of a lan of windows nt based computers. In Proceedings of the 18th IEEE Symposium on Reliable Distributed Systems SRDS 99, pages 178-187, 1999.
11. A. Ganapathi and D. Patterson. Crash data collection: A windows case study. In DSN '05: Proc. of the 2005 Int. Conference on Dependable Systems and Networks, pages 280-285, Washington, DC, USA, 2005. IEEE Computer Society.
12. C. Simache, M. Kaâniche, and A. Saidane. Event log based dependability analysis of windows nt and 2k systems. In PRDC '02: Proc. of the 2002 Pacific Rim Int. symp. on Dependable Computing, page 311, Washington, DC, USA, 2002. IEEE Computer Society.
13. J. Xu, Z. Kalbarczyk, and R. K. Iyer. Networked windows nt system field failure data analysis. In PRDC '99: Proc. of the 1999 Pacific Rim Int. symp. on Dependable Computing, page 178, Washington, DC, USA, 1999. IEEE Computer Society.
14. J. Stearley and A.J. Oliner. Bad words: Finding faults in spirit's syslogs. In Cluster Computing and the Grid, 2008. CCGRID '08. 8th IEEE International Symposium on, pages 765-770, may 2008.
15. D.D. Deavours, G. Clark, T. Courtney, D. Daly, S. Derisavi, J.M. Doyle, W.H. Sanders, and P.G. Webster. The mobius framework and its implementation. Software Engineering, IEEE Transactions on, 28(10):956 - 969, oct 2002.
16. Chinghway Lim, N. Singh, and S. Yajnik. A log mining approach to failure analysis of enterprise telephony systems. In Dependable Systems and Networks With FTCS and DCC, 2008. DSN 2008. IEEE International Conference on, pages 398-403, june 2008.
17. C. Di Martino, D. Cotroneo, Z. Kalbarczyk, and R. K. Iyer. A framework for assessing the dependability of supercomputers via automated log analysis. In Sup. volume of Proc. of the Int. Conference on Dependable Systems and Networks, Anchorage, AK., pages 383-384, 2008.
18. Thomas J. Hacker, Fabian Romero, and Christopher D. Carothers. An analysis of clustered failures on large supercomputing systems. Journal of Parallel and Distributed Computing, 69(7):652 - 665, 2009.
19. J. Brevik, D. Nurmi, and R. Wolski. Automatic methods for predicting machine availability in desktop grid and peer-to-peer systems. In Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid, CCGRID '04, pages 190-199, Washington, DC, USA, 2004. IEEE Computer Society.
20. B. Schroeder and G. A. Gibson. A large-scale study of failures in high-performance computing systems. In DSN '06: Proc. of the Int. Conference on Dependable Systems and Networks, pages 249-258, Washington, DC, USA, 2006. IEEE Computer Society.
21. Taliver Heath, Richard P. Martin, and Thu D. Nguyen. Improving cluster availability using workstation validation. In Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, SIGMETRICS '02, pages 217-227, New York, NY, USA, 2002. ACM.
22. Thomas J. Hacker and Zdzislaw Meglicki. Using queue structures to improve job reliability. In Proceedings of the 16th international symposium on High performance distributed computing, HPDC '07, pages 43-54, New York, NY, USA, 2007. ACM.
23. S. Fu and C. Xu. Exploring event correlation for failure prediction in coalitions of clusters. In SC '07: Proc. of the 2007 ACM/IEEE conference on Supercomputing, pages 1-12. ACM, 2007.
24. D. Tang and R.K. Iyer. Analysis and modeling of correlated failures in multicomputer systems. Computers, IEEE Transactions on, 41(5):567-577, may 1992.
25. X. Castillo and D.P. Siewiorek. Workload, performance, and reliability of digital computing systems. Fault-Tolerant Computing, 1995, 'Highlights from Twenty-Five Years'., Twenty-Fifth Int. symp. on, pages 367-, Jun 1995.
26. T. T. Y. Lin and D. P. Siewiorek. Error log analysis: statistical modeling and heuristic trend analysis. Reliability, IEEE Transactions on, 39(4):419-432, 1990.
27. Ramendra K. Sahoo, Anand Sivasubramaniam, Mark S. Squillante, and Yanyong Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proceedings of the 2004 International Conference on Dependable Systems and Networks, pages 772-, Washington, DC, USA, 2004. IEEE Computer Society.
28. Antonio Pecchia, Domenico Cotroneo, Zbigniew Kalbarczyk, and Ravishankar K. Iyer. Improving log-based field failure data analysis of multi-node computing systems. Dependable Systems and Networks, International Conference on, 0:97-108, 2011.
29. R. Vaarandi. Mining event logs with slct and loghound. In Network Operations and Management Symposium, 2008. NOMS 2008. IEEE, pages 1071-1074, april 2008.
30. Ziming Zheng, Zhiling Lan, B.H. Park, and A. Geist. System log pre-processing to improve failure prediction. In Dependable Systems Networks, 2009. DSN '09. IEEE/IFIP International Conference on, pages 572-577, 29 2009-july 2 2009.
31. M. F. Buckley and D. P. Siewiorek. A comparative analysis of event tupling schemes. In FTCS '96: Proc. of the The Twenty-Sixth Annual Int. symp. on Fault-Tolerant Computing (FTCS '96), page 294, Washington, DC, USA, 1996. IEEE Computer Society.
32. Ziming Zheng, Li Yu, Wei Tang, Zhiling Lan, Rinku Gupta, Narayan Desai, Susan Coghlan, and Daniel Buettner. Co-analysis of ras log and job log on blue gene/p. In Proceedings of the 2011 IEEE International Parallel & Distributed Processing Symposium, IPDPS '11, pages 840-851, Washington, DC, USA, 2011. IEEE Computer Society.
33. I. Lee and R. K. Iyer. Software dependability in the tandem guardian system. IEEE Trans. Softw. Eng., 21(5):455-467, 1995.
34. Xin Li, Kai Shen, Michael C. Huang, and Lingkun Chu. A memory soft error measurement on production systems. In 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, pages 21:1-21:6, Berkeley, CA, USA, 2007. USENIX Association.
35. Francesco Palmieri and Silvio Pardi. Towards a federated metropolitan area grid environment: The scope network-aware infrastructure. Future Generation Computer Systems, 26(8):1241 - 1256, 2010.
36. Francesco Palmieri, Silvio Pardi, and Paolo Veronesi. A fault avoidance strategy improving the reliability of the egi production grid infrastructure. In Proceedings of the 14th international conference on Principles of distributed systems, OPODIS'10, pages 159-172, Berlin, Heidelberg, 2010. Springer-Verlag.