What supercomputers say: A study of five system logs

Proceedings of the International Conference on Dependable Systems and Networks

Volumn , Issue , 2007, Pages 575-584

  Oliner, Adam ^a,c   Stearley, Jon ^b  

^a Stanford University   (United States)

^b SANDIA NATIONAL LABORATORIES   (United States)

^c SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FILTERING ALGORITHMS; FIVE SYSTEM LOGS; RED STORM; THUNDERBIRD;

ALGORITHMS; COMPUTER SYSTEMS; DATA ACQUISITION; INFORMATION ANALYSIS; PROGRAM PROCESSORS;

SUPERCOMPUTERS;

EID: 36049013419     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DSN.2007.103     Document Type: Conference Paper

References (28)

1. Cray red storm architecture documents. http://www.cray.com/products/xt3/ index.html, 2006.
2. Top500 supercomputing sites, http://www.top500.org/, June 2006.
3. N. R. Adiga and The BlueGene/L Team. An overview of the bluegene/l supercomputer. In Proceedings of ACM Supercomputing, 2002.
4. M. E Buckley and D. P. Siewiorek. A comparative analysis of event tupling schemes. In FTCS-26, Intl. Symp. on Fault Tolerant Computing, pages 294-303, June 1996.
5. D. G. Feitelson and D. Tsafrir. Workload sanitation for performance evaluation. In IEEE Intl. Symp. Performance Analysis Syst. & Software (ISPASS), pages 221-230, Mar 2006.
6. U. Flegel. Pseudonymizing unix log files. In Proceedings of the Infrastructure Security Conference (InfraSec), 2002.
7. J. L. Hellerstein, S. Ma, and C. Perng. Discovering actionable patterns in event data. IBM Systems Journal, 41(3), 2002.
8. I. Lee and R. K. Iyer. Faults, symptoms, and software fault tolerance in the tandem guardian90 operating system. In Fault-Tolerant Computing. FTCS-23. Digest of Papers., The Twenty-Third International Symposium on, pages 20-29, 1993.
9. Y. Liang, Y. Zhang, M. Jette, A. Sivasubramaniam, and R. K. Sahoo. Blue gene/l failure analysis and prediction models. In Proceedings of the Intl. Conf. on Dependable Systems and Networks (DSN), pages 425-434, 2006.
10. Y. Liang, Y. Zhang, A. Sivasubramaniam, R. K. Sahoo, J. Moreira, and M. Gupta. Filtering failure logs for a bluegene/l prototype. In Proceedings of the Intl. Conf. on Dependable Systems and Networks (DSN), pages 476-485, June 2005.
11. T. T. Y. Lin and D. P. Siewiorek. Error log analysis: statistical modeling and heuristic trend analysis. Reliability, IEEE Transactions on, 39(4):419-432, 1990.
12. S. Ma and J. Hellerstein. Mining partially periodic event patterns with unknown periods. In Proceedings of the International Conference on Data Engineering (ICDE), pages 409-416, 2001.
13. F. A. Nassar and D. M. Andrews. A methodology for analysis of failure prediction data. In Real-Time Systems Symposium, pages 160-166, December 1985.
14. A. Oliner, L. Rudolph, and R. Sahoo. Cooperative check-pointing theory. In Proceedings of the 20th Intl. Parallel and Distributed Processing Symposium (IPDPS), 2006.
15. A. Oliner, L. Rudolph, and R. K. Sahoo. Cooperative check-pointing: A robust approach to large-scale systems reliability. In Proceedings of the 20th Intl. Conf. on Supercomputing (ICS), Cairns, Australia, June 2006.
16. A. J. Oliner, L. Rudolph, R. K. Sahoo, J. E. Moreira, and M. Gupta. Probabilistic qos guarantees for supercomputing systems. In Proceedings of the Intl. Conf. on Dependable Systems and Networks (DSN), pages 634-643, 2005.
17. A. J. Oliner, R. K. Sahoo, J. E. Moreira, M. Gupta, and A. Sivasubramaniam. Fault-aware job scheduling for blue-gene/l systems. In Proceedings of the 18th Intl. Parallel and Distributed Processing Symposium (IPDPS), pages 64+, 2004.
18. J. Prewett. Analyzing cluster log files using logsurfer. In Proceedings of the 4th Annual Conference on Linux Clusters, 2003.
19. R. K. Sahoo, A. J. Oliner, I. Rish, M. Gupta, J. E. Moreira, S. Ma, R. Vilalta, and A. Sivasubramaniam. Critical event prediction for proactive management in large-scale computer clusters. In Proceedings of the 9th ACM SIGKDD, International Conference on Knowledge Discovery and Data Mining, pages 426-435. ACM Press, 2003.
20. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proceedings of the Intl. Conf. on Dependable Systems and Networks (DSN), pages 772-781, June 2004.
21. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance-computing systems. In Proceedings of the Intl. Conf. on Dependable Systems and Networks (DSN), Philadelphia, PA, June 2006.
22. B. Schroeder and G. Gibson. Disk failures in the real world: What does an mttf of 1,000,000 hours mean to you? In 5th Usenix Conference on File and Storage Technologies (FAST 2007), 2007.
23. J. Stearley. Towards informatic analysis of syslogs. In IEEE International Conference on Cluster Computing, pages 309-318, 2004.
24. J. Stearley. Defining and measuring supercomputer Reliability, Availability, and Serviceability (RAS). In Proceedings of the Linux Clusters Institute Conference, 2005. See http://www.cs.sandia.gov/~jrstear/ras.
25. D. Tang and R. K. Iyer. Analysis and modeling of correlated failures in multicomputer systems. Computers, IEEE Transactions on, 41(5):567-577, 1992.
26. M. M. Tsao. Trend Analysis and Fault Prediction. PhD dissertation, Carnegie-Mellon University, May 1983.
27. R. Vaarandi. A breadth-first algorithm for mining frequent patterns from event logs. In Proceedings of the 2004 IFIP International Conference on Intelligence in Communication Systems, volume 3283, pages 293-308, 2004.
28. K. Yamanishi and Y. Maruyama. Dynamic syslog mining for network failure monitoring. In Proceedings of the 11th ACM SIGKDD, International Conference on Knowledge Discovery and Data Mining, pages 499-508, New York, NY, USA, 2005. ACM Press.