System log pre-processing to improve failure prediction

Proceedings of the International Conference on Dependable Systems and Networks

Volumn , Issue , 2009, Pages 572-577

  Zheng, Ziming ^a   Lan, Zhiling ^a   Park, Byung H ^b   Geist, Al ^b  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Cray XT4; Event categorization; Event filtering; IBM Blue Gene L; Log preprocessing

Indexed keywords

APRIORI; ASSOCIATION RULE MINING; BLUE GENE/L SYSTEM; COMPRESSION RATES; EVENT FILTERING; FAILURE PATTERNS; FAILURE PREDICTION; FILTERING METHOD; PRE-PROCESSING; PRE-PROCESSING METHOD; PREDICTIVE METHODS;

ASSOCIATIVE PROCESSING; FAILURE ANALYSIS; SAFETY FACTOR;

QUALITY ASSURANCE;

EID: 70449794134     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DSN.2009.5270289     Document Type: Conference Paper

References (18)

1. A. Oliner and J. Stearly, "What Supercomputers Say: A Study of Five System Logs," Proc. of DSN, 2007.
2. Y. Liang, Y. Zhang, A. Sivasubramanium, R. Sahoo, J. Moreia, and M. Gupta, "Filtering Failure Logs for a Blue Gene/L Prototype," Proc. of DSN, 2005.
3. Y. Liang, Y. Zhang, H. Xiong, and R. Sahoo, "An Adaptive Semantic Filter for Blue Gene/L Failure Log Analysis," Workshop on SMTPS, 2007.
4. P. Gujrati, Y. Li, Z. Lan, R. Thakur, and J. White, "A Meta-Learning Failure Predictor for Blue Gene/L Systems," Proc. of ICPP, 2007.
5. R. Sahoo, A. Oliner, I. Rish, M. Gupta, J. Moreira, S. Ma, R. Vilalta, and A. Sivasubramaniam, "Critical Event Prediction for Proactive Management in Large-Scale Computer Clusters," Proc. of SIGKDD, 2003.
6. B. Schroeder and G. Gibson , "A Large-Scale Study of Failures in High Performance Computing Systems," Proc. of DSN, 2006.
7. Cray XT series documents. http://www.cray.com/products/XT.aspx, 2008.
8. A. Gara, M. Blumrich et al., "Overview of the Blue Gene/L System Architecture," IBM J. Res. & Dev., vol. 49(2/3), 2005.
9. The TOP500 supercomputing site. http://top500.org/, June 2008.
10. T. Lin and D. Siewiorek, "Error Log Analysis: Statistical Modeling and Heuristic Trend Analysis," IEEE Transactions on Reliability, vol. 39(4), 1990.
11. R. Agrawal, T. Imielinski, and A. Swami, "Mining Association Rules between Sets of Items in Large Database," Proc. of SIGMOD, 1993.
12. J. Hansen and D. Siewiorek, "Models for Time Coalescence in Event Logs," Proc. of FTCS, 1992.
13. M. Buckley and D. Siewiorek, "Comparative Analysis of Event Tupling Schemes," Proc. of FTCS, 1996.
14. F. Salfner and S. Tschirpke, "Error Log Processing for Accurate Failure Prediction," Proc. of WASL'08, in conjunction with OSDI, 2008.
15. J. Rouillard, "Real-time Log File Analysis Using the Simple Event Correlator (SEC)," Proc. of LISA, 2004.
16. C. Lim, N. Singh and S. Yajnik, "A Log Mining Approach to Failure Analysis of Enterprise Telephony Systems," Proc. of DSN, 2008.
17. The First USENIX Workshop on the Analysis of System Logs. http://www.usenix.org/events/wasl08/, Dec., 2008.
18. USENIX Computer Failure Data Repository. http://cfdr.usenix.org/data. html#cray.