Logmaster: Mining event correlations in logs of large-scale cluster systems

Proceedings of the IEEE Symposium on Reliable Distributed Systems

Volumn , Issue , 2012, Pages 71-80

  Fu, Xiaoyu ^a,b   Ren, Rui ^a   Zhan, Jianfeng ^a   Zhou, Wei ^a   Jia, Zhen ^a,b   Lu, Gang ^a,b  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

correlation mining; failure prediction; large scale systems; reliability

Indexed keywords

CORRELATION GRAPHS; CORRELATION MINING; EVENT CORRELATION; EVENT TYPES; FAILURE PREDICTION; HIGH PRECISION; INNOVATIVE ALGORITHMS; INNOVATIVE APPROACHES; LARGE-SCALE CLUSTERS; RECALL RATE; SEQUENCE MINING; TRANSACTIONAL DATA;

RELIABILITY; SYSTEMS SCIENCE;

ALGORITHMS;

EID: 84874306678     PISSN: 10609857     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SRDS.2012.40     Document Type: Conference Paper

References (25)

1. R. K. Sahoo et al. "Failure data analysis of a large-scale heterogeneous server environment". In Proc. of DSN, 2004.
2. A. Gainaru et al. "Adaptive event prediction strategy with dynamic time window for large-scale HPC systems". In Proc. of SLAML, 2011.
3. A. Gainaru et al. "Event log mining tool for large scale HPC systems". In Proc. of Euro-Par, 2011.
4. N. Yigitbasi et al. "Analysis and modeling of time-correlated failures in large-scale distributed systems". In Proc. of GRID, 2010.
5. Z. Zheng et al. "A practical failure prediction with location and lead time for Blue Gene/P". In Proc. of DSN-W, 2010.
6. J. Gu et al. "Dynamic Meta-Learning for Failure Prediction in Large-Scale Systems: A Case Study". In Proc. of ICPP, 2008.
7. S. Fu et al. "Exploring Event Correlation for Failure Prediction in Coalitions of Clusters". In Proc. of ICS, 2007.
8. S. Fu et al. "Quantifying Temporal and Spatial Correlation of Failure Events for Proactive Management". In Proc. of SRDS, 2007.
9. P. Gujrati et al. "A Meta-Learning Failure Predictor for Blue Gene/L Systems". In Proc. of ICPP, 2007.
10. Y. Liang et al. "Filtering Failure Logs for a BlueGene/L Prototype". In Proc. of DSN, 2005.
11. R. Ren et al. "LogMaster: Mining Event Correlations in Logs of Largescale Cluster Systems". CoRR abs/1003.0951, 2011.
12. The BlueGene/L Team, "An Overview of the BlueGene/L Supercomputer". In Proc. of Supercomputing, 2002.
13. B. Schroeder et al. "A Large-Scale Study of Failures in High-Performance Computing Systems". In Proc. of DSN, 2006.
14. J. L. Hellerstein et al. "Discovering actionable patterns in event data". In IBM Systems Journal, 41(3), 2002.
15. Y. Liang et al. "BlueGene/L Failure Analysis and Prediction Models". In Proc. of DSN, 2006.
16. Y. Liang et al. "Failure Prediction in IBM BlueGene/L Event Logs". In Proc. of ICDM, 2007.
17. Z. Zhang et al. "Precise Request Tracing and Performance Debugging for Multi-tier Services of Black Boxes". In Proc. of DSN, 2009.
18. Z. Zheng et al. "System Log Pre-processing to Improve Failure Prediction". In Proc. of DSN, 2009.
19. D. Tang et al. "Analysis and Modeling of Correlated Failures in Multicomputer Systems". IEEE Transactions on Computers. 41, 5 (May. 1992), 567-577.
20. R. Agrawal et al. "Fast Algorithms for Mining Association Rules", In Proc. VLDB, 1994.
21. R. Agrawal et al. "Mining Association Rules Between Sets of Items in Large Databases," In Proc. of SIGMODE, 1993.
22. Yu Li et al. "Practical online failure prediction for Blue Gene/P: Periodbased vs Event-driven," In Proc. of DSN-W, 2011.
23. N. Taerat et al. "Blue Gene/L Log Analysis and Time to Interrupt Estimation," In Proc. of ARES, 2009.
24. C. Chen et al. "Log analytics for dependable enterprise telephony," In Proc. of EDCC, 2011.
25. F. Salfner et al. "A survey of online failure prediction methods," In ACM Comput. Surv.,42(3): 1-42,March 2010.