Adaptive event prediction strategy with dynamic time window for large-scale HPC systems

Managing Large-Scale Systems via the Analysis of System Logs and the Application of Machine Learning Techniques, SLAML'11

Volumn , Issue , 2011, Pages

  Gainaru, Ana ^a,b   Cappello, Franck ^a,c   Fullop, Joshi ^a   Trausan Matu, Stefan ^b   Kramer, William ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b UNIVERSITY POLITEHNICA OF BUCHAREST   (Romania)

^c INRIA   (France)

Author keywords

Event prediction; HPC systems; Logfile analysis

Indexed keywords

CORRELATION EXTRACTION; DYNAMIC TIME; EVENT PREDICTION; FREQUENT SEQUENCES; LOG-FILE ANALYSIS; RUNTIMES; TIME WINDOWS;

LARGE SCALE SYSTEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; VEHICLE ROUTING;

FORECASTING;

EID: 81055139569     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2038633.2038637     Document Type: Conference Paper

References (24)

1. Y. Liang: BlueGene/L Failure Analysis and Prediction Models. Proceedings of the International Conference on Dependable Systems and Networks, 2006
2. N. Taerat et al: BlueGene/L Log Analysis and Time to Interrupt Estimation. International Conference on Availability, Reliability and Security, pp.173-180, 2009
3. R. K. Sahoo et al: Critical Event Prediction for Proactive Management In Large-scale Computer Clusters. International conference on Knowledge discovery and data mining, pp 426-435, 2003
4. N. Yigitbasi et al: Analysis and Modeling of Time-Correlated Failures in Large-Scale Distributed Systems. IEEE/ACM International Conference on Grid Computing, pp 65-72, 2010
5. J. G. Lou et al: Mining Dependency in Distributed Systems through Unstructured Logs Analysis ACM SIGOPS Volume 44 Issue 1, January 2010
6. W. Xu et al: Online System Problem Detection by Mining Patterns of Console Logs IEEE International Conference on Data Mining, pp 588-597, 2009
7. Z. Zheng et al: A Practical Failure Prediction with Location and Lead Time for Blue Gene/P International Conference on Dependable Systems and Networks Workshops, pp 15-22 2010
8. J. Gu et al: Dynamic Meta-Learning for Failure Prediction in Large-Scale Systems: A case Study International Conference on Parallel Processing, pp 157-164, 2008
9. E. Chuah et al: Diagnosing the Root-Cause of Failures from Cluster Log Files International Conference on High Performance Computing, pp 1-10, 2010
10. R. Ren et al: LogMaster: Mining Event Correlations in Logs of Large-scale Cluster Systems CoRR abs/1003.0951, 2010
11. N. Nakka et al: Predicting Node Failure in High Performance Computing Systems from Failure and Usage Logs IEEE Workshop on Dependable Parallel, Distributed and Network-Centric Systems, 2011
12. Ana Gainaru, Franck Cappello et al: Event log mining tool for large scale HPC systems. Euro-Par conference, Sep 2011.
13. N. Taerat and all: Blue gene/l log analysis and time to interrupt estimation. International Conference on Availability, Reliability and Security, pp 173-180 (2009).
14. The TOP500 Supercomputer Sites. www.top500.org
15. D. Kerbyson et al: Use of Predictive Performance Modeling During Large-scale Systems Installation International Workshop on Hardware/Software Support for Parallel and Distributed Scientific and Engineering Computing, 2002.
16. R.A. Oldfield et al: Modeling the impact of checkpoints on next-generation systems. IEEE Conference on Mass Storage Systems and Technologies, pp 30-46, Sept, 2007.
17. E. Elnozahy and J Plank: Checkpointing for peta-scale systems: a look into the future of practical rollback-recovery. Dependable and Secure Computing, pp 97-108, Apr, 2004.
18. Adam Oliner et al: What Supercomputers Say: A Study of Five System Logs. International Conference on Dependable Systems and Networks, June, 2007
19. Eric Heien, Derrick Kondo, Ana Gainaru, Franck Cappello: Modeling and Tolerating Heterogeneous Failures in Large Parallel Systems - Supercomputing 2011, November 2011
20. B. Schroeder et al: A large-scale study of failures in high-performance computing systems. IEEE DSN, pages 249-258, June 2006 (Pubitemid 44930426)
21. A. Gara et al: Overview of the blue gene/l system architecture. IBM Journal of Research and Development, 49(2):195-212, March 2005. (Pubitemid 40718128)
22. National Center for Supercomputing Applications at the University of Illinois. www.ncsa.illinois.edu. Accessed on 2010.
23. Y. Li and Z. Lan: Using Adaptive Fault Tolerance to Improve Application Robustness on the TeraGrid TeraGrid, 2007.
24. Y. Tan, X. Gu et al: Adaptive Runtime Anomaly Prediction for Dynamic Hosting Infrastructures Symposium on Principles of Distributed Computing, 2010.