Fault prediction under the microscope: A closer look into HPC systems

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn , Issue , 2012, Pages

  Gainaru, Ana ^a   Cappello, Franck ^b   Snir, Marc ^c   Kramer, William ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b INRIA   (France)

^c ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

fault detection; fault tolerance; large scale HPC systems; signal analysis

Indexed keywords

CLASSICAL APPROACH; COMPONENT FAILURES; EFFICIENT PREDICTIONS; FUTURE IMPROVEMENTS; HIGH PERFORMANCE COMPUTING SYSTEMS; PRECISION AND RECALL; PREDICTION METHODS; PREVENTIVE MEASURES;

FAULT DETECTION; FAULT TOLERANCE; FAULT TOLERANT COMPUTER SYSTEMS; RESEARCH; SIGNAL ANALYSIS;

FORECASTING;

EID: 84877693592     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2012.57     Document Type: Conference Paper

References (34)

1. N. Taerat, Y. Zhang, A. Sivasubramaniam, M. Jette, R. Sahoo: Blue-Gene/L Log Analysis and Time to Interrupt Estimation. International Conference on Availability, Reliability and Security, pp.173-180, 2009
2. L. Di Jorio, A. Laurent, M. Teisseire: Mining frequent gradual itemsets from large databases. International Conference on Intelligent Data Analysis, IDA09, 2009
3. A. Laurent, B. Negrevergne, N. Sicard, and A. Termier: PGP-mc: Towards a Multicore Parallel Approach for Mining Gradual Patterns Database Systems for Advanced Applications, volume 5981, pp 78-84, 2010
4. A. Gainaru, F. Cappello, W. Kramer: Taming of the Shrew: Modeling the Normal and Faulty Behavior of Large-scale HPC Systems International Parallel and Distributed Processing Symposium, 2012
5. Y. Liang: BlueGene/L Failure Analysis and Prediction Models. Proceedings of the International Conference on Dependable Systems and Networks, pp 425-434, 2006
6. R. K. Sahoo et al: Critical Event Prediction for Proactive Management In Large-scale Computer Clusters. International conference on Knowledge discovery and data mining, pp 426-435, 2003
7. N. Yigitbasi et al: Analysis and Modeling of Time-Correlated Failures in Large-Scale Distributed Systems. IEEE/ACM International Conference on Grid Computing, pp 65-72, 2010
8. J. G. Lou et al: Mining Dependency in Distributed Systems through Unstructured Logs Analysis ACM SIGOPS Volume 44 Issue 1, January 2010
9. W. Xu et al: Online System Problem Detection by Mining Patterns of Console Logs IEEE International Conference on Data Mining, pp 588-597, 2009
10. Z. Zheng et al: A Practical Failure Prediction with Location and Lead Time for Blue Gene/P International Conference on Dependable Systems and Networks Workshops, pp 15-22, 2010
11. J. Gu et al: Dynamic Meta-Learning for Failure Prediction in Large-Scale Systems: A case Study International Conference on Parallel Processing, pp 157-164, 2008
12. A. Gainaru, F. Cappello, S. Trausan-Matu, W. Kramer: Adaptive Event Prediction Strategy with Dynamic Time Window for Large-Scale HPC Systems. System Log Analysis with Machine Learning Workshop, SLAML, 2011
13. R. Ren et al: LogMaster: Mining Event Correlations in Logs of Large-scale Cluster Systems CoRR abs/1003.0951, 2010
14. N. Nakka et al: Predicting Node Failure in High Performance Computing Systems from Failure and Usage Logs IEEE Workshop on Dependable Parallel, Distributed and Network-Centric Systems, 2011
15. A. Gainaru, F. Cappello, S. Trausan-Matu, W. Kramer: Event log mining tool for large scale HPC systems. International Conference on Parallel Processing Euro-Par, volume 1, pp 52-64, 2011.
16. R. Iyer et al: Automatic recognition of intermittent failures: An experimental study of field data. IEEE Transactions on Computers, 39:525537, 1990.
17. A. Pecchia, D. Cotroneo, Z. Kalbarczyk, R. Iyer: Improving Log-Based Field Failure Data Analysis of Multi-Node Computing Systems International Conference on Dependable Systems and Networks (DSN), pp 97-108, 2011
18. Z. Zheng, Z. Lan, B. Park, and A. Geist: System log preprocessing to improve failure prediction. International Conference on Dependable Systems and Networks, pp 572-577, 2009.
19. A. Oliner et al: What Supercomputers Say: A Study of Five System Logs. International Conference on Dependable Systems and Networks, 2007
20. E. Heien, D. Kondo, A. Gainaru, D. LaPine, W. Kramer, F. Cappello: I Modeling and Tolerating Heterogeneous Failures in Large Parallel Systems. International Conference for High Performance Computing, Networking, Storage and Analysis, 2011
21. B. Schroeder et al: A large-scale study of failures in high-performance computing systems. International Conference on Dependable Systems and Networks, pages 249-258, June 2006
22. R. C. Milton: An Extended Table of Critical Values for the Mann-Whitney (Wilcoxon) Two-Sample Statistic. Journal of the American Statistical Association, Volume 59, Issue 3, 1978
23. National Center for Supercomputing Applications at the University of Illinois. www.ncsa.illinois.edu. Accessed on 2010.
24. The computer failure data repository. http://cfdr.usenix.org. Accessed on 2010.
25. L. B. Gomez et al: FTI: high performance Fault Tolerance Interface for hybrid systems. International Conference for High Performance Computing, Networking, Storage and Analysis, 2011.
26. G. Zheng et al: FTC-Charm++: An In-Memory Checkpoint-Based Fault Tolerant Runtime for Charm++ and MPI. International Conference on Cluster Computing CLUSTER, pp 93-103, 2004
27. R. Agarwal et al: Rebound: Scalable Checkpointing for Coherent Shared Memory. ACM SIGARCH Computer Architecture News, Volume 39 Issue 3, 2011
28. A. Moody, G. Bronevetsky, K. Mohror, B. R. de Supinski: Design, Modeling, and Evaluation of a Scalable Multi-level Checkpointing System. ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp.1-11, 2010
29. Z. Zheng, Z. Lan, R. Gupta, S. Coghlan, P. Beckman: A Practical Failure Prediction with Location and Lead Time for Blue Gene/P Proceedings of the 2010 International Conference on Dependable Systems and Networks Workshops, pp 15-22, 2010
30. C. Wang, F. Mueller, C. Engelmann, and S. Scott: Proactive process-level live migration in HPC environments. International Conference for High Performance Computing, Networking, Storage and Analysis, 2008.
31. J. Gu, Z. Zheng, Z. Lan, J. White, and B. Park: Dynamic meta-learning for failure prediction in large-scale systems: A case study. International Conference on Parallel Processing, 2008.
32. R. Sahoo and A. Oliner: Critical event prediction for proactive management in large-scale computer clusters. SIGKDD International Conference on Knowledge Discovery and Data mining, pp 426-435, 2003.
33. M. Steinder and A. Sethi: A survey of fault localization techniques in computer networks. Science of Computer Programming, volume 53, issue 2, 2004
34. F. Cappello, H. Casanova, and Y. Robert: Checkpointing vs. migration for post-petascale supercomputers. International Conference on Parallel Processing, 2010