Dynamic meta-learning for failure prediction in large-scale systems: A case study

Proceedings of the International Conference on Parallel Processing

Volumn , Issue , 2008, Pages 157-164

  Gu, Jiexing ^a   Zheng, Ziming ^a   Lan, Zhiling ^a   White, John ^b   Hocks, Eva ^c   Park, Byung Hoon ^d  

^a ILLINOIS INSTITUTE OF TECHNOLOGY   (United States)

^b Revisions Company

^c UNIVERSITY OF CALIFORNIA   (United States)

^d OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CASE STUDIES; COMPONENT RELIABILITIES; FAILURE PATTERNS; FAILURE PREDICTIONS; FALSE ALARM RATES; FAULT MANAGEMENTS; HIGH PERFORMANCE COMPUTING; OPEN PROBLEMS; PREDICTION ACCURACIES; PREDICTION ENGINES; RELIABLE COMPONENTS; SYSTEM OPERATIONS; SYSTEM SIZES; TRAINING SETS;

ALARM SYSTEMS; LARGE SCALE SYSTEMS; LEARNING SYSTEMS;

FORECASTING;

EID: 55849147399     PISSN: 01903918     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICPP.2008.17     Document Type: Conference Paper

References (35)

1. R. Agrawal, R. Srikant, "Fast Algorithms for Mining Association Rules", VLDB. Sep 12-15 1994, Chile, 487-99
2. A. Bouteiller, T. Herault, et al. , "MPICH-V: A Multiprotocol Automatic Fault Tolerant MPI", International Journal of High Performance Computing and Applications, 2005.
3. S. Chakravorty, C. L. Mendes and L. V. Kale, "Proactive Fault Tolerance in Large Systems", Proc. of HPCRI Workshop in conjunction with HPCA, 2005.
4. E. Elnozahy and J. S. Plank, "Checkpointing for PetaScale Systems: A Look into the Future of Practical Rollback-Recovery", IEEE Transactions on Dependable and Secure Computing, Volume 1, Number 2, 2004, pp. 97-108.
5. S. Fu, C. Z. Xu, "Exploring Event Correlation for Failure Prediction in Coalitions of Clusters", Proc of SC 2007, 2007.
6. A. Gara, M. A. Blumrich et a.l, "Overview of the Blue Gene/L System Architecture", IBM J. Res. & Dev. 49, No. 2/3, 195-212, 2005.
7. E. Gabriel, G. Fagg, and et al., "Open mpi: Goals, concept, and design of a next generation mpi implementation," in Proc. of The 11th European PVM/MPI Users' Group Meeting, 2004.
8. R. Gioiosa, J. Sancho, S. Jiang, F. Petrini, K. Davis, "Transparent Incremental Checkpointing at Kernel Level: A Foundation for Fault Tolerance for Parallel Computers", Proc. of SC2005, 2005.
9. A. Goyal, S. Lavenberg, and K. Trivedi, "Probabilistic Modeling of Computer System Availability", Annals of Operations Research, 1987.
10. P. Gujrati, Y. Li, Z. Lan, R. Thakur, and J. White, "A Meta-learning Failure Predictor for Bluegene/L Systems," Proc. of ICPP'07, 2001.
11. J. Han and M. Kamber "Data Mining: Concepts and Techniques", 2nd Edition, Morgan Kaufmann, 2006.
12. G. Hamerly and C. Elkan, "Bayesian Approaches to Failure Prediction for Disk Drives", Proc. of ICML, 2001.
13. J. Hellerstein, F. Zhang, P. Shahabuddin, "A Statistical Approach to Predictive Detection", Computer Networks: The International Journal of Computer and Telecommunications Networking, 2001.
14. R. K. Iyer, L. T. Young, V. Sridhar, "Recognition of error symptoms in large systems", Proceedings of 1986 ACM Fall joint computer conference, 1986.
15. Z. Lan, Y. Li, P. Gujrati, Z. Zheng, R. Thakur, and J. White, "A Fault Diagnosis and Prognosis Service for TeraGrid Clusters", Proc. of TeraGrid'07, 2007.
16. Y. Li, Z. Lan, "Exploit Failure Prediction for Adaptive Fault-Tolerance in Cluster Computing", Proc. of IEEE CCGrid'06, 2006.
17. Y. Liang, Y. Zhang, M. Jette, A. Sivasubramanium, R. Sahoo, "BlueGene/L Failure Analysis and Prediction Models", Proc. of DSN, 2006.
18. A. Oliner, L. Rudolph, and R. Sahoo, "Cooperative checkpointing theory," in Proc. of the International Parallel and Distributed Processing Symposium (IPDPS), 2006.
19. A. Oliner and J. Stearly, "What Supercomputers Say: A Study of Five System Logs", Proc. of DSN 2001.
20. A. Oliner, Ramendra K. Sahoo, José E. Moreira, Manish Gupta, Anand Sivasubramaniam, "Fault-Aware Job Scheduling for BlueGene/L Systems", IPDPS, 2004.
21. R. Polikar, "Ensemble Based Systems in Decision Making", IEEE Circuits and Systems Magazine, vol.6, no. 3, pp. 21-45, 2006.
22. D. Reed, C. Lu, and C. Mendes, "Big systems and big reliability challenges," in Proc. of Parallel Computing, Germany, 2003.
23. R.K. Sahoo, A.J. Oliner et al., "Critical event prediction for proactive management in large-scale computer clusters", Proc. of KDD, 2003, pp. 426-435.
24. Hoffmann, Salfner et al, "Advanced Failure Prediction in Complex Software Systems", Proc. of SRDS, 2004.
25. M. Schulz, G. Bronevetsky, R. Fernandes, D. Marques, K. Pingali, P. Stodghill, "Implementation and Evaluation of a Scalable Application-level Checkpoint-Recovery Scheme for MPI Programs", Supercomputing 2004. November 6-12, 2004.
26. th USENIX Conference on File and Storage Technologies, 2005.
27. D. Turnbull, N. Alldrin, "Failure Prediction in Hardware Systems", UCSD CSE22I Project, 2003.
28. K. Trivedi and K. Vaidyanathan, "A Measurement-based Model for Estimation of Resource Exhaustion in Operational Software Systems", Proc. of the 10th Int'l Symposium on Software Reliability Engineering, 1999.
29. K. Vaidyanathan and K. Gross, "MSET Performance Optimization for Detection of Softtware Aging", Proc. of ISSRE, 2003.
30. R. Vilalta and S. Ma, "Predicting Rare Events in Temporal Domains", Proc. of IEEE Intl. Conf. On Data Mining, 2002.
31. C. Wang and F. Mueller and C. Engelmann and S. Scott, "A Job Pause Service under LAM/MPI+BLCR for Transparent Fault Tolerance", Proc. of IPDPS ,2007.
32. Ian H. Witten and Eibe Frank (2005) "Data Mining: Practical machine learning tools and techniques", 2nd Edition, Morgan Kaufmann, San Francisco, 2005.
33. RAD Lab: Reliable Adaptive Distributed Systems Laboratory, http://radlab.cs berkeley.edu/
34. SDSC Blue Gene/L Homepage. www.sdsc.edu/us/resources/bluegene
35. The TOP500 Supercomputer Sites, www.top500.org