Predicting node failure in high performance computing systems from failure and usage logs

IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum

Volumn , Issue , 2011, Pages 1557-1566

  Nakka, Nithin ^a   Agrawal, Ankit ^b   Choudhary, Alok ^b  

^a University of Illinois at Urbana Champaign   (United States)

^b Northwestern University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION TREE CLASSIFIERS; FAILURE DATA; FAILURE INFORMATION; HIGH PERFORMANCE COMPUTERS; HIGH PERFORMANCE COMPUTING SYSTEMS; IDLE TIME; LOS ALAMOS NATIONAL LABORATORY; MINING CLASSIFICATION; NODE FAILURE; PREDICTION SYSTEMS; ROOT CAUSE; SEPARATE ANALYSIS; USAGE DATA;

COMPUTER SOFTWARE SELECTION AND EVALUATION; DATA MINING; DECISION TREES; DISTRIBUTED PARAMETER NETWORKS; FORECASTING; PARALLEL PROCESSING SYSTEMS; TREES (MATHEMATICS);

DISTRIBUTED COMPUTER SYSTEMS;

EID: 83455262121     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.310     Document Type: Conference Paper

References (27)

1. J. S. Plank and W. R. Elwasif. Experimental assessment of workstation failures and their impact on checkpointing systems. In Proceedings of FTCS-98.
2. S. Nath, H. Yu, P. B. Gibbons, and S. Seshan. Subtleties in tolerating correlated failures. In Proceedings of NSDI'06, 2006.
3. N. Nakka, A. Choudhary, "Failure data-driven selective node-level duplication to improve MTTF in High Performance Computing Systems", In Proceedings of HPCS 2009, June 2009.
4. T. Heath, R. P. Martin, and T. D. Nguyen. Improving cluster availability using workstation validation. In Proceedings of ACM SIGMETRICS, 2002.
5. th SRDS, 1995.
6. D. Nurmi, J. Brevik, and R. Wolski. Modeling machine availability in enterprise and wide-area distributed computing environments. In Euro-Par'05, 2005.
7. R. K. Sahoo, R. K., A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proceedings of DSN, June 2004.
8. D. Tang, R. K. Iyer, and S. S. Subramani. Failure analysis and modelling of a VAX cluster system. In Fault Tolerant Computing Systems, 1990.
9. J. Xu, Z. Kalbarczyk, and R. K. Iyer. Networked Windows NT system field failure data analysis. In Proc. of the PRDC, 1999.
10. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance-computing systems. In Proceedings of the DSN, June 2006.
11. R. K. Iyer, D. J. Rossetti, and M. C. Hsueh. Measurement and modeling of computer reliability as affected by system activity. ACM Transactions on Computing Systems, Vol. 4, No. 3, 1986.
12. th FTCS, 1981.
13. Adam J. Oliner, Jon Stearley: What Supercomputers Say: A Study of Five System Logs. In Proceedings of the DSN, Edinburgh, UK, June 2007, pp. 575-584.
14. Z. Lan, Y. Li, P. Gujrati, Z. Zheng, R. Thakur, and J. White, "A Fault Diagnosis and Prognosis Service for TeraGrid Clusters", In Proceedings of TeraGrid'07, 2007.
15. P. Gujrati, Y. Li, Z. Lan, R. Thakur, and J. White, "Exploring Meta-learning to Improve Failure Prediction in Supercomputing Clusters", In Proceedings of ICPP, 2007.
16. Y. Li and Z. Lan, "Using Adaptive Fault Tolerance to Improve Application Robustness on the TeraGrid", In Proceedings of TeraGrid'07, 2007.
17. Z. Lan and Y. Li, "Adaptive Fault Management of Parallel Applications for High Performance Computing", IEEE Transactions on Computers, Vol. 57, No. 12, pp. 1647-1660, 2008.
18. A. J. Oliner, R. K. Sahoo, J. E. Moreira, M. Gupta, and A. Sivasubramaniam. Fault-aware job scheduling for Bluegene/L systems. In Proceedings of the 18th IPDPS, 2004.
19. Ron Kohavi: The Power of Decision Tables. In: 8th European Conference on Machine Learning, 174-189, 1995.
20. G. Cooper, E. Herskovits (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning. 9(4):309-347.
21. George H. John, Pat Langley: Estimating Continuous Distributions in Bayesian Classifiers. In: Eleventh Conference on Uncertainty in Artificial Intelligence, San Mateo, 338-345, 1995.
22. Ross Quinlan (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo, CA.
23. Freund, Y., Mason, L.: The alternating decision tree learning algorithm. In: Proceeding of the Sixteenth International Conference on Machine Learning, Bled, Slovenia, 124-133, 1999.
24. Leo Breiman (2001). Random Forests. Machine Learning. 45(1):5-32.
25. I. Witten and E. Frank. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann Pub, 2005.
26. David Hosmer and Stanley Lemeshow.1989. Applied Logistic Regression. John Wiley and Sons, Inc.
27. le Cessie, S., van Houwelingen, J.C. (1992). Ridge Estimators in Logistic Regression. Applied Statistics. 41(1):191-201.