Proactive failure management by integrated unsupervised and semi-supervised learning for dependable cloud systems

Proceedings of the 2011 6th International Conference on Availability, Reliability and Security, ARES 2011

Volumn , Issue , 2011, Pages 83-90

  Guan, Qiang ^a   Zhang, Ziming ^a   Fu, Song ^a  

^a UNIVERSITY OF NORTH TEXAS   (United States)

Author keywords

Bayesian detector; Cloud systems; Decision tree; Dependable systems; Learning algorithms

Indexed keywords

ANOMALOUS BEHAVIOR; BAYESIAN DETECTORS; BAYESIAN MODEL; CLOUD SYSTEMS; COMPUTING SYSTEM; DECISION TREE CLASSIFIERS; DEPENDABLE SYSTEMS; EXECUTION ENVIRONMENTS; FAILURE DETECTION METHOD; FAILURE DYNAMICS; FAILURE MANAGEMENT; FAILURE PREDICTION; LABELED DATA; SEMI-SUPERVISED LEARNING; SYSTEM ADMINISTRATORS; SYSTEM COMPONENTS;

BAYESIAN NETWORKS; COMPUTER SYSTEMS; DECISION TREES; FORECASTING; LEARNING ALGORITHMS; PLANT EXTRACTS; SUPERVISED LEARNING;

CLOUD COMPUTING;

EID: 80455144683     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ARES.2011.20     Document Type: Conference Paper

References (40)

1. sysstat. Available at: http://sebastien.godard.pagesperso-orange.fr/.
2. A. Agbaria and R. Friedman. Model-based performance eval-uation of distributed checkpointing protocols. Performance Evaluation, 65(5):345-365, 2008. (Pubitemid 351400683)
3. L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classifi-cation and Regression Trees. Wadsworth and Brooks, 1984.
4. G. Bronevetsky, D. J. Marques, K. K. Pingali, R. Rugina, and S. A. McKee. Compiler-enhanced incremental checkpointing for openmp applications. In Proc. of ACM Symposium on Principles and Practice of Parallel Programming (PPoPP), 2008.
5. S. Chakravorty, C. Mendes, and L. Kale. Proactive fault tolerance in MPI applications via task migration. In Proc. of IEEE International Conference on High Performance Com-puting, 2006.
6. T. Cover and J. Thomas. Elements of Information Theory. John Wiley & Sons, 1991.
7. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classifica-tion. Wiley-Interscience, 2001.
8. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson. A survey of rollback-recovery protocols in message-passing systems. ACM Computing Surveys, 34(3):375-408, 2002.
9. S. Fu. Failure-aware construction and recon guration of distributed virtual machines for high availability computing. In Proc. of IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid), 2009.
10. S. Fu. Failure-aware resource management for high-availability computing clusters with distributed virtual ma-chines. Journal of Parallel and Distributed Computing, 70(4):384-393, 2010.
11. S. Fu and C. Xu. Exploring event correlation for failure pre-diction in coalitions of clusters. In Proceedings of ACM/IEEE Supercomputing Conference (SC), 2007.
12. S. Fu and C. Xu. Quantifying temporal and spatial correlation of failure events for proactive management. In Proc. of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2007.
13. S. Fu and C. Xu. Proactive resource management for failure resilient high performance computing clusters. In Proceedings of IEEE International Conference on Availability, Reliability and Security (ARES), 2009.
14. S. Fu and C. Xu. Quantifying event correlations for proac-tive failure management in networked computing systems. Journal of Parallel and Distributed Computing, 70(11):1100-1109, 2010.
15. S. S. Gokhale and K. S. Trivedi. Analytical models for architecture-based software reliability prediction: A uni ca-tion framework. IEEE Transactions on Reliability, 55(4):578-590, 2006. (Pubitemid 46405748)
16. Q. Guan and S. Fu. auto-AID: A data mining framework for autonomic anomaly identi cation in networked computer systems. In Proceedings of IEEE International Performance Computing and Communications Conference (IPCCC), 2010.
17. Q. Guan, D. Smith, and S. Fu. Anomaly detection in large-scale coalition clusters for dependability assurance. In Proceedings of IEEE International Conference on High Per-formance Computing (HiPC), 2010.
18. J. Han. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers Inc., 2005.
19. T. Heath, R. P. Martin, and T. D. Nguyen. Improving cluster availability using workstation validation. In Proc. of ACM Conference on Measurement and modeling of computer systems (SIGMETRICS), 2002.
20. O. Laadan and J. Nieh. Transparent checkpoint-restart of multiple processes on commodity operating systems. In Proc. of USENIX Annual Technical Conference (USENIX), 2007.
21. Y. Li and Z. Lan. Fast restart mechanism for check-point/recovery protocols in networked environments. In Proc. of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2008.
22. Y. Liang, Y. Zhang, A. Sivasubramaniam, M. Jette, and R. K. Sahoo. BlueGene/L failure analysis and prediction models. In Proc. of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2006.
23. Y. Liang, Y. Zhang, A. Sivasubramaniam, R. Sahoo, J. Moreira, and M. Gupta. Filtering failure logs for a BlueGene/L prototype. In Proc. of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2005.
24. J.W. Mickens and B. D. Noble. Exploiting availability predic-tion in distributed systems. In Proc. of USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2006.
25. A. B. Nagarajan, F. Mueller, C. Engelmann, and S. L. Scott. Proactive fault tolerance for HPC with Xen virtualization. In Proc. of ACM International Conference on Supercomputing (ICS), 2007.
26. A. J. Oliner, R. K. Sahoo, J. E. Moreira, and et al. Fault-aware job scheduling for BlueGene/L systems. In Proc. of the 18th International Parallel and Distributed Processing Symposium (IPDPS), 2004.
27. I. Philp. Software failures and the road to a peta!op machine. In Proc. of Symposium on High Performance Computer Architecture Workshop, 2005.
28. A. Polze, P. Tröger, and F. Salfner. Timely virtual machine migration for pro-active fault tolerance. In Proc. of IEEE International Workshop on Object/component/service-oriented Real-time Networked Ultra-dependable Systems (WORNUS), at IEEE 14th International Symposium on Object/Component/Service-oriented Real-time Distributed Computing (ISORC), 2011.
29. R. K. Sahoo, A. J. Oliner, I. Rish, and et al. Critical event prediction for proactive management in large-scale computer clusters. In Proc. of ACM International Conference on Knowledge Discovery and Data Dining (SIGKDD), 2003.
30. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proc. of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2004.
31. F. Salfner, M. Lenk, and M. Malek. A survey of online failure prediction methods. ACM Computing Surveys, 42:10:1-10:42, 2010.
32. F. Salfner and M. Malek. Proactive fault handling for system availability enhancement. In Proc. of the 19th IEEE International Parallel and Distributed Processing Sympo-sium (IPDPS), Workshop on Dependable Parallel Distributed Network-centric Systems, 2005.
33. F. Salfner and M. Malek. Using hidden semi-markov models for effective online failure prediction. In Proc. of the 26th IEEE International Symposium on Reliable Distributed Sys-tems (SRDS), 2007.
34. F. Salfner, P. Tröger, and S. Tschirpke. Cross-core event monitoring for processor failure prediction. In Proc. of IEEE International Conference on High Performance Computing & Simulation, Workshop on Dependable Multi-Core Comput-ing(DMCC), 2009.
35. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance-computing systems. In Proc. of IEEE/IFIP International Conference on Dependable Systems and Net-works (), 2006.
36. H. Song, C. Leangsuksun, and R. Nassar. Availability modeling and analysis on high performance cluster computing systems. In Proc. of IEEE International Conference on Availability, Reliability and Security (ARES), 2006.
37. C. Wang, F. Mueller, C. Engelmann, and S. L. Scott. Proactive process-level live migration in HPC environments. In Proc. of ACM/IEEE Conference on Supercomputing (SC), 2008.
38. P. Yalagandula, S. Nath, H. Yu, P. B. Gibbons, and S. Sesha. Beyond availability: Towards a deeper understanding of ma-chine failure characteristics in large distributed systems. In Proc. of USENIX WORLDS, 2004.
39. Z. Zhang and S. Fu. Failure prediction for autonomic management of networked computer systems with availability assurance. In Proceedings of IEEE Workshop on Depend-able Parallel, Distributed and Network-Centric Systems in conjunction with IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2010.
40. Z. Zhang and S. Fu. A hierarchical failure management framework for dependability assurance in compute clusters. International Journal of Computational Science, 4(4):313-326, 2010.