Ensemble of Bayesian predictors and decision trees for proactive failure management in cloud computing systems

Journal of Communications

Volumn 7, Issue 1, 2012, Pages 52-61

  Guan, Qiang ^a   Zhang, Ziming ^a   Fu, Song ^a  

^a UNIVERSITY OF NORTH TEXAS   (United States)

Author keywords

Cloud computing; Decision tree; Dependability assurance; Ensemble of Bayesian models; Failure prediction; Unsupervised and supervised learning

Indexed keywords

ANOMALOUS BEHAVIOR; BAYESIAN MODEL; CLOUD SYSTEMS; COMPUTING SYSTEM; CRUCIAL TECHNOLOGY; DECISION TREE CLASSIFIERS; DEPENDABILITY ASSURANCE; FAILURE DETECTION METHOD; FAILURE DYNAMICS; FAILURE MANAGEMENT; FAILURE PREDICTION; FALSE POSITIVE RATES; LABELED DATA; LEARNING-BASED APPROACH; MULTI-LAYER NETWORK; RUNTIME PERFORMANCE; SYSTEM ADMINISTRATORS; SYSTEM BEHAVIORS; TRUE POSITIVE RATES;

BAYESIAN NETWORKS; CLOUD COMPUTING; DECISION TREES; SUPERVISED LEARNING;

COMPUTER SYSTEMS;

EID: 84863074638     PISSN: 17962021     EISSN: None     Source Type: Journal    
DOI: 10.4304/jcm.7.1.52-61     Document Type: Article

References (49)

1. R. K. Sahoo, A. J. Oliner, I. Rish, and et al., Critical event prediction for proactive management in large-scale computer clusters, in Proceedings of ACM International Conference on Knowledge Discovery and Data Dining (SIGKDD), 2003.
2. A. J. Oliner, R. K. Sahoo, J. E. Moreira, and et al., Faultaware job scheduling for BlueGene/L systems, in Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS), 2004.
3. F. Salfner, M. Lenk, and M. Malek, A survey of online failure prediction methods, ACM Computing Surveys, vol. 42, pp. 10:1-10:42, 2010.
4. J. W. Mickens and B. D. Noble, Exploiting availability prediction in distributed systems, in Proceedings of USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2006.
5. S. Fu and C. Xu, Quantifying event correlations for proactive failure management in networked computing systems, Journal of Parallel and Distributed Computing, vol. 70, no. 11, pp. 1100-1109, 2010.
6. J. Gu, Z. Zheng, Z. Lan, J. White, E. Hocks, and B.-H. Park, Dynamic meta-learning for failure prediction in large-scale systems: A case study, in Proceedings of IEEE International Conference on Parallel Processing (ICPP), 2008.
7. H. Song, C. Leangsuksun, and R. Nassar, Availability modeling and analysis on high performance cluster computing systems, in Proceedings of IEEE International Conference on Availability, Reliability and Security (ARES), 2006.
8. J. Han, Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers Inc., 2005.
9. T. Cover and J. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
10. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. Wiley-Interscience, 2001.
11. L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. Wadsworth and Brooks, 1984.
12. "sysstat," available at: http://sebastien.godard.pagesperso-orange.fr/.
13. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson, A survey of rollback-recovery protocols in messagepassing systems, ACM Computing Surveys, vol. 34, no. 3, pp. 375-408, 2002.
14. O. Laadan and J. Nieh, Transparent checkpoint-restart of multiple processes on commodity operating systems, in Proceedings of USENIX Annual Technical Conference (USENIX), 2007.
15. Y. Li and Z. Lan, Fast restart mechanism for check-point/recovery protocols in networked environments, in Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2008.
16. A. Agbaria and R. Friedman, Model-based performance evaluation of distributed checkpointing protocols, Performance Evaluation, vol. 65, no. 5, pp. 345-365, 2008.
17. G. Bronevetsky, D. J. Marques, K. K. Pingali, R. Rugina, and S. A. McKee, Compiler-enhanced incremental checkpointing for openmp applications, in Proceedings of ACM Symposium on Principles and Practice of Parallel Programming (PPoPP), 2008.
18. I. Philp, Software failures and the road to a petaflop machine, in Proceedings of Symposium on High Performance Computer Architecture Workshop, 2005.
19. Y. Liang, Y. Zhang, A. Sivasubramaniam, M. Jette, and R. K. Sahoo, BlueGene/L failure analysis and prediction models, in Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2006.
20. F. Salfner, P. Tröger, and S. Tschirpke, Cross-core event monitoring for processor failure prediction, in Proceedings of IEEE International Conference on High Performance Computing & Simulation, Workshop on Dependable Multi-Core Computing(DMCC), 2009.
21. S. Fu and C. Xu, Exploring event correlation for failure prediction in coalitions of clusters, in Proceedings of ACM/IEEE Supercomputing Conference (SC), 2007.
22. S. Fu and C. Xu, Quantifying temporal and spatial correlation of failure events for proactive management, in Proceedings of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2007.
23. S. S. Gokhale and K. S. Trivedi, Analytical models for architecture-based software reliability prediction: A unification framework, IEEE Transactions on Reliability, vol. 55, no. 4, pp. 578-590, 2006.
24. F. Salfner and M. Malek, Using hidden semi-markov models for effective online failure prediction, in Proceedings of the 26th IEEE International Symposium on Reliable Distributed Systems (SRDS), 2007.
25. th IEEE International Parallel and Distributed Processing Symposium (IPDPS), Workshop on Dependable Parallel Distributed Network-centric Systems, 2005.
26. Z. Zhang and S. Fu, Failure prediction for autonomic management of networked computer systems with availability assurance, in Proceedings of IEEE Workshop on Dependable Parallel, Distributed and Network-Centric Systems in conjunction with IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2010.
27. Z. Zhang and S. Fu, A hierarchical failure management framework for dependability assurance in compute clusters, International Journal of Computational Science, vol. 4, no. 4, pp. 313-326, 2010.
28. Q. Guan and S. Fu, auto-AID: A data mining frame-work for autonomic anomaly identification in networked computer systems, in Proceedings of IEEE International Performance Computing and Communications Conference (IPCCC), 2010.
29. Q. Guan, D. Smith, and S. Fu, Anomaly detection in large-scale coalition clusters for dependability assurance, in Proceedings of IEEE International Conference on High Performance Computing (HiPC), 2010.
30. A. B. Nagarajan, F. Mueller, C. Engelmann, and S. L. Scott, Proactive fault tolerance for HPC with Xen virtualization, in Proceedings of ACM International Conference on Supercomputing (ICS), 2007.
31. A. Polze, P. Tröger, and F. Salfner, Timely virtual machine migration for pro-active fault tolerance, in Proceedings of IEEE International Workshop on Object/component/service-oriented Real-time Networked Ultra-dependable Systems (WORNUS), at IEEE 14th International Symposium on Object/Component/Serviceoriented Real-time Distributed Computing (ISORC), 2011.
32. S. Fu, Failure-aware construction and reconfiguration of distributed virtual machines for high availability computing, in Proceedings of IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid), 2009.
33. S. Chakravorty, C. Mendes, and L. Kale, Proactive fault tolerance in MPI applications via task migration, in Proceedings of IEEE International Conference on High Performance Computing, 2006.
34. C. Wang, F. Mueller, C. Engelmann, and S. L. Scott, Proactive process-level live migration in HPC environments, in Proceedings of ACM/IEEE Conference on Supercomputing (SC), 2008.
35. S. Fu, Failure-aware resource management for high-availability computing clusters with distributed virtual machines, Journal of Parallel and Distributed Computing, vol. 70, no. 4, pp. 384-393, 2010.
36. S. Fu and C. Xu, Proactive resource management for failure resilient high performance computing clusters, in Proceedings of IEEE International Conference on Availability, Reliability and Security (ARES), 2009.
37. S. Muppala, X. Zhou, and L. Zhang, Regression based multi-tier resource provisioning for session slowdown guarantees, in IEEE International Performance Computing and Communications Conference (IPCCC), 2010.
38. S. Muppala and X. Zhou, Coordinated session-based admission control with statistical learning for multi-tier internet applications, Journal of Network and Computer Applications, Elsevier, vol. 34, no. 1, pp. 20-29, 2011.
39. B. Schroeder and G. Gibson, A large-scale study of failures in high-performance-computing systems, in Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2006.
40. Y. Liang, Y. Zhang, A. Sivasubramaniam, R. Sahoo, J. Moreira, and M. Gupta., Filtering failure logs for a BlueGene/L prototype, in Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2005.
41. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang, Failure data analysis of a large-scale heterogeneous server environment, in Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2004.
42. P. Yalagandula, S. Nath, H. Yu, P. B. Gibbons, and S. Sesha, Beyond availability: Towards a deeper understanding of machine failure characteristics in large distributed systems, in Proceedings of USENIX WORLDS, 2004.
43. T. Heath, R. P. Martin, and T. D. Nguyen, Improving cluster availability using workstation validation, in Proceedings of ACM Conference on Measurement and modeling of computer systems (SIGMETRICS), 2002.
44. Y. Zhang, M. S. Squillante, A. Sivasubramaniam, and R. K. Sahoo, Performance implications of failures in large-scale cluster scheduling, in Proceedings of the 10th Workshop on Job Scheduling Strategies for Parallel Processing, 2004.
45. S. S. Gokhale and K. S. Trivedi, Analytical models for architecture-based software reliability prediction: A unification framework, IEEE Trans. on Reliability, vol. 55, no. 4, pp. 578-590, 2006.
46. W. Peng, T. Li, and S. Ma, Mining logs files for computing system management, in Proceedings of IEEE International Conference on Automatic Computing (ICAC), 2005.
47. A. Zheng, J. Lloyd, and E. Brewer, Failure diagnosis using decision trees, in Proceedings of IEEE International Conference on Automatic Computing (ICAC), 2004.
48. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase, Correlating instrumentation data to system states: a building block for automated diagnosis and control, in Proceedings of USENIX Symposium on Opearting Systems Design and Implementation (OSDI), 2004.
49. Q. Guan, Z. Zhang, and S. Fu, Ensemble of bayesian predictors for autonomic failure management in cloud computing, in Proceedings of IEEE International Conference on Computer Communications and Networks (ICCCN), 2011.