Anomaly detection in large-scale coalition clusters for dependability assurance

17th International Conference on High Performance Computing, HiPC 2010

Volumn , Issue , 2010, Pages

  Guan, Qiang ^a   Smith, Derek ^b   Fu, Song ^a  

^a Center for Advanced Research and Technology   (United States)

^b NEW MEXICO INSTITUTE OF MINING AND TECHNOLOGY   (United States)

Author keywords

Anomaly detection; Autonomic systems; Coalition clusters; Compute grids; System dependability

Indexed keywords

HEALTH; METADATA;

ANOMALY DETECTION; AUTONOMIC SYSTEMS; COALITION CLUSTER; COMPUTE GRID; DEPENDABILITY ASSURANCE; HEALTH DATA; HIGH PERFORMANCE COMPUTING SYSTEMS; LARGE-SCALES; SYSTEM COMPONENTS; SYSTEM DEPENDABILITY;

CLUSTERING ALGORITHMS;

EID: 79952786041     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HIPC.2010.5713169     Document Type: Conference Paper

References (37)

1. sysstat. Available at: http://pagesperso-orange.fr/sebastien.godard/.
2. H. Berenji, J. Ametha, and D. Vengerov. Inductive learning for fault diagnosis. In Proceedings of IEEE International Conference on Fuzzy Systems, 2003.
3. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41(3):1-58, 2009.
4. B. Chun and A. Vahdat. Workload and failure characterization on a large-scale federated testbed. Technical Report IRB-TR-03-040, Intel Research Berkeley, November 2003.
5. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: a building block for automated diagnosis and control. In Proceedings of USENIX Symposium on Opearting Systems Design and Implementation (OSDI), 2004.
6. T. Cover and J. Thomas. Elements of Information Theory. John Wiley & Sons, 1991.
7. N. Friedman, D. Geiger, and M. Goldszmidt. Bayesian network classifiers. Machine Learning, 29(2-3):131-163, 1997. (Pubitemid 127510036)
8. S. Fu. Failure-aware construction and reconfiguration of distributed virtual machines for high availability computing. In Proceedings of IEEE/ACM International Symposium on Cluster Computing and the Grid (CC-Grid), 2009.
9. S. Fu. Dependability enhancement for coalition clusters with autonomic failure management. In Proceedings of the 15th IEEE International Symposium on Computers and Communications (ISCC), 2010.
10. S. Fu. Failure-aware resource management for high-availability computing clusters with distributed virtual machines. Journal of Parallel and Distributed Computing, 70(4):384-393, 2010.
11. S. Fu and C.-Z. Xu. Exploring event correlation for failure prediction in coalitions of clusters. In Proceedings of ACM/IEEE Supercomputing Conference (SC), 2007.
12. S. Fu and C.-Z. Xu. Quantifying temporal and spatial correlation of failure events for proactive management. In Proceedings of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2007.
13. S. Fu and C.-Z. Xu. Proactive resource management for failure resilient high performance computing clusters. In Proceedings of IEEE International Conference on Availability, Reliability and Security (ARES), March 2009.
14. S. Fu and C.-Z. Xu. Quantifying event correlations for proactive failure management in networked computing systems. Journal of Parallel and Distributed Computing, 2010. doi:10.1016/j.jpdc.2010.06.010.
15. A. G. Ganek and T. A. Corbi. The dawning of the autonomic computing era. IBM Systems Journal, 42(1):5-18, 2003.
16. J. Han. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers Inc., 2005.
17. C.-H. Hsu and W.-C. Feng. A power-aware run-time system for high-performance computing. In Proceedings of ACM/IEEE Supercomputing Conference (SC), 2005.
18. J. O. Kephart and D. M. Chess. The vision of autonomic computing. IEEE Computer, 36(1):41-50, 2003.
19. Y. Liang, Y. Zhang, A. Sivasubramaniam, M. Jette, and R. K. Sahoo. BlueGene/L failure analysis and prediction models. In Proceedings of IEEE International Conference on Dependable Systems and Networks (DSN), 2006.
20. Y. Liang, Y. Zhang, A. Sivasubramaniam, R. Sahoo, J. Moreira, and M. Gupta. Filtering failure logs for a BlueGene/L prototype. In Proceedings of IEEE International Conference on Dependable Systems and Networks (DSN), 2005.
21. C. Lim, N. Singh, and S. Yajnik. A log mining approach to failure analysis of enterprise telephony systems. In Proceedings of IEEE International Conference on Dependable Systems and Networks (DSN), 2008.
22. S. Ma and J. L. Hellerstein. Mining partially periodic event patterns with unknown periods. In Proceedings of IEEE International Conference on Data Engineering (ICDE), 2001.
23. J. Meyer and L. Wei. Analysis of workload influence on dependability. In Proceedings of Symposium on Fault-Tolerant Computing (FTCS), 1988.
24. A. J. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In Proceedings of IEEE International Conference on Dependable Systems and Networks (DSN), 2007.
25. D. Oppenheimer, A. Ganapathi, and D. Patterson. Why do Internet services fail, and what can be done about it. In Proceedings of USENIX Symposium on Internet Technologies and Systems (USITS), 2003.
26. W. Peng, T. Li, and S. Ma. Mining logs files for computing system management. In Proceedings of IEEE International Conference on Automatic Computing (ICAC), 2005.
27. I. Philp. Software failures and the road to a petaflop machine. In Proceedings of International Symposium on High Performance Computer Architecture Workshop, 2005.
28. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proceedings of IEEE Conference on Dependable Systems and Networks (DSN), 2004.
29. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance-computing systems. In Proceedings of IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), June 2006.
30. J. Tucek, S. Lu, C. Huang, S. Xanthos, and Y. Zhou. Triage: diagnosing production run failures at the user's site. In Proceedings of ACM Symposium on Operating Systems Principles (SOSP), 2007.
31. R. Vilalta and S. Ma. Predicting rare events in temporal domains. In Proceedings of IEEE International Conference on Data Mining (ICDM), 2002.
32. P. Yalagandula, S. Nath, H. Yu, P. B. Gibbons, and S. Sesha. Beyond availability: Towards a deeper understanding of machine failure characteristics in large distributed systems. In Proceedings of USENIX Work- shop on Real, Large Distributed Systems (WORLDS), 2004.
33. K. Yamanishi and Y. Maruyama. Dynamic syslog mining for network failure monitoring. In Proceedings of ACM International Conference on Knowledge Discovery in Data Mining (KDD), 2005.
34. Y. Zhang, M. S. Squillante, A. Sivasubramaniam, and R. K. Sahoo. Performance implications of failures in large-scale cluster scheduling. In Proceedings of the 10th Workshop on Job Scheduling Strategies for Parallel Processing, 2004.
35. Z. Zhang and S. Fu. Failure prediction for autonomic management of networked computer systems with availability assurance. In Proceedings of IEEE Workshop on Dependable Parallel, Distributed and Network-Centric Systems in conjunction with IEEE International Parallel and Distributed Processing Sym- posium (IPDPS), 2010.
36. Z. Zhang and S. Fu. Proactive failure management for high availability computing in computer clusters. In Proceedings of IEEE International Conference on Computational Sciences and Optimization, 2010.
37. A. Zheng, J. Lloyd, and E. Brewer. Failure diagnosis using decision trees. In Proceedings of IEEE Inter- national Conference on Automatic Computing (ICAC), 2004.