Exploring event correlation for failure prediction in coalitions of clusters

Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC'07

Volumn , Issue , 2007, Pages

  Fu, Song ^a   Xu, Cheng Zhong ^a  

^a Wayne State University   (United States)

Author keywords

Coalition clusters; Failure prediction; Spatial correlation; System availability; Temporal correlation

Indexed keywords

COALITION CLUSTERS; FAILURE PREDICTION; SPATIAL CORRELATION; SYSTEM AVAILABILITY; TEMPORAL CORRELATION;

COMPUTERS; ELECTRIC FAULT CURRENTS; FORECASTING; STOCHASTIC MODELS;

CORRELATION METHODS;

EID: 56749178938     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1362622.1362678     Document Type: Conference Paper

References (41)

1. Data lifeguard. Available at: http://www.wdc.com/en/library/ 2579-850105.pdf.
2. gretl: GNU Regression, Econometrics and Time-series Library. Available at: http://gretl.sourceforge.net/.
3. jbnc: Bayesian network classifier toolbox in Java. Available at: http://jbnc.sourceforge.net/.
4. Siguardian. Available at: http://www.siguardian.com/.
5. System availability, failure and usage data sets. Los Alamos National Laboratory (LANL). Available at: http://institutes.lanl. gov/data/fdata/.
6. Wayne State University, Grid computing. Available at: https://www.grid.wayne.edu/.
7. Weka: The university of waikato. machine learning software in Java. Available at: http://www.cs.waikato.ac.nz/ml/weka/.
8. H. Berenji, J. Ametha, and D. Vengerov. Inductive learning for fault diagnosis. In Proceeding of IEEE International Conference on Fuzzy Systems, 2003.
9. J. O. Berger, V. D. Oliveira, and B. Sansó. Objective Bayesian analysis of spatially correlated data. Journal of the American Statistical Association, 96(456): 1361-1374, 2001.
10. G. Candea, A. B. Brown, A. Fox, and D. Patterson. Recovery-oriented computing: Building multitier dependability. IEEE Computer, 37(11): 60-67, 2004.
11. G. Candea, S. Kawamoto, Y. Fujiki, G. Friedman, and A. Fox. Microreboot-a technique for cheap recovery. In Proceeding of USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004.
12. X. CastiUo and D. P. Siewlorek. Workload, performance and reliability of digital computing systems. In Proceeding of IEEE Symposium on Fault-Tolerant Computing (FTCS), 1981.
13. R. Christodoulopoulou, K. Manassiev, A. Bilas, and C. Amza. Fast and transparent recovery for continuous availability of cluster-based servers. In Proceeding of ACM Symposium on Principles and practice of parallel programming (PPoPP), 2006.
14. B. Chun and A. Vahdat. Workload and failure characterization on a large-scale federated testbed. Technical Report IRB-TR-03-040, Intel Research Berkeley, 2003.
15. J. Dunagan, N. J. A. Harvey, M. B. Jones, D. Kostic, M. The-imer, and A. Wolman. FUSE: Lightweight guaranteed distributed failure notification. In Proceeding of USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004.
16. S. Fu and C.-Z. Xu. Quantifying temporal and spatial correlation of failure events for proactive management. In Proceeding of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2007.
17. A. G. Ganek and T. A. Corbi. The dawning of the autonomic computing era. IBM Systems Journal, 42(1):5-18, 2003.
18. T. Heath, R. P. Martin, and T. D. Nguyen. Improving cluster availability using workstation validation. In Proceeding of ACM International Conference on Measurement and modeling of computer systems (SIGMETRICS), 2002.
19. G. Hughes, J. Murray, K. Kreutz-Delgado, and C. Elkan. Improved disk-drive failure warnings. IEEE Transactions on Reliability, 51(3):350-357, 2002.
20. R. K. Iyer, D. J. Rossetti, and M. C. Hsueh. Measurement and modeling of computer reliability as affected by system activity. ACM Transactions on Computer Systems, 4(3):214-237, 1986.
21. J. O. Kephart and D. M. Chess. The vision of autonomic computing. IEEE Computer, 36(l):41-50, 2003.
22. Y. Liang, Y. Zhang, A. Sivasubramaniam, M. Jette, and R. K. Sahoo. BlueGene/L failure analysis and prediction models. In Proceeding of IEEE International Conference on Dependable Systems and Networks (DSN), 2006.
23. Y. Liang, Y Zhang, A. Sivasubramaniam, R. Sahoo, J. Moreira, and M. Gupta. Filtering failure logs for a BlueGene/L prototype. In Proceeding of IEEE International Conference on Dependable Systems and Networks (DSN), 2005.
24. J. Meyer and L. Wei. Analysis of workload influence on dependability. In Proceeding of IEEE International Symposium on Fault-Tolerant Computing (FTCS), 1988.
25. J. W. Mickens and B. D. Noble. Exploiting availability prediction in distributed systems. In Proceeding of USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2006.
26. S. Mourad and D. Andrews. On the reliability of the IBM MVS/XA operating system. IEEE Transactions on Software Engineering, 13(10): 1135-1139, 1987.
27. R. K. Sahoo, A. J. Oliner, I. Rish, and et al. Critical event prediction for proactive management in large-scale computer clusters. In Proceeding of ACM International Conference on Knowledge Discovery and Data Dining (SIGKDD), 2003.
28. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proceeding of IEEE International Conference on Dependable Systems and Networks (DSN). 2004.
29. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance-computing systems. In Proceeding of IEEE International Conference on Dependable Systems and Networks (DSN), 2006.
30. E. Schuchman and T. N. Vijaykumar. Blackjack: Hard error detection with redundant threads on SMT. In Proceeding of IEEE International Conference on Dependable Systems and Networks (DSN), 2007.
31. H. Song, C. Leangsuksun, and R. Nassar. Availability modeling and analysis on high performance cluster computing systems. In Proceeding of International Conference on Availability, Reliability and Security (ARES), 2006.
32. N. Sridhar. Decentralized local failure detection in dynamic distributed systems. In Proceeding of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2006.
33. J. Srinivasan, S. V. Adve, R Bose, and J. A. Rivers. A reliability odometer - lemon check your processor! In Proceeding of Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2004.
34. D. Tang and R. K. Iyer. Impact of correlated failures on dependability in a 'VAXcluster system. In Proceeding of IFIP Working Conference on Dependable Computing for Critical Applications, 1991.
35. D. Tang, R. K. Iyer, and S. S. Subramani. Failure analysis and modelling of a 'VAXcluster system. In Proceeding of IEEE International Symposium on Fault-Tolerant Computing (FTCS), 1990.
36. K. Vaidyanathan, R. E. Harper, S. W. Hunter, and K. S. Trivedi. Analysis and implementation of software rejuvenation in cluster systems. In Proceeding of ACM International Conference on Measurement and modeling of computer systems (SIGMETRICS), 2001.
37. R. Vilalta and S. Ma. Predicting rare events in temporal domains. In Proceeding of IEEE International Conference on Data Mining (ICDM), 2002.
38. G. M. Weiss and H. Hirsh. Learning to predict rare events in event sequences. In Proceeding of ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), 1998.
39. M. Wiesmann, P. Urban, and X. Defago. An SNMP based failure detection service. In Proceeding of IEEE International Symposium on Reliable Distributed Systems (SRDS), 2006.
40. J. Xu, Z. Kallbarczyk, and R. K. Iyer. Networked Windows NT system field failure data analysis. Technical Report CRHC 9808, UIUC, 1999.
41. P. Yalagandula, S. Nath, H. Yu, P. B. Gibbons, and S. Sesha. Beyond availability: Towards a deeper understanding of machine failure characteristics in large distributed systems. In Proceeding of USENIX WORLDS, 2004.