Anomaly localization in large-scale clusters

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2007, Pages 322-330

  Zheng, Ziming ^a   Li, Yawei ^a   Lan, Zhiling ^a  

^a Illinois Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADMINISTRATIVE DATA PROCESSING; BOOLEAN FUNCTIONS; FEATURE EXTRACTION; FINANCIAL DATA PROCESSING; LOCATION; PRINCIPAL COMPONENT ANALYSIS; STATISTICAL METHODS;

CELL-BASED; CLUSTER COMPUTING; COMPUTING ENVIRONMENTS; DATA ANALYSIS; DETECTION ALGORITHMS; FEATURE SPACES; HIGH-PERFORMANCE COMPUTING; INTERNATIONAL CONFERENCES; LARGE-SCALE CLUSTERS; LOCALIZATION METHODS; MASSIVE QUANTITIES; NUMBER OF COMPONENTS; OUTLIER DETECTION; PRINCIPAL COMPONENTS; ROOT CAUSES; TROUBLE SHOOTING;

COMPUTER SYSTEMS;

EID: 51049094584     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTR.2007.4629246     Document Type: Conference Paper

References (38)

1. Y. Liang, Y. Zhang, et al., "BlueGene /L Failure Analysis and Models", Proc. of DSN06, 2006.
2. G. Hoffmann, F. Salfner, M. Malek, "Advanced Failure Prediction in Complex Software Systems", Proc. of SRDS, 2004.
3. R. Sahoo, A. Oliner, et al., "Critical Event Prediction for Proactive Management in Large-scale Computer Clusters", Proc. of KDD 2003,pp. 426-435,2003.
4. K. Trivedi and K. Vaidyanathan, "A Measurement-based Model for Estimation of Resource Exhaustion in Operational Software Systems", Proc. of the 10th International Symposium on Software Reliability Engineering, 1999.
5. M. Chen, A. X. Zheng, J. Lloyd, M. I. Jordan, E. Brewer , "Failure Diagnosis Using Decision Trees", International Conference on Autonomic Computing (ICAC-04), 2004.
6. R. Vilalta and S. Ma, "Predicting Rare Events in Temporal Domains", Proc. of IEEE ICDM, 2002.
7. P. Gujrati, Y. Li, Z. Lan, R. Thakur, and J. White, "A meta-learning failure predictor for biuegene/l. systems," Proc. of ICPP"07, 2007.
8. J. Brevik, D. Nurmi, and R. Wolski, "Automatic Methods for Predicting Machine Availability in Desktop Grid and Peer-to-Peer Systems", Proc. of IEEE CCGrid, 2004.
9. J. Zhang and R. Figueiredo, "Application classification through monitoring and learning of resource consumption patterns", 10th IEEE International Paralle & Distributed Processing Symposium, Rhodes Island, Greece, Apr. 25-29, 2006.
10. Yawei Li and Zhiling Lan, "Exploit Failure Prediction for Adaptive Fault-Tolerance in Cluster Computing", Proc. of IEEE CCGrid'06, 2006.
11. R. Duda, P. Hart, and D. Stork. Pattern Classification. Wiley-Interscience, New York, NY, 2001. 2nd edition.
12. Edwin M. Knorr, Raymond T. Ng, Vladimir Tucakov, "Distance-based outliers: algorithms and applications", The VLDB Joumal,(2000) 8: 237-253.
13. The TOP500 Supercomputer Sites, http://www.top500.org
14. Hardware monitoring by lm. sensors http://secure.netroedge.com/lm78/info. html
15. B. Allen, "Monitoring Hard Disk with SMART", Linux Journal, January, 2004.
16. Greg Hamerly and Charles Elkan, "Bayesian approaches to failure prediction for disk drives", ICML 2001,pp. 1-9, 2001.
17. Kai Shen, Ming Zhong, and Chuanpeng Li, "I/O System Performance Debugging Using Model-driven. Anomaly Characterization", 4th USENIX Conference on File and Storage Technologies, pp. 309C322, 2005.
18. A. V. Mirgorodskiy, N. Maruyama, B.P. Miller, "Problem Diagnosis in Large-Scale Computing Environments", Supercomputing, 2006, pp. 11-24, 2006.
19. R. Chillarege and N. S. Bowen, "Understanding large system failure - A fault injection experiment", Proc. 19th Int. Symp. Fault-Tolerant Comput., pp. 355-363, June 1989.
20. Nancy Leveson, "Safeware: System safety and computers", New York: Addison-Wesley, 1995.
21. A. Lakhina, M. Crovella, C. Diot, "Mining anomalies using traffic feature distributions",Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, 2005.
22. K. Vaidyanathan, K. S. Trivedi, "A measurement-based model for estimation of resource exhaustion in operational software systems", Proceedings of the 10th International Symposium on Software Reliability Engineering, pp.84-93, 1999.
23. B. Schroeder, G. A. Gibson , "A Large-scale Study of Failures in High-performance-computing Systems", Proceedings of the International Conference on Dependable Systems and Networks, pp. 249-258, 2006.
24. H. Casanova, G. Obertelli, F. Berman and R. Wolski, "The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid", Supercomputing, 2000, pp. 60-69, 2000.
25. Girard, A. , Brunilde, S., "Multicommodity flow models, failure propagation, and reliable loss network design", IEEE/ACM Transactions on Networking (TON), Volume 6, Issue 1, pp. 82-93, 1998.
26. Ganglia Monitoring System, http://ganglia.sourceforge.net/
27. M. Scottile and R. Minnich, "Supermon: A High- Speed Cluster Monitoring System", Proc. IEEE Cluster, 2002.
28. S. Smallen, C. Olschanowski, K. Ericson, P. Bechman, and J. Schopf, "The Inca test harness and reporting Framework", Proc. of SC04, 2004.
29. A. Lakhina, M. Crovella, C. Diot, "Diagnosing Network-Wide Traffic Anomalies", Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, 2004.
30. Rao. Y. N. and Principe. J. C., "A fast, On-Line Algorithm for PCA and Its Convergence Characteristics", Proceedings of the 2000 IEEE Signal Processing Society Workshop, vol.1, pp. 299-307, 2000.
31. Ananth Grama, Vipin Kumar, Anshul Gupta, George Karpis, "Introduction to Parallel Computing (2nd Edition)", Addison-Wesley, 2003.
32. P. Bodik, G. Friedman, L. Biewald, H. Levine, G. Candea, K. Patel, G. Tolle, J. Hui, A. Fox, M. I. Jordan and D. Patterson, "Combining Visualization and Statistical Analysis to Improve Operator Confidence and Efficiency for Failure Detection and Localization", The 2nd IEEE International Conference on Autonomic Computing (ICAC '05), 2005.
33. S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, Vol. 290: 2323-2326
34. R. Vilalta and S. Ma, "Predicting rare events in temporal domains," in Proc. of IEEE Intl. Conf. On Data Mining, 2002.
35. G. Hamerly and C. Elkan, "Bayesian approaches to failure prediction for disk drives," in Proc. of ICML, 2001.
36. J. Hellerstein, F. Zhang, and P. Shahabuddin, "A statistical approach to predictive detection," Computer Networks: The International Journal of Computer and Telecommunications Networking, 2001.
37. K. Vaidyanathan and K. Gross, "MSET Performance Optimization for Detection of Softtware Aging", Proc. of ISSRE, 2003.
38. J. Sancho, F. Petrini, G. Johnson, J. Fernandez, and E. Frachtenberg, "On the feasibility of incremental checkpointing for scientific computing," in Proc. of IPDPS'04, 2004.