Exploring time and frequency domains for accurate and automated anomaly detection in cloud computing systems

Proceedings of IEEE Pacific Rim International Symposium on Dependable Computing, PRDC

Volumn , Issue , 2013, Pages 196-205

  Guan, Qiang ^a   Fu, Song ^a   De Bardeleben, Nathan ^b   Blanchard, Sean ^b  

^a Center for Advanced Research and Technology   (United States)

^b Ultrascale Systems Research Center Los Alamos National Laboratory1   (United States)

Author keywords

Anomaly detection; Autonomic management; Cloud computing; Dependable systems; Wavelet analysis

Indexed keywords

WAVELET ANALYSIS;

ANOMALY DETECTION; ANOMALY IDENTIFICATION; AUTONOMIC MANAGEMENT; CLOUD COMPUTING ENVIRONMENTS; DEPENDABILITY ASSURANCE; DEPENDABLE SYSTEMS; SLIDING DETECTION WINDOWS; TIME AND FREQUENCY DOMAINS;

CLOUD COMPUTING;

EID: 84906748850     PISSN: 15410110     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/PRDC.2013.40     Document Type: Conference Paper

References (62)

1. S. Agarwala, F. Alegre, K. Schwan, and J. Mehalingham. E2EProf: Automated end-To-end performance management for enterprise systems. In Proc. of IEEE/IFIP Intl. Conf. on Dependable Systems and Networks (DSN), 2007
2. S. Agarwala and K. Schwan. Sysprof: Online distributed behavior diagnosis through fine-grain system monitoring. In Proc. of IEEE Intl. Conf. on Distributed Computing Systems (ICDCS), 2006
3. E. Amaldi and V. Kann. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, 209(1-2):237-260, 1998
4. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. A view of cloud computing. Communications of the ACM, 53:50-58, 2010
5. P. Barford, J. Kline, D. Plonka, and A. Ron. A signal analysis of network traffic anomalies. In Proc. of ACM SIGCOMM Workshop on Internet measurment, 2002
6. D. Bernick, B. Bruckert, P. D. Vigna, D. Garcia, R. Jardine, J. Klecka, and J. Smullen. Nonstop advanced architecture. In Proc. of IEEE Conf. on Dependable Systems and Networks (DSN), 2005
7. B. Boashash. Time frequency signal analysis and processing : A comprehensive reference. Elsevier, 2003
8. J. a. Carreira, H. Madeira, and J. a. G. Silva. Xception: A technique for the experimental evaluation of dependability in modern computers. IEEE Transactions on Software Engineering, 24(2):125-136, 1998
9. E. Cecchet, J. Marguerite, and W. Zwaenepoel. Performance and scalability of EJB applications. In Proc. of ACM Conf. on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), 2002
10. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41(3):15:1-15:58, 2009
11. D. Chen, G. Jacques-Silva, Z. Kalbarczyk, R. K. Iyer, and B. Mealey. Error behavior comparison of multiple computing systems: A case study using linux on pentium, solaris on sparc, and aix on power. In Proc. of IEEE Pacific Rim Intl. Symp. on Dependable Computing (PRDC), 2008
12. M. Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer. Pinpoint: Problem determination in large, dynamic internet services. In Proc. of IEEE/IFIP Intl. Conf. on Dependable Systems and Networks (DSN), 2002
13. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: A building block for automated diagnosis and control. In Proc. of USENIX Symp. on Opearting Systems Design and Implementation (OSDI), 2004
14. J. Combes, A. Grossmann, and P. Tchamitchian. Wavelets: Timefrequency methods and phase space. Springer-Verlag, 1990
15. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, New York, 1991
16. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008
17. N. DeBardeleben, S. Blanchard, Q. Guan, Z. Zhang, and S. Fu. Experimental framework for injecting logic errors in a virtual machine to profile applications for soft error resilience. In Proc. of Workshop on Resiliency in High Performance Computing in conjunction with Intl. European Conf. on Parallel and Distributed Computing (Euro-Par), 2011
18. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification (Second Edition). Wiley-Interscience, 2000
19. S. Fu. Failure-Aware construction and reconfiguration of distributed virtual machines for high availability computing. In Proc. of IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid), 2009
20. S. Fu. Failure-Aware resource management for high-Availability computing clusters with distributed virtual machines. Journal of Parallel and Distributed Computing, 70(4):384-393, 2010
21. S. Fu. Performance metric selection for autonomic anomaly detection on cloud computing systems. In Proc. of IEEE Global Communications Conf. (GLOBECOM), 2011
22. S. Fu and C. Xu. Exploring event correlation for failure prediction in coalitions of clusters. In Proc. of ACM/IEEE Supercomputing Conf. (SC), 2007
23. S. Fu and C. Xu. Quantifying event correlations for proactive failure management in networked computing systems. Journal of Parallel and Distributed Computing, 70(11):1100-1109, 2010
24. S. Garg, A. Puliafito, and K. S. Trivedi. Analysis of software rejuvenation using markov regenerative stochastic petri net. In Proc. of IEEE Intl. Symp. on Software Reliability Engineering (ISSRE), 1995
25. W. Gu, Z. Kalbarczyk, R. K. Iyer, and Z. Yang. Characterization of linux kernel behavior under errors. In Proc. of IEEE/IFIP Intl. Conf. on Dependable Systems and Networks (DSN), 2003
26. Q. Guan, C.-C. Chiu, and S. Fu. A cloud dependability analysis framework for assessing system dependability in cloud computing infrastructures. In Proc. of IEEE/IFIP Pacific Rim Intl. Symp. on Dependable Computing (PRDC), 2012
27. Q. Guan, C.-C. Chiu, Z. Zhang, and S. Fu. Efficient and accurate anomaly identification using reduced metric space in utility clouds. In Proc. of IEEE Intl. Conf. on Networking, Architecture and Storage (NAS), 2012
28. Q. Guan and S. Fu. auto-AID: A data mining framework for autonomic anomaly identification in networked computer systems. In Proc. of IEEE Intl. Performance Computing and Communications Conf. (IPCCC), 2010
29. Q. Guan and S. Fu. Adaptive anomaly identification by exploring metric subspace in cloud computing infrastructures. In Proc. of IEEE Symp. on Reliable Distributed Systems (SRDS), 2013
30. Q. Guan, Z. Zhang, and S. Fu. Proactive failure management by integrated unsupervised and semi-supervised learning for dependable cloud systems. In Proc. of IEEE Intl. Conf. on Availability, Reliability and Security (ARES), 2011
31. Q. Guan, Z. Zhang, and S. Fu. Ensemble of bayesian predictors and decision trees for proactive failure management in cloud computing systems. Journal of Communications, 7(1):52-61, 2012
32. G. Hamerly and C. Elkan. Bayesian approaches to failure prediction for disk drives. In Proc. of Conf. on Machine Learning (ICML), 2001
33. T. Heath, R. P. Martin, and T. D. Nguyen. Improving cluster availability using workstation validation. In Proc. of ACM Intl. Conf. on Measurement and modeling of computer systems (SIGMETRICS), 2002
34. J. L. Hellerstein, F. Zhang, and P. Shahabuddin. A statistical approach to predictive detection. Computer Networks: The Intl. Journal of Computer and Telecommunications Networking, 35(1):77-95, 2001
35. V. Hodge and J. Austin. A survey of outlier detection methodologies. Artificial Intelligence Review, 22:85-126, 2004
36. K. Joshi, G. Bunker, F. Jahanian, A. Van Moorsel, and J. Weinman. Dependability in the cloud: Challenges and opportunities. In Proc. of IEEE/IFIP Intl. Conf. on Dependable Systems and Networks (DSN), 2009
37. A. Lakhina, M. Crovella, and C. Diot. Diagnosing network-wide traffic anomalies. In Proc. of ACM SIGCOMM Conf., 2004
38. Z. Lan, J. Gu, Z. Zheng, R. Thakur, and S. Coghlan. A study of dynamic meta-learning for failure prediction in large-scale systems. Journal of Parallel and Distributed Computing, 70(6):630-643, 2010
39. Y. Liang, Y. Zhang, A. Sivasubramaniam, R. Sahoo, J. Moreira, and M. Gupta. Filtering failure logs for a BlueGene/L prototype. In Proc. of IEEE Conf. on Dependable Systems and Networks (DSN), 2005
40. F. Longo, R. Ghosh, V. Naik, and K. Trivedi. A scalable availability model for infrastructure-As-A-service cloud. In Proc. of IEEE Conf. on Dependable Systems and Networks (DSN), 2011
41. J. W. Mickens and B. D. Noble. Exploiting availability prediction in distributed systems. In Proc. of USENIX Symp. on Networked Systems Design and Implementation (NSDI), 2006
42. H. Pannu, J. Liu, and S. Fu. AAD: Adaptive anomaly detection system for cloud computing infrastructures. In Proc. of IEEE Symp. on Reliable Distributed Systems (SRDS), 2012
43. H. Pannu, J. Liu, and S. Fu. A self-evolving anomaly detection framework for developing highly dependable utility clouds. In Proc. of IEEE Global Communications Conf. (GLOBECOM), 2012
44. W. Peng and T. Li. Mining logs files for computing system management. In Proc. of IEEE Intl. Conf. on Autonomic Computing (ICAC), 2005
45. perf Subsystem. perf: Linux profiling with performance counters. Available at: Http://perf.wiki.kernel.org/.
46. C. Pham, D. Chen, Z. Kalbarczyk, and R. K. Iyer. Cloudval: A framework for validation of virtualization environment in cloud infrastructure. In Proc. of IEEE/IFIP Intl. Conf. on Dependable Systems and Networks (DSN), 2011
47. R. D. Reed and R. J. Marks. Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks. MIT Press, 1998
48. M. Rosenblum and T. Garfinkel. Virtual machine monitors: Current technology and future trends. IEEE Computer, 38(5):39-47, 2005
49. R. K. Sahoo, A. J. Oliner, I. Rish, M. Gupta, J. E. Moreira, S. Ma, R. Vilalta, and A. Sivasubramaniam. Critical event prediction for proactive management in large-scale computer clusters. In Proc. of ACM Intl. Conf. on Knowledge Discovery and Data Dining (KDD), 2003
50. R. K. Sahoo, A. Sivasubramaniam, M. S. Squillante, and Y. Zhang. Failure data analysis of a large-scale heterogeneous server environment. In Proc. of IEEE Conf. on Dependable Systems and Networks (DSN), 2004
51. F. Salfner, M. Lenk, and M. Malek. A survey of online failure prediction methods. ACM Computing Surveys, 42(3):10:1-10:42, 2010
52. B. Schroeder and G. A. Gibson. A large-scale study of failures in highperformance computing systems. IEEE Transactions on Dependable and Secure Computing, 7:337-351, 2010
53. D. P. Siewiorek, R. Chillarege, and Z. T. Kalbarczyk. Reflections on industry trends and experimental research in dependability. IEEE Transactions on Dependable and Secure Computing, 1(2):109-127, 2004
54. J. E. Smith and R. Nair. The architecture of virtual machines. IEEE Computer, 38(5):32-38, 2005
55. M. Steinder and A. Sethi. A survey of fault localization techniques in computer networks. Science of Computer Programming, 53(2):165-194, 2004
56. SYSSTAT Utilities. SYSSTAT: Performance monitoring tools. Available at: Http://sebastien.godard.pagesperso-orange.fr/.
57. K. Vaidyanathan and K. Gross. Mset performance optimization for detection of software aging. In Proc. of IEEE Intl. Symp. on Software Reliability Engineering (ISSRE), 2003
58. B. Vidakovic. Statistical Modeling by Wavelets. 2010
59. R. Vilalta and S. Ma. Predicting rare events in temporal domains. In Proc. of IEEE Intl. Conf. on Data Mining (ICDM), 2002
60. K. V. Vishwanath and N. Nagappan. Characterizing cloud computing hardware reliability. In Proc. of ACM Symp. on Cloud computing (SOCC), 2010
61. X. Yao. Evolving artificial neural networks, 1999
62. A. X. Zheng, J. Lloyd, and E. Brewer. Failure diagnosis using decision trees. In Proc. of IEEE Conf. on Autonomic Computing (ICAC), 2004.