Linking resource usage anomalies with system failures from cluster log data

Proceedings of the IEEE Symposium on Reliable Distributed Systems

Volumn , Issue , 2013, Pages 111-120

  Chuah, Edward ^b   Jhumka, Arshad ^a   Narasimhamurthy, Sai ^b   Hammond, John ^c   Browne, James C ^d   Barth, Bill ^c  

^a UNIVERSITY OF WARWICK   (United Kingdom)

^b Xyratex Technology Limited   (United Kingdom)

^c Texas Advanced Computing Center   (United States)

^d UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Cluster log data; Large clusters; Linux O S; Lustre file system; Resource Anomalies and Failures

Indexed keywords

FILESYSTEM; LARGE CLUSTER SYSTEMS; LARGE CLUSTERS; LINKING RESOURCES; LINUX O/S; LOG DATA; MULTIPLE SOURCE; RESOURCE-MONITORING TOOLS;

COMPUTER OPERATING SYSTEMS; DISTRIBUTED COMPUTER SYSTEMS; SUPERCOMPUTERS; SYSTEMS ENGINEERING;

NATURAL RESOURCES MANAGEMENT;

EID: 84891501027     PISSN: 10609857     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SRDS.2013.20     Document Type: Conference Paper

References (41)

1. B. Schroeder and G. A. Gibson, "A large-scale study of failures in highperformance computing systems," in Proceedings of IEEE/IFIP DSN, 2006, pp. 249-258.
2. T. J. Hacker, F. Romero, and C. D. Carothers, "An analysis of clustered failures on large supercomputing systems," Journal of Parallel and Distributed Computing, vol. 69, no. 7, pp. 652-665, 2009.
3. G. Bronevetsky, I. Laguna, B. R. de Supinski, and S. Bagchi, "Automatic fault characterization via abnormality-enhanced classification," in Proceedings of IEEE/IFIP DSN, 2012, pp. 1-12.
4. I. Laguna, D. H. Anh, B. R. de Supinski, S. Bagchi, and T. Gamblin, "Probabilistic diagnosis of performance faults in large scale parallel applications," in Proceedings of PACT, 2012, pp. 1-10.
5. Z. Lan, Z. Zheng, and Y. Li, "Toward automated anomaly identification in large-scale systems," IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 2, pp. 174-187, 2010.
6. Q. Guan, D. Smith, and S. Fu, "Anomaly detection in large-scale coalition clusters for dependability assurance," in Proceedings of IEEE HiPC, 2010, pp. 1-10.
7. T. Reidemeister, M. A. Munawar, M. Jiang, and P. A. Ward, "Diagnosis of recurrent faults using log files," in Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research, 2009, pp. 12-23.
8. A. J. Oliner, A. V. Kulkarni, and A. Aiken, "Using correlated surprise to infer shared influence," in Proceedings of IEEE/IFIP DSN, 2010, pp. 191-200.
9. Z. Zheng, L. Yu, Z. Lan, and T. Jones, "3-dimensional root cause diagnosis via co-analysis," in Proceedings of ACM ICAC, 2012, pp. 181-190.
10. Y. Liang, Y. Zhang, H. Xiong, and R. Sahoo, "Failure prediction in ibm bluegene/l event logs," in Proceedings of IEEE ICDM, 2007, pp. 583-588.
11. F. Salfner, P. Troeger, and S. Tschirpke, "Cross-core event monitoring for processor failure prediction," in Proceedings of HPCS DMCC Workshop, 2009, pp. 67-73.
12. Z. Lan, J. Gu, Z. Zheng, R. Thakur, and S. Coghlan, "A study of dynamic meta-learning for failure prediction in large-scale systems," Journal of Parallel and Distributed Computing, vol. 70, no. 6, pp. 630-643, 2010.
13. X. Fu, R. Ren, J. Zhan, W. Zhou, Z. Jia, and G. Lu, "Logmaster: Mining event correlations in logs of large-scale cluster systems," in Proceedings of IEEE SRDS, 2012, pp. 1-10.
14. J. Hammond, "Tacc stats: I/o performance monitoring for the intransigent," in Invited Keynote for the 3rd IASDS Workshop, 2011, pp. 1-29.
15. E. Chuah, G. Lee, W.-C. Tjhi, S.-H. Kuo, T. Hung, J. Hammond, T. Minyard, and J. C. Browne, "Establishing hypothesis for recurrent system failures from cluster log files," in Proceedings of IEEE DASC, Dec 12-14 2011, pp. 1-8.
16. J. L. Hammond, T. Minyard, and J. Browne, "End-to-end framework for fault management for open source clusters: Ranger," in Proceedings of ACM TeraGrid, no. 9, 2010.
17. A. Avizienis, J.-C. Lapire, B. Randell, and C. Landwehr, "Basic concepts and taxonomy of dependable and secure computing," IEEE Transactions on Dependable and Secure Computing, vol. 1, no. 1, pp. 11-33, 2004.
18. T. T. Project, http://sebastien.godard.pagesperso-orange.fr/.
19. Z. Zheng, L. Yu, W. Tang, and Z. Lan, "Co-analysis of ras log and job log on blue gene/p," in Proceedings of IEEE IPDPS, 2011, pp. 840-851.
20. A. Oliner and J. Stearley, "What supercomputers say: A study of five system logs," in Proceedings of IEEE/IFIP DSN, June 2007, pp. 575-584.
21. P.-N. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Addison-Wesley, 2006.
22. C. Croux, P. Filzmoser, and M. R. Oliveira, "Algorithms for projectionpursuit robust principal component analysis," Chemometrics and Intelligent Laboratory Systems, vol. 87, no. 2, pp. 218-225, 2007.
23. A. Hyvarinen and E. Oja, "Independent component analysis: Algorithms and applications," Neural Networks, vol. 13, no. 4-5, pp. 411-430, 2000.
24. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. Wiley-Interscience, 2000.
25. E. H. Moore, "On the reciprocal of the general algebraic matrix," Bulletin of the AMS, vol. 26, no. 9, p. 394395, 1920.
26. A. A. Mahimkar, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and Q. Zhao, "Towards automated performance diagnosis in a large iptv network," in Proceedings of the ACM SIGCOMM 2009 conference on Data communication, 2009, pp. 231-242.
27. A. J. Oliner and A. Aiken, "Online detection of multi-component interactions in production systems," in Proceedings of IEEE/IFIP DSN, 2011, pp. 49-60.
28. R. K. Iyer, L. T. Young, and V. Sridhar, "Recognition of error symptoms in large systems," in 1986 ACM Fall joint computer conference, 1986, pp. 797-806.
29. Lustre, http://http://wiki.lustre.org/manual/LustreManual18 HTML/ index.html.
30. M. M. Mano, Computer System Architecture. Prentice Hall International Editions, 1993.
31. H. M. Deitel, Operating Systems. Addison-Wesley, 1990.
32. F. Halsall, Data Communications, Computer Networks and Open Systems. Addison-Wesley, 1996.
33. A. Chester, M. Leeke, M. Al-Ghamdi, S. A. Jarvis, and A. Jhumka, "A modular failure-aware resource allocation architecture for cloud computing," in Proceedings of UKPEW, 2011.
34. D. Banerjee, V. Madduri, and M. Srivatsa, "A framework for distributed monitoring and root cause analysis for large ip networks," in Proceedings of IEEE SRDS, 2009, pp. 246-255.
35. J. Dipu, P. Prakash, R. R. Kompella, and R. Chandra, "Shedding light on enterprise network failures using spotlight," in Proceedings of IEEE SRDS, 2010, pp. 167-176.
36. S. P. Kavulya, S. Daniels, K. Joshi, M. Hiltunen, R. Gandhi, and P. Narasimhan, "Draco: Statistical diagnosis of chronic problems in large distributed systems," in Proceedings of IEEE/IFIP DSN, 2012, pp. 1-12.
37. S. Tati, B. J. Ko, G. Cao, A. Swami, and T. L. Porta, "Adaptive algorithms for diagnosing large-scale failures in computer networks," in Proceedings of IEEE/IFIP DSN, 2012, pp. 1-12.
38. G. Khanna, M. Y. Cheng, P. Varadharajan, S. Bagchi, M. P. Correia, and P. J. Verissimo, "Automated rule-based diagnosis through a distributed monitor system," IEEE Transactions on Dependable and Secure Computing, vol. 4, no. 4, pp. 266-279, 2007.
39. J. Tan, X. Pan, E. Marinelli, S. Kavulya, R. Gandhi, and P. Narasimhan, "Problem diagnosis for mapreduce-based cloud computing environments," in Proceedings of IEEE/IFIP NOMS, 2010.
40. W. Xu, L. Huang, A. Fox, D. Patterson, and M. Jordan, "Mining console logs for large-scale system problem detection," in Proceedings of 3rd Workshop on Tackling Computer Systems Problems with Machine Learning Techniques, December 2008.
41. D. Yuan, J. Zheng, S. Park, Y. Zhou, and S. Savage, "Improving software diagnosibility via log enhancement," in Proceedings of ACM ASPLOS, 2011, pp. 3-14.