3-Dimensional root cause diagnosis via co-analysis

ICAC'12 - Proceedings of the 9th ACM International Conference on Autonomic Computing

Volumn , Issue , 2012, Pages 181-190

  Zheng, Ziming ^a   Yu, Li ^a   Lan, Zhiling ^a   Jones, Terry ^b  

^a Illinois Institute of Technology   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Co analysis; Diagnosis; Large scale system

Indexed keywords

3-DIMENSIONAL; BLUE GENE; CO-ANALYSIS; DIAGNOSIS INFORMATION; ERROR PRONES; LOCATION INFORMATION; MANUAL PROCESSING; OAK RIDGE NATIONAL LABORATORY; ROOT CAUSE; ROOT CAUSE ANALYSIS; SYSTEM ADMINISTRATORS; SYSTEM SIZE;

DIAGNOSIS; SUPERCOMPUTERS;

LARGE SCALE SYSTEMS;

EID: 84867695274     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2371536.2371571     Document Type: Conference Paper

References (35)

1. S. Alam, R. Barrett, M. Bast, M. Fahey, J. Kuehn, C. McCurdy, J. Rogers, P. Roth, R. Sankaran, J. Vetter, P. Worley, and W. Yu. Early evaluation of IBM BlueGene/P. Proc. of Supercomputing, 2008.
2. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using magpie for request extraction and workload modelling. In Proceedings of OSDI, 2004.
3. J. Brandt, A. Gentile, C. Houf, J. Mayo, P. Pebay, D. Roe, D. Thompson, and M. Wong. OVIS 3.2 user's guide. SAND 2010-7109, Sandia National Laboratories, October 2010.
4. G. Bronevetsky, I. Laguna, S. Bagchi, R. Bronis, D. Ahn, and M. Schulz. AutomaDeD: Automata-based debugging for dissimilar parallel tasks. In Proceedings of DSN, 2010.
5. M. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer. Pinpoint: problem determination in large, dynamic Internet services. In Proceedings of DSN, 2002.
6. T. Chilimbi, B. Liblit, K. Mehra, A. Nori, and K. Vaswani. Holmes: Effective statistical debugging via efficient path profiling. In Proceedings of ICSE, 2009.
7. E. Chuah, S. Kuo, P. Hiew, W. Tjhi, G. Lee, J. Hammond, M. Michalewicz, T. Hung, and J. Browne. Diagnosing the root-causes of failures from cluster log files. In Proceedings of HiPC, 2010.
8. N. DeBardeleben, J. Laros, J. Daly, S. Scott, C. Engelmann, and B. Harrod. High-end computing resilience: Analysis of issues facing the HEC community and path-forward for research and development. White Paper, 2009.
9. N. Desai, R. Bradshaw, C. Lueninghoener, A. Cherry, S. Coghlan, and W. Scullin. Petascale systemmanagement experiences. In Proceedings of LISA, 2008.
10. A. Gainaru, F. Cappello, F. J., and S. Trausan. Adaptive event prediction strategy with dynamic time window for large-scale HPC systems. In Proceedings of SLAML, 2011.
11. Q. Gao, F. Qin, and D. Panda. DMTracker: Finding bugs in large-scale parallel programs by detecting anomaly in data movements. In Proceedings of Supercomputing, 2006.
12. J. Han and M. Kamber. Data Mining:Concepts and Techniques. Morgan Kaufmann, 2000.
13. M. Kasick, J. Tan, R. Gandhi, and P. Narasimhan. Black-box problem diagnosis in parallel file systems. In Proceedings of FAST, 2010.
14. M. Khan, H. Le, H. Ahmadi, T. Abdelzaher, and J. Han. Dustminer: troubleshooting interactive complexity bugs in sensor networks. In Proceedings of SenSys, 2008.
15. E. Kiciman and A. Fox. Detecting application-level failures in component-based internet services. IEEE Trans. Neural Networks, 16(5):1027-1041, 2005.
16. P. Kogge and et al. Exascale computing study: Technology challenges in achieving exascale systems. White Paper, 2008.
17. G. Lakner and G. Mullen-Schultz. IBM BlueGene solution: System administration. IBM Redbook, 2007.
18. Z. Lan, Z. Zheng, and Y. Li. Toward automated anomaly identification in large-scale systems. IEEE Trans. on Parallel and Distributed Systems, 21(2):174-187, 2010.
19. Y. Liang, Y. Zhang, A. Sivasubramanium, R. Sahoo, J. Moreia, and M. Gupta. Filtering failure logs for a BlueGene/L prototype. In Proceedings of DSN, 2005.
20. A. Mahimkar, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and Q. Zhao. Towards automated performance diagnosis in a large IPTV network. In Proceedings of SIGCOMM, 2009.
21. N. Maruyama and S. Matsuoka. Model-based fault localization: Finding behavioral outliers in large-scale computing systems. New Generation Comput, 28:237-255, 2010.
22. A. Mirgorodskiy, N. Maruyama, and B. Miller. Problem diagnosis in large-scale computing environments. In Proceedings of Supercomputing, 2006.
23. A. Oliner, A. Kulkarni, and A. Aiken. Using correlated surprise to infer shared influence. In Proceedings of DSN, 2010.
24. A. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In Proceedings of DSN, 2007.
25. X. Pan, J. Tan, S. Kalvulya, R. Gandhi, and P. Narasimhan. Blind men and the elephant: Piecing together hadoop for diagnosis. In Proceedings of ISSRE, 2009.
26. J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000.
27. A. Pecchia, D. Cotroneo, Z. Kalbarczyk, and R. Iyer. Improving log-based field failure data analysis of multi-node computing systems. In Proceedings of DSN, 2011.
28. X. Rao, H. Wang, D. Shi, Z. Chen, H. Cai, and Q. Zhou. Identifying faults in large-scale distributed systems by filtering noisy error logs. In Proceedings of DSNW, 2011.
29. S. Sabato, E. Yomtov, and A. Tsherniak. Analyzing system logs: A new view of what's important. In USENIX SysML workshop, 2007.
30. R. Sambasivan, A. Zheng, M. Rosa, E. Krevat, S. Whitman, M. Stroucken, W. Wang, L. Xu, and G. Ganger. Diagnosing performance changes by comparing request flows. In Proceedings of NSDI, 2011.
31. B. Schroeder and G. Gibson. A large-scale study of failures in high-performance computing systems. In Proceedings of DSN, 2006.
32. J. Stearley and A. Oliner. Bad words: Finding faults in spirit's syslogs. In Proceedings of the Workshop on Resiliency in High Performance Computing, 2008.
33. L. Yu, Z. Zheng, Z. Lan, T. Jones, J. Brandt, and A. Gentile. Filtering log data: Finding the needles in the haystack. In Proceedings of DSN, 2012.
34. Z. Zheng, Z. Lan, B. Park, and A. Geist. System log pre-processing to improve failure prediction. In Proceedings of DSN, 2009.
35. Z. Zheng, L. Yu, W. Tang, Z. Lan, R. Gupta, N. Desai, S. Coghlan, and D. Buettner. Co-analysis of RAS log and job log on Blue Gene/P. In Proceedings of IPDPS, 2011.