Mining invariants from console logs for system problem detection

Proceedings of the 2010 USENIX Annual Technical Conference, USENIX ATC 2010

Volumn , Issue , 2019, Pages 231-244

  Lou, Jian Guang ^a   Fu, Qiang ^a   Yang, Shengqi ^b   Xu, Ye ^c   Li, Jiang ^a  

^a MICROSOFT RESEARCH ASIA   (China)

^b BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^c NANJING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; WELL LOGGING;

DISTRIBUTED SYSTEMS; HIGH-ACCURACY; LARGE-SCALE DISTRIBUTED SYSTEM; LINEAR CHARACTERISTICS; NOVEL ALGORITHM; PROBLEM DETECTION; PROGRAM INVARIANTS; STRUCTURED LOGS;

ANOMALY DETECTION;

EID: 85077114796     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. th Usenix LISA‟04, Nov. 14-19, 2004.
2. J. E. Prewett, Analyzing Cluster Log Files Using Logsurfer, In Proc. of Annual Conference on Linux Clusters, 2003.
3. th Usenix LISA‟93, 1993.
4. A. Oliner and J. Stearley. What supercomputers say: A Study of Five System Logs. In Proc. of IEEE DSN‟07, 2007.
5. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan, Detecting Large-Scale System Problems by Mining Console Logs, In Proc. of ACM SIGOPS SOSP‟09, Big Sky, MT, Oct. 11-14, 2009.
6. W. Xu, L. Huang, A. Fox, D. Patterson, and M. Jordan, Mining Console Logs for Large-Scale System Problem Detection, In Proc. of SysML, Dec. 2008.
7. W. Dickinson, D. Leon, and A. Podgurski, Finding Failures by Cluster Analysis of Execution Profiles, In Proc. of ICSE, May 2001.
8. A.V. Mirgorodskiy, N. Maruyama, and B.P. Miller, Problem Diagnosis in Large-Scale Computing Environments, In Proc. of the ACM/IEEE SC 2006 Conference, Nov. 2006.
9. C. Yuan, N. Lao, J.R. Wen, J. Li, Z. Zhang, Y.M. Wang, and W. Y. Ma, Automated Known Problem Diagnosis with Event Traces, In Proc. of EuroSys‟06, Apr. 2006.
10. M. D. Ernst, J. Cockrell, W. G. Griswold, and D. Notkin, Dynamically Discovering Likely Program Invariants to Support Program Evolution, In IEEE Trans. on Software Engineering, pp. 99-123, Vol.27, No.2, Feb. 2001.
11. H. Chen, H. Cheng, G. Jiang, and K. Yoshihira, Exploiting Local and Global Invariants for the Management of Large Scale Information Systems, In Proc. of ICDM‟08, Pisa, Italy, Dec. 2008.
12. G. Jiang, H. Chen, and K. Yoshihira, Efficient and Scalable Algorithms for Inferring Likely Invariants in Distributed Systems, In IEEE Trans. on Knowledge and Data Engineering, pp. 1508-1523, Vol.19, No. 11, Nov. 2007.
13. H. Chen, G. Jiang, C. Ungureanu, and K. Yoshihira, Failure Detection and Localization in Component Based Systems by Online Tracking, In Proc. of SIGKDD, pp. 750-755, 2005.
14. Q. Fu, J.-G. Lou, Y. Wang, and J. LI, Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis, In Proc. of ICDM, Florida, Dec. 2009.
15. J. Tan, X. Pan, S. Kavulya, R. Gandhi, and P. Narasim-han, SALSA: Analyzing Logs as State Machines, In Proc. of WASL, Dec. 2008.
16. D. Cotroneo, R. Pietrantuono, L. Mariani, and F. Pastore, Investigation of Failure Causes in Workload Driven Reliability Testing, In Proc. of the 4th Workshop on Software Quality Assurance, Sep. 2007.
17. G. Jiang, H. Chen, C. Ungureanu, and K. Yoshihira. Multi-resolution Abnormal Trace Detection Using Varied-length N-grams and Automata, In Proc. of ICAC, Jun. 2005.
18. Hadoop. http://hadoop.apache.org/core.
19. D. Donoho, Y. Tsaig, I. Drori, and J. Starck, Sparse Solutions of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit, Technical Report of Department of Statistics, Standford, TR2006-02, 2006.