Mining dependency in distributed systems through unstructured logs analysis

Operating Systems Review (ACM)

Volumn 44, Issue 1, 2010, Pages 91-96

  Lou, Jian Guang ^a   Fu, Qiang ^a   Wang, Yi ^b   Li, Jiang ^a  

^a MICROSOFT RESEARCH ASIA   (China)

^b BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

Author keywords

Cooccurrence analysis; Dependency graph; Log analysis; Root error localization

Indexed keywords

APPLICATION-SPECIFIC KNOWLEDGE; BAYESIAN DECISION THEORY; BELONG TO; CO-OCCURRENCE ANALYSIS; DEPENDENCY GRAPHS; DISTRIBUTED SYSTEMS; FALSE POSITIVE DETECTION; INTER-COMPONENT DEPENDENCIES; LOG ANALYSIS; ROOT ERROR LOCALIZATION; SYSTEM COMPONENTS;

DECISION THEORY;

EID: 77958132122     PISSN: 01635980     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1740390.1740411     Document Type: Article

References (14)

1. th Workshop on Hot Topics in Networks, 2006.
2. X. Chen, M. Zhang, Z. M. Mao, and P. Bahl, "Automating Network Application Dependency Discovery: Experiences, Limitations, and New Solutions", USENIX OSDI'08, 2008.
3. M. Gupta, A. Neogi, M. K. Agarwal, and G. Kar, "Discovering Dynamic Dependencies in Enterprise Environments for Problem Determination", in Proc. of 14th Int. Workshop on Distributed systems: Operations and Management (DSOM), pp. 221-233, Oct. 2003.
4. P. Barham, R. Black, M. Goldszmidt, R. Isaacs, J. MacCormick, R. Mortier, and A. Simma, "Constellation: automated discovery of service and host dependencies in networked systems", TechReport, MSR-TR-2008-67, April, 2008.
5. S. Kandula, R. Chandra, and D. Katabi, "What's Going On? Learning Communication Rules in Edge Networks", SIGCOMM'08, 2008.
6. V. Bahl, R. Chandra, A. Greenberg, S. Kandula, D. A. Maltz, M. Zhang, "Towards Highly Reliable Enterprise Network Services Via Inference of Multi-level Dependencies", SIGCOMM'07, 2007.
7. th ACM conf. on Internet measurement, pp. 119-132, 2006.
8. th Int. Workshop on Distributed Systems: Operations and Management (DSOM'04), pp. 171-182, Nov. 2004.
9. A. Simma, M. Goldszmidt, J. MacCormick, P. Barham, R. Black, R. Isaacs, and R. Mortier, "CT-NOR: representing and reasoning about events in continuous time", In UAI'08, July 2008.
10. Hadoop. http://hadoop. apache.org/core.
11. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters", In USENIX Symposium on Operating Systems Design and Implementation, Dec. 2004.
12. S. Ghemawat and S. Leung, "The Google File System", In ACM Symposium on Operating Systems Principles", Oct. 2003.
13. Q. Fu, J.-G. Lou, Y. Wang, and J. Li, "Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis", submitted to ICDM'09.
14. Hadoop bug reporting portal. http://issues.apache.org/jira/browse/HADOOP