Mining temporal invariants from partially ordered logs

Managing Large-Scale Systems via the Analysis of System Logs and the Application of Machine Learning Techniques, SLAML'11

Volumn , Issue , 2011, Pages

  Beschastnikh, Ivan ^a   Brun, Yuriy ^a   Ernst, Michael D ^a   Krishnamurthy, Arvind ^a   Anderson, Thomas E ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONCURRENT SYSTEMS; DISTRIBUTED SYSTEMS; LOG ANALYSIS; PARTIAL ORDER; PARTIALLY ORDERED EVENTS; TEMPORAL INVARIANTS;

LARGE SCALE SYSTEMS; LEARNING ALGORITHMS;

LEARNING SYSTEMS;

EID: 81055155516     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2038633.2038636     Document Type: Conference Paper

References (37)

1. BATES, P. High-level Debugging of Distributed Systems: The Behavioral Abstraction Approach. Journal of Systems and Software 3, 4 (Dec. 1983), 255-264. (Pubitemid 15547401)
2. BECKER, D., RABENSEIFNER, R., WOLF, F., AND LINFORD, J. C. Scalable Timestamp Synchronization for Event Traces of Message-Passing Applications. Parallel Comput. 35 (Dec. 2009), 595-607.
3. BESCHASTNIKH, I., BRUN, Y., SCHNEIDER, S., SLOAN, M., AND ERNST, M. D. Leveraging Existing Instrumentation to Automatically Infer Invariant-Constrained Models. In Proceedings of the the 8th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering (2011), ESEC/FSE '11.
4. CHEN, Y., AND CHEN, Y. An Efficient Algorithm for Answering Graph Reachability Queries. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering (Washington, DC, USA, 2008), IEEE Computer Society, pp. 893-902.
5. CONSENS, M. C., HASAN, M. Z., AND MENDELZON, A. O. Debugging Distributed Programs by Visualizing and Querying Event Traces. In Applications of Databases, First International Conference, ADB-94 (1993), vol. 819, pp. 181-183.
6. DAO, D., ALBRECHT, J., KILLIAN, C., AND VAHDAT, A. Live Debugging of Distributed Systems. In Proceedings of the 18th International Conference on Compiler Construction: Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2009 (Berlin, Heidelberg, 2009), CC '09, Springer-Verlag, pp. 94-108.
7. DWYER, M. B., AVRUNIN, G. S., AND CORBETT, J. C. Patterns in Property Specifications for Finite-State Verification. In Proceedings of the 21st international conference on Software engineering (New York, NY, USA, 1999), ICSE '99, ACM, pp. 411-420.
8. ERNST, M. D., COCKRELL, J., GRISWOLD, W. G., AND NOTKIN, D. Dynamically Discovering Likely Program Invariants to Support Program Evolution. IEEE Transactions on Software Engineering 27, 2 (Feb. 2001), 99-123. (Pubitemid 32254259)
9. FIDGE, C. J. Timestamps in Message-Passing Systems that Preserve the Partial Ordering. In 11th Australian Computer Science Conference (University of Queensland, Australia, 1988), pp. 55-66.
10. FLOYD, R. W. Algorithm 97: Shortest path. Commun. ACM 5, 6 (June 1962), 345+.
11. GABEL, M., AND SU, Z. Javert: Fully Automatic Mining of General Temporal Properties from Dynamic Traces. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering (New York, NY, USA, 2008), SIGSOFT '08/FSE-16, ACM, pp. 339-349.
12. GARAVEL, H., LANG, F., MATEESCU, R., AND SERWE, W. CADP 2010: A toolbox for the Construction and Analysis of Distributed Processes. In Tools and Algorithms for the Construction and Analysis of Systems, P. Abdulla and K. Leino, Eds., vol. 6605 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg, Berlin, Heidelberg, 2011, ch. 33, pp. 372-387.
13. GEELS, D., ALTEKAR, G., MANIATIS, P., ROSCOE, T., AND STOICA, I. Friday: Global Comprehension for Distributed Replay. In Networked Systems Design and Implementation (NSDI) (2007).
14. GORALČÍKOVÁ, A., AND KOUBEK, V. A Reduct-and-Closure Algorithm for Graphs. In Mathematical Foundations of Computer Science 1979, J. Becvár, Ed., vol. 74 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg, 1979, pp. 301-307.
15. GUO, Z., WANG, X., TANG, J., LIU, X., XU, Z., WU, M., KAASHOEK, M. F., AND ZHANG, Z. R2: An Application-Level Kernel for Record and Replay. In Proceedings of the 8th USENIX conference on Operating systems design and implementation (Berkeley, CA, USA, 2008), OSDI'08, USENIX Association, pp. 193-208.
16. Welcome to Apache Hadoop! http://hadoop.apache. org/. Accessed April 6, 2011.
17. JIANG, G., CHEN, H., UNGUREANU, C., AND YOSHIHIRA, K. Multi-resolution Abnormal Trace Detection Using Varied-length N-grams and Automata. In Proceedings of the Second International Conference on Automatic Computing (Washington, DC, USA, 2005), IEEE Computer Society, pp. 111-122. (Pubitemid 43959616)
18. JIANG, G., CHEN, H., AND YOSHIHIRA, K. Efficient and Scalable Algorithms for Inferring Likely Invariants in Distributed Systems. IEEE Transactions on Knowledge and Data Engineering 19, 11 (Nov. 2007), 1508-1523. (Pubitemid 47573794)
19. LIU, X., GUO, Z., WANG, X., CHEN, F., LIAN, X., TANG, J., WU, M., KAASHOEK, M. F., AND ZHANG, Z. D3S: Debugging Deployed Distributed Systems. In Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation (Berkeley, CA, USA, 2008), NSDI'08, USENIX Association, pp. 423-437.
20. LOU, J. G., FU, Q., YANG, S., LI, J., AND WU, B. Mining Program Workflow from Interleaved Traces. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (New York, NY, USA, 2010), KDD '10, ACM, pp. 613-622.
21. MATTERN, F. Virtual Time and Global States of Distributed Systems. In Parallel and Distributed Algorithms (1989), pp. 215-226.
22. PERKINS, J. H., KIM, S., LARSEN, S., AMARASINGHE, S., BACHRACH, J., CARBIN, M., PACHECO, C., SHERWOOD, F., SIDIROGLOU, S., SULLIVAN, G., WONG, W.-F., ZIBIN, Y., ERNST, M. D., AND RINARD, M. Automatically Patching Errors in Deployed Software. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (Big Sky, MT, USA, Oct. 2009), pp. 87-102.
23. REYNOLDS, P., KILLIAN, C., WIENER, J. L., MOGUL, J. C., SHAH, M. A., AND VAHDAT, A. Pip: Detecting the Unexpected in Distributed Systems. In Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3 (Berkeley, CA, USA, 2006), NSDI'06, USENIX Association, p. 9.
24. SEN, A., AND GARG, V. K. Detecting Temporal Logic Predicates on the Happened-Before Model. In Proceedings of the 16th International Parallel and Distributed Processing Symposium (Washington, DC, USA, 2002), IPDPS '02, IEEE Computer Society.
25. STONE, J. M. A Graphical Representation of Concurrent Processes. SIGPLAN Not. 24 (Nov. 1988), 226-235.
26. TAN, J., PAN, X., KAVULYA, S., GANDHI, R., AND NARASIMHAN, P. SALSA: Analyzing Logs as State Machines. In Proceedings of the First USENIX conference on Analysis of system logs (Berkeley, CA, USA, 2008), WASL'08, USENIX Association, p. 6.
27. TAN, J., PAN, X., KAVULYA, S., GANDHI, R., AND NARASIMHAN, P. Mochi: Visual Log-analysis based Tools for Debugging Hadoop. In Proceedings of the 2009 conference on Hot topics in cloud computing (Berkeley, CA, USA, 2009), HotCloud'09, USENIX Association, p. 18.
28. THERESKA, E., AND GANGER, G. R. IRONModel: Robust Performance Models in the Wild. SIGMETRICS Perform. Eval. Rev. 36, 1 (June 2008), 253-264.
29. VAARANDI, R. A Breadth-First Algorithm for Mining Frequent Patterns from Event Logs. In Proceedings of the 2004 IFIP International Conference on Intelligence in Communication Systems (2004), vol. 3283, pp. 293-308. (Pubitemid 39737227)
30. XU, W., HUANG, L., FOX, A., PATTERSON, D., AND JORDAN, M. I. Detecting Large-Scale System Problems by Mining Console Logs. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (New York, NY, USA, 2009), SOSP '09, ACM, pp. 117-132.
31. YABANDEH, M., ANAND, A., CANINI, M., AND KOSTIĆ, D. Finding Almost-Invariants in distributed systems. In Proceedings of the 30th IEEE Symposium on Reliable Distributed Systems (Oct. 2011).
32. YABANDEH, M., KNEZEVIC, N., KOSTIĆ, D., AND KUNCAK, V. CrystalBall: Predicting and Preventing Inconsistencies in Deployed Distributed Systems. In Proceedings of the 6th USENIX symposium on Networked systems design and implementation (Berkeley, CA, USA, 2009), USENIX Association, pp. 229-244.
33. YANG, J., CHEN, T., WU, M., XU, Z., LIU, X., LIN, H., YANG, M., LONG, F., ZHANG, L., AND ZHOU, L. MODIST: Transparent Model Checking of Unmodified Distributed Systems. In Proceedings of the 6th USENIX symposium on Networked systems design and implementation (Berkeley, CA, USA, 2009), USENIX Association, pp. 213-228.
34. YANG, J., AND EVANS, D. Automatically Inferring Temporal Properties for Program Evolution. In Proceedings of the 15th International Symposium on Software Reliability Engineering (Washington, DC, USA, Nov. 2004), IEEE Computer Society, pp. 340-351. (Pubitemid 40455706)
35. YANG, J., AND EVANS, D. Dynamically Inferring Temporal Properties. In Proceedings of the 5th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering (New York, NY, USA, 2004), PASTE '04, ACM, pp. 23-28.
36. YUAN, D., MAI, H., XIONG, W., TAN, L., ZHOU, Y., AND PASUPATHY, S. SherLog: Error Diagnosis by Connecting Clues from Run-Time Logs. SIGPLAN Not. 45 (Mar. 2010), 143-154.
37. ZHU, K. Q., FISHER, K., AND WALKER, D. Incremental Learning of System Log Formats. SIGOPS Oper. Syst. Rev. 44 (Mar. 2010), 85-90.