Online detection of multi-component interactions in production systems

Proceedings of the International Conference on Dependable Systems and Networks

Volumn , Issue , 2011, Pages 49-60

  Oliner, Adam J ^a   Aiken, Alex ^a  

^a Stanford University   (United States)

Author keywords

anomalies; modeling; signal compression; statistical correlation; System management

Indexed keywords

ANOMALIES; LARGE SYSTEM; LOG FILE; MULTICOMPONENTS; ON-LINE DETECTION; PRODUCTION SYSTEM; REAL TIME; SCALABLE METHODS; SIGNAL COMPRESSION; STATISTICAL CORRELATION; SYSTEM ADMINISTRATORS; SYSTEM MANAGEMENT;

PRODUCTION ENGINEERING; REAL TIME SYSTEMS; SCALABILITY;

ONLINE SYSTEMS;

EID: 80051926966     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DSN.2011.5958206     Document Type: Conference Paper

References (42)

1. M. K. Aguilera, J. C. Mogul, J. L. Wiener, P. Reynolds, and A. Methitacharoen. Performance debugging for distributed systems of black boxes. In SOSP, 2003.
2. P. Bahl, R. Chandra, A. Greenberg, S. Kandula, D. A. Maltz, and M. Zhang. Towards highly reliable enterprise network services via inference of multi-level dependencies. In SIGCOMM, 2007.
3. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using Magpie for request extraction and workload modelling. In OSDI, 2004.
4. M. Brodie, I. Rish, and S. Ma. Optimizing probe selection for fault localization. In Workshop on Distributed Systems: Operations and Management (DSOM), 2001.
5. A. Brown, G. Kar, and A. Keller. An active approach to characterizing dynamic dependencies for problem determination in a distributed environment. In IEEE IM, 2001.
6. M. Y. Chen, A. Accardi, E. Kiciman, J. Lloyd, D. Patterson, A. Fox, and E. Brewer. Path-based failure and evolution management. In NSDI, 2004.
7. M. Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer. Pinpoint: problem determination in large, dynamic internet services. In DSN, June 2002.
8. S. Chutani and H. Nussbaumer. On the distributed fault diagnosis of computer networks. In IEEE Symposium on Computers and Communications, 1995.
9. I. Cohen, S. Zhang, M. Goldszmidt, J. Symons, T. Kelly, and A. Fox. Capturing, indexing, clustering, and retrieving system history. In SOSP, 2005.
10. C. Ensel. New approach for automated generation of service dependency models. In Latin American Network Operation and Management Symposium (LANOMS), 2001.
11. H. H. Feng, O. M. Kolesnikov, P. Fogla, W. Lee, and W. Gong. Anomaly detection using call stack information. In IEEE Symposium on Security and Privacy, 2003.
12. R. Fonseca, G. Porter, R. H. Katz, S. Shenker, and I. Stoica. X-Trace: A pervasive network tracing framework. In NSDI, 2007.
13. D. Gao, M. K. Reiter, and D. Song. Gray-box extraction of execution graphs for anomaly detection. In CCS, pages 318-329, 2004. (Pubitemid 40338213)
14. D. Geels, G. Altekar, S. Shenker, and I. Stoica. Replay debugging for distributed applications. In USENIX Technical, 2006.
15. J. T. Giffin, S. Jha, and B. P. Miller. Detecting manipulated remote call streams. In USENIX Security, pages 61-79, 2002.
16. K. Glerum, K. Kinshumann, S. Greenberg, G. Aul, V. Orgovan, G. Nichols, D. Grant, G. Loihle, and G. Hunt. Debugging in the (very) large: Ten years of implementation and experience. In SOSP, 2009.
17. I. T. Jolliffe. Principal Component Analysis. Springer, 2002.
18. S. Kandula, D. Katabi, and J.-P. Vasseur. Shrink: A tool for failure diagnosis in IP networks. In MineNet Workshop at SIGCOMM, 2005.
19. R. R. Kompella, J. Yates, A. Greenberg, and A. C. Snoeren. IP fault localization via risk modeling. In NSDI, 2005.
20. C. Kruegel, D. Mutz, F. Valeur, and G. Vigna. On the detection of anomalous system call arguments. LNCS, 2003.
21. X. Liu, Z. Guo, X. Wang, F. Chen, X. Lian, J. Tang, M. Wu, M. F. Kaashoek, and Z. Zhang. D3S: debugging deployed distributed systems. In NSDI, 2008.
22. X. Liu, W. Lin, A. Pan, and Z. Zhang. WiDS Checker: Combating bugs in distributed systems. In NSDI, 2007.
23. J. C. Mogul. Emergent (mis)behavior vs. complex software systems. In EuroSys, 2006.
24. M. Montemerlo et al. Junior: The Stanford entry in the Urban Challenge. Journal of Field Robotics, 2008.
25. A. J. Oliner and A. Aiken. A query language for understanding component interactions in production systems. In ICS, 2010.
26. A. J. Oliner, A. Aiken, and J. Stearley. Alert detection in system logs. In ICDM, December 2008.
27. A. J. Oliner, A. V. Kulkarni, and A. Aiken. Using correlated surprise to infer shared influence. In DSN, 2010.
28. A. J. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In DSN, 2007.
29. X. Pan, J. Tan, S. Kavulya, R. Gandhi, and P. Narasimhan. Ganesha: Black-box fault diagnosis for MapReduce systems. Technical report, CMU-PDL-08-112, 2008.
30. S. Papadimitriou, J. Sun, and C. Faloutsos. Streaming pattern discovery in multiple time-series. In VLDB, 2005.
31. P. Reynolds, C. Killian, J. L. Wiener, J. C. Mogul, M. A. Shah, and A. Vahdat. Pip: Detecting the unexpected in distributed systems. In NSDI, 2006.
32. P. Reynolds, J. L. Wiener, J. C. Mogul, M. K. Aguilera, and A. Vahdat. WAP5: black-box performance debugging for wide-area systems. In WWW, 2006.
33. I. Rish, M. Brodie, N. Odintsova, S. Ma, and G. Grabarnik. Real-time problem determination in distributed systems using active probing. In NOMS, 2004.
34. Y. Sakurai, S. Papadimitriou, and C. Faloutsos. BRAID: Stream mining through group lag correlations. In SIGMOD, 2005.
35. R. Schwarz and F. Mettern. Detecting causal relationships in distributed computations: in search of the holy grail. Distributed Computing, 1994.
36. A. Singh, P. Maniatis, T. Roscoe, and P. Druschel. Using queries for distributed monitoring and forensics. In EuroSys, 2006.
37. The Computer Failure Data Repository (CFDR). The HPC4 data. http://cfdr.usenix.org/data.html,2009.
38. S. Thrun and M. Montemerlo, et al. Stanley: The robot that won the DARPA Grand Challenge. Journal of Field Robotics, 2006.
39. W. Xu, L. Huang, A. Fox, D. Patterson, and M. Jordan. Online system problem detection by mining patterns of console logs. In ICDM, 2009.
40. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan. Detecting large-scale system problems by mining console logs. In SOSP, 2009.
41. E. S. K. Yu and J. Mylopoulos. Understanding "why" in software process modelling, analysis, and design. In ICSE, Sorrento, Italy, May 1994.
42. Y. Zhao, Z. Zhu, Y. Chen, D. Pei, and J. Wang. Towards efficient largescale VPN monitoring and diagnosis under operational constraints. In INFOCOM, 2009.