Clustering intrusion detection alarms to support root cause analysis

ACM Transactions on Information and System Security

Volumn 6, Issue 4, 2003, Pages 443-471

  Julisch, Klaus ^a  

^a IBM RESEARCH ZURICH   (Switzerland)

Author keywords

Cluster analysis; Data mining; False positives; Intrusion detection; Root cause analysis

Indexed keywords

CLUSTER ANALYSIS; FALSE POSITIVES; INTRUSION DETECTION; ROOT CAUSE ANALYSIS;

ALARM SYSTEMS; COMPUTER SOFTWARE; DATA MINING; INFORMATION MANAGEMENT; RISK MANAGEMENT; SECURITY OF DATA;

COMPUTER NETWORKS;

EID: 3142623031     PISSN: 10949224     EISSN: None     Source Type: Journal    
DOI: 10.1145/950191.950192     Document Type: Review

References (70)

1. AGRAWAL, R., GEHRKE, J., GUNOPULOS, D., AND RAGHAVAN, P. 1998. Automatic subspace clustering of high dimensional data for data mining applications. In ACM SIGMOD International Conference on Management of Data, 94-105.
2. ALLEN, J., CHRISTIE, A., FITHEN, W., McHuGH, J., PICKEL, J., AND STONER, E. 2000. State of the practice of intrusion detection technologies. Tech. Rep., Carnegie Mellon University. http://www.cert.org/archive/pdf/99tr028.pdf.
3. ALMGREN, M., DEBAR, H., AND DACIER, M. 2000. A lightweight tool for detecting web server attacks. In Network and Distributed System Security Symposium (NDSS 2000), 157-170.
4. ANDERBERG, M. R. 1973. Cluster Analysis for Applications. Academic Press, New York.
5. AXELSSON, S. 2000. The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security 3, 3, 186-205.
6. BACE, R. 2000. Intrusion Detection. Macmillan Technical Publishing.
7. BARBARÁ, D. AND JAJODIA, S., (Eds.). 2002. Applications of Data Mining in Computer Security. Kluwer Academic Publishers, Boston, MA.
8. BARBARÁ, D., WU, N., AND JAJODIA, S. 2001. Detecting Novel network intrusions using Bayes estimators. In 1st SIAM International Conference on Data Mining (SDM'01).
9. BELLOVIN, S. M. 1993. Packets found on an Internet. Computer Communications Review 23, 3, 26-31.
10. BLOEDORN, E., HILL, B., CHRISTIANSEN, A., SKORUPKA, C., TALBOOT, L., AND TIVEL, J. 2000. Data mining for improving intrusion detection, http://www.mitre.org/support/papers/tech_papers99_00/.
11. BOULOUTAS, A., CALO, S., AND FINKEL, A. 1994. Alarm correlation and fault identification in communication networks. IEEE Transactions on Communications 42, 2-4, 523-533.
12. BRODERICK, J. (ed.) 1998. IBM outsourced solution. http://www.infoworld. com/cgi-bin/displayTC.pl?/980504sb3-ibm.htm.
13. CERT. 1996. Advisory CA-1996-26: Denial-of-service attack via ping. http://www.cert.org/advisories/CA-1996-26.html.
14. CERT. 2001. Advisory CA-2001-19: "Code Red" worm exploiting buffer overflow in US indexing service DLL. http://www.cert.org/advisories/CA- 2001-19.html.
15. CHAN, P. AND STOLFO, S. 1998. Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection. In 4th International Conference on Knowledge Discovery and Data Mining, 164-168.
16. CHEUNG, D., HWANG, H., FU, A., AND HAN, J. 2000. Efficient rule-based attribute-oriented induction for data mining. Journal of Intelligent Information Systems 15, 2, 175-200.
17. CLIFTON, C. AND GENGO, G. 2000. Developing custom intrusion detection filters using data mining. In Military Communications International Symposium (MILCOM2000).
18. CUPPENS, F. 2001. Managing alerts in a multi-intrusion detection environment. In 17th Annual Computer Security Applications Conference (ACSAC), 22-31.
19. CUPPENS, F. AND MIÈGE, A. 2002. Alert correlation in a cooperative intrusion detection framework. In IEEE Symposium on Security and Privacy, Oakland, CA.
20. DAIN, O. AND CUNNINGHAM, R. K. 2002. Fusing heterogeneous alert streams into scenarios. See Barbará and Jajodia [2002].
21. DEBAR, H., DACIER, M., AND WESPI, A. 2000. A revised taxonomy for intrusion detection systems. Annales des Télécommunications 55, 7-8, 361-378.
22. DEBAR, H. AND WESPI, A. 2001. Aggregation and correlation of intrusion-detection alerts. In 4th Workshop on Recent Advances in Intrusion Detection (RAID). LNCS. Springer Verlag, Berlin, 85-103.
23. DOUGHERTY, J., KOHAVI, R., AND SAHAMI, M. 1995. Supervised and unsupervised discretization of continuous features. In. Proceedings of the 12th International Conference on Machine Learning, 194-202.
24. ERLINGER, M. AND STANIFORD-CHEN, S. Intrusion detection exchange format (IDWG). http://www.ietf.org/html.charters/idwg-charter.html.
25. FAWCETT, T. AND PROVOST, F. 1997. Adaptive fraud detection. Data Mining and Knowledge Discovery 1, 291-316.
26. HAN, J., CAI, Y., AND CERCONE, N. 1992. Knowledge discovery in databases: An attribute-oriented approach. In 18th International Conference on Very Large Databases, 547-559.
27. HAN, J., CAI, Y., AND CERCONE, N. 1993. Data-driven discovery of quantitative rules in relational databases. IEEE Transactions on Knowledge and Data Engineering 5, 1, 29-40.
28. HAN, J. AND FU, Y. 1994. Dynamic generation and refinement of concept hierarchies for knowledge discovery in databases. In Proceedings of the AAAI Workshop on Knowledge Discovery in Databases, 157-168.
29. HAN, J. AND KAMBER, M. 2000. Data Mining: Concepts and Techniques. Morgan Kaufmann Publisher, San Mateo, CA.
30. HELLERSTEIN, J. L. AND MA, S. 2000. Mining event data for actionable patterns. In The Computer Measurement Group.
31. HOUCK, K., CALO, S., AND FINKEL, A. 1995. Towards a practical alarm correlation system. In 4th International Symposium on Integrated Network Management. A. S. Sethi, Y. Raynaud, and F. Faure-Vincent, Eds. Chapman & Hall, London, 226-237.
32. ILUNG, K. 1993. USTAT: A real-time intrusion detection system for UNIX. In IEEE Symposium on Security and Privacy, Oakland, CA, 16-28.
33. JAIN, A. AND DUBES, R. 1988. Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs, NJ.
34. JAKOBSON, G. AND WEISSMAN, M. D. 1993. Alarm Correlation. IEEE Network 7, 6, 52-59.
35. JAKOBSON, G. AND WEISSMAN, M. D. 1995. Real-time telecommunication network management: Extending event correlation with temporal constraints. In 4th International Symposium on Integrated Network Management. A. S. Sethi, Y. Raynaud, and F. Faure-Vincent, Eds. Chapman & Hall, London, 290-301.
36. JULISCH, K. 2001. Mining alarm clusters to improve alarm handling efficiency. In 17th Annual Computer Security Applications Conference (ACSAC), 12-21.
37. JULISCH, K. AND DACIER, M. 2002. Mining intrusion detection alarms for actionable knowledge. In 8th ACM International Conference on Knowledge Discovery and Data Mining, 366-375.
38. KLEMETTINEN, M. 1999. A knowledge discovery methodology for telecommunication network alarm data. Ph.D. Thesis, University of Helsinky, Finland.
39. KUMAR, S. 1995. Classification and detection of computer intrusions. Ph.D. Thesis, Purdue University.
40. LAPRIE, J., Ed. 1992. Dependability: Basic Concepts and Terminology. Dependable Computing and Fault-Tolerant Systems, vol. 5. Springer-Verlag, Vienna.
41. LEE, W. AND STOLFO, S. J. 2000. A framework for constructing features and models for intrusion detection systems. ACM Transactions on Information and System Security 3, 4, 227-261.
42. LEWIS, L. 1993. A case-based reasoning approach to the resolution of faults in communication networks. In 3th International Symposium on Integrated Network Management. H.-G. Hegering and Y. Yemini, Eds. North Holland, Amsterdam, 671-682.
43. LIN, D. 1998. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, 296-304.
44. LIPPMANN, R., WEBSTER, S., AND STETSON, D. 2002. The effect of identifying vulnerabilities and patching software on the utility of network intrusion detection. In 5th Workshop on Recent Advances in Intrusion Detection (RAID). LNCS, vol. 2516. Springer Berlin, Verlag, 307-326.
45. LU, Y. 1997. Concept hierarchy in data mining: Specification, generation, and implementation. M.S. Thesis, Simon Fraser University, Canada.
46. MANGANARIS, S., CHRISTENSEN, M., ZERKLE, D., AND HERMIZ, K. 2000. A data mining analysis of RTID alarms. Computer Networks 34, 4, 571-577.
47. MCHUGH, J. 2000. Testing intrusion detection systems: A critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory. ACM Transactions on Information and System Security 3, 4, 262-294.
48. MILLER, R. AND YANG, Y. 1997. Association rules over interval data. In Proceedings of the ACM SIGMOD International Conference on Management of Data, 452-461.
49. MILLIGAN, G. W. 1996. Clustering validation: Results and implications for applied analyses. World Scientific Publishing, Singapore, 341-375.
50. MOUNJI, A. 1997. Languages and tools for rule-based distributed intrusion detection. Ph.D. Thesis, Facultes Universitaires Notre-Dame de la Paix Namur, Belgium.
51. NING, P., JAJODIA, S., AND WANG, X. 2001. Abstraction-based intrusion detection in distributed environments. ACM Transactions on Information and System Security 4, 4, 407-452.
52. NYGATE, Y. A. 1995. Event correlation using rule and object based techniques. In 4th International Symposium on Integrated Network Management. A. S. Sethi, Y. Raynaud, and F. Faure-Vincent, Eds. Chapman & Hall, London, 278-289.
53. OHSIE, D. A. 1998. Modeled abductive inference for event management and correlation. Ph.D. Thesis, Columbia University.
54. PAPADIMITRIOU, C. H. 1994. Computational Complexity. Addison-Wesley, Reading, MA.
55. PAXSON, V. 1999. Bro: A system for detecting network intruders in real-time. Computer Networks 31, 23/24, 2435-2463.
56. PENG, Y. AND REGGIA, J. A. 1987a. A probabilistic causal model for diagnostic problem solving -Part I: Diagnostic strategy. IEEE Transactions on Syst. Man Cybern. 17, 3, 395-404.
57. PENG, Y. AND REGGIA, J. A. 1987b. A probabilistic causal model for diagnostic problem solving -Part I: Integrating symbolic causal inference with numeric probabilistic inference. IEEE Trans. Syst. Man. Cybern. 17, 2, 146-162.
58. POWELL, D. AND STROUD, R. 2001. Architecture and Revised Model of MAFTIA. Tech. Rep. CS-TR-749, University of Newcastle upon Tyne.
59. PRICE, K. E. 1997. Host-based misuse detection and conventional operating systems' audit data collection. M.S. Thesis, Purdue University.
60. PTACEK, T. H. AND NEWSHAM, T. N. 1998. Insertion, evasion, and denial of service: Eluding network intrusion detection. Tech. Rep., Secure Networks, Inc.
61. RADA, R. AND BICKNELL, B. 1989. Ranking documents with a thesaurus. Journal of the American Society for Information Science 40, 5, 304-310.
62. RADA, R., MILL, H., BICKNELL, E., AND BLETTNER, M. 1989. Development and application of a metric on semantic nets. IEEE Transactions on Syst. Man Cybern. 19, 1, 17-30.
63. RESNIK, P. 1999. Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11, 95-130.
64. RIGOUTSOS, I. AND FLORATOS, A. 1998. Combinatorial pattern discovery in biological sequences: The TEIRESIAS algorithm. Bioinformatics 14, 1, 55-67.
65. SEKAR, R., GUANG, Y., VERMA, S., AND SHANBHAG, T. 1999. A high-performance network intrusion detection system. In 6th ACM Conference on Computer and Communications Security, 8-17.
66. STANIFORD, S., HOAGLAND, J. A., AND MCALERNEY, J. M. 2000. Practical automated detection of stealthy portscans. In ACM Computer and Communications Security IDS Workshop, 1-7.
67. TANENBAUM, A. S. 1996. Computer Networks. Prentice-Hall, Englewood Cliffs, NJ.
68. VALDES, A. AND SKINNER, K. 2001. Probabilistic alert correlation. In 4th Workshop on Recent Advances in Intrusion Detection (RAID). LNCS. Springer Verlag, Berlin, 54-68.
69. YEMINI, S., KLIGER, S., MOZES, E., YEMINI, Y., AND OHSIE, D. 1996. High speed & robust event correlation. IEEE Communications Magazine 34, 5, 82-90.
70. ZAMBONI, D. 2001. Using internal sensors for computer intrusion detection. Ph.D. Thesis, Purdue University.