Behavior-based modeling and its application to Email analysis

ACM Transactions on Internet Technology

Volumn 6, Issue 2, 2006, Pages 187-221

  Stolfo, Salvatore J ^a,b   Hershkop, Shlomo ^a,b   Hu, Chia Wei ^a,b   Li, Wei Jen ^a,b   Nimeskern, Oliver ^a,b   Wang, K E ^a,b  

^a Och Spine at New York Presbyterian Hospitals   (United States)

^b Columbia University ^*  (United States)

Author keywords

Anomaly detection; Behavior profiling; Email virus propagations

Indexed keywords

ANOMALY DETECTION; BEHAVIOR PROFILING; EMAIL VIRUS PROPAGATIONS; USER'S (SOCIAL) CLIQUES;

COMPUTER VIRUSES; ELECTRONIC COMMUNITIES; ELECTRONIC MAIL; INTELLIGENT AGENTS; LAW ENFORCEMENT; SPAMMING; USER INTERFACES;

DATA MINING;

EID: 33747144912     PISSN: 15335399     EISSN: 15335399     Source Type: Journal    
DOI: 10.1145/1149121.1149125     Document Type: Article

References (40)

1. AGRAWAL, R., IMIELINSKI, T., AND SWAMI, A. 1993. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD International Conference on the Management of Data. pp. 207-216.
2. APAP, P., ANDREW HONIG, A., SHLOMO HERSHKOP, S., ELEAZAR ESKIN, E., AND STOLFO, S. J. 2002. Detecting malicious software by monitoring anomalous windows registry accesses. In Proceedings of the Fifth International Symposium on Recent Advances in Intrusion Detection (RAID-2002, Zurich, Switzerland, Oct.). 16-18.
3. BHATTACHARYYA, M., HERSHKOP, S., ESKIN, E., AND STOLFO, S. J. 2002. MET: An experimental system for malicious email tracking. In Proceedings of the 2002 New Security Paradigms Workshop (NSPW-2002), Virginia Beach, VA, Sept.).
4. BI, Z., FALOUSTOS, C., AND KORN, F. 2001. The DGX distribution for mining massive, skewed data. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 17-26.
5. BRON, C. AND KERBOSCH, J. 1973. Finding all cliques of an undirected graph. Commun. ACM 16, 9, 575-577.
6. DAMASHEK, M. 1995. Gauging similarity with n-grams: Language independent categorization of text. In Science, 267, 843-848.
7. DAVIS, P. T. 2003. Finding friends and enemies through the analysis of clique dynamics. Tech. rep., Computer Science Department, Columbia University, New York, NY.
8. DENNING, D. E. 1987. An intrusion-detection model. IEEE Trans. Softw. Eng., SE-13, 222-232.
9. ESKIN, E. 2000. Anomaly detection over noisy data using learned probability distributions. In Proceedings of the 17th International Conference on Machine Learning (ICML-2000).
10. ESKIN, E., ARNOLD, A., PRERAU, M., PORTNOY, L., AND STOLFO, S. J. 2002. A geometric framework for unsupervised anomaly detection: Detecting intrusions in unlabeled data. Data Mining for Security Applications.(Jajodia, Barbara, Eds.), Kluwer, Norwell, MA.
11. GHOSH, A. K., SCHWARTZBARD, A., AND SCHATZ, M. 1999. Learning Program Behavior Profiles for Intrusion Detection. In Proceedings of the Workshop Intrusion Detection and Network Monitoring 1999.51-62.
12. HERSHKOP, S., FERSTER, R., BUI, L. H., WANG, K., AND STOLFO, S. J. 2003. Host-based anomaly detection by wrapping file system accesses. Tech. rep. Columbia University, New York, NY. Go online to http://www.cs.columbia.edu/ids/ publications/.
13. HOFMEYR, S. A., FORREST, S., AND SOMAYAJI, A. 1998. Intrusion detection using sequences of system calls. J. Comput. Secur. 6, 151-180.
14. HOGG, R. V. AND CRAIG, A. T. 1994. Introduction to Mathematical Statistics, Prentice Hall, Englewood Cliffs, N.J., 293-301.
15. JAVITZ, H. S. AND VALDES, A. 1993. The NIDES Statistical Component: Description and Justification. Tech. rep. SRI International, Menlo Park, CA.
16. JOHN, G. H. AND LANGLEY, P. 1995. Estimating continuous distributions in Bayesian classifiers. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence. 338-345.
17. KLEINBERG, J. 2002. Bursty and hierarchical structure in streams. In Proceedings 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 91-101.
18. LANE, T. AND BRODLEY, C. E. 1999. Temporal sequence learning and data reduction for anomaly detection. ACM Trans. Inform. Syst. Secur., 2, 295-331.
19. LEE, W. AND STOLFO, S. 1999. A framework for constructing features and models for intrusion detection systems. In Proceedings of the 1999 IEEE Symposium on Computer Security and Privacy and Proceedings of the 8th ACM SICKDD International Conference on Knowledge Discovery and Data Mining.
20. LEE, W., STOLFO, S., AND CHAN, P. 1997. Learning patterns from Unix process execution traces for intrusion detection. In Proceedings of the AAAI Workshop: AI Approaches to Fraud Detection and Risk Management (July).
21. LEE, W., STOLFO, S., AND MOK, K. 1998. Mining audit data to build intrusion detection models. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD '98), New York, NY, Aug.)
22. LEE, W., STOLFO, S. J., AND MOK, K. 1999. Mining in a data-flow environments: Experiences in intrusion detection. In Proceedings of the 1999 Conference on Knowledge Discovery and Data Mining (KDD-99).
23. LEE, W. AND XIANG, D. 2001. Information-theoretic measures for anomaly detection. In Proceedings of the 2001 IEEE Symposium on Security and Privacy (May).
24. MAHONEY, M. V. AND CHAN, P. K. 2001. Detecting novel attacks by identifying anomalous network packet headers. Tech. rep. Florida Institute of Technology, Melbourne, FL. CS-2001-2.
25. MITCHELL, T. M. 1997. Machine Learning, McGraw-Hill, New York, NY, 180-183.
26. MYSQL. 2002. Go online to www.mysql.org.
27. NEWMAN, M. E., FORREST, S., AND BALTHRUP, J. 2002. Email networks and the spread of computer viruses. Phys. Rev. E 66, 3 (Sept.).
28. NIBLACK, W., ET AL. 1993. The QBIC project: Querying images by content using color, texture, and shape. In Proceedings of the SPIE (Feb.).
29. SCHONLAU, M., DUMOUCHEL, W., JU, W., KARR, A. F., THEUS, M., AND VARDI, Y. 2001. Computer intrusion detecting masquerades. Statist. Sci. 16, 1, 1-17.
30. SCHULTZ, M. G., ESKIN, E., AND STOLFO, S. J. 2001. Malicious email filter - A UNIX mail filter that detects malicious windows executables. In Proceedings of USENIX Annual Technical Conference-FREENIX Track (Boston, MA).
31. SMITH, J. R. 1997. Integrated spatial and feature image systems: Retrieval, compression and analysis. Ph. D. deissertation. Columbia University, New York, NY.
32. STOLFO, S. J., HERSHKOP, S., WANG, K., NIMESKERN, D., AND HU, C.-W. 2003. Behavior profiling of email. In Proceedings of the 1st NSF/NIJ Symposium on Intelligence & Security Informatics (ISI 2003, Tucson, AZ).
33. STOLFO, S. J., CHAN, P., AND PRODROMIDIS, A. 1999. Distributed data mining in credit card fraud detection, IEEE Intell. Syst. 14, 6, 67-74.
34. TAN, K. M. C. AND MAXION, R. A. 2002. Why 6? Defining the operational limits of stide, an anomaly-based intrusion detector. In Proceedings of the IEEE Symposium on Security and Privacy. IEEE Computer Society Press, Los Alamitos, CA, 188-201.
35. TAYLOR, C. AND ALVES-FOSS, J. 2001. NATE: Network analysis of anomalous traffic events, a low-cost approach. In Proceedings of the New Security Paradigms Workshop. 89-96.
36. WAGNER, D. AND SOTO, P. 2002. Mimicry attacks on host-based intrusion detection systems. In Proceedings of the 9th ACM Conference on Computer and Communications Security (CCS, Washington, DC). 255-264.
37. WARRENDER, C., FORREST, S., AND PEARLMUTTER, B. 1999. Detecting intrusions using system calls: Alternative data models. In Proceedings of the IEEE Symposium Security and Privacy.
38. WATTS, D. J. 2003. Six Degrees: The Science of a Connected Age. W.W. Norton & Company, New York, NY.
39. WILLIAMSON, M. M. 2002. Throttling viruses: Restricting propagation to defeat malicious mobile code. In Proceedings of the ACSAC Security Conference (Las Vegas, NV).
40. YE, N. 2000. A markov chain model of temporal behavior for anomaly detection. In Proceedings of the 2000 IEEE Workshop on Information Assurance and Security (U. S. Military Academy, West Point, NY).