Sequential anomaly detection based on temporal-difference learning: Principles, models and case studies

Applied Soft Computing Journal

Volumn 10, Issue 3, 2010, Pages 859-867

  Xu, Xin ^a  

^a NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Anomaly detection; Computer security; Learning prediction; Markov reward processes; Reinforcement learning; Temporal difference

Indexed keywords

ANOMALY DETECTION; ANOMALY DETECTION METHODS; ANOMALY DETECTION MODELS; APPLICATION DOMAINS; BEHAVIOR PATTERNS; COMPUTER SECURITY; CYBER-ATTACKS; DETECTION ACCURACY; HOST COMPUTERS; LABELED TRAINING DATA; LEARNING PREDICTION; MACHINE LEARNING METHODS; MARKOV REWARD PROCESS; MARKOV REWARD PROCESS MODEL; MULTI-STAGE; OPEN PROBLEMS; PREDICTION ACCURACY; RESEARCH AREAS; REWARD FUNCTION; SEQUENTIAL DATA; SYSTEM CALLS; TD-LEARNING; TEMPORAL DIFFERENCE LEARNING; VALUE FUNCTIONS;

COMPUTER CRIME; INTRUSION DETECTION; NETWORK SECURITY; REINFORCEMENT; REINFORCEMENT LEARNING; RESEARCH; SECURITY SYSTEMS;

LEARNING ALGORITHMS;

EID: 77649270156     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2009.10.003     Document Type: Article

References (29)

1. Chandola V., Arindam Banerjee, and Vipin Kumar. Anomaly detection: a survey. ACM Computing Surveys (2009) 1-72
2. Joshi M.V., Agarwal R.C., and Kumar V. Predicting rare classes: can boosting make any weak learner strong?. Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA (2002) 297-306
3. Chawla N.V., Japkowicz N., and Kotcz A. Editorial: special issue on learning from imbalanced data sets. SIGKDD Explorations 6 1 (2004) 1-6
4. Steinwart I., Hush D., and Scovel C. A classification framework for anomaly detection. Journal of Machine Learning Research 6 (2005) 211-232
5. Sekar R., Gupta A., Frullo J., Shanbhag T., Tiwari A., Yang H., and Zhou S. Specification-based anomaly detection: a new approach for detecting network intrusions. Proceedings of the 9th ACM Conference on Computer and Communications Security (2002), ACM Press 265-274
6. Lee W., Stolfo S.J., and Mok K.W. Adaptive intrusion detection: a data mining approach. Artificial Intelligence Review 14 6 (2000) 533-567
7. P. Laskov, P. Düssel, C. Schäfer, K. Rieck, Learning intrusion detection: supervised or unsupervised? Proc. ICIAP 2005, September. Lecture Notes in Computer Science, LNCS 3617 (2005) 50-57.
8. Mahoney M., and Chan: P. Learning nonstationary models of normal network traffic for detecting novel attacks. Proceedings of 8th International Conference on Knowledge Discovery and Data Mining (2002) 376-385
9. Shah H., Undercoffer J., and Joshi: A. Fuzzy clustering for intrusion detection. Proceedings of the 12th IEEE International Conference on Fuzzy Systems (2003) 1274-1278
10. X. Xu, X.N. Wang, Adaptive network intrusion detection method based on PCA and support vector machines, ADMA 2005, Lecture Notes in Artificial Intelligence, LNAI 3584 (2005) 696-703.
11. Jha S., Tan K., and Maxion R. Markov chains, classifiers, and intrusion detection. Proceedings of the Computer Security Foundations Workshop (CSFW). June (2001)
12. Yeung D.Y., and Ding Y.X. Host-based intrusion detection using dynamic and static behavioral models. Pattern Recognition 36 (2003) 229-243
13. Ye N., Zhang Y., and Borror C.M. Robustness of the Markov-Chain model for cyber-attack detection. IEEE Transactions on Reliability 53 1 (2004) 116-123
14. Lane T. A decision-theoretic, semi-supervised model for intrusion detection. Machine Learning & Data Mining for Computer Security: Methods & Applications (2006), Springer pp. 157-178
15. Kaelbling L.P., Littman M.L., and Moore A.W. Reinforcement learning: a survey. Journal of Artificial Intelligence Research 4 (1996) 237-285
16. Cannady J. Next generation intrusion detection: autonomous reinforcement learning of network attacks. 23th National Information Systems Security Conference (2000)
17. http://www.cs.unm.edu/∼immsec/data-sets.htm.
18. Ourston D., Matzner S., et al. Applications of hidden Markov models to detecting multi-stage network attacks. Proc. of the 36th Hawaii International Conference on System Science (2002) 334-343
19. Sutton R. Learning to predict by the method of temporal differences. Machine Learning 3 1 (1988) 9-44
20. Hofmeyr S., et al. Intrusion detection using sequences of systems call. Journal of Computer Security 6 (1998) 151-180
21. Liao Y.H., and Vemuri V.R. Using text categorization techniques for intrusion detection. Proceedings of the 11th USENIX Security Symposium. August (2002) 51-59
22. Tsitsiklis J.N., and Roy B.V. An analysis of temporal difference learning with function approximation. IEEE Transactions on Automatic Control 42 5 (1997) 674-690
23. Boyan J.A. Technical update: least-squares temporal difference learning. Machine Learning 49 (2002) 233-246
24. Egan J.P. Signal Detection Theory and ROC Analysis (1975), Academic Press, New York (Series in Cognition and Perception)
25. D.K. Kang, D. Fuller, V. Honavar, Learning classifiers for misuse detection using a bag of system calls representation, in: P. Kantor et al. (Eds.), ISI 2005, Lecture Notes in Computer Science, 3495 (2005) 511-516.
26. Xu X., Tao T., and Lu X.C. Kernel least-squares temporal difference learning. International Journal of Information Technology 11 9 (2005) 54-63
27. Lee W., and Xiang D. Information-theoretic measures for anomaly detection. IEEE Symposium on Security and Privacy (2001)
28. Tajbakhsh A. Mohammad Rahmati, and Abdolreza Mirzaei, intrusion detection using fuzzy association rules. Applied Soft Computing 9 2 (2009) 462-469
29. Lazarevic A., Ertoz L., Kumar V., Ozgur A., and Srivastava J. A comparative study of anomaly detection schemes in network intrusion detection. Proceedings of the Third SIAM International Conference on Data Mining (2003) 25-36