Toward credible evaluation of anomaly-based intrusion-detection methods

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 40, Issue 5, 2010, Pages 516-524

  Tavallaee, Mahbod ^a   Stakhanova, Natalia ^a   Ghorbani, Ali Akbar ^a  

^a UNIVERSITY OF NEW BRUNSWICK   (Canada)

Author keywords

Anomaly detection; intrusion detection

Indexed keywords

ANOMALY DETECTION; ANOMALY-BASED INTRUSION DETECTION; COMMERCIAL APPLICATIONS; DETECTION METHODS; DETECTION TECHNIQUE; EXPERIMENTAL RESEARCH; FALSE ALARM RATE; HIGH DETECTION RATE; INTRUSION DETECTION SYSTEMS; RESEARCH COMMUNITIES;

CHEMICAL DETECTION; RESEARCH; SURVEYS;

INTRUSION DETECTION;

EID: 77955847605     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2010.2048428     Document Type: Article

References (31)

1. R. Sekar, A. Gupta, J. Frullo, T. Shanbhag, A. Tiwari, H. Yang, and S. Zhou, "Specification-based anomaly detection: A new approach for detecting network intrusions," in Proc. 9th ACM Conf. Comput. Commun. Security (CCS), 2002, pp. 265-274.
2. C. Gates and C. Taylor, "Challenging the anomaly detection paradigm: A provocative discussion," in Proc. 2006 Workshop N. Security Paradigms (NSPW), pp. 21-29.
3. H. Ringberg, M. Roughan, and J. Rexford, "The need for simulation in evaluating anomaly detectors," SIGCOMM Comput. Commun. Rev., vol.38, no.1, pp. 55-59, 2008.
4. P. J. Denning, "ACM President's Letter: What is experimental computer science?" Commun. ACM, vol.23, no.10, pp. 543-544, 1980.
5. W. F. Tichy, P. Lukowicz, L. Prechelt, and E. A. Heinz, "Experimental evaluation in computer science: a quantitative study," J. Syst. Softw., vol.28, no.1, pp. 9-18, 1995.
6. J. Wainer, C. N. Barsottini, D. Lacerda, and L. R. M. de Marco, "Empirical evaluation in computer science research published by ACM," Inf. Softw. Technol., vol.51, no.6, pp. 1081-1085, 2009.
7. M. V. Zelkowitz and D. R. Wallace, "Experimental models for validating technology," Computer, vol.31, no.5, pp. 23-31, 1998.
8. M. V. Zelkowitz, "An update to experimental models for validating computer technology," J. Syst. Softw., vol.82, no.3, pp. 373-376, 2009.
9. J. E. Hannay, O. Hansen, V. By Kampenes, A. Karahasanovic, N.-K. Liborg, and A. C. Rekdal, "A survey of controlled experiments in software engineering," IEEE Trans. Softw. Eng., vol.31, no.9, pp. 733-753, Sep. 2005.
10. C. Zannier, G. Melnik, and F. Maurer, "On the success of empirical studies in the international conference on software engineering," in Proc. 28th Int. Conf. Softw. Eng. (ICSE), New York, NY: ACM, 2006, pp. 341-350.
11. K. Pawlikowski, H.-D. Jeong, and J.-S. R., "On credibility of simulation studies of telecommunication networks," IEEE Commun. Mag., vol.40, no.1, pp. 132-139, Jan. 2001. (Pubitemid 34110003)
12. S. Kurkowski, T. Camp, and M. Colagrosso, "MANET simulation studies: the incredibles," SIGMOBILE Mob. Comput. Commun. Rev., vol.9, no.4, pp. 50-61, 2005.
13. Estimated Venue Impact Factors. (2009, Jan.). [Online]. Available: http://citeseerx.ist.psu.edu/stats/venues
14. W. Trochim and J. P. Donnelly, The Research Methods Knowledge Base. Mason, OH: Atomic Dog, 2006.
15. MIT Lincoln Labs. (2008, Feb.). DARPA intrusion detection evaluation [Online]. Available: http://www.ll.mit.edu/mission/communications/ist/corpora/ ideval/index.html
16. KDD Cup 1999. (2008, Oct.) [Online]. Available: http://kdd.ics.uci.edu/ databases/kddcup99/kddcup99.html
17. J. McHugh, "The 1998 Lincoln Laboratory IDS evaluation," in Proc. 3rd Int. Workshop Recent Adv. Intrusion Detection (RAID). London, U.K.: Springer-Verlag, 2000, pp. 145-161.
18. M. V. Mahoney and P. K. Chan, "An analysis of the 1999 darpa/lincoln laboratory evaluation data for network anomaly detection," in Proc. 6th Int. Symp. Recent Adv. Intrusion Detection. Berlin, Germany: Springer-Verlag, 2003, pp. 220-237.
19. The Internet Traffic Archive. (2009, Apr.). [Online]. Available: http://ita.ee.lbl.gov/
20. A. H. Sung and S. Mukkamala, "Identifying important features for intrusion detection using support vector machines and neural networks," in Proc. Symp. Appl. Internet (SAINT). Washington, DC: IEEE Comput. Soc., 2003, pp. 209-216.
21. N. Hohn and D. Veitch, "Inverting sampled traffic," IEEE/ACM Trans. Netw., vol.14, no.1, pp. 68-80, Jan. 2006.
22. J. Mai, C.-N. Chuah, A. Sridharan, T. Ye, and H. Zang, "Is sampled data sufficient for anomaly detection?" in Proc. 6th ACM SIGCOMM Conf. Internet Meas. (IMC). New York, NY: ACM, 2006, pp. 165-176.
23. P. Tune and D. Veitch, "Towards optimal sampling for flow size estimation," in Proc. 8th ACM SIGCOMM Conf. Internet Meas. (IMC). New York, NY: ACM, 2008, pp. 243-256.
24. V. Yegneswaran, P. Barford, and J. Ullrich, "Internet intrusions: global characteristics and prevalence," SIGMETRICS Perform. Eval. Rev., vol.31, no.1, pp. 138-147, 2003.
25. K. Xu, Z.-L. Zhang, and S. Bhattacharyya, "Profiling internet backbone traffic: behavior models and applications," SIGCOMM Comput. Commun. Rev., vol.35, no.4, pp. 169-180, 2005.
26. C.-C. Chang and C.-J. Lin. (2001). LIBSVM: A library for support vector machines [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/libsvm
27. R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation,and Modeling. New York, NY: Wiley-Interscience, 1991.
28. A. A. Cárdenas, J. S. Baras, and K. Seamon, "A framework for the evaluation of intrusion detection systems," in Proc. 2006 IEEE Symp. Security Privacy (SP). Washington, DC: IEEE Comput. Soc., 2006, pp. 63-77.
29. J. W. Ulvila and J. E. Gaffney, "Evaluation of intrusion detection systems," J. Res. Nat. Inst. Standards Technol., vol.108, no.6, pp. 453-471, 2003.
30. S. Axelsson, "The base-rate fallacy and its implications for the difficulty of intrusion detection," in Proc. 6th ACM Conf. Comput. Commun. Security (CCS), New York, NY: ACM, 1999, pp. 1-7.
31. A. Lazarevic, L. Ertoz, V. Kumar, A. Ozgur, and J. Srivastava, "A comparative study of anomaly detection schemes in network intrusion detection," presented at the 3rd SIAM Int. Conf. Data Mining, San Francisco, CA, 2003.