Event labeling combining ensemble detectors and background knowledge

Progress in Artificial Intelligence

Volumn 2, Issue 2-3, 2014, Pages 113-127

  Fanaee T, Hadi ^a   Gama, Joao ^a  

^a UNIVERSITY OF PORTO   (Portugal)

Author keywords

Background knowledge; Ensemble learning; Event detection; Event labeling

Indexed keywords

TIME SERIES;

BACK-GROUND KNOWLEDGE; COMPARATIVE STUDIES; DISTANCE FUNCTIONS; ENSEMBLE LEARNING; EVENT DETECTION; STATIC EVALUATION; TIME-CONSUMING TASKS; UNSUPERVISED APPROACHES;

PREDICTIVE ANALYTICS;

EID: 85040007509     PISSN: 21926352     EISSN: 21926360     Source Type: Journal    
DOI: 10.1007/s13748-013-0040-3     Document Type: Article

References (56)

1. Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433–459 (2010)
2. Aggarwal, C.C.: Outlier ensembles: position paper. SIGKDD Explor. Newsl. 14(2), 49–58 (2013)
3. Anantharam, P., Thirunarayan, K., Sheth, A.: Topical anomaly detection from twitter stream. In Proceedings of the 3rd Annual ACMWeb Science Conference, WebSci ’12, pp. 11–14, NewYork, ACM (2012)
4. Ashfaq, A., Javed, M., Khayam, S., Radha, H.: An informationtheoretic combining method for multi-classifier anomaly detection systems. In: Communications (ICC), 2010 IEEE international conference on, pp. 1–5 (2010)
5. Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: Proceedings of the 2nd ACM SIGCOMM workshop on internet measurment, IMW ’02, pp. 71–82, New York, ACM (2002)
6. Borgnat, P., Abry, P., Flandrin, P., Robardet, C., Rouquier, J.-B., Fleury, E.: Shared bicycles in a city: a signal processing and data analysis perspective. Adv. Complex Syst. 14(03), 415–438 (2011)
7. Brauckhoff, D., Dimitropoulos, X., Wagner, A., Salamatian, K.: Anomaly extraction in backbone networks using association rules. In: Proceedings of the 9thACMSIGCOMMconference on Internet measurement conference, IMC ’09, pp. 28–34, New York, ACM (2009)
8. Buckeridge, D.L., Burkom, H., Campbell, M., Hogan, W.R., Moore, A.W.: Algorithms for rapid outbreak detection: a research synthesis. J. Biomed. Inform. 38(2), 99–113 (2005)
9. Capital Bike Share System: Capital bike sharing trip history data. http://www.capitalbikeshare.com/trip-history-data (2013)
10. Carletta, J.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22(2), 249–254 (1996)
11. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009)
12. Collier, N., Doan, S., Kawazoe, A., Goodwin, R.M., Conway, M., Tateno, Y., Ngo, Q.-H., Dien, D., Kawtrakul, A., Takeuchi, K., et al.: Biocaster: detecting public health rumors with aweb-based text mining system. Bioinformatics 24(24), 2940–2941 (2008)
13. Dembski, W.A.: The Design Inference: Eliminating Chance Through Small Probabilities. Cambridge University Press (1998)
14. Department of Human Resources: District of Columbia. Washington D.C. holiday schedule. http://dchr.dc.gov/page/ holiday-schedule (2013)
15. Dewaele, G., Fukuda, K., Borgnat, P., Abry, P., Cho K.: Extracting hidden anomalies using sketch and non gaussian multiresolution statistical detection procedures. In: Proceedings of the 2007 workshop on Large scale attack defense, LSAD ’07, pp. 145–152, New York, ACM (2007)
16. Dietterich, T.G.: Ensemble methods in machine learning. In: Proceedings of the first international workshop on multiple classifier systems, MCS ’00, pp. 1–15. Springer, London (2000)
17. Fanaee-T H., Gama, J.: Bike sharing data set. http://fanaee.com/research/datasets/bike/ (2013)
18. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378–382 (1971)
19. Floyd, S., Paxson, V.: Difficulties in simulating the internet. IEEE/ACM Trans. Netw. 9(4), 392–403 (2001)
20. Fontugne, R., Borgnat, P., Abry, P., Fukuda, K.: Mawilab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking. In: Proceedings of the 6th International COnference, Co-NEXT ’10, pp. 8:1–8:12. New York, ACM (2010)
21. Freemeteo: Washington D.C. weather history. http://www.freemeteo.com (2013)
22. Ghosh, A.K., Schwartzbard, A., Schatz, M.: Learning program behavior profiles for intrusion detection. In: Proceedings of the 1st conference on Workshop on Intrusion Detection and Network Monitoring. USENIX Association, vol. 1, pp. 6–6. ID’99, Berkeley (1999).
23. Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9(1), 69–82 (2008)
24. Guralnik, V., Srivastava, J.: Event detection from time series data. In: Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 33–42. ACM (1999)
25. Hastie, T., Tibshirani, R., Friedman, J.J.H.: The elements of statistical learning. vol. 1, Springer, New York (2001)
26. Witten, I.H., Eibe Frank, M.A.H.: Data mining: practical machine learning tools and technique. In: Kaufmann, M. 3rd edn (2011)
27. Ikonomovska, E., Gama, J., Dzeroski, S.: Learning model trees from evolving data streams. Data Min. Knowl. Discov. 23(1), 128–168 (2011)
28. Jackson, M.L., Baer, A., Painter, I., Duchin, J.: A simulation study comparing aberration detection algorithms for syndromic surveillance. BMC Med. Inform. Decis. Mak. 7(1), 6 (2007)
29. Kerman, M., Jiang, W., Blumberg, A., Buttrey, S.: Event detection challenges, methods, and applications in natural and artificial systems. In: Proceedings of 14th international command and control research and technology symposium, ICCRTS, Lockheed Martin MS2, pp. 1–19 (2009)
30. Kuncheva, L.I.: Combining pattern classifiers: methods and algorithms. Wiley-Interscience (2004)
31. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. SIGCOMM Comput. Commun. Rev. 34(4), 219–230 (2004)
32. Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. In: Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, SIGCOMM ’05, pp. 217–228. New York, ACM (2005)
33. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)
34. Li, X., Bian, F., Crovella, M., Diot, C., Govindan, R., Iannaccone, G., Lakhina, A.: Detection and identification of network anomalies using sketch subspaces. In: Proceedings of the 6th ACMSIGCOMM conference on Internet measurement, IMC ’06, pp. 147–152. New York, ACM (2006)
35. Lippmann, R., Haines, J.W., Fried, D.J., Korba, J., Das, K.: The: darpa off-line intrusion detection evaluation. Comput. Netw. 34(4), 579–595 (1999)
36. Marins, A., Casanova, M.A., Furtado, A., Breitman, K.: Modeling provenance for semantic desktop applications. In: SEMISH-Anais do Seminario Integrado de Software eHardware 34, pp. 2101–2112 (2007)
37. McHugh, J.: Testing intrusion detection systems: a critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Trans. Inform. Syst. Secur. 3(4), 262–294 (2000)
38. Nychis, G., Sekar, V., Andersen, D.G., Kim, H., Zhang, H.: An empirical evaluation of entropy-based traffic anomaly detection. In: Proceedings of the 8thACMSIGCOMMconference on Internet measurement, IMC ’08, pp. 151–156. New York, ACM (2008)
39. Patterson, K., Hassani, H., Heravi, S., Zhigljavsky, A.: Multivariate singular spectrum analysis for forecasting revisions to real-time data. J. Appl. Stat. 38(10), 2183–2211 (2011)
40. Policy Institute: Bike-sharing programs hit the streets in over 500 cities worldwide. http://www.earth-policy.org/plan_b_updates/2013/update112 (2013)
41. Ringberg, H., Soule, A., Rexford, J.: Webclass: adding rigor to manual labeling of traffic anomalies. SIGCOMM Comput. Commun. Rev. 38(1), 35–38 (2008)
42. Rubinstein, B.I., Nelson, B., Huang, L., Joseph, A.D., Lau, S.-H., Rao, S., Taft, N., Tygar, J.D.: Antidote: understanding and defending against poisoning of anomaly detectors. In: Proceedings of the 9th ACMSIGCOMM conference on Internet measurement conference, IMC ’09, pp. 1–14. New York, ACM (2009)
43. San Miguel, J.C., Martinez, J.M., Garcia, A.: An ontology for event detection and its application in surveillance video. In: Proceedings of the 2009 sixth IEEE international conference on advanced video and signal based surveillance, AVSS ’09, IEEE Computer Society, pp. 220–225. Washington, DC (2009)
44. Scherrer, A., Larrieu, N., Owezarski, P., Borgnat, P., Abry, P.: Nongaussian and long memory statistical characterizations for internet traffic with anomalies. IEEE Trans. Dependable Secur. Comput. 4(1), 56–70 (2007)
45. Scott, S.L.: A bayesian paradigm for designing intrusion detection systems. Comput. Stat. Data Anal. 45(1), 69–83 (2004)
46. Senin, P.: Dynamic time warping algorithm review. In: University of Hawaii at Manoa, Technical report series (2008)
47. Shanbhag, S., Wolf, T.: Accurate anomaly detection through parallelism. Netw. Mag. Glob. Internetwkg. 23(1), 22–28 (2009)
48. Tan, K., Maxion, R.: The effects of algorithmic diversity on anomaly detector performance. Dependable systems and networks, 2005. DSN2005. In: Proceedings of international conference on, pp. 216–225 (2005)
49. Tan, K.M.C., Maxion, R.A.: Performance evaluation of anomalybased detection mechanisms. In: Technical report series CS-TR- 870, University of Newcastle upon Tyne, Newcastle upon Tyne, NE1 7RU, UK (2004)
50. Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the kdd cup 99 data set. In: Proceedings of the second IEEE international conference on Computational intelligence for security and defense applications, CISDA’09, Piscataway, NJ, pp. 53–58. IEEE Press (2009)
51. Vogel, P., Greiser, T., Mattfeld, D.C.: Understanding bike-sharing systems using data mining: exploring activity patterns. Procedia Soc. Behav. Sci. 20, 514–523 (2011)
52. Warrender, C., Forrest, S., Pearlmutter, B.: Detecting intrusions using system calls: alternative datamodels. In security and privacy, 1999. In: Proceedings of the 1999 IEEE symposium on, pp. 133–145 (1999)
53. Wong, C., Bielski, S., Studer, A., Wang, C.: Empirical analysis of rate limiting mechanisms. In: Proceedings of the 8th international conference on Recent Advances in Intrusion Detection, RAID’ 05, pp. 22–42. Springer, Heidelberg (2006)
54. Wong, W.-K., Moore, A., Cooper, G., Wagner, M.: What’s strange about recent events (wsare): an algorithm for the early detection of disease outbreaks. J. Mach. Learn. Res. 6, 1961–1998 (2005)
55. Xu, C., Zhang, Y.-F., Zhu, G., Rui, Y., Lu, H., Huang, Q.: Using webcast text for semantic event detection in broadcast sports video. Multimedia IEEE Trans. 10(7), 1342–1355 (2008)
56. Zheng, V.W., Zheng, Y., Xie, X., Yang, Q.: Collaborative location and activity recommendations with gps history data. In: Proceedings of the 19th international conference on World wide web, WWW’10, pp. 1029–1038. New York, ACM (2010)