Sketch-based streaming PCA algorithm for network-wide traffic anomaly detection

Proceedings - International Conference on Distributed Computing Systems

Volumn , Issue , 2010, Pages 807-816

  Liu, Yang ^a   Zhang, Linfeng ^a   Guan, Yong ^a  

^a Iowa State University   (United States)

Author keywords

Data streams; Principle component analysis; Traffic anomaly

Indexed keywords

BOUNDED ERRORS; DAILY LIVES; DATA STREAM; FLASH CROWD; LOW PROFILE; NETWORK FAILURE; NETWORK OPERATION CENTERS; NETWORK SERVICES AND APPLICATIONS; PCA ALGORITHMS; PRINCIPLE COMPONENT ANALYSIS; RUNNING TIME; SCALABILITY PROBLEMS; SECURITY PROBLEMS; SLIDING WINDOW; SPATIAL ANALYSIS; SPATIAL DETECTION; TRAFFIC ANOMALIES; TRAFFIC DYNAMICS; TRAFFIC MEASUREMENTS;

DATA COMMUNICATION SYSTEMS;

ALGORITHMS;

EID: 77955890394     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDCS.2010.45     Document Type: Conference Paper

References (27)

1. A. Lakhina, M. Crovella, and C. Diot, "Diagnosing networkwide traffic anomalies," SIGCOMM Comput. Commun. Rev., vol. 34, no. 4, pp. 219-230, 2004.
2. H. Ringberg, A. Soule, J. Rexford, and C. Diot, "Sensitivity of pca for traffic anomaly detection," SIGMETRICS '07, pp. 109-120, 2007.
3. B. I. P. Rubinstein, B. Nelson, L. Huang, A. D. Joseph, S. Lau, S. Rao, N. Taft, and J. D. Tygar, "Stealthy poisoning attacks on pca-based anomaly detectors," SIGMETRICS '09 (poster), 2009.
4. A. Lakhina, M. Crovella, and C. Diot, "Mining anomalies using traffic feature distributions," SIGCOMM '05, pp. 217-228, 2005.
5. A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and N. Taft, "Structural analysis of network traffic flows," SIGMETRICS '04/Performance '04, pp. 61-72, 2004.
6. "Abilene observatory data collections," www.internet2.edu/ observatory/.
7. X. Li, F. Bian, M. Crovella, C. Diot, R. Govindan, G. Iannaccone, and A. Lakhina, "Detection and identification of network anomalies using sketch subspaces," IMC '06, pp. 147-152, 2006.
8. L. Huang, X. L. Nguyen, M. Garofalakis, J. Hellerstein, M. Jordan, A. Joseph, and N. Taft, "Communication-efficient online detection of network-wide anomalies," INFOCOM '07, pp. 134-142, 2007.
9. Y. Huang, N. Feamster, A. Lakhina, and J. J. Xu, "Diagnosing network disruptions with network-wide analysis," SIGMETRICS '07, pp. 61-72, 2007.
10. P. Chhabra, C. Scott, E. Kolaczyk, and M. Crovella, "Distributed spatial anomaly detection," INFOCOM '08, pp. 1705-1713, 2008.
11. J. Kline, S. Nam, P. Barford, D. Plonka, and A. Ron, "Traffic anomaly detection at fine time scales with bayes nets," ICIMP '08, pp. 37-46, 2008.
12. D. Brauckhoff, K. Salamatian, and M. May, "Applying pca for traffic anomaly detection: Problems and solutions," in INFOCOM 2009, IEEE, 2009, pp. 2866-2870.
13. P. Barford, J. Kline, D. Plonka, and A. Ron, "A signal analysis of network traffic anomalies," IMW '02, pp. 71-82, 2002.
14. C. C. Aggarwal, Data Streams: Models and Algorithms (Advances in Database Systems). Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006.
15. K. Kumar, J. Xu, J. Wang, O. Spatschek, and L. Li, "Spacecode bloom filter for efficient per-flow traffic measurement," INFOCOM '04, pp. 1762-1773, 2004.
16. Q. G. Zhao, A. Kumar, J. Wang, and J. J. Xu, "Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices," SIGMETRICS '05, pp. 350-361, 2005.
17. S. Guha, D. Gunopulos, and N. Koudas, "Correlating synchronous and asynchronous data streams," KDD '03, pp. 529-534, 2003.
18. S. Papadimitriou, J. Sun, and C. Faloutsos, "Streaming pattern discovery in multiple time-series," VLDB '05, pp. 697-708, 2005.
19. S. Papadimitriou and P. Yu, "Optimal multi-scale patterns in time series streams," SIGMOD '06, pp. 647-658, 2006.
20. M. Datar, A. Gionis, P. Indyk, and R. Motwani, "Maintaining stream statistics over sliding windows: (extended abstract)," SODA '02, pp. 635-644, 2002.
21. J. E. Jackson and G. S. Mudholkar, "Control procedures for residuals associated with principal component analysis," Thechnometrics, pp. 341-349, 1979.
22. S. S. Vempala, The Random Projection Method. Rhode Island: American Mathematical Society, 2004.
23. L. Zhang and Y. Guan, "Variance estimation over sliding windows," PODS '07, pp. 225-232, 2007.
24. N. Alon, P. B. Gibbons, Y. Matias, and M. Szegedy, "Tracking join and self-join sizes in limited storage," PODS '99, pp. 10-20, 1999.
25. D. Achlioptas, "Database-friendly random projections: Johnson-lindenstrauss with binary coins," Journal of computer and System Sciences, vol. 66, no. 4, pp. 671-687, 2003.
26. P. Li, T. J. Hastie, and K. W. Church, "Very sparse random projections," KDD' 06, pp. 287-296, 2006.
27. G. Stewart and J. guang Sun, Matrix perturbation theory. Boston: Academic Press, 1990.