Reversible sketches for efficient and accurate change detection over network data streams

Proceedings of the 2004 ACM SIGCOMM Internet Measurement Conference, IMC 2004

Volumn , Issue , 2004, Pages 207-212

  Schweller, Robert ^a   Gupta, Ashish ^a   Parsons, Elliot ^a   Chen, Yan ^a  

^a Northwestern University   (United States)

Author keywords

Change detection; Data stream computation; IP mangling; Modular hashing; Network anomaly detection; Reverse hashing; Sketch

Indexed keywords

ALGORITHMS; ERROR ANALYSIS; GRAPH THEORY; MATHEMATICAL MODELS; NETWORK PROTOCOLS; ROUTERS; SET THEORY; TELECOMMUNICATION LINKS; TELECOMMUNICATION TRAFFIC; TIME SERIES ANALYSIS;

CHANGE DETECTION; DATA STREAM COMPUTATION; IP MANGLING; MODULAR HASHING; NETWORK ANOMALY DETECTION; REVERSE HASHING; REVERSIBLE SKETCHES;

TELECOMMUNICATION NETWORKS;

EID: 14944385135     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1028788.1028814     Document Type: Conference Paper

References (27)

1. CORMEN, T., LEISERSON, C., AND RIVEST, R. Introduction to Algorithms. The MIT Press, Cambridge, MA, 1989.
2. CORMODE, G., ET AL. Finding hierarchical heavy hitters in data streams. In Proc. of VLDB (2003).
3. CORMODE, G., ET AL. Holistic UDAFs at streaming speeds. In Proc. of ACM SIGMOD (2004).
4. CORMODE, G., AND MUTHUKRISHNAN, S. Improved data stream summaries: The count-min sketch and its applications. Tech. Rep. 2003-20, DIMACS, 2003.
5. CORMODE, G., AND MUTHUKRISHNAN, S. What's new: Finding significant differences in network data streams. In Proc. of IEEE Infocom (2004).
6. DUFFIELD, N., LUND, C., AND THORUP, M. Properties and prediction of flow statistics from sampled packet streams. In Proc. of ACM SIGCOMM Internet Measurement Workshop (IMW) (2002).
7. DUFFIELD, N., LUND, C., AND THORUP, M. Flow sampling under hard resource constraints. In Proc. of ACM SIGMETRICS (2004).
8. ESTAN, C., AND VARGHESE, G. New directions in traffic measurement and accounting. In Proc. of ACM SIGCOMM (2002).
9. FLAJOLET, P., AND MARTIN, G. N. Probabilistic counting algorithms for data base applications. J. Comput. Syst. Sci. 31, 1 (1985), 182-209.
10. GILBERT, A. C., ET AL. QuickSAND: Quick summary and analysis of network data. Tech. Rep. 2001-43, DIMACS, 2001.
11. GILBERT, A. C., KOTIDIS, Y., MUTHUKRISHNAN, S., AND STRAUSS, M. J. QuickSAND: Quick summary and analysis of network data. Tech. Rep. 2001-43, DIMACS, 2001.
12. HAMMING, R. W. Coding and Information Theory. Prentice Hall, 1986.
13. HOFMEYR, S., AND FORREST, S. Intrusion detection using sequences of system calls. Journal of Computer Security 6 (1998).
14. KRISHNAMURTHY, B., SEN, S., ZHANG, Y., AND CHEN, Y. Sketch-based change detection: Methods, evaluation, and applications. In Proc. of ACM SIGCOMM IMC (2003).
15. LOYALL, J. P., ET AL. Building adaptive and agile applications using intrusion detection and response. In Proc. of NDSS (2000).
16. MANKU, G. S., AND MOTWANI, R. Approximate frequency counts over data streams. In Proc. of IEEE VLDB (2002).
17. MIRKOVIC, J., AND REIHER, P. Data streams: algorithms and applications. In Proc. of ACM SODA (2003).
18. MOORE, D., ET AL. The spread of the Sapphire/Slammer worm. IEEE Magazine on Security and Privacy (August 2003).
19. MUTHUKRISHNAN. Data streams: Algorithms and applications (short). In Proc. of ACM SODA (2003).
20. PAXSON, V. Bro: A system for detecting network intruders in real-time. Computer Networks 31, 23-24 (1999), 2435-2463.
21. ROESCH, M. Snort: The lightweight network intrusion detection system, 2001. http://www.snort.org/.
22. RYUTOV, T., NEUMAN, C., KIM, D., AND ZHOU, L. Integrated access control and intrusion detection for web servers. IEEE Trans. on Parallel and Distributed Systems 14, 9 (2003), 841-850.
23. SCHWELLER, R., CHEN, Y., PARSONS, E., GUPTA, A., MEMIK, G., AND ZHANG, Y. Reverse hashing for sketch-based change detection on high-speed networks. Tech. Rep. NWU-CS-2004-45, Northwestern University, 2004.
24. STANIFORD, S., PAXSON, V., AND WEAVER, N. How to own the Internet in your spare time. In Proceedings of the 11th USENIX Security Symposium (2002).
25. WAGNER, D., AND DEAN, D. Intrusion detection via static analysis. In Proc. of the IEEE Symposium on Security and Privacy (2001).
26. WEAVER, N., PAXSON, V., STANIFORD, S., AND CUNNINGHAM, R. Large scale malicious code: A research agenda. Tech. Rep. DARPA-sponsored report, 2003.
27. XILINX INC. Virtex-II Pro and Virtex-II Pro X Platform FPGAs: Complete data sheet, 2004. www.xilinx.com/bvdocs/publications/ds083.pdf.