Online identification of hierarchical heavy hitters: Algorithms, evaluation, and applications

Proceedings of the 2004 ACM SIGCOMM Internet Measurement Conference, IMC 2004

Volumn , Issue , 2004, Pages 101-114

  Zhang, Yin ^a   Singh, Sumeet ^b   Sen, Subhabrata ^a   Duffield, Nick ^a   Lund, Carsten ^a  

^a AT AND T LABS RESEARCH   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Change Detection; Data Stream Computation; Hierarchical Heavy Hitters; Network Anomaly Detection; Packet Classification

Indexed keywords

ALGORITHMS; DATA STRUCTURES; ERROR ANALYSIS; HIERARCHICAL SYSTEMS; INTERNET; NETWORK PROTOCOLS; ONLINE SYSTEMS; PACKET NETWORKS; ROUTERS; SET THEORY;

CHANGE DETECTION; DATA STREAM COMPUTATION; HIERARCHICAL HEAVY HITTERS (HHH); NETWORK ANOMALY DETECTION; PACKET CLASSIFICATION;

TELECOMMUNICATION TRAFFIC;

EID: 14944356227     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. H. Arsham. Time series analysis and forecasting techniques. http://obelia.jde.aca.mmu.ac.uk/resdesgn/arsham/opre330Forecast.htm.
2. F. Baboescu, S. Singh, and G. Varghese. Packet classification for core routers: Is there an alternative to CAMs. In INFOCOM, 2003. http://citeseer.ist. psu.edu/baboescu03packet.html.
3. F. Baboescu and G. Varghese. Scalable packet classification. In Proc. ACM SIGCOMM, 2001. http://citeseer.ist.psu.edu/baboescu01packet.html.
4. P. Barford, J. Kline, D. Plonka, and A. Ron. A signal analysis of network traffic anomalies. In Proceedings of the ACM SIGCOMM Internet Measurement Workshop, Marseille, France, November 2002.
5. P. Barford and D. Plonka. Characteristics of network traffic flow anomalies. In Proceedings of the ACM SIGCOMM Internet Measurement Workshop, San Francisco, CA, November 2001.
6. G. E. P. Box and G. M. Jenkins. Time Series Analysis, Forecasting and Control. Holden-Day, 1976.
7. G. E. P. Box, G. M. Jenkins, and G. C. Reinsel. Time Series Analysis, Forecasting and Control. Prentice-Hall, Englewood Cliffs, 1994.
8. J. Brutlag. Aberrant behavior detection in time series for network monitoring. In Proc. of the 14th USENIX System Administration Conference (LISA XIV), New Orleans, LA, December 2000.
9. C. Chen and L.-M. Liu. Forecasting time series with outliers. Journal of Forecasting, 12:13-35, 1993.
10. Cisco. Random Sampled NetFlow. http://www.cisco.com/univercd/cc/td/doc/ product/software/ios123/123newft/123t/123t_2/nfstatsa.pdf.
11. G. Cormode, F. Korn, S. Muthukrishnan, and D. Srivastava. Finding hierarchical heavy hitters in data streams. In International Conference on Very Large Data Bases, 2003.
12. G. Cormode, F. Korn, S. Muthukrishnan, and D. Srivastava. Diamond in the rough: Finding hierarchical heavy hitters in multi-dimensional data. In Proc. ACM SIGMOD, June 2004.
13. G. Cormode and S. Muthukrishnan. What's hot and what's not: Tracking most frequent items dynamically. In Proc. ACM PODC '2003, July 2003.
14. G. Cormode and S. Muthukrishnan. Improved data stream summaries: The count-min sketch and its applications. In Journal of Algorithms, 2004. In press. http://dimacs.rutgers.edu/~graham/pubs/cm-full.pdf.
15. N. Duffield and C. Lund. Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure. In ACM SIGCOMM Internet Measurement Workshop, Miami Beach, FL, October 2003.
16. N. Duffield, C. Lund, and M. Thorup. Charging from sampled network usage. In ACM SIGCOMM Internet Measurement Workshop, San Francisco, CA, November 2001.
17. C. Estan and G. Varghese. New directions in traffic measurement and accounting. In Proc. ACM SIGCOMM, Pittsburgh, PA, August 2002.
18. C. Estan and G. Varghese. Automatically inferring patterns of resource consumption in network traffic. In Proc. ACM SIGCOMM, Karlsruhe, Germany, August 2003.
19. F. Feather, D. Siewiorek, and R. Maxion. Fault detection in an ethernet network using anomaly signature matching. In Proc. ACM SIGCOMM, 1993.
20. A. Feldmann and S. Muthukrishnan. Tradeoffs for packet classification. In INFOCOM (3), pages 1193-1202, 2000. http://citeseerist.psu.edu/ feldmann00tradeoffs.html.
21. P. Gupta and N. McKeown. Packet classification on multiple fields. In Proc. ACM SIGCOMM, pages 147-160, 1999. http://citeseer.ist.psu.edu/ gupta99packet.html.
22. C. Hood and C. Ji. Proactive network fault detection. In Proc. IEEE INFOCOM '97, Kobe, Japan, April 1997.
23. K. J. Houle, G. M. Weaver, N. Long, and R. Thomas. Trends in Denial of Service Attack Technology. http://www.cert.org/archive/pdf/DoS_trends.pdf.
24. J. Jung, B. Krishnamurthy, and M. Rabinovich. Flash crowds and denial of service attacks: Characterization and implications for CDNs and web sites. In Proceedings of the World Wide Web Conference, Honolulu, Hawaii, May 2002. http://www.research.att.com/~bala/papers/www02-fc.html.
25. I. Katzela and M. Schwartz. Schemes for fault identification in communication networks. IEEE/ACM Transactions on Networking, 3(6):753-764, December 1995.
26. B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen. Sketch-based change detection: Methods, evaluation, and applications. In Proc. ACM/USENIX Internet Measurement Conference, 2003. http://www.research.att.com/~yzhang/papers/nad- imc03.pdf.
27. G. Manku and R. Motwani. Approximate frequency counts over data streams. In International Conference on Very Large Data Bases, 2002.
28. D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, and N. Weaver. The spread of the Sapphire/Slammer worm. Technical report, CAIDA, February 2003. http://www.cs.berkeley.edu/~nweaver/sapphire/.
29. S. Muthukrishnan. Data streams: Algorithms and applications, 2003. Manuscript based on invited talk from 14th SODA.
30. S. Singh, F. Baboescu, G. Varghese, and J. Wang. Packet classification using multidimensional cutting. In Proc. ACM SIGCOMM, 2003. http://citeseer.ist. psu.edu/singh03packet.html.
31. V. Srinivasan, S. Suri, and G. Varghese. Packet classification using tuple space search. In Proc. ACM SIGCOMM, pages 135-146, 1999. http://citeseer.ist.psu.edu/srinivasan99packet.html.
32. V. Srinivasan and G. Varghese. Faster IP lookups using controlled prefix expansion. In ACM Transactions on Computer Systems, 1999.
33. V. Srinivasan, G. Varghese, S. Suri, and M. Waldvogel. Fast and scalable layer four switching. In Proc. ACM SIGCOMM, 1998. http://citeseer.ist.psu.edu/ srinivasan98fast.html.
34. R. S. Tsay. Outliers, level shifts, and variance changes in time series. Journal of Forecasting, 7:1-20, 1988.
35. A. Ward, P. Glynn, and K. Richardson. Internet service performance failure detection. Performance Evaluation Review, August 1998.