Data streaming algorithms for estimating entropy of network traffic

Performance Evaluation Review

Volumn 34, Issue 1, 2006, Pages 145-156

  Lall, Ashwin ^a   Sekar, Vyas ^b   Ogihara, Mitsunori ^a   Xu, Jun ^c   Zhang, Hui ^b  

^a UNIVERSITY OF ROCHESTER   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Data streaming; Traffic analysis

Indexed keywords

DATA STREAMING; ENTROPY ESTIMATION; TRAFFIC ANALYSIS; TRAFFIC CLASSIFICATION;

ALGORITHMS; DATA REDUCTION; ENTROPY; MATHEMATICAL MODELS; PROBLEM SOLVING; TELECOMMUNICATION LINKS; TELECOMMUNICATION NETWORKS;

TELECOMMUNICATION TRAFFIC;

EID: 33750373763     PISSN: 01635999     EISSN: 01635999     Source Type: Conference Proceeding    
DOI: 10.1145/1140103.1140295     Document Type: Conference Paper

References (26)

1. A. Chakrabarti, K. Do Ba, and S. Mutlmkrishnan. Estimating entropy and entropy norm on data streams. In Proceedings of the 23rd International Symposium on Theoretical Aspects of Computer Science (STACS), 2006.
2. N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency moments. In Proceedings of ACM Symposium on Theory of Computing (STOC), 1996.
3. P. Barford, J. Kline, D. Plonka, and A. Ron. A Signal Analysis of Network Traffic Anomalies. In Proceedings of ACM SIGCOMM Internet Measurement Workshop (IMW), 2002.
4. J. D. Brutlag. Aberrant behavior detection in time series for network monitoring. In Proceedings of USENIX Large Installation System Administration Conference (LISA), 2000.
5. N. Duffield, C. Lund, and M. Thorup. Estimating flow distributions from sampled How statistics. In Proceedings of ACM SIGCOMM, 2003.
6. C. Estan and G. Varghese. New directions in traffic measurement and accounting. In Proceedings of ACM SIGCOMM, 2002.
7. L. Feinstein, D. Schnackenberg, R. Balupari, and D. Kindred. Statistical approaches to DDoS attack detection and response. In Proceedings of the DARPA Information Survivability Conference, and Exposition. 2003.
8. S. Guha, A. McGregor, and S. Venkatasubramanian. Streaming and sublinear approximation of entropy and information distances. In Proceedings of ACM Symposium on. Discrete Algorithms (SODA), 2006.
9. F. Hao, M. Kodialam, and T. V. Lakshman. ACCEL-RATE: a faster mechanism for memory efficient per-flow traffic estimation. In Proceedings of ACM SIGMETRICS, 2004.
10. N. Holm and D. Veitch. Inverting sampled traffic. In Proceedings of ACM/USENIX Internet. Measurement Conference (IMC), 2003.
11. B. Kalyanasundaram and G. Schnitger. The probabilistic communication complexity of set intersection. SIAM Journal on Discrete Mathematics, 5(4):545-557, 1992.
12. V. Karamcheti, D. Geiger. Z. Kedem, and S. Muthukrishman. Detecting malicious network traffic using inverse distributions of packet contents. In Proceedings of ACM SIGCOMM Workthop on Mining Network Data (MineNet), 2005.
13. A. Kumar, M. Sung, J. Xu, and J. Wang. Data streaming algorithms for efficient and accurate estimation of flow distribution. In Proceedings of ACM SIGMETRICS/IFIP WG 7.3 Performance,2004.
14. A. Kumar, M. Sung, J. Xu, and E. Zegura, A data streaming algorithm for estimating subpopulation flow sizes distribution. In Proceedings of ACM SIGMETRICS, 2005.
15. E. Kushilevitz und N. Nisan. Communication complexity. Cambridge University Press. New York, NY, USA, 1997.
16. A. Lakhina, M. Crovella. and C. Diot. Diagnosing network-wide traffic anomalies. In Proceedings of ACM SIGCOMM, 2004.
17. A. Lakhina, M. Crovella, and C. Diot. Mining anomalies using traffic feature distributions. In Proceedings of ACM SIGCOMM, 2005.
18. K. Levchenko, R. Paturi, and G. Varghese. On the difficulty of scalably detecting network attacks. In Proceedings of ACM Conference on Computer and Communications Security (CCS), 2004.
19. S. Muthukrishnan. Data streams: algorithms and applications. http://athos.rutgers.edu/~muthu/stream-1-1.ps.
20. Cisco Netflow. http://www.cisco.com/warp/public/ 732/Tech/nfflp/netflow/ index.shtml.
21. M. Ronghan. A. Greenberg, C. Kalmanek, M. Rumsewicz. J. Yates, and Y. Zhang. Experience in measuring internet backbone traffic variability: Models, metrics, measurements and meaning. In Proceedings of International Teletraffic Congress (ITC), 2003.
22. S. Venkataraman, D. Song, P. B. Gibbons, and A. Blmn. New Streaming Algorithms for Fast Detection of Superspreaders . In Proceedings of Network and Distributed System Security Symposium (NDSS). 2005.
23. A. Wagner and B. Plattner. Entropy Based Worm and Anomaly Detection in Fast IP Networks. In Proceedings of IEEE International Workshop on Enabling Technologies. Infrastructures for Collaborative Enterprises, 2005.
24. K. Xu, Z.-L. Zlmng, and S. Bhattachnrya. Profiling internet backbone traffic: Behavior models and applications. In Proceedings of ACM SIGCOMM, 2005.
25. Y. Zhang, Z. Ge, M. Roughan, and A. Greenberg. Network anomography. In Proceedings of ACM/USENIX Internet Measurement, Conference (IMC), 2005.
26. Y. Zhang, S. Singh, S. Sen, N. Duffield, and C. Lund. Online identification of hierarchical heavy hitters: algorithms, evaluations, and applications. In Proceedings of ACM/USENIX Internet. Measurement Conference (IMC), 2004.