Revisiting the case for a minimalist approach for network flow monitoring

Proceedings of the ACM SIGCOMM Internet Measurement Conference, IMC

Volumn , Issue , 2010, Pages 328-341

  Sekar, Vyas ^a   Reiter, Michael K ^b   Zhang, Hui ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

Anomaly detection; Data streaming; Sampling; Traffic monitoring

Indexed keywords

'CURRENT; ANOMALY DETECTION; DATA STREAMING; FLOW MONITORING; FLOW-LEVEL; MANAGEMENT APPLICATIONS; NETWORKS FLOWS; NETWORKS MANAGEMENT; PACKET SAMPLING; TRAFFIC MONITORING;

COMPLEX NETWORKS;

EID: 78650910238     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1879141.1879186     Document Type: Conference Paper

References (46)

1. Flexible Netflow. http://www.cisco.com/en/US/products/ps6965/products- ios-protocol-option-home.html,.
2. Juniper cflowd. http://www.juniper.net/techpubs/software/junos/junos91/ swconfig-policy/cflowd.html.
3. Narus Intercept Solution. http://www.narus.com/index.php/solutions/ intercept.
4. NetFlow Input Filters. http://www.cisco.com/en/US/docs/ios/12-3t/12-3t4/ feature/guide/gtnfinpf.html.
5. Packet Sampling, IETFWorking Group Charter. http://www.ietf.org/html. charters/psamp/charter/.
6. K. Argyraki, P. Maniatis, O. Irzak, S. Ashish, and S. Shenker. Loss and Delay Accountability for the Internet. In Proc. ICNP, 2007.
7. D. Brauckhoff, B. Tellenbach, A. Wagner, A. Lakhina, and M. May. Impact of Traffic Sampling on Anomaly Detection Metrics. In Proc. IMC, 2006.
8. C. Cranor, T. Johnson, O. Spatscheck, and V. Shkapenyuk. Gigascope: A stream database for network applications. In Proc. ACM SIGMOD, 2003.
9. G. R. Cantieni, G. Iannaccone, C. Barakat, C. Diot, and P. Thiran. Reformulating the Monitor Placement problem: Optimal Network-Wide Sampling. In Proc. CoNeXT, 2006.
10. L. D. Carli, Y. Pan, A. Kumar, C. Estan, and K. Sankaralingam. PLUG: Flexible Lookup Modules for Rapid Deployment of New Protocols in High-speed Routers. In Proc. SIGCOMM, 2010.
11. B. Claise. Cisco Systems NetFlow Services Export Version 9. RFC 3954.
12. M. P. Collins and M. K. Reiter. Finding Peer-to-Peer File-sharing using Coarse Network Behaviors. In Proc. ESORICS, 2006.
13. G. Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms, 55, 2005.
14. N. Duffield, C. Lund, and M. Thorup. Charging from sampled network usage. In Proc. IMW, 2001.
15. N. Duffield, C. Lund, and M. Thorup. Estimating Flow Distributions from Sampled Flow Statistics. In Proc. of ACM SIGCOMM, 2003.
16. C. Estan, K. Keys, D. Moore, and G. Varghese. Building a Better NetFlow. In Proc. ACM SIGCOMM, 2004.
17. C. Estan and G. Varghese. New Directions in Traffic Measurement and Accounting. In Proc. ACM SIGCOMM, 2002.
18. A. Feldmann, A. G. Greenberg, C. Lund, N. Reingold, J. Rexford, and F. True. Deriving Traffic Demands for Operational IP Networks: Methodology and Experience. In Proc. ACM SIGCOMM, 2000.
19. Google sparse hash project. http://code.google.com/p/google-sparsehash/.
20. H. Song, S. Dharmapurikar, J. Turner, and J. Lockwood. Fast Hash Table Lookup Using Extended Bloom Filter: An Aid to Network Processing. In Proc. ACM SIGCOMM, 2005.
21. N. Hohn and D. Veitch. Inverting Sampled Traffic. In Proc. IMC, 2003.
22. T. Karagiannis, D. Papagiannaki, and M. Faloutsos. BLINC: Multilevel Traffic Classification in the Dark. In Proc. ACM SIGCOMM, 2005.
23. K. Keys, D. Moore, and C. Estan. A Robust System for Accurate Real-time Summaries of Internet Traffic. In Proc. SIGMETRICS, 2005.
24. R. Kompella and C. Estan. The Power of Slicing in Internet Flow Measurement. In Proc. IMC, 2005.
25. B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen. Sketch-based change detection: Methods, evaluation, and applications. In Proc. ACM IMC, 2003.
26. A. Kumar, M. Sung, J. Xu, and J. Wang. Data Streaming Algorithms for Efficient and Accurate Estimation of Flow Distribution. In Proc. ACM SIGMETRICS, 2004.
27. A. Kumar and J. Xu. Sketch Guided Sampling - Using On-Line Estimates of Flow Size for Adaptive Data Collection. In Proc. IEEE Infocom, 2006.
28. A. Lakhina, M. Crovella, and C. Diot. Diagnosing Network-Wide Traffic Anomalies. In Proc. ACM SIGCOMM, 2004.
29. A. Lakhina, M. Crovella, and C. Diot. Mining anomalies using traffic feature distributions. In Proc. ACM SIGCOMM, 2005.
30. A. Lall, V. Sekar, J. Xu, M. Ogihara, and H. Zhang. Data Streaming Algorithms for Estimating Entropy of Network Traffic. In Proc. ACM SIGMETRICS, 2006.
31. Y. Lu, A. Montanari, B. Prabhakar, S. Dharmapurikar, and A. Kabbani. Counter Braids: A Novel Counter Architecture for Per-Flow Measurement. In Proc. SIGMETRICS, 2008.
32. M. P. Collins and M. K. Reiter. Hit-listWorm Detection and Bot Identification in Large Networks Using Protocol Graphs. In Proc. RAID, 2007.
33. H. Madhyastha and B. Krishnamurthy. A Generic Language for Application-Specific Flow Sampling. ACM CCR, 38(2):7-15, Apr. 2008.
34. J. Mai, C.-N. Chuah, A. Sridharan, T. Ye, and H. Zang. Is Sampled Data Sufficient for Anomaly Detection? In Proc. IMC, 2006.
35. A. Ramachandran, S. Seetharaman, and N. Feamster. Fast Monitoring of Traffic Subpopulations. In Proc. IMC, 2008.
36. B. Ribeiro, D. Towsley, T. Ye, and J. Bolot. Fisher information of sampled packets: an application to flow size estimation. In Proc. IMC, 2006.
37. S. Kumar and P. Crowley. Segmented Hash: An Efficient Hash Table Implementation for High Performance Networking Subsystems. In Proc. ACM ANCS, 2005.
38. V. Sekar, N. Duffield, K. van der Merwe, O. Spatscheck, and H. Zhang. LADS: Large-scale Automated DDoS Detection System. In Proc. USENIX ATC, 2006.
39. V. Sekar, M. K. Reiter,W. Willinger, H. Zhang, R. Kompella, and D. G. Andersen. cSamp: A System for Network-Wide Flow Monitoring. In Proc. NSDI, 2008.
40. V. Sekar, A. Gupta, M. K. Reiter and H. Zhang. Coordinated Sampling sans Origin-Destination Identifiers: Algorithms and Analysis. In Proc. COMSNSETS, 2010.
41. S. Venkataraman, D. Song, P. B. Gibbons, and A. Blum. New Streaming Algorithms for Fast Detection of Superspreaders. In Proc. NDSS, 2005.
42. Y. Xie, V. Sekar, D. A. Maltz, M. K. Reiter, and H. Zhang. Worm Origin Identification Using Random Moonwalks. In Proc. IEEE Symposium on Security and Privacy, 2005.
43. K. Xu, Z.-L. Zhang, and S. Bhattacharya. Profiling Internet Backbone Traffic: Behavior Models and Applications. In Proc. ACM SIGCOMM, 2005.
44. Y. Gao, Y. Zhao, R. Schweller, S. Venkataraman, Y. Chen, D. Song and M.-Y. Kao. Detecting Stealthy Attacks Using Online Histograms. In Proc. IWQoS, 2007.
45. L. Yuan, C.-N. Chuah, and P. Mohapatra. ProgME: Towards Programmable Network MEasurement. In Proc. SIGCOMM, 2007.
46. Q. Zhao, J. Xu, and Z. Liu. Design of a novel statistics counter architecture with optimal space and time efficiency. In Proc. ACM SIGMETRICS, 2006.