SCRIPT: A framework for scalable real-time IP flow record analysis

Proceedings of the 2010 IEEE/IFIP Network Operations and Management Symposium, NOMS 2010

Volumn , Issue , 2010, Pages 278-285

  Morariu, Cristian ^a   Racz, Peter ^a   Stiller, Burkhard ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Distributed analysis; Ip flow accounting; Peer to Peer

Indexed keywords

INTRUSION DETECTION; PROGRAM PROCESSORS;

CENTRALIZED APPROACHES; DISTRIBUTED ANALYSIS; IP (INTERNET PROTOCOL); IP FLOW; MANAGEMENT OPERATION; NETWORK MONITORING; PEER TO PEER; SCALABILITY PROBLEMS;

INTERNET PROTOCOLS;

EID: 77957801263     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/NOMS.2010.5488476     Document Type: Conference Paper

References (27)

1. BOB hashing function: http://burtleburtle.net/bob/hash/doobs.html, Last accessed: September 2009.
2. D. Brauckhoff, B. Tellenbach, A. Wagner, M. May, A. Lakhina, Impact of Packet Sampling on Anomaly Detection Metrics, 6th ACM SIGCOMM Conference on Internet Measurements, Rio de Janeiro, Brazil, October 25-17, 2006, pp 159-164.
3. Cisco White Paper: Approaching the Zettabyte Era, June 2008.
4. Cisco Application Extension Platform: http://www.cisco.com/en/US/ products/ps9701/, Last accessed: September 2009.
5. K. C. Claffy, G. C. Polyzos, H. W. Braun: Application of Sampling Methodologies to Network Traffic Characterization, ACM SIGCOMM'93, San Francisco, California, U.S.A., September 13-17, 1993, pp 194-203.
6. B. Claise: Cisco Systems NetFlow Services Export Version 9, RFC3954, October 2004.
7. B. Claise: Specification of the IP Flow Information Export (IP-FIX) Protocol for the Exchange of IP Traffic Flow Information, RFC5101, January 2008.
8. B. Claise (Ed.), A. Johnson, J. Quittek: Packet Sampling (PSAMP) Protocol Specifications, IETF RFC 5476, March 2009.
9. W. Cochran: Sampling Techniques, Wiley, 1987.
10. N. Duffield (Ed.), D. Chiou, B. Claise, A. Greenberg, M. Grossglauser, J. Rexford: A Framework for Packet Selection and Reporting, IETF RFC 5474, March 2009.
11. N.G. Duffield, C. Lund, M. Thorup: Learn More, Sample Less: Control of Volume and Variance in Network Measurement, IEEE Transactions of Information Theory, Vol. 51, 2005, pp 1756-1775.
12. N. Duffield, C. Lund, M. Thorup: Charging from Sampled Network Usage, 1st ACM SIGCOMM Workshop on Internet Measurements, San Francisco, California, U.S.A., November 1-2, 2001, pp 245-256.
13. N. G. Duffield, M. Grossglauser: Trajectory Sampling for Direct Traffic Observation, IEEE/ACM Transactions on Networking, Vol. 9, No. 3, 2001, pp 280-292.
14. C. Estan, G. Varghese: New Directions in Traffic Measurement and Accounting: Focusing on the Elephants, Ignoring the Mice, ACM Transactions on Computer Systems, Volume 21, Issue 3, Aug. 2003.
15. Flow-Tools Homepage: http://www.splintered.net/sw/flowtools/, June 2008.
16. S.H. Han, M.S. Kim, H.T Ju, J. W. K. Hong: The Architecture of NG-MON: A Passive Network Monitoring System for High-Speed IP Networks, 13th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM'02), Montreal, Canada, October 21-23, 2002, pp 16-27.
17. Y. Kitatsuji, K. Yamazaki: A Distributed Real-time Tool for IP-flow Measurement, International Symposium on Applications and the Internet (SAINTW'04), Tokyo, Japan, January 26-30, 2004, pp 91-98.
18. J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, B. Zhao: OceanStore: An Architecture for Global-Scale Persistent Storage, 9th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2000), Cambridge, Massachusetts, USA, November 12-15, 2000, pp 190-201.
19. J. Liang, R. Kumar, K.W. Ross: The Kazaa Overlay: A Measurement Study, Computer Networks Journal (Elsevier), 2005, Vol. 50, No. 6.
20. J. Mai, C. Chuah, A. Sridharan, H. Z. T. Ye: Is Sampled Data Sufficient for Anomaly Detection?, 6th ACM SIGCOMM Conference on Internet Measurements, Rio de Janeiro, Brazil, October 25-17, 2006, pp 165-176.
21. P. Maymounkov, D. Mazières: Kademlia: A Peer-to-Peer Information System Based on the XOR Metric, IPTPS 2002, March 2002, Cambridge, MA, USA.
22. C. Morariu: Distributed Packet Capturing Architecture for High-Speed Network Links, 33rd Annual IEEE Conference on Local Computer Networks (LCN), Montreal, Canada, October 2008.
23. NFDump Project: http://nfdump.sourceforge.net/, June 2008.
24. B. Trammell, E. Boschi: Bidirectional Flow Export Using IP Flow Information Export (IPFIX), RFC 5103, January 2008.
25. Wuala Homepage: http://wua.la/, April 2008.
26. T. Zseby: Statistical Sampling for Non-Intrusive Measurements in IP Networks, Ph. D. Thesis, Technische Universität Berlin, Germany, Universitätsbibliothek (Diss.-Stelle), Fakultät IV - Elektrotechnik und Informatik, 2005.
27. T. Zseby, M. Molina, N. Duffield, S. Niccolini, F. Raspall: Sampling and Filtering Techniques for IP Packet Selection, IETF RFC 5475, March 2009.