FlowInfra: A fault-resilient scalable infrastructure for network-wide flow level measurement

APNOMS 2011 - 13th Asia-Pacific Network Operations and Management Symposium: Managing Clouds, Smart Networks and Services, Final Program

Volumn , Issue , 2011, Pages

  Gao, Lei ^a   Yang, Jiahai ^a   Zhang, Hui ^a   Zhang, Bin ^a   Qin, Donghong ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

distributed system; flow level measurement; hash; IPv6; load balance; peer to peer

Indexed keywords

DISTRIBUTED SYSTEMS; FLOW LEVEL MEASUREMENT; HASH; IPV6; LOAD-BALANCE; PEER TO PEER;

CLOUDS; DISTRIBUTED COMPUTER SYSTEMS; FLOW MEASUREMENT; INTERNATIONAL TRADE; INTERNET PROTOCOLS; LEVEL MEASUREMENT; MEASUREMENTS; ROUTERS; SCALABILITY;

NETWORK MANAGEMENT;

EID: 83255176962     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/APNOMS.2011.6077019     Document Type: Conference Paper

References (32)

1. N. G. Duffield, C. Lund, and M. Thorup, Estimating flow distributions from sampled flow statistics, presented at the ACM SIGCOMM, Karlsruhe, Germany, Aug. 2003.
2. C. Estan, K. Keys, D. Moore, and G. Varghese. Building a Better NetFlow. In Proc. of SIGCOMM'04, Portland, Oregon, USA, Aug. 2004.
3. N. Duffield, C. Lund, and M. Thorup. Properties and Prediction of Flow Statistics from Sampled Packet Streams. In Proc. ACM SIGCCOMM IMW'02, Marseille, France, Nov. 2002.
4. P. Barford, J. Kline, D. Plonka, and A. Ron. A Signal Analysis of Network Traffic Anomalies. In Proc. ACM SIGCOMM IMW'02, pages 71-82, Marseille, France, Nov. 2002.
5. N. Duffield. Sampling for Passive Internet Measurement: A Review. Statistical Science, 19(3):472-498, 2004. (Pubitemid 40472647)
6. A. Lakhina, M. Crovella, and C. Diot. Mining Anomalies Using Traffic Feature Distributions. In Proc. ACM SIGCOMM '05, Philadelphia, PA, USA, Aug. 2005.
7. MAI, J., CHUAH, C.-N., SRIDHARAN, A., YE, T., AND ZANG, H. Is sampled data sufficient for anomaly detection? In IMC 2006, Rio de Janeiro, Brazil, October 2006.
8. C. Estan and G. Varghese, New directions in traffic measurement and accounting, presented at the ACM SIGCOMM, Pittsburgh, PA, Aug. 2002.
9. N. G. Duffield, C. Lund, and M. Thorup, Charging from sampled network usage, presented at the ACM SIGCOMM Internet Measurement Workshop, San Francisco, CA, Nov. 2001.
10. N. G. Duffield and M. Grossglauser. Trajectory Sampling for Direct Traffic Observation. IEEE/ACM Transactions on Networking, 9(3):280-292, June 2001.
11. Nick Duffield and Matthias Grossglauser, Trajectory sampling with unreliable reporting, IEEE/ACM Transactions on Networking (TON), v.16 n.1, p.37-50, February 2008
12. Cisco NetFlow. http://www.cisco.com/en/US/products/ps6601/products-ios- protocol-group-home.html
13. BOB hashing function: http://burtleburtle.net/bob/hash/doobs.html.
14. th ACM SIGCOMM Conference on Internet Measurements, Rio de Janeiro, Brazil, October 25-17, 2006, pp 159-164.
15. B. Claise: Cisco Systems NetFlow Services Export Version 9, RFC3954, October 2004.
16. B. Claise: Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information, RFC5101, January 2008.
17. cSamp: A System for Network-Wide Flow Monitoring" by Vyas Sekar, Michael K. Reiter, Walter Willinger, Hui Zhang, Ramana Rao Kompella, and David G. Andersen. In Proc. 5th USENIX NSDI, (San Francisco, CA), Apr. 2008.
18. B. Claise (Ed.), A. Johnson, J. Quittek: Packet Sampling (PSAMP) Protocol Specifications, IETF RFC 5476, March 2009.
19. S.H. Han, M.S. Kim, H.T Ju, J. W. K. Hong: The Architecture of NG-MON: A Passive Network Monitoring System for High-Speed IP Networks, 13th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM'02), Montreal, Canada, October 21-23, 2002, pp 16-27.
20. Y. Kitatsuji, K. Yamazaki: A Distributed Real-time Tool for IPflow Measurement, International Symposium on Applications and the Internet (SAINTW'04), Tokyo, Japan, January 26-30, 2004, pp 91-98.
21. J. Kubiatowicz, D. Bindel, Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer, C. Wells, B. Zhao: OceanStore: An Architecture for Global-Scale Persistent Storage, 9th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2000), Cambridge, Massachusetts, USA, November 12-15, 2000, pp 190-201.
22. J. Liang, R. Kumar, K.W. Ross: The Kazaa Overlay: A Measurement Study, Computer Networks Journal (Elsevier), 2005, Vol. 50, No. 6.
23. P. Maymounkov, D. Mazières: Kademlia: A Peer-to-Peer Information System Based on the XOR Metric, in IPTPS 2002, March 2002, Cambridge, MA, USA.
24. C. Morariu: Distributed Packet Capturing Architecture for High-Speed Network Links, 33rd Annual IEEE Conference on Local Computer Networks (LCN), Montreal, Canada, October 2008.
25. NFDump Project: http://nfdump.sourceforge.net/.
26. C. Morariu, T. Kramis, and B. Stiller: DIPStorage: Distributed Storage of IP Flow Records. 16th IEEE Workshop on Local and Metropolitan Area Networks, 2008, pp. 108-113.
27. T. Zseby, M. Molina, N. Duffield, S. Niccolini, F. Raspall: Sampling and Filtering Techniques for IP Packet Selection, IETF RFC 5475, March 2009.
28. T. Karagiannis, K. Papagiannaki, and M. Faloutsos: BLINC: Multilevel Traffic Classification in the Dark, in SIGCOMM'05, Philadelphia, USA, August 21-26, 2005.
29. A. Moore and D. Zuev: Internet Traffic Classification Using Bayesian Analysis Techniques, in SIGMETRICS'05, Banff, Canada, June 6-10, 2005.
30. J. Erman, A. Mahanti, and M. Arlitt. Internet Traffic Identification using Machine Learning. In GLOBECOM'06, San Francisco, USA, November 2006.
31. C. Estan, G. Varghese: New Directions in Traffic Measurement and Accounting: Focusing on the Elephants, Ignoring the Mice, ACM Transactions on Computer Systems, Volume 21, Issue 3, Aug. 2003.
32. Cristian Morariu, Peter Racz, Burkhard Stiller: SCRIPT: A Framework for Scalable Real-time IP Flow Record Analysis, in NOMS'10, Osaka, Japan, April 2010.