Commodity Packet Capture Engines: Tutorial, Cookbook and Applicability

IEEE Communications Surveys and Tutorials

Volumn 17, Issue 3, 2015, Pages 1364-1390

  Moreno, Victor ^a   Ramos, Javier ^a   Santiago Del Rio, Pedro M ^a   Garcia Dorado, Jose Luis ^a   Gomez Arribas, Francisco J ^a   Aracil, Javier ^a  

^a UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

Commodity hardware; high performance networking; network traffic monitoring; packet capture engine

Indexed keywords

ENGINES; INTERNET;

CAPITAL EXPENDITURES; COMMODITY HARDWARE; HIGH-PERFORMANCE NETWORKING; NETWORK PROCESSING; NETWORK TRAFFIC MONITORING; PACKET CAPTURE; RESEARCH COMMUNITIES; SPECIALIZED HARDWARE;

HARDWARE;

EID: 84940103361     PISSN: None     EISSN: 1553877X     Source Type: Journal    
DOI: 10.1109/COMST.2015.2424887     Document Type: Article

References (78)

1. J. L. García-Dorado, A. Finamore, M. Mellia, M. Meo, and M. Munafo, "Characterization of ISP traffic: Trends, user habits, and access technology impact, " IEEE Trans. Netw. Serv. Manag., vol. 9, no. 2, pp. 142-155, Jun. 2012
2. B. Li, J. Springer, G. Bebis, and M. H. Gunes, "A survey of network flow applications, " J. Netw. Comput. Appl., vol. 36, no. 2, pp. 567-581, Mar. 2013
3. M.-S. Kim, H.-J. Kong, S.-C. Hong, S.-H. Chung, and J. Hong, "A flowbased method for abnormal network traffic detection, " in Proc. IEEE/IFIP Netw. Oper. Manag. Symp., 2004, pp. 599-612
4. M. Mellia, R. Lo Cigno, and F. Neri, "Measuring IP and TCP behavior on edge nodes with tstat, " Comput. Netw., vol. 47, no. 1, pp. 1-21, Jan. 2005
5. M. Polychronakis, E. Markatos, K. Anagnostakis, and A. Oslebo, "Design of an application programming interface for IP network monitoring, " in Proc. IEEE/IFIP Netw. Opera. Manag. Symp., 2004, pp. 483-496
6. C. Fraleigh et al., "Packet-level traffic measurements from the sprint IP backbone, " IEEE Netw., vol. 17, no. 6, pp. 6-16, Nov./Dec. 2003
7. J. Yu andX. Zhou, "Ultra-high-capacity DWDM transmission system for 100G and beyond, " IEEE Commun. Mag., vol. 48, no. 4, pp. S56-S64, Mar. 2010
8. S. Dharmapurikar, P. Krishnamurthy, T. Sproull, and J. Lockwood, "Deep packet inspection using parallel bloom filters, " IEEE Micro, vol. 24, no. 1, pp. 52-61, Jan./Feb. 2004
9. G. Antichi, S. Giordano, D. Miller, and A. Moore, "Enabling open-source high speed network monitoring on NetFPGA, " in Proc. IEEE/IFIP Netw. Oper. Manag. Symp., 2012, pp. 1029-1035
10. F. Yu, R. Katz, and T. Lakshman, "Gigabit rate packet pattern-matching using TCAM, " in Proc. 12th IEEE Int. Conf. Netw. Protocols, 2004, pp. 174-184
11. C. Meiners, J. Patel, E. Norige, E. Torng, and A. Liu, "Fast regular expression matching using small TCAMs for network intrusion detection and prevention systems, " in Proc. USENIX Conf. Security, 2010, pp. 1-16
12. Endace, "Packet capture performance evaluation, " 2014 [Online]. Available: http://www.emulex.com
13. " Cisco network convergence system, " Cisco, San Jose, CA, USA, Feb. 15, 2015 [Online]. Available: http://www.cisco.com/en/US/products/ps13132/index.html
14. N. McKeown et al., "Openflow: Enabling innovation in campus networks, " ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 2, pp. 69-74, Apr. 2008
15. L. Braun, A. Didebulidze, N. Kammenhuber, and G. Carle, "Comparing and improving current packet capturing solutions based on commodity hardware, " in Proc. ACM Internet Meas. Conf., 2010, pp. 206-217
16. J. L. García-Dorado et al., "High-performance network traffic processing systems using commodity hardware, " in Data Traffic Monitoring and Analysis. Berlin, Germany, Springer-Verlag, 2013, ch. 1, pp. 3-27
17. L. Rizzo, L. Deri, and A. Cardigliano, "10 Gbit/s line rate packet processing using commodity hardware: Survey and new proposals, " Feb. 15, 2015 [Online]. Available: http://luca.ntop.org/10g.pdf
18. S. Alcock, P. Lorier, and R. Nelson, "Libtrace: A packet capture and analysis library, " ACM SIGCOMM Computer Commun. Rev., vol. 42, no. 2, pp. 42-48, Apr. 2012
19. S. Lee, K. Levanti, and H. Kim, "Network monitoring: Present and future, " Comput. Netw., vol. 65, no. 1, pp. 84-98, Jun. 2014
20. R. Hofstede et al., "Flow monitoring explained: From packet capture to data analysis with netflow and IPFIX, " IEEE Commun. Surveys Tuts., vol. 16, no. 4, pp. 2037-2064, 4th Quart. 2014
21. S. Han, K. Jang, K. S. Park, and S. Moon, "PacketShader: A GPU-Accelerated software router, " in Proc. ACM SIGCOMM, 2010, pp. 1-12
22. "82599 10 Gbe controller datasheet, " Intel, Santa Clara, CA, USA, Feb. 15, 2015 [Online]. Available: http://www.intel.com/content/www/us/en/ethernet-controllers/82599-10-gbe-controller-datasheet.html
23. Receive Side Scaling, Microsoft, Redmond, WA, USA, Feb. 15, 2015 [Online]. Available: http://msdn.microsoft.com/en-us/library/windows/hardware/ff567236(v=vs.85).aspx
24. S. Woo and K. Park, "Scalable TCP session monitoring with symmetric receive-side scaling, " Tech. Rep. KAIST, Daejeon, Korea, 2012 [Online]. Available: http://www.ndsl.kaist.edu/shinae/papers/TR-symRSS.pdf
25. M. Dobrescu, K. Argyraki, and S. Ratnasamy, "Toward predictable performance in software packet-processing platforms, " in Proc. USENIX Symp. Netw. Syst. Des. Implementation, 2012, pp. 1-14
26. A. Lucent, "FP3: breakthrough 400G network processor, " Alcatel Lucent, Boulogne-Billancourt, France, 2014 [Online]. Available: http://www3. alcatel-lucent.com/products/fp3/
27. "APP3000 network, " LSI, San Jose, CA, USA, 2014 [Online]. Available: http://www.lsi.com/products/mobile-communication-processors/pages/app-network-processors.aspx
28. " IXP4XX product line of network processors, " Intel, Santa Clara, CA, USA, 2014 [Online]. Available: http://www.intel.com/p/en-US/embedded/hwsw/hardware/ixp-4xx
29. NetFPGA, "NetFPGA project, " 2014 [Online]. Available: http://www.netfpga.org
30. "Endace EMULEX, " Endace, Auckland, New Zealand, 2014 [Online]. Available: http://www.endace.com/
31. "Network analysis module (NAM) products, " Cisco, San Jose, CA, USA, 2014 [Online]. Available: http://www.cisco.com/go/nam
32. L. Zabala, A. Ferro, and A. Pineda, "Modelling packet capturing in a traffic monitoring system based on Linux, " in Proc. Perform. Eval. Comput. Telecommun. Syst., 2012, pp. 1-6
33. L. Foundation, "NAPI, " 2014 [Online]. Available: http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
34. L. Rizzo, "NETMAP: A novel framework for fast packet I/O, " in Proc. USENIX Annu. Techn. Conf., 2012, pp. 1-9
35. C. Schroder, Linux Networking Cookbook. Newton, MA, USA: O'Reilly Media, 2007
36. C. Benvenuti, Understanding Linux Network Internals. Newton, MA, USA: O'Reilly Media, 2005
37. G. Liao, X. Znu, and L. Bnuyan, "A new server I/O architecture for high speed networks, " in Proc. Symp. High-Perform. Comput. Archit., 2011, pp. 255-265
38. A. Papadogiannakis, G. Vasiliadis, D. Antoniades, M. Polychronakis, and E. Markatos, "Improving the performance of passive network monitoring applications with memory locality enhancements, " Comput. Commun., vol. 35, no. 1, pp. 129-140, Jan. 2012
39. W. Wenji, P. DeMar, and M. Crawford, "Why can some advanced Ethernet NICs cause packet reordering?" IEEE Commun. Lett., vol. 15, no. 2, pp. 253-255, Feb. 2011
40. V. Moreno, P. M. Santiago del Río, J. Ramos, J. Garnica, and J. L. García-Dorado, "Batch to the future: Analyzing timestamp accuracy of highperformance packet I/O engines, " IEEE Commun. Lett., vol. 16, no. 11, pp. 1888-1891, Nov. 2012
41. W. Su, L. Zhang, D. Tang, and X. Gao, "Using direct cache access combined with integrated NIC architecture to accelerate network processing, " in Proc. IEEE Conf. High Perform. Comput. IEEE Conf. Embedded Softw. Syst., 2012, pp. 509-515
42. L. Deri, "Improving passive packet capture: Beyond device polling, " in Proc. Syst. Admin. Netw. Eng. Conf., 2004, pp. 1-12
43. L. Deri, "nCap: Wire-speed packet capture and transmission, " in Proc. IEEE/IFIP Workshop End-to-End Monit. Techn. Services, 2005, pp. 47-55
44. L. Rizzo, "Revisiting network I/O apis: The netmap framework, " ACM Queue, vol. 10, no. 1, pp. 30-39, 2012
45. L. Rizzo, "Portable packet processing modules for OS kernels, " IEEE Netw., vol. 28, no. 2, pp. 6-11, Mar./Apr. 2014
46. N. Bonelli, A. Di Pietro, S. Giordano, and G. Procissi, "On multi-gigabit packet capturing with multi-core commodity hardware, " in Proc. Passive Active Meas. Conf., 2012, pp. 64-73
47. N. Bonelli, S. Giordano, G. Procissi, and L. Abeni, "A purely functional approach to packet processing, " in Proc. ACM/IEEE Symp. Archit. Netw. Commun. Syst., 2014, pp. 219-230
48. " Intel Data Plane Development Kit (Intel DPDK) Release Notes, " Intel, Santa Clara, CA, USA, 2014 [Online]. Available: http://www.intel.com/content/dam/www/public/us/en/documents/release-notes/inteldpdk-release-notes.pdf
49. V. Moreno, "Development and evaluation of a low-cost scalable architecture for network traffic capture and storage for 10Gbps networks, " M.S. thesis, Univ. Autónoma de Madrid, Madrid, Spain, 2012 [Online]. Available: http://www.ii.uam.es/vmoreno/Publications/morenoTFM2012. pdf
50. V. Moreno et al., "Packet storage at multi-gigabit rates using off-The-shelf systems, " in Proc. IEEE Int. Conf. High Perform. Comput. Commun., 2014, pp. 486-489
51. M. Dobrescu et al., "Routebricks: Exploiting parallelism to scale software routers, " in Proc. ACM SIGOPS Symp. Oper. Syst. Principles, 2009, pp. 15-28
52. M. Krasnyansky, "UIO-IXGBE, " 2012 [Online]. Available: https://opensource.qualcomm.com/wiki/UIO-IXGBE
53. F. Fusco and L. Deri, "High speed network traffic analysis with commodity multi-core systems, " in Proc. ACM Internet Meas. Conf., 2010, pp. 1-7
54. ntop, "Libzero for DNA, " 2014 [Online]. Available: http://www.ntop.org/products/pf-ring/libzero-for-dna
55. PacketShader, "Packet I/O engine, " 2012 [Online]. Available: http://shader.kaist.edu/packetshader/io-engine/index.html
56. netmap, "The fast packet I/O framework, " 2014 [Online]. Available: http://info.iet.unipi.it/luigi/netmap
57. PFQ, "PFQ homepage, " 2015 [Online]. Available: http://netserv.iet.unipi. it/software/pfq
58. DPDK, "Data plane development kit, " 2015 [Online]. Available: http://dpdk.org
59. "Intel Data Plane Development Kit (Intel DPDK) Programmer's Guide, " Intel, Santa Clara, CA, USA, 2014 [Online]. Available: http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdkprogrammers-guide.pdf
60. HPCAP, "High-Performance 10G Network Capture Engine, " 2015 [Online]. Available: http://github.com/hpcn-uam/HPCAP
61. V. Moreno et al., "Multi-granular, multi-purpose and multi-Gb/s monitoring on off-The-shelf systems, " Int. J. Netw. Manag., vol. 24, no. 4, pp. 221-234, Jul./Aug. 2014
62. M.-S. Kim, Y. J. Won, and J.W. Hong, "Characteristic analysis of Internet traffic from the perspective of flows, " Comput. Commun., vol. 29, no. 10, pp. 1639-1652, Jun. 2006
63. C. Walsworth, E. Aben, K. Claffy, and D. Andersen, "The CAIDA anonymized 2009 Internet traces, " CAIDA, La Jolla, CA, USA, 2009 [Online]. Available: http://www.caida.org/data/passive/passive-2009-dataset.xml
64. "Traffic Analysis Research, " CAIDA, La Jolla, CA, USA [Online]. Available: http://www.caida.org/research/traffic-Analysis/
65. E. Kohler, R. Morris, B. Chen, J. Jannotti, and M. Kaashoek, "The Click modular router, " ACM Trans. Comput. Syst., vol. 18, no. 3, pp. 263-297, 2000
66. L. Rizzo, M. Carbone, and G. Catalli, "Transparent acceleration of software packet forwarding using netmap, " in Proc. IEEE INFOCOM, 2012, pp. 2471-2479
67. J. L. García-Dorado et al., "Low-cost and high-performance: VoIP monitoring and full-data retention at multi-Gb/s rates using commodity hardware, " Int. J. Netw. Manag., vol. 24, no. 3, pp. 181-199, 2014
68. G. Vasiliadis, M. Polychronakis, and S. Ioannidis, "MIDeA: a multiparallel intrusion detection architecture, " in Proc. ACM Conference Comput. Commun. Security, 2011, pp. 297-308
69. G. Szabó, I. Gódor, A. Veres, S. Malomsoky, and S. Molnar, "Traffic classification over Gbit speed with commodity hardware, " J. Commun. Softw. Syst., vol. 5, no. 3, 2010
70. P. M. Santiago del Río et al., "Wire-speed statistical classification of network traffic on commodity hardware, " in Proc. ACM Internet Meas. Conf., 2012, pp. 65-72
71. M. Danelutto, L. Deri, and D. De Sensi, "Network monitoring on multicores with algorithmic skeletons." in Proc. Int. Conf. PARCO, 2011
72. A. Di Pietro et al., "Toward composable network traffic measurement, " in Proc. IEEE INFOCOM, 2013, pp. 70-74
73. F. Schneider, J. Wallerich, and A. Feldmann, "Packet capture in 10-Gigabit Ethernet environments using contemporary commodity hardware, " in Proc. Passive Active Meas. Conf., 2007, pp. 1-11
74. J. Nickolls and W. Dally, "The GPU computing era, " IEEE Micro, vol. 30, no. 2, pp. 56-69, Mar./Apr. 2010
75. A. Callado et al., "A survey on Internet traffic identification, " IEEE Commun. Surveys Tuts., vol. 11, no. 3, pp. 37-52, 3rd Quart. 2009
76. T. Nguyen and G. Armitage, "A survey of techniques for Internet traffic classification using machine learning, " IEEE Commun. Surveys Tuts., vol. 10, no. 4, pp. 56-76, 4th Quart. 2008
77. C. Systems, "White paper: Introduction to Cisco IOS netFlow, " 2012 [Online]. Available: http://www.cisco.com/c/en/us/products/ios-nx-ossoftware/ios-netflow/white-paper-listing.html
78. M. Dusi et al., "Blockmon: Flexible and high-performance big data stream analytics platform and its use cases, " NEC Techn. J., vol. 7, no. 2, pp. 102-106, 2012.