A survey of techniques for internet traffic classification using machine learning

IEEE Communications Surveys and Tutorials

Volumn 10, Issue 4, 2008, Pages 56-76

  Nguyen, Thuy T T ^a   Armitage, Grenville ^a  

^a SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

Author keywords

Flow clustering; Internet protocol; Machine learning; Payload inspection; Real time; Statistical traffic properties; Traffic classification

Indexed keywords

FLOW CLUSTERING; MACHINE-LEARNING; PAYLOAD INSPECTION; REAL TIME; STATISTICAL TRAFFIC PROPERTIES; TRAFFIC CLASSIFICATION;

DATA MINING; INTERNET; LEARNING SYSTEMS; SURVEYS; TELECOMMUNICATION NETWORKS; TRANSMISSION CONTROL PROTOCOL;

TELECOMMUNICATION TRAFFIC;

EID: 62849120844     PISSN: None     EISSN: 1553877X     Source Type: Journal    
DOI: 10.1109/SURV.2008.080406     Document Type: Review

References (68)

1. Snort-The de facto standard for intrusion detection/prevention, http://www.snort.org, as of August 14, 2007.
2. Bro intrusion detection system-Bro overview, http://bro-ids.org, as of August 14, 2007.
3. V. Paxson, "Bro: A system for detecting network intruders in real-time," Computer Networks, no. 31(23-24), pp. 2435-2463, 1999.
4. L. Stewart, G. Armitage, P. Branch, and S. Zander, "An architecture for automated network control of QoS over consumer broadband links," in IEEE International Region 10 Conference (TENCON 05), Melbourne, Australia, November 2005.
5. F. Baker, B. Foster, and C. Sharp, "Cisco architecture for lawful intercept in IP networks," Internet Engineering Task Force, RFC 3924, 2004.
6. T. Karagiannis, A. Broido, N. Brownlee, and K. Claffy, "Is P2P dying or just hiding?" in Proc. 47th annual IEEE Global Telecommunications Conference (Globecom 2004), Dallas, Texas, USA, November/December 2004.
7. S. Sen, O. Spatscheck, and D. Wang, "Accurate, scalable in network identification of P2P traffic using application signatures," in WWW2004, New York, NY, USA, May 2004.
8. R. Braden, D. Clark, and S. Shenker, "Integrated services in the Internet architecture: An overview," RFC 1633, IETF, 1994.
9. S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss, "An architecture for differentiated services," RFC 2475, IETF, 1998.
10. G. Armitage, Quality of Service In IP Networks: Foundations for a Multi-Service Internet. Macmillan Technical Publishing, April 2000.
11. L. Burgstahler, K. Dolzer, C. Hauser, J. Jahnert, S. Junghans, C. Macian, and W. Payer, "Beyond technology: The missing pieces for QoS success," in RIPQoS '03: Proc. ACM Special Interest Group on Data Communication (SIGCOMM) workshop on Revisiting IP QoS. New York, NY, USA: ACM Press, August, 2003, pp. 121-130.
12. Ubicom Inc., Solving Performance Problems with Interactive Applications in a Broadband Environment using StreamEngine Technology, http://www.ubicom.com/ pdfs/whitepapers/StreamEngine-WP-20041031.pdf, as of August 14, 2007.
13. J. But, N. Williams, S. Zander, L. Stewart, and G. Armitage, "ANGEL-Automated Network Games Enhancement Layer," in Proc. of 5th Annual Workshop on Network and Systems Support for Games (Netgames) 2006, Singapore, October 2006.
14. A. Moore and D. Zuev, "Internet traffic classification using Bayesian analysis techniques," in ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) 2005, Banff, Alberta, Canada, June 2005.
15. T. Karagiannis, K. Papagiannaki, and M. Faloutsos, "Blinc: Multilevel traffic classification in the dark," in Proc. of the Special Interest Group on Data Communication conference (SIGCOMM) 2005, Philadelphia, PA, USA, August 2005.
16. J. Erman, A. Mahanti, and M. Arlitt, "Byte me: A case for byte accuracy in traffic classification," in MineNet '07: Proc. 3rd annual ACM workshop on Mining network data. New York, NY, USA: ACM Press, June 2007, pp. 35-38. (Pubitemid 47317571)
17. Internet Assigned Numbers Authority (IANA), http://www.iana.org/ assignments/port-numbers, as of August 14, 2007.
18. M. Roughan, S. Sen, O. Spatscheck, and N. Duffield, "Class-of- service mapping for QoS: A statistical signature-based approach to IP traffic classification," in Proc. ACM/SIGCOMM Internet Measurement Conference (IMC) 2004, Taormina, Sicily, Italy, October 2004.
19. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications," RFC 1889, IETF, 1996.
20. A. Moore and K. Papagiannaki, "Toward the accurate identification of network applications," in Proc. Passive and Active Measurement Workshop (PAM2005), Boston, MA, USA, March/April 2005.
21. A. Madhukar and C. Williamson, "A longitudinal study of P2P traffic classification," in 14th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, September 2006.
22. V. Paxson, "Empirically derived analytic models of wide-area TCP connections," IEEE/ACM Trans. Networking, vol. 2, no. 4, pp. 316-336, 1994.
23. C. Dewes, A. Wichmann, and A. Feldmann, "An analysis of Internet chat systems," in ACM/SIGCOMM Internet Measurement Conference 2003, Miami, Florida, USA, October 2003.
24. K. Claffy, "Internet traffic characterisation," PhD Thesis, University of California, San Diego, 1994.
25. T. Lang, G. Armitage, P. Branch, and H.-Y. Choo, "A synthetic traffic model for Half-life," in Proc. Australian Telecommunications Networks and Applications Conference 2003 ATNAC2003, Melbourne, Australia, December 2003.
26. T. Lang, P. Branch, and G. Armitage, "A synthetic traffic model for Quake 3," in Proc. ACM SIGCHI International Conference on Advances in computer entertainment technology (ACE2004), Singapore, June 2004.
27. Z. Shi, Principles of Machine Learning. International Academic Publishers, 1992.
28. H. Simon, "Why should machines learn?" in R. S. Michalski, J. G. Carbonell, and T. M. Mitchell (editors) Machine Learning: An Artificial Intelligence Approach. Morgan Kaufmann, 1983.
29. I. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations (Second Edition). Morgan Kaufmann Publishers, 2005.
30. B. Silver, "Netman: A learning network traffic controller," in Proc. Third International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Association for Computing Machinery, 1990.
31. J. Frank, "Machine learning and intrusion detection: Current and future directions," in Proc. National 17th Computer Security Conference, Washington,D.C., October 1994.
32. Y. Reich and J. S. Fenves, "The formation and use of abstract concepts in design," in Fisher, D. H. and Pazzani, M. J. (editors), Concept Formation: Knowledge and Experience in Unsupervised Learning. Morgan Kaufmann, 1991.
33. H. D. Fisher, J. M. Pazzani, and P. Langley, Concept Formation: Knowledge and Experience in Unsupervised Learning. Morgan Kaufmann, 1991.
34. R. Duda, P. Hart, and D. Stork, Pattern Classification (2nd edition). JWiley-Interscience, 2001.
35. O. Carmichael and M. Hebert, "Shape-based recognition of wiry objects," IEEE Trans. Pattern Anal. Machine Intell., vol. 26, no. 12, pp. 1537-1552, 2004.
36. W. Rand, "Objective criteria for the evaluation of clustering methods," J. American Statistical Association, vol. 66, no. 336, pp. 846-850, 1971.
37. M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "Cluster validity methods: Part I," SIGMOD Rec., vol. 31, no. 2, pp. 40-45, 2002.
38. R. Xu and D. Wunsch, "Survey of clustering algorithms," IEEE Trans. Neural Networks, no. Vol.16, Issue 3, pp. 645-678, May 2005. (Pubitemid 40718010)
39. M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "Clustering validity checking methods: Part II," SIGMOD Rec., vol. 31, no. 3, pp. 19-27, 2002.
40. M. Hall and G. Holmes, "Benchmarking attribute selection techniques for discrete class data mining," IEEE Trans. Knowledge Data Eng., vol. 15, no. 6, pp. 1437-1447, 2003.
41. D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1989.
42. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artificial Intelligent, vol. 97, no. 1-2, pp. 273-324, 1997. (Pubitemid 127401107)
43. P. H. Winston, Artificial intelligence (2nd ed.). Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1984.
44. J. Park, H.-R. Tyan, and K. C.-C.J., "Internet traffic classification for scalable QoS provision," in IEEE International Conference on Multimedia and Expo, Toronto, Ontario, Canada, July 2006.
45. J. Erman, M. Arlitt, and A. Mahanti, "Traffic classification using clustering algorithms," in MineNet '06: Proc. 2006 SIGCOMM workshop on Mining network data. New York, NY, USA: ACM Press, 2006, pp. 281-286. (Pubitemid 46740690)
46. S. Zander, T. Nguyen, and G. Armitage, "Automated traffic classification and application identification using machine learning," in IEEE 30th Conference on Local Computer Networks (LCN 2005), Sydney, Australia, November 2005.
47. J. Erman, A. Mahanti, M. Arlitt, and C. Williamson, "Identifying and discriminating between web and peer-to-peer traffic in the network core," in WWW '07: Proc. 16th international conference on World Wide Web. Banff, Alberta, Canada: ACM Press, May 2007, pp. 883-892. (Pubitemid 47582318)
48. A. McGregor, M. Hall, P. Lorier, and J. Brunskill, "Flow clustering using machine learning techniques," in Proc. Passive and Active Measurement Workshop (PAM2004), Antibes Juan-les-Pins, France, April 2004.
49. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Statistical Society, vol. 30, no. 1, 1997.
50. P. Cheeseman and J. Stutz, "Bayesian classification (AutoClass): Theory and results," in Advances in Knowledge Discovery and Data Mining, 1996.
51. NetMate, http://sourceforge.net/projects/netmate-meter/, as of August 14, 2007.
52. The National Laboratory for Applied Network Research (NLANR), Traffic Measurement Data Repository, http://pma.nlanr.net/Special/, as of August 14, 2007.
53. L. Bernaille, R. Teixeira, I. Akodkenou, A. Soule, and K. Salamatian, "Traffic classification on the fly," ACM Special Interest Group on Data Communication (SIGCOMM) Computer Communication Review, vol. 36, no. 2, 2006.
54. T. Nguyen and G. Armitage, "Synthetic sub-flow pairs for timely and stable IP traffic identification," in Proc. Australian Telecommunication Networks and Application Conference, Melbourne, Australia, December 2006.
55. T. Auld, A. W. Moore, and S. F. Gull, "Bayesian neural networks for Internet traffic classification," IEEE Trans. Neural Networks, no. 1, pp. 223-239, January 2007. (Pubitemid 46062929)
56. T. Nguyen and G. Armitage, "Training on multiple sub-flows to optimise the use of Machine Learning classifiers in real-world IP networks," in Proc. IEEE 31st Conference on Local Computer Networks, Tampa, Florida, USA, November 2006.
57. P. Haffner, S. Sen, O. Spatscheck, and D. Wang, "ACAS: Automated construction of application signatures," in MineNet '05: Proceeding of the 2005 ACM SIGCOMM workshop on Mining network data. Philadelphia, Pennsylvania, USA: ACM Press, August 2005, pp. 197-202.
58. The University of Twente, Traffic Measurement Data Repository, http://arch.cs.utwente.nl/projects/m2c/m2c-D15.pdf, as of 17th August 2007.
59. Wolfenstein Enemy Territory, http://www.enemyterritory.com/, as of 17th August 2007.
60. J. Park, H.-R. Tyan, and K. C.-C.J., "GA-Based Internet Traffic Classification Technique for QoS Provisioning," in Proc. 2006 International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Pasadena, California, December 2006.
61. M. Crotti, M. Dusi, F. Gringoli, and L. Salgarelli, "Traffic classification through simple statistical fingerprinting," SIGCOMM Comput. Commun. Rev., vol. 37, no. 1, pp. 5-16, 2007.
62. J. Erman, A. Mahanti, M. Arlitt, I. Cohen, and C. Williamson, "Semisupervised network traffic classification," ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) Performance Evaluation Review, vol. 35, no. 1, pp. 369-370, 2007.
63. "Offline/realtime network traffic classificatioin using semisupervised learning," Department of Computer Science, University of Calgary, Tech. Rep., February 2007.
64. J. Erman, A. Mahanti, and M. Arlitt, "Internet traffic identification using machine learning techniques," in Proc. of 49th IEEE Global Telecommunications Conference (GLOBECOM 2006), San Francisco, USA, December 2006.
65. N. Williams, S. Zander, and G. Armitage, "A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification," Special Interest Group on Data Communication (SIGCOMM) Computer Communication Review, vol. 36, no. 5, pp. 5-16, 2006.
66. J. Ma, K. Levchenko, C. Kreibich, S. Savage, and G. Voelker, "Unexpected means of protocol inference," in IMC '06: Proc. 6th ACM SIGCOMM on Internet measurement. Rio de Janeriro, Brazil: ACM Press, October 2006, pp. 313-326. (Pubitemid 47165614)
67. D. Bonfiglio, M. Mellia, M. Meo, D. Rossi, and P. Tofanelli, "Revealing Skype traffic: when randomness plays with you," in SIGCOMM '07: Proc. 2007 conference on Applications, technologies, architectures, and protocols for computer communications. New York, NY, USA: ACM, August 2007, pp. 37-48. (Pubitemid 350239771)
68. N. Williams, S. Zander, and G. Armitage, "Evaluating machine learning methods for online game traffic identification," Centre for Advanced Internet Architectures, http://caia.swin.edu.au/reports/060410C/CAIA- TR-060410C.pdf, Tech. Rep. 060410C, as of August 14, 2007.