Dynamic feature analysis and measurement for large-scale network traffic monitoring

IEEE Transactions on Information Forensics and Security

Volumn 5, Issue 4, 2010, Pages 905-919

  Guan, Xiaohong ^a,b   Qin, Tao ^a   Li, Wei ^a   Wang, Pinghui ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Correlation analysis; dynamic changes; network traffic monitoring; regional flow model; Renyi cross entropy

Indexed keywords

CORRELATION ANALYSIS; CROSS ENTROPY; DYNAMIC CHANGES; FLOW MODEL; NETWORK TRAFFIC;

DYNAMIC ANALYSIS; ENTROPY; MONITORING; TELECOMMUNICATION TRAFFIC;

NETWORK MANAGEMENT;

EID: 78649408044     PISSN: 15566013     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIFS.2010.2066970     Document Type: Article

References (45)

1. K. M. Carley, "Dynamic network analysis", in Dynamic Social Network Modeling and Analysis: Workshop Summary and Papers, Washington, DC, 2003, pp. 133-145.
2. M. Roughan, T. Griffin, M. Mao, A. Greenberg, and B. Freeman, "Combining routing and traffic data for detection of IP forwarding anomalies", ACM SIGMETRICS Performance Evaluation Rev., vol. 32, no. 1, pp. 416-417, 2004.
3. M. S. Kim, H. J. Kang, C. S. Hong, H. S. Chung, and J. W. Hong, "A flow-based method for abnormal network traffic detection", in Proc. IEEE/IFIP Network Operations and Management Symp., Seoul, Korea, 2004, pp. 599-612.
4. P. Barford, J. Kline, and D. Plonka, "A signal analysis of network traffic anomalies", in Proc. ACM SIGCOMM Internet Measurement Workshop, Marseilles, France, 2002, pp. 71-82.
5. M. V. Mahoney, "Network traffic anomaly detection based on packet bytes", in Proc. ACM-SAC 2003, Melbourne, FL, 2003, pp. 346-350.
6. G. Giorgi and C. Narduzzi, "Detection of anomalous behaviors in networks from traffic measurements", IEEE Trans. Instrum. Meas., vol. 57, no. 12, pp. 2782-2791, Dec. 2008.
7. S. Kim and A. Reddy, "A study of analyzing network traffic as images in real-time", in Proc. IEEE INFOCOM, Miami, FL, 2005, pp. 2056-2067.
8. W. John and S. Tafvelin, "Analysis of internet backbone traffic and header anomalies observed", in Proc. 7th ACM SIGCOMM Conf. Internet Measurement, San Diego, CA, 2007, pp. 111-116.
9. E. Kohler, J. Li, V. Paxson, and S. Shenker, "Observed structure of addresses in IP traffic", IEEE/ACM Trans. Netw., vol. 14, no. 6, pp. 1207-1218, Dec. 2006.
10. G. Tan, M. Poletto, F. Kaashoek, and J. Guttag, "Role classification of hosts within enterprise networks based on connection patterns", in Proc. 2003 USENIX Annu. Tech. Conf., Washington, DC, 2003, pp. 15-28.
11. P. McDaniel, S. Sen, and O. Spatscheck, "Enterprise security: A community of interest based approach", in Proc. 3th Annu. Network and Distributed System Security Symp., California, 2006, pp. 1-15.
12. W. Aiello, C. Kalmanek, P. McDaniel, S. Sen, and O. Spatscheck, "Analysis of communities of interest in data networks", in Proc. Passive and Active Network Measurement, Boston, MA, 2005, pp. 83-96.
13. CISCO NetFlow [Online]. Available: http://www.cisco.com/en/US/products/ ps6601/products-white-paper09186a00800a3db9.shtml
14. N. Ye, S. Vilbert, and Q. Chen, "Computer intrusion detection through EWMA for autocorrelated and uncorrelated data", IEEE Trans. Reliability, vol. 52, no. 1, pp. 75-82, Mar. 2003.
15. M. Thottan and C. Ji, "Anomaly detection in IP networks", IEEE Trans. Signal Process., vol. 51, no. 8, pp. 2191-2204, Aug. 2003.
16. W. Lee and D. Xiang, "Information-theoretic measures for anomaly detection", in Proc. IEEE Symp. Security and Privacy, Oakland, CA, 2001, pp. 130-143.
17. K. Xu, Z. Zhang, and S. Bhattacharyya, "Profiling internet backbone traffic: Behavior models and applications", ACM SIGCOMM Computer Communication Rev., vol. 35, no. 4, pp. 169-180, 2005. (Pubitemid 46323502)
18. Y. Liu, D. Towsley, T. Ye, and J. Bolot, "An information-theoretic approach to network monitoring and measurement", in Proc. 5th ACM SIGCOMM Internet Measurement Conf., Berkeley, CA, 2005, pp. 1-14.
19. J. Jung, V. Paxson, A. Berger, and H. Balakrishnan, "Fast port scan detection using sequential hypothesis testing", in Proc. IEEE Symp. Security and Privacy, California, 2004, pp. 211-225.
20. C. Zou, W. Gong, D. Towsley, and L. Gao, "The monitoring and early detection of internet worms", IEEE/ACM Trans. Netw., vol. 13, no. 5, pp. 961-974, Oct. 2005.
21. X. Qin, D. Dagon, G. Gu, W. Lee, M. Warfield, and P. Allor, "Worm detection using local networks", in Proc. Recent Advances of Intrusion Detection, Riviera, France, 2004, pp. 1-18.
22. D. Lee and N. Brownlee, "Passive measurement of one-way and two-way flow lifetimes", ACM SIGCOMM Comput. Commun. Rev., vol. 37, no. 3, pp. 17-28, 2007.
23. V. Paxson, "End-to-end internet packet dynamics", IEEE/ACM Trans. Netw., vol. 7, no. 3, pp. 277-292, Jun. 1999.
24. D. Zuev and A. Moore, "Traffic classification using a statistical approach", Lecture Notes Comput. Sci., vol. 3431, pp. 321-324, 2005.
25. H. Liu, W. Feng, Y. Huang, and X. Li, "A peer-to-peer traffic identification method using machine learning", in Proc. Networking, Architecture, and Storage, Guilin, China, 2007, pp. 155-160.
26. L. Bernaille, R. Teixeira, and I. Akodkenou, "Traffic classification on the fly", Comput. Commun. Rev., vol. 36, no. 2, pp. 23-26, 2006.
27. Y. Zhang, Z. Ge, A. Greenberg, and M. Roughan, "Network anomography", in Proc. 5th ACM SIGCOMM Conf. Internet Measurement, Berkeley, CA, pp. 317-330.
28. N. G. Duffield, C. Lund, and M. Thorup, "Estimating flow distributions from sampled flow statistics", in Proc. ACM SIGCOMM, Karlsruhe, Germany, 2003, pp. 325-336.
29. N. Duffield, C. Lund, and M. Thorup, "Properties and prediction of flow statistics from sampled packet streams", in Proc. ACM SIG-COMM Internet Measurement Workshop, Marseilles, France, 2002, pp. 159-171.
30. N. Duffield, C. Lund, and M. Thorup, "Flow sampling under hard resource constraints", in Proc. ACM SIGMETRICS, New York, 2004, pp. 85-96.
31. D. Brauckhoff, B. Tellenbach, A. Wagner, A. Lakhina, and M. May, "Impact of traffic sampling on anomaly detection metrics", in Proc. Internet Measurement Conf., Janeriro, Brazil, 2006, pp. 159-164.
32. J. Mai, C. Chuah, A. Sridharan, T. Ye, and H. Zang, "Is sampled data sufficient for anomaly detection", in Proc. ACM SIGCOMM Internet Measurement Conf., Janeriro, Brazil, 2006, pp. 165-176.
33. R. Schweller, Z. Li, Y. Chen, Y. Gao, A. Gupta, Y. Zhang, P. Dinda, M. Kao, and G. Memik, "Reversible sketches: Enabling monitoring and analysis over high-speed data streams", IEEE/ACM Trans. Netw., vol. 15, no. 5, pp. 1059-1072, Oct. 2007.
34. G. Cormode and S. Muthukrishnan, "An improved data stream summary: The count-min sketch and its applications", in Proc. Latin American Theoretical Informatics, Buenos Aires, Argentina, 2004, pp. 29-38.
35. P. Flajolet and G. Martin, "Probabilistic counting algorithms for data base applications", J. Comput. Syst. Sci., vol. 31, no. 2, pp. 182-209, 1985.
36. B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen, "Sketch-based change detection: Methods, evaluation, and applications", in Proc. ACM SIGCOMM Internet Measurement Conf., Miami, FL, 2003, pp. 234-247.
37. A. Lakhina, M. Crovella, and C. Diot, "Characterization of networkwide anomalies in traffic flows", in Proc. 4th ACM SIGCOMM Internet Measurement Conf., Taormina, Italy, 2004, pp. 201-206.
38. A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. Kolaczyk, and N. Taft, "Structural analysis of network traffic flows", ACM SIGMET-RICS, vol. 32, no. 1, pp. 61-72, 2004.
39. T. Qin, X. Guan, W. Li, and P. Wang, "Dynamic features measurement and analysis for large-scale networks", in Proc. 2008 IEEE Int. Conf. Communication Workshops, Beijing, China, 2008, pp. 212-216.
40. M. Freedman, M. Vutukuru, N. Feamster, and H. Balakrishnan, "Geographic locality of IP prefixes", in Proc. ACM Internet Measurement Conf. 2005, Berkeley, CA, 2005, pp. 153-158.
41. K. Keys, D. Moore, R. Koga, E. Lagache, M. Tesch, and K. Claffy, "The architecture of the CoralReef: Internet traffic monitoring software suite", in Proc. 2nd Passive and Active Measurement Workshop, Amsterdam, The Netherlands, 2001.
42. F. H. Edward, "Measuring network change: Renyi cross entropy and the second order degree distribution", in Proc. Passive and Active Measurement Conf. 2006, Adelaide, Australia, Apr. 2006.
43. G. Nychis, V. Sekar, D. Andersen, H. Kim, and H. Zhang, "An empirical evaluation of entropy-based traffic anomaly detection", in Proc. 8th ACM SIGCOMM Internet Measurement Conf., Vouliagmeni, Greece, 2008, pp. 151-156.
44. Y. Gu, A. McCallum, and D. Towsley, "Detecting anomalies in network traffic using maximum entropy estimation", in Proc. 5th ACM SIGCOMM Conf. Internet Measurement, Berkeley, CA, 2005, pp. 32-37.
45. D. Lee and N. Brownlee, "A methodology for finding significant network hosts", in Proc. 32nd IEEE Conf. Local Computer Networks, Dublin, Ireland, 2007, pp. 981-988.