Benford's law behavior of internet traffic

Journal of Network and Computer Applications

Volumn 40, Issue 1, 2014, Pages 194-205

  Arshadi, Laleh ^a   Jahangir, Amir Hossein ^a  

^a SHARIF UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Anomaly detection; Benford's law; Computer network traffic analysis; Weibull distribution

Indexed keywords

WEIBULL DISTRIBUTION;

ANOMALY DETECTION; BENFORD'S LAW; DISTRIBUTED DATA; INTENTIONAL ATTACKS; INTER-ARRIVAL TIME; NATURALLY OCCURRING; NETWORK TRAFFIC ANALYSIS; SHAPE PARAMETERS;

INTERNET;

EID: 84896318444     PISSN: 10848045     EISSN: 10958592     Source Type: Journal    
DOI: 10.1016/j.jnca.2013.09.007     Document Type: Article

References (101)

1. Arshadi L, Jahangir AH. On the TCP flow inter-arrival times distribution. In: Proceedings of the 5th European symposium on computer modeling and simulation, Madrid, Spain. November 2011. p. 360-5.
2. P. Andriotis, G. Oikonomou, and T. Tryfonas JPEG steganography detection with Benford's law Digital Investigation 9 3-4 2013 246 257
3. Arshadi L, Jahangir AH. Entropy based SYN flooding detection. In: Proceedings of the 36th annual IEEE conference on local computer networks (LCN), Bonn, Germany. October 2011. p. 139-42.
4. Allen WH, Marin GA. The loss technique for detecting new denial of service attacks, In: Proceedings of IEEE SoutheastCon, Greensboro, NC, USA. March 2004. p. 302-9.
5. A. Abraham, C. Grosan, and C. Martin-Vide Evolutionary design of intrusion detection programs International Journal of Network Security 4 3 2007 328 339
6. Ahmed T, Oreshkin B, Coates M. Machine learning approaches to network anomaly detection, In: SYSML'07 proceedings of the 2nd USENIX workshop on tackling computer systems problems with machine learning techniques, Cambridge, MA, USA. April 2007.
7. S. Ali, I.U. Haq, S. Rizvi, N. Resheed, U. Sarfraz, and S.A. Khayam et al. On mitigating sampling-induced accuracy loss in traffic anomaly detection systems ACM SIGCOMM Computer Communication Review (CCR) 40 3 2010 5 16
8. Azadmanesh M. Design of realistic traffic generator and traffic anomalies [MSc. Thesis]. Sharif University of Technology, January 2013 [in Persian].
9. F. Benford The law of anomalous numbers Proceedings of the American Philosophical Society 78 4 1938 551 572
10. J. Boyle An application of Fourier series to the most significant digit problem American Mathematical Monthly 101 2 1994 879 886
11. A. Berger, P. Hill, B. Kaynar, and A. Ridder Finite-state Markov chains obey Benford's law SIAM Journal of Matrix Analysis and Applications 32 3 2011 665 684
12. P. Becker Patterns in listings of failure-rate and MTTF values and listings of other data IEEE Transactions on Reliability 31 2 1982 132 134 (Pubitemid 12549107)
13. B. Buck, A. Merchant, and S. Perez An illustration of Benford's first digit law using alpha decay half lives European Journal in Physics 14 2 1993 59 63
14. C. Breunig, and A. Goerres Searching for electoral irregularities in an established democracy: applying Benford's Law tests to Bundestag elections in Unified Germany Electoral Studies 30 3 2011 534 545
15. A. Berger, and T.P. Hill A basic theory of Benford's law Probability Surveys 8 2011 1 126
16. Benford Online Bibliography. 2013. Available from: 〈 http://www.benfordonline.net [accessed 06.13.]
17. Beebe NHF. A bibliography of publications about Benford's law, Heap's law and Zipf's law. 2013. Available from: 〈ftp://ftp.math.utah. edu/public-html/public-html/pub/tex/bib/benfords-law.pdf [accessed 06.13].
18. Barford P, Kline J, Plonka D, Ron A. A signal analysis of network traffic anomalies. In: Proceedings of the 2nd ACM SIGCOMM workshop on Internet measurement. Marseille, France. November 2002.
19. Bellaïche M, Grégoire JC., SYN flooding attack detection based on entropy computing. In: Proceedings of the IEEE GLOBECOM. Honolulu, Hawaii, USA. December 2009. p. 1-6.
20. D. Brauckhoff, X. Dimitropoulos, A. Wagner, and K. Salamatian Anomaly extraction in backbone networks using association rules IEEE/ACM Transactions on Networking 20 6 2012 1788 1799
21. D.R. Cox Long-range dependence: a review H.A. David, H.T David, Statistics: an appraisal 1984 Iowa State University Press 55 74
22. Cao J, Cleveland WS, Lin D, Son DX. On the nonstationarity of Internet traffic. In: Proceedings of ACM SIGMETRICS international conference on measurement and modeling of computer systems. New York, NY, USA. June 2001. p. 102-12.
23. V. Chandola, A. Banerjee, and V. Kumar Anomaly detection: a survey ACM Computing Surveys 41 3 2009 15:1 15:58
24. Callegari C, Giordano S, Pagano M, Pepe T. Detecting heavy change in the heavy hitter distribution of network traffic. In: Proceedings of the 7th International wireless communications and mobile computing conference (IWCMC). Istanbul, Turkey. July 2011. p. 1298-303.
25. C. Callegari, S. Giordano, and M. Pagano Application of wavelet packet transform to network anomaly detection S. Balandin, D. Moltchanov, Y. Koucheryavy, Next Generation teletraffic and wired/wireless advanced networking. Lecture notes in computer science vol. 5174 2008 Springer Berlin, Heidelberg 246 257
26. Cheng CM, Kung HT, Tan KS. Use of spectral analysis in defense against DoS attacks. In: Proceedings of IEEE GLOBECOM. Taipei, Taiwan. November 2002. p. 2143-8.
27. Cheng X, Xie K, Wang D. Network traffic anomaly detection based on self-similarity using HHT and wavelet transform. In: Proceedings of the fifth international conference on information assurance and security. Xian, China. August 2009.
28. V. Chatzigiannakis, S. Papavassiliou, and G. Androulidakis Improving network anomaly detection effectiveness via an integrated multi-metric-multi- link (m3l) PCA-based approach Security and Communication Networks 2 3 2009 289 304
29. Callegari C, Gazzarrini L, Giordano S, Pagano M, Pepe T. A novel multi time-scales PCA-based anomaly detection system. In: Proceedings of the international symposium on performance evaluation of computer and telecommunication systems, SPECTS. Ottawa, Canada. July 2010. p. 156-62.
30. C. Callegari et al. A methodological overview on anomaly detection, data traffic monitoring and analysis, LNCS vol. 7754 2013 Springer Berlin, Heidelberg 148 183
31. Dewaele G, Fukuda K, Borgnat P, Abry P, Cho K. Extracting hidden anomalies using sketch and non-Gaussian multiresolution statistical detection procedures. In: Proceedings of the SIGCOMM LSAD workshop. Kyoto, Japan. August 2007.
32. Erlang AK. The theory of probabilities and telephone conversations. Transactions of the Danish Academy of Technical Sciences, no. 2, 1948. (First published in: Nyt Tidsskrift for Matematik B 1909; 20: 33-39s.)
33. Ensafi R, Dehghanzadeh S, Akbarzadeh MR, Optimizing fuzzy k-means for network anomaly detection using PSO. In: Proceedings of the IEEE/ACS international conference on computer systems and applications. Doha. April 2008. p. 686-93.
34. H. Engel, and C. Leuenberger Benford's law for exponential random variables Statistics & Probability Letters 63 4 2003 361 365 (Pubitemid 36742502)
35. A. Feldmann Characteristics of TCP connection arrivals K. Park, W. Willinger, Self-similar network traffic and performance evaluation 2000 John Wiley & Sons Inc. New York, USA 367 400
36. R.M. Fewster A simple explanation of Benford's law The American Statistician 63 1 2009 26 32
37. Fu D, Shi YQ, Su W. A generalized Benford's law for JPEG coefficients and its applications in image forensics. In: Proceedings of SPIE6505, security, steganography, and watermarking of multimedia contents IX, 65051L. San Jose, CA. February 2007.
38. E. Feitosa, E. Souto, and D.H. Sadok An orchestration approach for unwanted Internet traffic identification Computer Networks 56 12 2012 2805 2831
39. N. Gauvrit, and J-P. Delahaye Scatter and regularity imply Benford's law. and more. Randomness Through Computation 2011 World Scientific Singapore p. 53 69
40. González FP, Heileman GL, Abdallah CT. Benford's law in image processing. In: Proceedings of IEEE international conference on image processing. Texas. September 2007.
41. A.K. Ganame, J. Bourgeoisa, R. Bidou, and F. Spiesa A global security architecture for intrusion detection on computer networks Computers & Security 27 2008 30 47
42. P. Garica-Teodoro, J. DIaz-Verdejoa, G. Macia-Fernandeza, and E. Vazquez Anomaly-based network intrusion detection: techniques, systems and challenges Computers & Security 28 1 2009 18 28
43. T. Hill A statistical derivation of the significant-digit law Statistical Science 10 4 1996 354 363
44. T.P. Hill Base invariance implies Benford's law Proceedings of the American Mathematical Society 123 3 1995 887 895
45. Ide T, Kashima H. Eigenspace-based anomaly detection in computer systems. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining. Seattle, WA, USA. August 2004. p. 440-9.
46. G. Judge, and L. Schechter Detecting problems in survey data using Benford's law Journal of Human Resources 44 1 2009 1 24
47. James C, Murthy HA. Decoupling non-stationary and stationary components in long range network time series in the context of anomaly detection. In: Proceedings of the 37th annual IEEE Conference on local computer networks (LCN). Florida, USA. October 2012.
48. S.S. Kim, and R.AL Narasimha Statistical techniques for detecting traffic anomalies through packet header data IEEE/ACM Transactions on Networking 16 3 2008 562 575
49. Kruegel C, Mutz D, Robertson W, Valeur F. Bayesian event classification for intrusion detection. In: Proceedings of the 19th annual computer security applications conference. Las Vegas, NV, USA, December 2003. p. 14-23.
50. Kline J, Nam S, Barford P, Plonka D, Ron A. Traffic anomaly detection at fine time scales with Bayes nets. In: Proceedings of the international conference on Internet monitoring and protection. Bucharest, Romania. 2008. p. 37-46.
51. W.E. Leland, M.S. Taqqu, W. Willinger, and D.V. Wilson On the self-similar nature of Ethernet traffic (extended version) IEEE/ACM Transactions on Networking 2 1 1994 1 15
52. I.W.C. Lee, and A.O. Fapojuwo Analysis and modeling of a campus wireless network TCP/IP traffic Computer Networks 53 15 2009 2674 2687
53. L.M. Leemis, B.W. Schmeiser, and D.L. Evans Survival distributions satisfying Benford's law The American Statistician 54 3 2000 236 241 (Pubitemid 33204717)
54. W. Lu, and A. Ghorbani Network anomaly detection based on wavelet analysis EURASIP Journal on Advances in Signal Processing 2009 2009 (article no. 4)
55. Li K, Zhou W, Yu S, Dai B. Effective DDoS attacks detection using generalized entropy metric. In: Proceedings of the 9th international conference on algorithms and architectures for parallel processing, ICA3PP 2009. Taipei, Taiwan. June 2009. p. 266-80.
56. M. Li Change trend of averaged Hurst parameter of traffic under DDOS flood attacks Computers & Security 25 3 2006 213 220 (Pubitemid 43796072)
57. Lakhina A, Crovella M, Diot C. Mining anomalies using traffic feature distributions. In: Proceedings of the 2005 conference on applications, technologies, architectures, and protocols for computer communications (ACM SIGCOMM'05). New York, NY, USA. October 2005. p. 217-28.
58. Liu Y, Zhang L, Guan Y. Sketch-based streaming PCA algorithm for network-wide traffic anomaly detection. In: Proceedings of the international conference on distributed computing systems. Genova, June 2010. p. 807-16.
59. Buyse Marc, Stephen L. George, Evans Stephen, Nancy L. Geller, Ranstam Jonas, Scherrer Bruno, Lesaffre Emmanuel, Murray Gordon, Edler Lutz, Hutton Jane, Colton Theodore, Lachenbruch Peter, and Verma Babul The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials Statistics in Medicine 18 24 1999 3435 3451 (Pubitemid 30041358)
60. Mahoney MV, Chan PK. Learning nonstationary models of normal network traffic for detecting novel attacks. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. Edmonton, Canada. July 2002. p. 376-85.
61. MAWI. Working Group Traffic Archive, samplepoint-F. 2013. Available from 〈http://mawi.wide.ad.jp/mawi/samplepoint-F/2009 [accessed 06.13].
62. Mebane Jr WR. Election forensics: the second-digit Benford's law test and recent American presidential elections. In: Proceedings of the election fraud conference. Salt Lake City. September 2006.
63. Nigrini MJ. The detection of income tax evasion through an analysis of digital frequencies [Ph.D. thesis]. University of Cincinnati, 1992.
64. M. Nigrini A taxpayer compliance application of Benford's law Journal of the American Taxation Association 18 1 1996 72 91
65. M. Nigrini The use of Benford's law as an aid in analytical procedures Auditing: A Journal of Practice & Theory 16 2 1997 52 67
66. Nigrini M. Benford's Law: Applications for forensic accounting, auditing, and fraud detection. New Jersey, US: Wiley, Hoboken; 2012.
67. Nychis G, Sekar V, Andersen DG, Kim H, Zhang H. An empirical evaluation of entropy-based traffic anomaly detection. In: Proceedings of the 8th ACM SIGCOMM Internet measurement conference. Vouliagmeni, Greece. October 2008. p. 151-6.
68. NUST Traffic Dataset. 2013. 〈http://wisnet.seecs.nust. edu.pk/projects/ENS/DataSets.html [accessed 06.13].
69. M. Orita, Y. Hagiwara, A. Moritomo, K. Tsunoyama, T. Watanabe, and K. Ohno Agreement of drug discovery data with Benford's law Expert Opinion on Drug Discovery 8 1 2013 1 5
70. V. Paxson, and S. Floyd Wide area traffic: the failure of Poisson modeling IEEE/ACM Transactions on Networking 3 3 1995 226 244
71. Pustisek M, Humar I, Bester J. Empirical analysis and modeling of peer-to-peer traffic flows. In: Proceedings of the 14th IEEE Mediterranean electrotechnical conference. Ajaccio. 2008. p. 169-75.
72. A. Patcha, and J.M. Park An overview of anomaly detection techniques: existing solutions and latest technological trends Computer Networks 51 12 2007 3448 3470 (Pubitemid 46921030)
73. R.S. Pinkham On the distribution of first significant digits The Annuals of Mathematical Statistics 32 4 1961 1223 1230
74. L. Pericchi, and D. Torres Quick anomaly detection by the Newcomb-Benford law, with applications to electoral processes data from the USA, Puerto Rico and Venezuela Statistical Science 26 4 2011 502 516
75. VV. Phoha Internet security dictionary 2002 Springer-Verlag New York, US
76. Pukkawanna S, Fukuda K. Combining sketch and wavelet models for anomaly detection. In: Proceedings of the 2010 IEEE international conference on intelligent computer communication and processing (ICCP). Cluj-Napoca. August 2010. p. 313-9.
77. Portnoy L, Eskin E, Stolfo SJ. Intrusion detection with unlabeled data using clustering. In: Proceedings of the ACM workshop on data mining applied to security. Philadelphia, PA, USA, 2001.
78. A. Papoulis, and S.U. Pillai Probability, random variables and stochastic processes 4th ed. 2002 McGraw Hill
79. Rehak M, Pechoucek M, Celeda P, Krmicek V, Minarik P, Medvigy D. Collaborative attack detection in high-speed networks. In: Proceedings of the 5th international central and eastern European conference on multi-agent systems and applications (CEEMAS'07). Leipzig, Germany: Springer; 2007. p. 73-82.
80. B.P. Sarkar An observation on the significant digits of the binomial coefficients and factorials Sankhya: The Indian Journal of Statistics Series B 35 3 1973 363 364
81. Sanches J, Marques JS. Image reconstruction using the Benford law. In: Proceedings of the IEEE international conference on image processing. Atlanta, GA. October 2006.
82. M. Sambridge, H. Tkalcic, and A. Jackson Benford's law in the natural sciences Geophysical Research Letters 37 2010 L22301
83. S. Staniford, J.A. Hoagland, and J.M. McAlerney Practical automated detection of stealthy portscans Journal of Computer Security 10 1-2 2002 105 136 (Pubitemid 34531414)
84. Soule A, Salamatian K, Taft N. Combining filtering and statistical methods for anomaly detection. In: Proceedings of ACM SIGCOMM Internet measurement conference. Berkeley, CA, October 2005. p. 331-344.
85. I. Steinwart, D. Hush, and C. Scovel A classification framework for anomaly detection Journal of Machine Learning Research 6 1 2005 211 232
86. A. Servin, and D. Kudenko Multi-agent reinforcement learning for intrusion detection Lecture Notes in Computer Science 4865 2008 211 223
87. SERPRO. Brazilian Federal Service of data processing. 2011. 〈http://www.serpro.gov.br.
88. Tong S, Zhang Z, Xie Y, Wu X. Image splicing detection based on statistical properties of Benford model. In: Proceedings of the 2nd international conference on computer science and electronics engineering. 2013.
89. Tavallaee M, Lu W, Iqbal SA, Ghorbani A. A novel covariance matrix based approach for detecting network anomalies. In: Proceedings of the 6th annual communication networks and services research conference. Halifax, Canada. May 2008. p. 75-81.
90. Z. Tan, A. Jamdagni, X. He, P. Nanda, and R.P. Liu Denial-of-service attack detection based on multivariate correlation analysis Neural information processing, Lecture notes in computer science vol. 7064 2011 Springer Berlin, Heidelberg 756 765
91. A. Tajbakhsh, M. Rahmati, and A. Mirzaei Intrusion detection using fuzzy association rules Applied Soft Computing 9 2 2009 462 469
92. M. Thottan, G. Liu, and C. Ji Anomaly detection approaches for communication networks, algorithms for next generation networks 2010 Springer Berlin, Heidelberg 239 261
93. Wu H, Zhou M, Gong J. Investigation on the IP flow inter-arrival time in large-scale network. In: Proceedings of the international conference on wireless communications, networking and mobile computing. Shanghai, China. September 2007.
94. J. Wlodarski Fibonacci and Lucas numbers tend to obey Benford's law Fibonacci Quarterly 9 1 1971 87 88
95. Y. Xiang, L. Ke, and W. Zhou Low-rate DDoS attacks detection and traceback by using new information metrics IEEE Transactions on Information Forensics and Security 6 2 2011 426 437
96. D.S. Yeung, S. Jin, and X. Wang Covariance-matrix modeling and detecting various flooding attacks IEEE Transactions on. Systems, Man and Cybernetics, Part A 37 2 2007 157 169 (Pubitemid 46485241)
97. Ye D, Quan B, Ye Z. P2P distributed intrusion detections by using mobile agents, In: Proceedings of the seventh IEEE/ACIS international conference on computer and information science (ICIS 08). May 2008. p. 259-65.
98. N. Ye, Y. Zhang, and C.M. Borror Robustness of the Markov-chain model for cyber-attack detection IEEE Transactions on Reliability 53 1 2004 116 123
99. Yang Y, Deng F Yang FH. An unsupervised anomaly detection approach using subtractive clustering and hidden Markov model. In: Proceedings of the second international conference in communications and networking in China. Shanghai, China, August 2007. p. 313-6.
100. Zhang Z, Li J, Manikopoulos CN, Jorgenson J, Ucles J. HIDE: a hierarchical network intrusion detection system using statistical preprocessing and neural network classification. In: Proceedings of the 2001 IEEE workshop on information assurance and security. NY, USA, June 2001. p. 85-90.
101. C.V. Zhou, C. Leckie, and S. Karunasekera Decentralized multidimensional alert correlation for collaborative intrusion detection Journal of Network and Computer Applications 2009 1106 1123