Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices

Performance Evaluation Review

Volumn 33, Issue 1, 2005, Pages 350-361

  Zhao, Qi ^a   Kumar, Abhishek ^a   Wang, Jia ^b   Xu, Jun ^a  

^a Georgia Institute of Technology   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

Data Streaming; Network Measurement; Sampling; Statistical Inference; Traffic Matrix

Indexed keywords

ALGORITHMS; DATA TRANSFER; PROBLEM SOLVING; SAMPLING; STATISTICAL METHODS; TELECOMMUNICATION TRAFFIC;

DATA STREAMING; NETWORK MEASUREMENT; STATISTICAL INFERENCE; TRAFFIC MATRIX;

COMPUTER NETWORKS;

EID: 33244459866     PISSN: 01635999     EISSN: 01635999     Source Type: Conference Proceeding    
DOI: 10.1145/1071690.1064258     Document Type: Conference Paper

References (28)

1. S. Bhattacharyya, C. Diot, J. Jetcheva, and N. Taft. Geographical and temporal characteristics of inter-pop flows: View from a single pop. European Transactions on Telecommunications, 2000.
2. J. Cao, D. Davis, S. Vander Wiel, and B. Yu. Time-varing network tomography:router link data. Journal of American Statistics Association, pages 1063-1075, 2000.
3. J. Carter and M. Wegman. Universal classes of hash functions. Journal of Computer and System Sciences, pages 143-154, 1979.
4. N. Duffield and M. Grossglauser. Trajectory sampling for direct traffic observation. IEEE transaction of Networking, pages 280-292, June 2001.
5. N. Duffield, C. Lund, and M. Thorup. Properties and prediction of flow statistics from sampled packet streams. In Proc. of ACM/SIGCOMM IMW, August 2002.
6. N. Duffield, C. Lund, and M. Thorup. Estimating flow distribution from sampled flow statistics. In Proc. of ACM SIGCOMM, August 2003.
7. C. Estan and G. Varghese. New directions in traffic measurement and a ccounting. In Proc. of ACM SIGCOMM, August 2002.
8. C. Estan and G. Varghese. Bitmap algorithms for counting active flows on high speed links. In Proc. of ACM/SIGCOMM IMC, October 2003.
9. A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and F.True. Deriving traffic demand for operational IP networks: Methodology and experience. In Proc. of ACM SIGCOMM, August 2000.
10. D.A. Huffman. A method for the construction of minimum-redundancy codes. Proc. of I.R.E, pages 1098-1102, 1952.
11. S. Jaiswal, G. Iannaccone, C. Diot, J. Kurose, and D. Towsley. Measurement and classification of out-of-sequence packets in a tier-1 IP backbone. In Proc. of IEEE INFOCOM, March 2003.
12. A. Kumar, M. Sung, J. Xu, and J. Wang. Data streaming algorithms for efficient and accurate estimation of flow size distribution. In Proc. of ACM SIGMETRICS, 2004.
13. A. Kumar, J. Xu, J. Wang, O. Spatschek, and L. Li. Space-code bloom filter for efficient per-flow traffic measurement. In Proc. of IEEE INFOCOM, March 2004.
14. A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot. Traffic matrix estimation:existing techniques and new directions. In Proc. of ACM SIGCOMM, August 2002.
15. http://pma.nlanr.net.
16. A. Nucci, R. Cruz, N. Taft, and C. Diot. Design of IGP link weight changes for estimation of traffic matrices. In Proc. of IEEE INFOCOM, March 2004.
17. A. Pasztor and D. Veitch. PC based precision timing without GPS. In Proc. of ACM SIGMETRICS, June 2002.
18. S. Ramabhadran and G. Varghese. Efficient implementation of a statistics counter architecture. In Proc. of ACM SIGMETRICS, June 2003.
19. M. Ramakrishna, E. Fu, and E. Bahcekapili. Efficient hardware hashing functions for high performance computers. IEEE Transactions on Computers, pages 1378-1381, 1997.
20. http://www.ripe.net.
21. A. Snoeren, C. Partridge, L. Sanchez, C. Jones, F. Tchakountio, S. Kent, and W. Strayer. Hash-based IP traceback. In Proc. of ACM SIGCOMM, August 2001.
22. A. Soule, A. Nucci, R. Cruz, E. Leonardi, and N. Taft. How to identify and estimate the largest traffic matrix elements in a dynamic environment. In Proc. of ACM SIGMETRICS, June 2004.
23. C. Tebaldi and M. West. Bayesian inference on network traffic using link count data. Journal of American Statistics Association, pages 557-576, 1998.
24. Y. Vardi. Internet tomography: estimating source-destination traffic intensities from link data. Journal of American Statistics Association, pages 365-377, 1996.
25. K.Y. Whang, B.T. Vander-zanden, and H.M. Taylor. A linear-time probabilistic counting algorithm for database applications. IEEE transaction of Database Systems, pages 208-229, June 1990.
26. J. Xu, M. Singhal, and Joanne Degroat. A novel cache architecture to support layer-four packet classification at memory access speeds. In Proc. of IEEE INFOCOM, March 2000.
27. Y. Zhang, M. Roughan, N. Duffield, and A. Greenberg. Fast accurate computation of large-scale IP traffic matrices from link loads. In Proc. of ACM SIGMETRICS, June 2003.
28. Y. Zhang, M. Roughan, C. Lund, and D. Donoho. An information-theoretic approach to traffic matrix estimation. In Proc. of ACM SIGCOMM, August 2003.