Optimal sampling in state space models with applications to network monitoring

SIGMETRICS'08: Proceedings of the 2008 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems

Volumn 36, Issue 1 SPECIAL ISSUE, 2008, Pages 145-156

  Singhal, Harsh ^a   Michailidis, George ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Internet traffic matrix estimation; Kalman filtering; Optimal design of experiments; State space models

Indexed keywords

CONDITIONAL MEANS; CONVEX PROGRAMS; EXPERIMENT DESIGNS; INTERNET TRAFFIC MATRIX ESTIMATION; KALMAN FILTERING; NETWORK ENGINEERS; NETWORK FLOWS; NETWORK MONITORING; NETWORK TOMOGRAPHIES; NETWORK TRAFFIC FLOWS; NETWORKING TECHNOLOGIES; NUMERICAL STUDIES; OPTIMAL DESIGN OF EXPERIMENTS; OPTIMAL DESIGNS; OPTIMAL SAMPLING RATES; OPTIMAL SAMPLINGS; RESOURCE CONSTRAINTS; SAMPLED DATUMS; SAMPLING RATES; STATE SPACE MODELS; TRACKING PROBLEMS; TRAFFIC VOLUMES;

AGGLOMERATION; COMPUTER SYSTEMS; CONTROL THEORY; DESIGN OF EXPERIMENTS; DIAGNOSTIC RADIOGRAPHY; INTERNET; INTERNET PROTOCOLS; MEDICAL IMAGING; NUMERICAL ANALYSIS; OPTIMAL SYSTEMS; ROUTERS; TOMOGRAPHY; TRAFFIC SURVEYS;

TRAFFIC CONTROL;

EID: 57349120523     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1375457.1375474     Document Type: Conference Paper

References (27)

1. P. Barford, J. Kline, D. Plonka, and A. Ron. A signal analysis of network traffic anomalies. In IMW '02: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, pages 71-82, New York, NY, USA, 2002. ACM Press.
2. S. Boyd and L. Vandenberglie. Convex Optimization. Cambridge University Press, 2004.
3. J. Cao, D. Davis, S. Wiel, and B. Yu. Time-varying network tomography : Router link data. Journal of the American Statistical Association, 95:1063-75, 2000.
4. A. Chen and J. Cao. Method of one dimensional projections for network tomography. Statistical Inverse Problems, R. Liu, W. Strawderman, C. H. Zhang (editors), IMS Lecture Note Series, 2006.
5. B. Choi and S. Bhattacharrya. On the accuracy and overhead of Cisco sampled netflow. In ACM Sigmetrics Workshop on Large-Scale Network Inference (LSNI), Banff, Canada, June 2005.
6. N. Duffield, C. Lund, and M. Thorup. Properties and prediction of flow statistics from sampled packet streams. In IMW '02: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, pages 159-171, New York, NY, USA, 2002. ACM Press.
7. N. Duffield, C. Lund, and M. Thorup. Flow sampling under hard resource constraints. In SIGMETRICS '04/'Performance '04: Proceedings of the joint international conference on Measurement and, modeling of computer systems, pages 85-96, New York, NY, USA, 2004. ACM.
8. N. Duffield, C. Lund, and M. Thorup. Optimal combination of sampled network measurements. In IMC'05: Proceedings of the Internet Measurement Conference 2005, pages 8-8, Berkeley, CA, USA, 2005. USENIX Association.
9. N. G. Duffield, C. Lund, and M. Thorup. Learn more, sample less: control of volume and variance in network measurement. IEEE Transactions on Information Theory, 51(5):1756-1775, 2005.
10. V. Erramilli, M. Crovella, and N. Taft. An independent-connection model for traffic matrices. In IMC '06: Proceedings of the 6th ACM SIGCOMM on Internet measurement, pages 251-256, New York, NY, USA, 2006. ACM Press.
11. A. C. Harvey. Forecasting, structural time series models and the Kalman filter. Cambridge University Press, 1990.
12. A. Lakhina, M. Crovella, and C. Diot. Diagnosing network-wide traffic anomalies. In SIGCOMM '04: Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, pages 219-230, New York, NY, USA, 2004. ACM Press.
13. E. Lawrence, G. Michailidis, V. Nair, and B. Xi. Network tomograph: a review and recent developments. In Frontiers in Statistics, pages 345-364. Fan and H. Koul (eds), Imperial College Press, London, 2006.
14. J. Lee. Semidefmite programming in experimental design. In H. Wolkowicz, R. Saigal and L. Vandenberghe, editors, "Handbook of Semidefinite Programming", International Series in Operations Research and Management Science, Vol. 27. Kluwer, 2000.
15. G. Liang, N. Taft, and B. Yu. A fast lightweight approach to origin-destination IP traffic estimation using partial measurements. IEEE Transactions on Information Theory, 52(6):2634-2648, 2006.
16. H. Lutkepohl. Introduction to multiple time series analysis. Springer-Verlag, New York, 1993.
17. A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot. Traffic matrix estimation: existing techniques and new directions. In Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications. ACM Press, 2002.
18. L. Peterson and B. Davie. Computer Networks: A Systems Approach. Morgan Kaufmann, San Francisco, 2003.
19. F. Pukelsheim. Optimal Design of Experiments. John Wiley & Sons, Inc., 1993.
20. H. Singhal and G. Michailidis. Identifiability of flow distributions from link measurements with applications to computer networks. Inverse Problems, 23(5):1821-1849, 2007.
21. A. Soule, A. Lakhina, N. Taft, K. Papagiannaki, K. Salamatian, A. Nucci, M. Crovella, and C. Diot. Traffic matrices: Balancing measurements, inference and modeling. In Proceedings of the joint international conference on Measurement and modeling of computer systems. ACM Press, 2005.
22. A. Soule, A. Nucci, R. Cruz, E, Leonardi, and N. Taft. How to identify and estimate the largest traffic matrix elements in a dynamic environment. In SIGMETRICS '04/Performance '04: Proceedings of the joint international conference on Measurement and modeling of computer systems, pages 73-84, New York, NY, USA, 2004. ACM Press.
23. L. Yang and G. Michailidis. Estimation of flow lengths from sampled data. In Proceedings of IEEE GLOBECOM, 2006.
24. L. Yang and G. Michailidis. Sample based estimation of network traffic characteristics. In Proceedings of IEEE INFOCOM, 2007.
25. Y. Zhang, M. Roughan, N. G. Duffield, and A. G. Greenberg. Fast accurate computation of large-scale IP traffic matrices from link loads. In Proceedings of the International Conference on Measurements and Modeling of Computer Systems, SIGMETRICS 2003, pages 206-217, 2003.
26. Q. Zhao. Towards Ideal Network Traffic Measurement: A Statistical Algorithmic Approach. PhD thesis, College of Computing, Georgia Institute of Technology, Atlanta, Georgia, USA, 2007.
27. Q. Zhao. Z. Ge, J. Wang, and J. Xu. Robust traffic matrix estimation with imperfect information: making use of multiple data sources. In SIGMETRICS '06/Performance '06: Proceedings of the joint international conference on Measurement and modeling of computer systems, pages 133-144, New York, NY, USA, 2006. ACM.