Composable, scalable, and accurate weight summarization of unaggregated data sets

Proceedings of the VLDB Endowment

Volumn 2, Issue 1, 2009, Pages 431-442

  Cohen, Edith ^a   Duffield, Nick ^a   Kaplan, Haim ^b   Lund, Carsten ^a   Thorup, Mikkel ^a  

^a AT AND T LABS RESEARCH   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

AGGREGATES; DATA COMMUNICATION SYSTEMS; FLOW MEASUREMENT; SENSOR NETWORKS;

AGGREGATED DATUM; CONTEXT DISTRIBUTION; DISTRIBUTED DATA STREAMS; MEASUREMENTS OF; MULTIPLE SERVERS; RESOURCE CONSTRAINT; SYNTHETIC AND REAL DATA; UNBIASED ESTIMATES;

WEBSITES;

EID: 81255183511     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/1687627.1687677     Document Type: Article

References (32)

1. M. T. Chao. A general purpose unequal probability sampling plan. Biometrika, 69(3):653-656, 1982.
2. S. Chaudhuri, R. Motwani, and V.R. Narasayya. On random sampling over joins. In Proc. ACM SIGMOD Conference, pages 263-274, 1999.
3. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Algorithms and estimators for accurate summarization of Internet traffic. In Proceedings of the 7th ACM SIGCOMM conference on Internet measurement (IMC), 2007.
4. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Algorithms and estimators for accurate summarization of Internet traffic. Manuscript, 2007.
5. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Sketching unaggregated data streams for subpopulation-size queries. In Proc. of the 2007 ACM Symp. on Principles of Database Systems (PODS 2007). ACM, 2007.
6. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Stream sampling for variance-optimal estimation of subset sums. In Proc. 20th ACM-SIAM Symposium on Discrete Algorithms. ACM-SIAM, 2009.
7. E. Cohen, N. Duffield, C. Lund, and M. Thorup. Confident estimation for multistage measurement sampling and aggregation. In ACM SIGMETRICS, 2008. June 2-6, 2008, Annapolis, Maryland, USA.
8. E. Cohen and H. Kaplan. Summarizing data using bottom-k sketches. In Proc. 26th ACM PODC, 2007.
9. E. Cohen and H. Kaplan. Tighter estimation using bottom-k sketches. In Proceedings of the 34th VLDB Conference, 2008.
10. N. Duffield and C. Lund. Predicting resource usage and estimation accuracy in an ip flow measurement collection infrastructure. In ACM SIGCOMM Internet Measurement Workshop, 2003. Miami Beach, Fl, October 27-29, 2003.
11. N. Duffield, M. Thorup, and C. Lund. Priority sampling for estimating arbitrary subset sums. J. Assoc. Comput. Mach., 54(6), 2007.
12. N.G. Duffield, C. Lund, and M. Thorup. Learn more, sample less: control of volume and variance in network measurements. IEEE Transactions on Information Theory, 51(5):1756-1775, 2005.
13. C. Estan and G. Varghese. New directions in traffic measurement and accounting. In Proceedings of the ACM SIGCOMM'02 Conference. ACM, 2002.
14. C.T. Fan, M.E. Muller, and I. Rezucha. Development of sampling plans by using sequential (item by item) selection techniques and digital computers. J. Amer. Stat. Assoc., 57:387-402, 1962.
15. P. Gibbons and Y. Matias. New sampling-based summary statistics for improving approximate query answers. In SIGMOD. ACM, 1998.
16. J. Hájek. Asymptotic theory of rejective sampling with varying probabilities from a finite population. The Annals of Mathematical Statistics, 35(4):1491-1523, 1964.
17. D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260):663-685, 1952.
18. T. Johnson, S. Muthukrishnan, and I. Rozenbaum. Sampling algorithms in a stream operator. In Proc. ACM SIGMOD, pages 1-12, 2005.
19. D.E. Knuth. The Art of Computer Programming, Vol. 2: Seminumerical Algorithms. Addison-Wesley, 1969.
20. D. Moore, V. Paxson, S. Savage, C. Shannon, S. Staniford, and N. Weaver. Inside the slammer worm. IEEE Security and Privacy Magazine, 1(4):33-39, 2003.
21. The Netflix Prize. http://www.netflixprize.com/.
22. Cisco NetFlow. http://www.cisco.com/warp/public/732/Tech/netflow.
23. E. Ohlsson. Sequential poisson sampling. J. Official Statistics, 14(2):149-162, 1998.
24. B. Rosén. Asymptotic theory for successive sampling with varying probabilities without replacement, i. The Annals of Mathematical Statistics, 43(2):373-397, 1972.
25. B. Rosén. Asymptotic theory for order sampling. J. Statistical Planning and Inference, 62(2):135-158, 1997.
26. S. Sampath. Sampling Theory and Methods. CRC press, 2000.
27. R. Singh and N. S. Mangat. Elements of survey sampling. Springer-Verlag, New York, 1996.
28. A. B. Sunter. List sequential sampling with equal or unequal probabilites without replacement. Applied Statistics, 26:261-268, 1977.
29. M. Szegedy and M. Thorup. On the variance of subset sum estimation. In Proc. 15th ESA, LNCS 4698, pages 75-86, 2007.
30. Y. Tillé. An elimination procedure for unequal probability sampling without replacement. Biometrika, 83(1):238-241, 1996.
31. Y. Tillé. Sampling Algorithms. Springer-Verlag, New York, 2006.
32. J.S. Vitter. Random sampling with a reservoir. ACM Trans. Math. Softw., 11(1):37-57, 1985.