Priority sampling for estimation of arbitrary subset sums

Journal of the ACM

Volumn 54, Issue 6, 2007, Pages

  Duffield, Nick ^a   Lund, Carsten ^a   Thorup, Mikkel ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

Reservoir sampling; Sampling without replacement; Subset sum estimation; Weighted sampling

Indexed keywords

RESERVOIR SAMPLING; SAMPLING WITHOUT REPLACEMENT; SUBSET SUM ESTIMATION; WEIGHTED SAMPLING;

INTERNET; OPTIMAL SYSTEMS; PARAMETER ESTIMATION; RANDOM NUMBER GENERATION; SAMPLING; SET THEORY;

STATISTICAL TESTS;

EID: 37049036831     PISSN: 00045411     EISSN: 1557735X     Source Type: Journal    
DOI: 10.1145/1314690.1314696     Document Type: Article

References (28)

1. ADLER, R., FELDMAN, R., AND TAQQU, M. 1998. A Practical Guide to Heavy Tails. Birkhauser.
2. ALON, N., DUFFIELD, N., LUND, C., AND THORUP, M. 2005. Estimating arbitrary subset sums with few probes. In Proceedings of the 24th ACM Symposium on Principles of Database Systems (PODS). ACM, New York, 317-325.
3. ARNOLD, B., AND BALAKRISHNAN, N. 1988. Relations, Bounds and Approximations for Order Statistics. Lecture Notes in Statistics, vol. 53. Springer, New York.
4. BREWER, K., AND HANIF, M. 1983. Sampling with unequal probabilities. Lecture Notes in Statistics, vol. 15. Springer-Verlag, New York.
5. CHAUDHURI, S., MOTWANI, R., AND NARASAYYA, V. 1999. On random, sampling over joins. In Proceedings of the ACM SIGMOD Conference. ACM, New York, 263-274.
6. COHEN, E. 1997. Size-estimation framework with applications to transitive closure and reachability. J. Comput. Syst. Sci. 55, 3, 441-453.
7. COHEN, E., DUFFIELD, N., KAPLAN, H., LUND, C., AND THORUP, M. 2007. Sketching unaggregated data streams for subpopulation-size queries. In Proceedings of the 26th ACM Symposium on Principles of Database Systems (PODS). ACM, New York, 253-262.
8. COHEN, E., AND KAPLAN, H. 2007. Bottom-k sketches: Better and more efficient estimation of aggregates (poster). In Proceedings of the ACM IFIP Conference on Measurement and Modeling of Computer Systems (SIGMETRICS/Performance). ACM, New York, 353-354.
9. CORMEN, T. H., LEISERSON, C. E., RIVEST, R. L., AND STEIN, C. 2001. Introduction to Algorithms, 2nd ed. MIT Press, McGraw-Hill. ISBN 0-262-03293-7, 0-07-013151-1.
10. DAVID, H. 1981. Order Statistics, 2nd ed. Wiley, New York.
11. DUFFIELD, N., LUND, C., AND THORUP, M. 2004. Flow sampling under hard resource constraints. In Proceedings of the ACM IFIP Conference on Measurement and Modeling of Computer Systems (SIGMETRICS/Performance). ACM, New York, 85-96.
12. DUFFIELD, N., LUND, C., AND THORUP, M. 2005a. Learn more, sample less: control of volume and variance in network measurements. IEEE Trans. Inf. Theory 51, 5, 1756-1775.
13. DUFFIELD, N., LUND, C., AND THORUP, M. 2005b. Optimal combination of sampled network measurements. In Proceedings of the ACM SIGCOMM Internet Measurement Conference (IMC). ACM, New York, 91-104.
14. DUFFIELD, N., LUND, C., AND THORUP, M. 2005C. Sampling to estimate arbitrary subset sums. Tech. Rep. cs.DS/0509026, Computing Research Repository (CoRR). http://arxiv.org/abs/cs.DS/0509026.
15. FAN, C., MÜLLER, M., AND REZUCHA, I. 1962. Development of sampling plans by using sequential (item by item) selection techniques and digital computers. J. Amer. Stat. Assoc. 57, 387-402.
16. HELLERSTEIN, J., HAAS, P., AND WANG, H. 1997. Online aggregation. In Proceedings of the ACM SIGMOD Conference. ACM, New York, 171-182.
17. JOHNSON, T., MUTHUKRISHNAN, S., AND ROZENBAUM, I. 2005. Sampling algorithms in a stream operator. In Proceedings of the ACM SIGMOD Conference. ACM, New York, 1-12.
18. KNUTH, D. 1969. The Art of Computer Programming, Vol. 2: Seminumerical Algorithms. Addison-Wesley, Reading, MA.
19. MOORE, D., PAXSON, V., SAVAGE, S., SHANNON, C., STAMFORD, S., AND WEAVER, N. 2003. Inside the slammer worm. IEEE Sec. Priv. Mag. 1, 4, 33-39.
20. MUTHUKRISHNAN, S. 2005. Data streams: Algorithms and applications. Foundat. Trends Theoret. Comput. Sci. 1, 2.
21. PARK, K., KIM, G., AND CROVELLA, M. 1996. On the relationship between file sizes, transport protocols, and self-similar network traffic. In Proceedings of the 4th IEEE International Conference on Network Protocols (ICNP). IEEE Computer Society Press, Los Alamitos, CA.
22. SÄRNDAL, C.-E., SWENSSON, B., AND WRETMAN, J. 1992. Model Assisted Survey Sampling. Springer-Verlag, New York.
23. SUNTER, A. B. 1977. List sequential sampling with equal, or unequal probabilites without replacement. Applied Statistics 26, 261-268.
24. SZEGEDY, M. 2006. The DLT priority sampling is essentially optimal. In Proceedings of the 38th ACM Symposium on the Theory of Computing (STOC). ACM, New York, 150-158.
25. SZEGEDY, M., AND THORUP, M. 2007. On the variance of subset sum estimation. In Proceedings of the 15th European Symposium on Algorithms (ESA). Lecture Notes in Computer Science, vol. 4698. Springer-Verlag, New York, 75-86.
26. THORUP, M. 2006. Confidence intervals for priority sampling. In Proceedings of the ACMIFIP Conference on Measurement and Modeling of Computer Systems (SIGMETRICS/Performance). ACM, New York, 252-253.
27. THORUP, M. 2007. Equivalence between priority queues and sorting. J. ACM 54, 6, Article 28.
28. VITTER, J. 1985. Random sampling with a reservoir. ACM Trans. Math. Softw. 11, 1, 37-57.