Tighter estimation using bottom k sketches

Proceedings of the VLDB Endowment

Volumn 1, Issue 1, 2008, Pages 213-229

  Cohen, Edith ^a   Kaplan, Haim ^b  

^a AT AND T LABS RESEARCH   (United States)

^b TEL AVIV UNIVERSITY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATE QUERY PROCESSING; DISTRIBUTED DATABASE; HORVITZ-THOMPSON ESTIMATOR; MASSIVE DATA SETS; PRIORITY SAMPLINGS; SAMPLING WITHOUT REPLACEMENTS; SYNTHETIC AND REAL DATA; UNBIASED ESTIMATOR;

EID: 70349672710     PISSN: None     EISSN: 21508097     Source Type: Journal    
DOI: 10.14778/1453856.1453884     Document Type: Article

References (37)

1. N. Alon, N. Duffield, M. Thorup, and C. Lund. Estimating arbitrary subset sums with few probes. In Proceedings of the 24th ACM Symposium on Principles of Database Systems, pages 317-325, 2005.
2. K. S. Beyer, P. J. Haas, B. Reinwald, Y. Sismanis, and R. Gemulla. On synopses for distinct-value estimation under multiset operations. In SIGMOD, pages 199-210. ACM, 2007.
3. K. Bharat and A. Z. Broder. Mirror, mirror on the web: A study of host pairs with replicated content. In Proceedings of the 8th International World Wide Web Conference (WWW), pages 501-512, 1999.
4. A. Z. Broder. On the resemblance and containment of documents. In Proceedings of the Compression and Complexity of Sequences, pages 21-29. ACM, 1997.
5. A. Z. Broder. Identifying and ffiltering near-duplicate documents. In Proceedings of the 11th Annual Symposium on Combinatorial Pattern Matching, volume 1848 of LLNCS, pages 1-10. Springer, 2000.
6. E. Cohen. Size-estimation framework with applications to transitive closure and reachability. J. Comput. System Sci., 55:441-453, 1997.
7. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Algorithms and estimators for accurate summarization of Internet traffic. In Proceedings of the 7th ACM SIGCOMM conference on Internet measurement (IMC), 2007.
8. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Sketching unaggregated data streams for subpopulation-size queries. In Proc. of the 2007 ACM Symp. on Principles of Database Systems (PODS 2007). ACM, 2007.
9. E. Cohen, N. Duffield, H. Kaplan, C. Lund, and M. Thorup. Variance optimal sampling based estimation of subset sums. Technical Report cs.DS/0803.0473, Computing Research Repository (CoRR), 2008.
10. E. Cohen, N. Duffield, C. Lund, M. Thorup, and H. Kaplan. Summarization framework for unaggregated data. Submitted, 2008.
11. E. Cohen and H. Kaplan. Efficient estimation algorithms for neighborhood variance and other moments. In Proc. 15th ACM-SIAM Symposium on Discrete Algorithms. ACM-SIAM, 2004.
12. E. Cohen and H. Kaplan. Spatially-decaying aggregation over a network: model and algorithms. In SIGMOD. ACM, 2004.
13. E. Cohen and H. Kaplan. Bottom-k sketches: Better and more efficient estimation of aggregates. In Proceedings of the ACM SIGMETRICS'07 Conference, 2007. poster.
14. E. Cohen and H. Kaplan. Spatially-decaying aggregation over a network: model and algorithms. J. Comput. System Sci., 73:265-288, 2007.
15. E. Cohen and H. Kaplan. Summarizing data using bottom-k sketches. In Proceedings of the ACM PODC'07 Conference, 2007.
16. E. Cohen and H. Kaplan. Estimating aggregates over multiple subsets. Manuscript, 2008.
17. E. Cohen and H. Kaplan. Sketch-based estimation of subpopulation-weight. Technical Report 802.3448, CORR, 2008.
18. E. Cohen and M. Strauss. Maintaining time-decaying stream aggregates. In Proc. of the 2003 ACM Symp. on Principles of Database Systems (PODS 2003). ACM, 2003.
19. E. Cohen, Y.-M. Wang, and G. Suri. When piecewise determinism is almost true. In Proc. Paciffic Rim International Symposium on Fault-Tolerant Systems, pages 66-71, Dec. 1995.
20. T. Dasu, T. Johnson, S. Muthukrishnan, and V. Shkapenyuk. Mining database structure; or, how to build a data quality browser. In SIGMOD Conference, pages 240-251, 2002.
21. N. Duffield, M. Thorup, and C. Lund. Learn more, sample less, control of volume and variance in network measurement. IEEE Transactions in Information Theory, 51(5):1756-1775, 2005.
22. N. Duffield, M. Thorup, and C. Lund. Priority sampling for estimating arbitrary subset sums. J. Assoc. Comput. Mach., 54(6), 2007.
23. P. S. Efraimidis and P. G. Spirakis. Weighted random sampling with a reservoir. Inf. Process. Lett., 97(5):181-185, 2006.
24. P. Gibbons and Y. Matias. New sampling-based summary statistics for improving approximate query answers. In SIGMOD. ACM, 1998.
25. D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a ffinite universe. Journal of the American Statistical Association, 47(260):663-685, 1952.
26. M. Hua, J. Pei, A. W. C. Fu, X. Lin, and H.-F. Leung. Efficiently answering top-k typicality queries on large databases. In Proceedings of the 33rd VLDB Conference, 2007.
27. H. Kaplan and M. Sharir. Randomized incremental constructions of three-dimensional convex hulls and planar voronoi diagrams, and approximate range counting. In SODA '06: Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm, pages 484-493, New York, NY, USA, 2006. ACM Press.
28. D. Mosk-Aoyama and D. Shah. Computing separable functions via gossip. In Proceedings of the ACM PODC'06 Conference, 2006.
29. R. Motwani, E. Cohen, M. Datar, S. Fujiware, A. Gronis, P. Indyk, J. Ullman, and C. Yang. Finding interesting associations without support pruning. IEEE Transactions on Knowledge and Data Engineering, 13:64-78, 2001.
30. The Netix Prize. http://www.netflixprize.com/.
31. Cisco NetFlow. http://www.cisco.com/warp/public/732/Tech/netflow.
32. S. Sampath. Sampling Theory and Methods. CRC press, 2000.
33. D. W. Scott. Multivariate Density Estimation: Theory, Practice and Visualization. John Wiley & Sons, New York, 1992.
34. R. Singh and N. S. Mangat. Elements of survey sampling. Springer-Verlag, New York, 1996.
35. N. T. Spring and D. Wetherall. A protocol-independent technique for eliminating redundant network traffic. In Proceedings of the ACM SIGCOMM'00 Conference. ACM, 2000.
36. M. Szegedy. The DLT priority sampling is essentially optimal. In Proc. 38th Annual ACM Symposium on Theory of Computing. ACM, 2006.
37. M. Thorup. Conffidence intervals for priority sampling. In ACM SIGMETRICS Performance Evaluation Review, 2006.