Optimal sampling from sliding windows

Journal of Computer and System Sciences

Volumn 78, Issue 1, 2012, Pages 260-272

  Braverman, Vladimir ^a   Ostrovsky, Rafail ^a   Zaniolo, Carlo ^a  

^a Engineering IV   (United States)

Author keywords

Data streams; Random sampling; Sliding windows

Indexed keywords

DATA STREAMS; FORESTRY;

COMPLEXITY BOUNDS; FUNDAMENTAL TOOLS; RANDOM SAMPLING; SAMPLING WITH REPLACEMENT; SAMPLING WITHOUT REPLACEMENTS; SLIDING WINDOW; SMALL PROBABILITY EVENTS; STREAMING ALGORITHM;

IMPORTANCE SAMPLING;

EID: 81955168067     PISSN: 00220000     EISSN: 10902724     Source Type: Journal    
DOI: 10.1016/j.jcss.2011.04.004     Document Type: Conference Paper

References (62)

1. C. Aggarwal Data Streams: Models and Algorithms 2007 Springer-Verlag
2. C. Aggarwal, On biased reservoir sampling in the presence of stream evolution, in: Proceedings of the 32nd International Conference on Very Large Data Bases, 2006, pp. 607-618.
3. N. Alon, N. Duffield, C. Lund, M. Thorup, Estimating arbitrary subset sums with few probes, in: Proceedings of the 24th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2005, pp. 317-325. (Pubitemid 43275494)
4. N. Alon, Y. Matias, M. Szegedy, The space complexity of approximating the frequency moments, in: Proceedings of the 28th Annual ACM Symposium on Theory of Computing, 1996, pp. 20-29.
5. A. Arasu, B. Babcock, S. Babu, J. Cieslewicz, M. Datar, K. Ito, R. Motwani, U. Srivastava, and J. Widom STREAM: The Stanford Data Stream Management System. Data-Stream Management: Processing High-Speed Data Streams 2005 Springer-Verlag Book chapter
6. A. Arasu, G.S. Manku, Approximate counts and quantiles over sliding windows, in: Proceedings of the 23rd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2004.
7. A.M. Ayad, J.F. Naughton, Static optimization of conjunctive queries with sliding windows over infinite streams, in: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, 2004.
8. B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in: Proceedings of the 21st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2002.
9. B. Babcock, S. Babu, M. Datar, R. Motwani, and D. Thomas Operator scheduling in data stream systems VLDB J. 13 4 2004 333 353 (Pubitemid 39653741)
10. B. Babcock, M. Datar, R. Motwani, Sampling from a moving window over streaming data, in: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, 2002, pp. 633-634.
11. B. Babcock, M. Datar, R. Motwani, Load shedding for aggregation queries over data streams, in: Proceedings of the 20th International Conference on Data Engineering, 2004.
12. B. Babcock, M. Datar, R. Motwani, L. O'Callaghan, Maintaining variance and k-medians over data stream windows, in: Proceedings of the 22nd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2003, pp. 633-634.
13. Z. Bar-Yossef, Sampling lower bounds via information theory, in: STOC, 2003.
14. Z. Bar-Yossef, T.S. Jayram, R. Kumar, D. Sivakumar, An information statistics approach to data stream and communication complexity, in: Proceedings of the 43rd Symposium on Foundations of Computer Science, 2002, pp. 209-218.
15. Z. Bar-Yossef, R. Kumar, D. Sivakumar, Reductions in streaming algorithms, with an application to counting triangles in graphs, in: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, 2002, pp. 623-632.
16. Z. Bar-Yossef, R. Kumar, D. Sivakumar, Sampling algorithms: lower bounds and applications, in: STOC, 2001.
17. L. Bhuvanagiri, S. Ganguly, D. Kesh, C. Saha, Simpler algorithm for estimating frequency moments of data streams, in: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithm, 2006, pp. 708-713. (Pubitemid 43275292)
18. V. Braverman, R. Ostrovsky, Smooth histograms on stream windows, in: Proceedings of the 48th Symposium on Foundations of Computer Science, 2007.
19. L.S. Buriol, G. Frahling, S. Leonardi, A. Marchetti-Spaccamela, C. Sohler, Counting triangles in data streams, in: Proceedings of the 25th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2006, pp. 253-262. (Pubitemid 46946487)
20. A. Chakrabarti, G. Cormode, A. McGregor, A near-optimal algorithm for computing the entropy of a stream, in: Proceedings of ACM-SIAM Symposium on Discrete Algorithms, 2007.
21. A. Chakrabarti, K. Do Ba, S. Muthukrishnan, Estimating entropy and entropy norm on data streams, in: Proceedings of the 23rd International Symposium on Theoretical Aspects of Computer Science, 2006.
22. A. Chakrabarti, S. Khot, X. Sun, Near-optimal lower bounds on the multi-party communication complexity of set-disjointness, in: Proceedings of the 18th Annual IEEE Conference on Computational Complexity, 2003.
23. K. Chaudhuri, N. Mishra, When random sampling preserves privacy, in: CRYPTO, 2006.
24. S. Chaudhuri, R. Motwani, V. Narasayya, On random sampling over joins, in: Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, 1999, pp. 263-274.
25. E. Cohen Size-estimation framework with applications to transitive closure and reachability J. Comput. System Sci. 55 3 1997 441 453 (Pubitemid 127432363)
26. E. Cohen, H. Kaplan, Summarizing data using bottom-k sketches, in: Proceedings of the 26th Annual ACM Symposium on Principles of Distributed Computing, 2007.
27. G. Cormode, S. Muthukrishnan, I. Rozenbaum, Summarizing and mining inverse distributions on data streams via dynamic inverse sampling, in: Proceedings of the 31st International Conference on Very Large Data Bases, 2005.
28. D. Coppersmith, R. Kumar, An improved data stream algorithm for frequency moments, in: Proceedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms, 2004, pp. 151-156.
29. A. Das, J. Gehrke, and M. Riedewald Semantic approximation of data stream joins IEEE Trans. Knowl. Data Eng. 17 1 2005 44 59 (Pubitemid 40536470)
30. lp regression, in: SODA, 2008.
31. M. Datar, A. Gionis, P. Indyk, R. Motwani, Maintaining stream statistics over sliding windows: (extended abstract), in: Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, 2002, pp. 635-644.
32. M. Datar, S. Muthukrishnan, Estimating rarity and similarity over data stream windows, in: Proceedings of the 10th Annual European Symposium on Algorithms, 2002, pp. 323-334.
33. N. Duffield, C. Lund, M. Thorup, Flow sampling under hard resource constraints, in: ACM SIGMETRICS Performance Evaluation Review, vol. 32, no. 1, 2004.
34. J. Feigenbaum, S. Kannan, and J. Zhang Computing diameter in the streaming and sliding-window models Algorithmica 41 2005 25 41
35. J. Feigenbaum, S. Kannan, M. Strauss, and M. Viswanathan Testing and spot-checking of data streams Algorithmica 34 1 2002 67 80
36. G. Frahling, P. Indyk, C. Sohler, Sampling in dynamic data streams and applications, in: Proceedings of the 21st Annual Symposium on Computational Geometry, 2005.
37. S. Ganguly, Estimating frequency moments of update streams using random linear combinations, in: Proceedings of the 8th International Workshop on Randomized Algorithms, 2004, pp. 369-380.
38. S. Gandhi, S. Suri, E. Welzl, Catching elephants with mice: sparse sampling for monitoring sensor networks, in: SenSys, 2007.
39. R. Gemulla, Sampling algorithms for evolving datasets, PhD dissertation.
40. R. Gemulla, W. Lehner, Sampling time-based sliding windows in bounded space, in: Proceedings of the 2008 ACM SIGMOD Intl. Conf. on Management of Data, pp. 379-392.
41. P.B. Gibbons, Y. Matias, New sampling-based summary statistics for improving approximate query answers, in: Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data, 1998, pp. 331-342. (Pubitemid 128655980)
42. P.B. Gibbons, S. Tirthapura, Distributed streams algorithms for sliding windows, in: Proceedings of the 14th Annual ACM Symposium on Parallel Algorithms and Architectures, 2002, pp. 10-13.
43. L. Golab, D. DeHaan, E.D. Demaine, A. Lopez-Ortiz, J.I. Munro, Identifying frequent items in sliding windows over on-line packet streams, in: Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement, 2003.
44. L. Golab, M.T. Özsu, Processing sliding window multi-joins in continuous queries over data streams, in: Proceedings of the 29th International Conference on Very Large Data Bases, 2003, pp. 500-511.
45. S. Guha, A. McGregor, S. Venkatasubramanian, Streaming and sublinear approximation of entropy and information distances, in: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithm, 2006, pp. 733-742. (Pubitemid 43275295)
46. S. Guha, A. Meyerson, N. Mishra, R. Motwani, and L. O'Callaghan Clustering data streams: Theory and practice IEEE Trans. Knowl. Data Eng. 15 2003
47. P.J. Haas Data stream sampling: Basic techniques and results M. Garofalakis, J. Gehrke, R. Rastogi, Data Stream Management: Processing High Speed Data Streams 2011 Springer
48. N. Harvey, J. Nelson, K. Onak, Sketching and streaming entropy via approximation theory, in: The 49th Annual Symposium on Foundations of Computer Science (FOCS 2008).
49. P. Indyk, D. Woodruff, Optimal approximations of the frequency moments of data streams, in: Proceedings of the 37th Annual ACM Symposium on Theory of Computing, 2005, pp. 202-208. (Pubitemid 43098588)
50. M. Kolonko, and D. Wäsch Sequential reservoir sampling with a nonuniform distribution ACM Trans. Math. Software 32 2 2006 257 273 (Pubitemid 44076011)
51. H. Jowhari, M. Ghodsi, New streaming algorithms for counting triangles in graphs, in: Proceedings of the 11th COCOON, 2005, pp. 710-716. (Pubitemid 41450248)
52. L.K. Lee, H.F. Ting, Maintaining significant stream statistics over sliding windows, in: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithm, 2006, pp. 724-732. (Pubitemid 43275294)
53. K. Li Reservoir-sampling algorithms of time complexity O (n (1 + log (N / n))) ACM Trans. Math. Software 20 4 December 1994 481 493
54. J. Li, D. Maier, K. Tufte, V. Papadimos, P.A. Tucker, No pane, no gain: efficient evaluation of sliding-window aggregates over data streams, in: ACM SIGMOD Record, vol. 34, no. 1, 2005.
55. S. Muthukrishnan Data streams: Algorithms and applications Found. Trends Theor. Comput. Sci. 1 2 2005
56. C.R. Palmer, C. Faloutsos, Density biased sampling: an improved method for data mining and clustering, in: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 2000, pp. 82-92.
57. C. Procopiuc, O. Procopiuc, Density estimation for spatial data streams, in: Proceedings of the 9th International Symposium on Spatial and Temporal Databases, 2005, pp. 109-126. (Pubitemid 41433161)
58. M. Szegedy, The DLT priority sampling is essentially optimal, in: Proceedings of the 38th Annual ACM Symposium on Theory of Computing, 2006, pp. 150-158. (Pubitemid 44306548)
59. N. Tatbul, S. Zdonik, Window-aware load shedding for aggregation queries over data streams, in: Proceedings of the 32nd International Conference on Very Large Data Bases, 2006.
60. J.S. Vitter Random sampling with a reservoir ACM Trans. Math. Software 11 1 1985 37 57
61. L. Zhang, Z. Li, M. Yu, Y. Wang, Y. Jiang, Random sampling algorithms for sliding windows over data streams, in: Proceedings of the 11th Joint International Computer Conference, 2005, pp. 572-575.
62. H. Zhao, A. Lall, M. Ogihara, O. Spatscheck, J. Wang, J. Xu, A data streaming algorithm for estimating entropies of od flows, in: Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, 2007.