Finding global icebergs over distributed data sets

Proceedings of the ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems

Volumn , Issue , 2006, Pages 298-307

  Zhao, Qi ^a   Ogihara, Mitsunori ^b   Wang, Haixun ^c   Xu, Jun ^a  

^a Georgia Institute of Technology   (United States)

^b University of Rochester   (United States)

^c IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Data streaming; Icebergs; Statistical inference

Indexed keywords

COMMUNICATION OVERHEAD; DATA STREAMING; ICEBERGS; STATISTICAL INFERENCE;

ALGORITHMS; QUERY LANGUAGES; SAMPLING; SERVERS; STATISTICS;

DATA STRUCTURES;

EID: 34250634879     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1142351.1142394     Document Type: Conference Paper

References (29)

1. Akamai technologies inc. http://www.akamai.coni/.
2. Cisco netflow. http://www.cisco.com/warp/public/732/netflow/.
3. Internet2 abilene network, http://abilene.internet2.edu/.
4. N. Alon and J. H. Spencer. The Probabilistic Method. A Wiley-lnterscience Publication, 2000.
5. A. Arasu and G. S. Manku. Approximate quantiles and frequency counts over sliding windows. In Proc. ACM SIGACT-SIQMOD-SIGART Symposium on Principles of Database Systems, 2004.
6. B. Babcock, S. Babu, M. Datar, and R. Motwani. Chain : Operator scheduling for memory minimization in data stream systems. In Proc. ACM SIGMOD, June 2003.
7. B. Babcock and C. Olston. Distributed Top-K monitoring. In Proceedings of ACM SIGMOD international conference on Management of data, 2003.
8. B. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the Association for Computing Machinery, 13(7):422-426, L970.
9. M. Charikar, K. Chen, and M. Farach-Colton. Finding frequent items in data streams. In Proc. 29th ICALP, July 2002.
10. C. Chen and Y. Ling. A sampling-based estimator for top-k query. In Proc. International Conference on Data Engineering (ICDE), 2002.
11. M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Cetintemel, Y. Xing, and S. Zdonik. Scalable distributed stream processing. In Proc. CIDR - Biennial Conference on Innovative Data Systems Research, January 2003.
12. H. Chernoff. A measure of the asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Stat., 23:493-509.
13. S. Cohen and Y. Matias. Spectral bloom filters. In Proc. ACM SIGMOD Conference on Management of Data, 2003.
14. S. Cohen and Y. Matias. Spectral bloom filters. In Proc. ACM SIGMOD Conference on Management of Data, 2003.
15. G. Cormode and S. Muthukrishnan. What's hot and what's not: Tracking most frequent items dynamically. In Proc. ACM PODS, 2003.
16. G. Cormode and S. Muthukrishnan. An improved data stream summary:the count-min sketch and its applications. Journal of Algorithms, April 2005.
17. A. Das, S. Ganguly, M. Garofalakis, and R. Rastogi. Distributed set-expression cardinality estimation. In Proc. VLDB, 2004.
18. E. Demaine, J. Munro, and A. Lopez-Ortiz. Frequency estimation of internet packet streams with limited space. In Proc. European Symposium on Algorithms (ESA). Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, Germany, 2002.
19. L. Fan, P. Cao, J. Almeida, and A.Z. Broder. Summary cache: a scalable wide-area Web cache sharing protocol. IEEE/ACM Transactions on Networking, 8(3):281-293, 2000.
20. M. Fang, N. Shivakumar, H. Garcia-Molina, R. Motwani, and J. Ullman. Computing iceberg queries efficiently. In Proc. the. 24th International Conference on Very Large. Data Bases (VLDB), 1998.
21. J. Feigenbaum and S. Kannan. Streaming algorithms for distributed, massive data sets. In In proceedings of FOCS, 1999.
22. R. Karp, S. Shenker, and C. Papadimitriou. A simple algorithm for finding frequent elements in streams and bags. ACM Transactions on Database Systems (TODS), 28:51-55, 2003.
23. A. Kumar, M. Sung, J. Xu, and J. Wang. Data streaming algorithms for efficient and accurate estimation of flow size distribution. In Proc. ACM SIGMETRICS, June 2004.
24. A. Kumar, J. Xu, J. Wang, O. Spatschek, and L. Li. Space-Code Bloom Filter for Efficient per-flow Traffic Measurement. In Proc. IEEE INFOCOM, March 2004. Extended abstract appeared in Proc. ACM IMC 2003.
25. A. Manjhi, V. Shkapenyuk, K. Dhamdhere, and C. Olston. Finding (recently) frequent items in distributed data streams. In Proc. International Conference on Data Engineering (ICDE), 2005.
26. G. Manku and R. Motwani. Approximate frequency counts over data streams. In Proc. 28th International Conference on Very Large Data Bases (VLDB), 2002.
27. C. Olston, J. Jiang, and J. Widom. Adaptive filters for continuous queries over distributed data streams. In Proc. ACM SIGMOD, June 2003.
28. Y. Wang. Personal communication. Dec 2004.
29. Q. Zhao, M. Ogihara, H. Wang, and J. Xu. Finding global icebergs over distributed data sets. In Technical report, April 2006.