Comparing data streams using Hamming norms (how to zero in)

IEEE Transactions on Knowledge and Data Engineering

Volumn 15, Issue 3, 2003, Pages 529-540

  Cormode, Graham ^a   Datar, Mayur ^b   Indyk, Piotr ^c   Muthukrishnan, S ^a,d  

^a RUTGERS UNIVERSITY   (United States)

^b Stanford University   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^d AT AND T LABS RESEARCH   (United States)

Author keywords

Approximate query processing; Data reduction; Data stream analysis; Data structures and algorithms

Indexed keywords

APPROXIMATION THEORY; DATA PROCESSING; DATA REDUCTION; INTERNET; ROUTERS;

DATA STREAMS;

DATA STRUCTURES;

EID: 0038633398     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2003.1198388     Document Type: Article

References (44)

1. N. Alon, Y. Matias, and M. Szegedy, "The Space Complexity of Approximating the Frequency Moments," JCSS: J. Computer and System Sciences, vol. 58, pp. 137-147, 1999.
2. Amazon Project, http://www.cs.cornell.edu/database/amazon/amazon.htm, 2002.
3. S. Babu, L. Subramanian, and J. Widom, "A Data Stream Management System for Network Traffic Management," Proc. Workshop Network-Related Data Management, 2001.
4. S. Babu and J. Widom, "Continuous Queries over Data Streams," ACM Sigmod Record, vol. 30, no. 3, pp. 109-120, 2001.
5. Z. Bar-Yossef, T.S. Jayram, R. Kumar, D. Sivakumar, and L. Trevisian, "Counting Distinct Elements in a Data Stream." Proc. RANDOM 2002, pp. 1-10, 2002.
6. Z. Bar-Yossef, R. Kumar, and D. Sivakumar, "Reductions in Streaming Algorithms, with an Application to Counting Triangles in Graphs," Proc. ACM-SIAM Ann. Symp. Discrete Algorithms, pp. 623-632, 2002.
7. J. Bunge and M. Fitzpatrick, "Estimating the Number of Species: A Review," J. Am. Statistical Assoc., vol. 88, no. 421, p. 364, 1993.
8. J.M. Chambers, C.L. Mallows, and B.W. Stuck, "A Method for Simulating Stable Random Variables," J. Am. Statistical Assoc., vol. 71, no. 354, pp. 340-344, 1976.
9. M. Charikar, S. Chaudhuri, R. Motwani, and V.R. Narasayya, "Towards Estimation Error Guarantees for Distinct Values," Proc. 19th Symp. Principles of Database Systems, pp. 268-279, 2000.
10. S. Chaudhuri, R. Motwani, and V. Narasayya, "Random Sampling for Histogram Construction: How Much is Enough?" Proc. ACM SIGMOD Int'l Conf. Management of Data, vol. 27, no. 2, pp. 436-447, 1998.
11. E. Cohen, M. Datar, S. Fujiwara, A. Gionis, P. Indyk, R. Motwani, J. Ullman, and C. Yang, "Finding Interesting Associations without Support Pruning," Proc. 16th Int'l Conf. Data Eng., pp. 489-500, 2000.
12. G. Cormode, P. Indyk, N. Koudas, and S. Muthukrishnan, "Fast Mining of Tabular Data via Approximate Distance Computations," Proc. Int'l Conf. Data Eng., 2002.
13. G. Cormode and S. Muthukrishnan, "Estimating Dominance Norms of Multiple Data Streams," Technical Report 2002-35, DIMACS, 2002.
14. C. Cranor, L. Gao, T. Johnson, and O. Spatscheck, "Gigascope: High Performance Network Monitoring with an SQL Interface," Proc. ACM SIGMOD Int'l Conf. Management of Data, p. 262, 2002.
15. P. Dasu, T. Johnson, S. Muthukrishnan, and V. Shkapenyuk, "Mining Database Structures or How to Build a Data Quality Browser," Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 240-251, 2002.
16. DIMACS Workshop Streaming Data Analysis and Mining; and DIMACS Workshop Sublinear Algorithms, Center for Discrete Mathematics and Computer Science, http://dimacs.rutgers.edu/Workshops/Streaming/, http://dimacs.rutgers.edu/Work shops/Sublinear, 2001.
17. P. Domingos, G. Hulten, and L. Spencer, "Mining Time-Changing Data Streams," Proc. Seventh Int'l Conf. Knowledge Discovery and Data Mining, pp. 97-106, 2001.
18. 1-Difference Algorithm for Massive Data Streams," Proc. 40th Ann. Symp. Foundations of Computer Science, pp. 501-511, 1999.
19. A. Feldmann, A. Greenberg, C. Lund, N. Reingold, and J. Rexford, "Netscope: Traffic Engineering for IP Networks," IEEE Network Magazine, pp. 11-19, 2000.
20. A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and F. True, "Deriving Traffic Demands for Operational IP Networks: Methodology and Experience," IEEE/ACM Trans. Networking, pp. 265-279, 2001.
21. P. Flajolet and G.N. Martin, "Probabilistic Counting," Proc. 24th Ann. Symp. Foundations of Computer Science, pp. 76-82, 1983.
22. P. Flajolet and G.N. Martin, "Probabilistic Counting Algorithms for Database Applications," J. Computer and System Sciences, vol. 31, pp. 182-209, 1985.
23. P. Gibbons, "Distinct Sampling for Highly-Accurate Answers to Distinct Values Queries and Event Reports," Proc. 27th Int'l Conf. Very Large Databases, pp. 541-550, 2001.
24. P. Gibbons and Y. Matias, "Synopsis Structures for Massive Data Sets," DIMACS, Series in Discrete Math. and Theoretical Computer Science, A, 1999.
25. P. Gibbons and S. Tirthapura, "Estimating Simple Functions on the Union of Data Streams," Proc. 13th ACM Symp. Parallel Algorithms and Architectures, pp. 281-290, 2001.
26. A. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss, "QuickSAND: Quick Summary and Analysis of Network Data," Technical Report 2001-43, DIMACS, 2001.
27. A. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss, "Surfing Wavelets on Streams: One-Pass Summaries for Approximate Aggregate Queries," Proc. 27th Int'l Conf. Very Large Data Bases, pp. 79-88, 2001.
28. M. Grossglauser, N. Koudas, Y. Park, and A. Varriot, "Falcon: Fault Management via Alarm Warehousing and Mining," Proc. Workshop Network-Related Data Management, 2001.
29. S. Guha, N. Koudas, and K. Shim, "Data Streams and Histograms," Proc. Symp. Theory of Computing, pp. 471-475, 2001.
30. S. Guha, N. Mishra, R. Motwani, and L. O'Callaghan, "Clustering Data Streams," Proc. 41st Ann. Symp. Foundations of Computer Science, 2000.
31. P.J. Haas, J.F. Naughton, S. Seshadri, and L. Stokes, "Sampling-Based Estimation of the Number of Distinct Values of an Attribute," Proc. 21st Int'l Conf. Very Large Databases, pp. 311-322, 1995.
32. P. Indyk, "Stable Distributions, Pseudorandom Generators, Embeddings and Data Stream Computation," Proc. 40th Symp. Foundations of Computer Science, pp. 189-197, 2000.
33. P. Indyk, N. Koudas, and S. Muthukrishnan, "Identifying Representative Trends in Massive Time Series Data Sets Using Sketches," Proc. 26th Int'l Conf. Very Large Databases, pp. 363-372, 2000.
34. W.B. Johnson and J. Lindenstrauss, "Extensions of Lipshitz Mapping into Hubert Space," Contemporary Math., vol. 26, pp. 189-206, 1984.
35. S. Madden and M.J. Franklin, "Fjording the Stream: An Architecture for Queries over Streaming Sensor Data," Proc. 18th Int'l Conf. Data Eng., 2002.
36. D. Moore, G.M. Voelker, and S. Savage, "Inferring Internet Denial of Service Activity," Proc. Usenix Security Symp., pp. 9-22, 2001.
37. Cisco NetFlow, more details at http://www.cisco.com/warp/public/732/Tech/netflow/, 2002.
38. NOAA, National Oceanic and Atmospheric Administration, US National Weather Service, http://www.nws.noaa.gov/, 2003.
39. J. Nolan Stable Distributions, available from http://academic2.american.edu/̃jpnolan/stable/chapl.pdf, 2002.
40. J. Nolan, personal communication, 2002.
41. W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, second ed., Cambridge Univ. Press, 1992.
42. California PATH Smart-AHS, more details at http://path.berkeley.edu/SMART-AHS/index.html, 1997.
43. N. Thaper, S. Guha, P. Indyk, and N. Koudas, "Multidimensional Dynamic Histograms," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2002.
44. Unidata, Atmospheric Data Repository, http://www.unidata.ucar.edu/, 2003.