Summarizing and mining Skewed data streams

Proceedings of the 2005 SIAM International Conference on Data Mining, SDM 2005

Volumn , Issue , 2005, Pages 44-55

  Cormode, Graham ^a   Muthukrishnan, S ^b  

^a RUTGERS UNIVERSITY   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Data mining; Data stream analysis; Heavy hitters; Massive data; Power laws; Zipf distribution

Indexed keywords

DATA STREAM; HEAVY-HITTER; MASSIVE DATA; POWER-LAW; ZIPF DISTRIBUTION;

ALGORITHMS; DATA COMMUNICATION SYSTEMS; METADATA; QUERY PROCESSING;

DATA MINING;

EID: 84880101277     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (55)

1. D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, C. Erwin, E. Galvez, M. Hatoun, A. Maskey, A. Rasin, A. Singer, M. Stonebraker, N. Tatbul, Y. Xing, R. Yan, and S. Zdonik. Aurora: a data stream management system. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages 666-666, 2003.
2. L. Adamic. Zipf, power-law, pareto - a ranking tutorial. http://www.hpl.hp.com/research/idl/papers/ranking/, 2000.
3. N. Alon, P. Gibbons, Y. Matias, and M. Szegedy. Tracking join and self-join sizes in limited storage. In Proceedings of the Eighteenth ACM Symposium on Principles of Database Systems, pages 10-20, 1999.
4. N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency moments. In Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, pages 20-29, 1996.
5. Journal version in Journal of Computer and System Sciences, 58:137-147, 1999.
6. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and issues in data stream systems. In Proceedings of ACM Principles of Database Systems, pages 1-16, 2002.
7. Z. Bar-Yossef, R. Kumar, and D. Sivakumar. Reductions in streaming algorithms, with an application to counting triangles in graphs. In Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 623-632, 2002.
8. J. Baumes, M. Goldberg, M. Magdon-Ismail, and W. Wallace. Discovering hidden groups in communication networks. In 2nd NSF/NIJ Symposium on Intelligence and Security Informatics, pages 126-137, 2004.
9. A. Bestavros, M. Crovella, and T. Taqqu. Heavy-Tailed Probability Distributions in the World Wide Web, pages 3-25. Birkhäuser, 1999.
10. L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and Zipf-like distributions: Evidence and implications. In INFOCOM, pages 126-134, 1999.
11. M. Charikar, K. Chen, and M. Farach-Colton. Finding frequent items in data streams. In Procedings of the International Colloquium on Automata, Languages and Programming (ICALP), pages 693-703, 2002.
12. G. Cormode, F. Korn, S. Muthukrishnan, T. Johnson, O. Spatscheck, and D. Srivastava. Holistic UDAFs at streaming speeds. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages 35-46, 2004.
13. G. Cormode and S. Muthukrishnan. What's hot and what's not: Tracking most frequent items dynamically. In Proceedings of ACM Principles of Database Systems, pages 296-306, 2003.
14. G. Cormode and S. Muthukrishnan. An improved data stream summary: The count-min sketch and its applications. In Latin American Informatics, pages 29-38, 2004.
15. G. Cormode and S. Muthukrishnan. What's new: Finding significant differences in network data streams. In Proceedings of IEEE Infocom, 2004.
16. C. Cranor, T. Johnson, O. Spatscheck, and V. Shkapenyuk. Gigascope: A stream database for network applications. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages 647-651, 2003.
17. A. Dobra, M. Garofalakis, J. E. Gehrke, and R. Rastogi. Processing complex aggregate queries over data streams. In Proceedings of the 2002 ACM Sigmod International Conference on Management of Data, pages 61-72, 2002.
18. C. Estan and G. Varghese. Data streaming in computer networks. In Proceedings of Workshop on Management and Processing of Data Streams, http://www.research.att.com/conf/mpds2003/schedule/estanV.ps, 2003.
19. W. Fei, P. S. Yu, and H. Wang. Mining extremely skewed trading anomalies. In International Conference on Extending Database Technology, pages 801-810, 2004.
20. 1-difference algorithm for massive data streams. In Proceedings of the 40th Annual Symposium on Foundations of Computer Science, pages 501-511, 1999.
21. M. Garofalakis, J. Gehrke, and R. Rastogi. Querying and mining data streams: You only get one look. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2002.
22. A. Gilbert, S. Guha, P. Indyk, Y. Kotidis, S. Muthukrishnan, and M. Strauss. Fast, small-space algorithms for approximate histogram maintenance. In Proceedings of the 34th ACM Symposium on Theory of Computing, pages 389-398, 2002.
23. A. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss. Surfing wavelets on streams: One-pass summaries for approximate aggregate queries. In Proceedings of the International Conference on Very Large Data Bases, pages 79-88, 2001.
24. Journal version in IEEE Transactions on Knowledge and Data Engineering, 15(3):541-554, 2003.
25. A. C. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss. How to summarize the universe: Dynamic maintenance of quantiles. In Proceedings of the International Conference on Very Large Data Bases, pages 454-465, 2002.
26. L. Golab and M. T. Özsu. Issues in data stream management. SIGMOD Record (ACM Special Interest Group on Management of Data), 32(2):5-14, June 2003.
27. M. Henzinger, P. Raghavan, and S. Rajagopalan. Computing on data streams. Technical Report SRC 1998-011, DEC Systems Research Centre, 1998.
28. The himalaya project. http://www.cs.cornell.edu/database/himalaya/ Himalaya.htm.
29. B. Huberman, P. Pirolli, J. Pitkow, and R. Lukose. Strong regularities in world wide web surfing. Science, pages 95-97, April 1998.
30. IBM Research - stream data mining. http://www.research.ibm.com/compsci/ project-spotlight/kdd/.
31. W.B. Johnson and J. Lindenstrauss. Extensions of Lipshitz mapping into Hilbert space. Contemporary Mathematics, 26:189-206, 1984.
32. NASA jet propulsion laboratory. http://www7.nationalacademies.org/bms/ BravermannasPDF.pdf.
33. H. Kargupta and V. Puttagunta. An efficient randomized algorithm for distributed principal component analysis for heterogenous data. In Workshop on high performance data mining at SIAM Intl Conf on Data mining, 2004.
34. H. Kargupta and V. Puttagunta. Onboard vehicle data stream monitoring and fast computation of sparse correlation matrices. In Workshop on data mining in resource constrained environments at SIAM Intl Conf on Data mining, 2004.
35. E. Kohler, J. Li, V. Paxson, and S. Shenker. Observed structure of addresses in IP traffic. In ACM SIGCOMM Internet Measurement Workshop, pages 253-266, 2002.
36. N. Koudas and D. Srivastava. Data stream query processing: A tutorial. In Proceedings of the International Conference on Very Large Data Bases, page 1149, 2003.
37. B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen. Sketch-based change detection: Methods, evaluation and applications. In Proceedings of the ACM SIGCOMM conference on Internet measurement, pages 234-247, 2003.
38. R. Kumar, J. Novak, P. Raghavan, and A. Tomkins. On the bursty evolution of blogspace. In Proceedings of the WWW Conference, pages 568-576, 2003.
39. E. Kushilevitz and N. Nisan. Communication Complexity. Cambridge University Press, 1997.
40. Internet traffic archive. http://ita.ee.lbl.gov/.
41. S. Madden and M. J. Franklin. Fjording the stream: An architecture for queries over streaming sensor data. In Proceedings of 18th International Conference on Data Engineering, pages 555-566, 2002.
42. D. Madigan. DIMACS working group on monitoring message streams. http://stat.rutgers.edu/~madigan/mms/, 2003.
43. MAIDS: Mining alarming incidents in data streams. http://maids.ncsa.uiuc. edu/.
44. G.S. Manku and R. Motwani. Approximate frequency counts over data streams. In Proceedings of the International Conference on Very Large Data Bases, pages 346-357, 2002.
45. R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, 1995.
46. S. Muthukrishnan. Data streams: Algorithms and applications. In Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms, 2003.
47. Ohio State CSE - algorithms for mining data streams. http://www.cse.ohio- state.edu/~agrawal/Research-new/mining.htm.
48. V. Paxson. Empirically derived analytic models of wide-area TCP connections. IEEE ACM Transactions on Networking, 2(4):316-336, 1994.
49. S. Redner. How popular is your paper? An empirical study of the citation distribution. The European Physical Journal B, pages 131-134, 1998.
50. Stanford stream data manager. http://www-db.stanford.edu/stream/sqr.
51. M. Thorup and Y. Zhang. Tabulation based 4-universal hashing with applications to second moment estimation. In Proceedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 615-624, 2004.
52. H. Wang. Bibliography on mining data streams. http://wis.cs.ucla.edu/ ~hxwang/stream/bib.html.
53. D. Woodruff. Optimal space lower bounds for all frequency moments. In Proceedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 167-175, 2004.
54. Y. Zhu and D. Shasha. StatStream: Statistical monitoring of thousands of data streams in real time. In Proceedings of the International Conference on Very Large Data Bases, pages 358-369, 2002.
55. G. Zipf. Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Addison Wesley, 1949.