Order statistics and estimating cardinalities of massive data sets

Discrete Applied Mathematics

Volumn 157, Issue 2, 2009, Pages 406-427

  Giroire, Frédéric ^a,b  

^a INRIA ROCQUENCOURT   (France)

^b MASCOTTE   (France)

Author keywords

Algorithm analysis; Cardinality estimates; Traffic analysis; Very large multisets

Indexed keywords

DATA STORAGE EQUIPMENT;

ALGORITHM ANALYSIS; BINARY REPRESENTATIONS; BIT PATTERNS; CARDINALITIES; CARDINALITY; CARDINALITY ESTIMATES; INTERNAL LOOPS; INTERNET TRAFFICS; MASSIVE DATUMS; MATHEMATICAL ANALYSES; MULTI SETS; NEW CLASSES; ONE PASSES; ORDER STATISTICS; PROCESSING SPEEDS; STANDARD ERRORS; TRAFFIC ANALYSIS; VERY LARGE MULTISETS;

ALGORITHMS;

EID: 56349119757     PISSN: 0166218X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dam.2008.06.020     Document Type: Article

References (20)

1. Bar-Yossef Z., Jayram T.S., Kumar R., Sivakumar D., and Trevisan L. Counting distinct elements in a data stream. RANDOM '02: Proceedings of the 6th International Workshop on Randomization and Approximation Techniques (2002), Springer-Verlag 1-10
2. Broder A. On the resemblance and containment of documents. SEQUENCES '97: Proceedings of the Compression and Complexity of Sequences 1997 (1997), IEEE Computer Society, Washington, DC, USA 21
3. Broder A.Z. Identifying and filtering near-duplicate documents. COM '00: Proceedings of the 11th Annual Symposium on Combinatorial Pattern Matching (2000), Springer-Verlag, London, UK 1-10
4. P. Chassaing, L. Gerin, Efficient estimation of the cardinality of large data sets, in: math.ST/0701347, 2007. Extended abstract in the proceedings of the 4th Colloquium on Mathematics and Computer Science, 2007, pp. 419-422
5. Considine J., Li F., Kollios G., and Byers J. Approximate aggregation techniques for sensor databases. ICDE '04: Proceedings of the 20th International Conference on Data Engineering (2004), IEEE Computer Society, Washington, DC, USA 449
6. Durand M., and Flajolet P. Loglog counting of large cardinalities. In: Di Battista G., and Zwick U. (Eds). Annual European Symposium on Algorithms, ESA03. Lecture Notes in Computer Science vol.~2832 (2003) 605-617
7. Estan C., Varghese G., and Fisk M. Bitmap algorithms for counting active flows on high-speed links. IEEE/ACM Trans. Netw. 14 5 (2006) 925-937
8. Flajolet P. Adaptive sampling. In: Hazewinkel M. (Ed). Encyclopaedia of Mathematics vol.~Suppl.~I (1997), Kluwer Academic Publishers, Dordrecht 28
9. P. Flajolet, E. Fusy, O. Gandouet, F. Meunier, Hyperloglog: The analysis of a near-optimal cardinality estimation algorithm, in: Philippe Jacquet (Ed.) Analysis of Algorithm 2007, AofA07, Discrete Mathematics and Theoretical Computer Science Proceedings, 2007 (in press)
10. Flajolet P., and Martin P.N. Probabilistic counting. Proceedings of the 24th Annual Symposium on Foundations of Computer Science (1983), IEEE Computer Society Press 76-82
11. C. Fraleigh, C. Diot, B. Lyles, S. Moon, P. Owezarski, K. Papagiannaki, F. Tobagi, Design and deployment of a passive monitoring infrastructure, in: Passive and Active Measurement Workshop, Amsterdam, April 2001
12. Fusy É., and Giroire F. Estimating the number of active flows in a data stream over a sliding window. In: Appelgate D. (Ed). Proceedings of the Ninth Workshop on Algorithm Engineering and Experiments and the Fourth Workshop on Analytic Algorithmics and Combinatorics. Proceedings of the New Orleans Conference (2007), SIAM Press 223-231
13. L. Getoor, B. Taskar, D. Koller, Selectivity estimation using probabilistic models, in: SIGMOD Conference, 2001
14. P.B. Gibbons, Distinct sampling for highly-accurate answers to distinct values queries and event reports, VLDB J. (2001) 541-550
15. Giroire F. Order statistics and estimating cardinalities of massive data sets. In: Martnez C. (Ed). International Conference on Analysis of Algorithms, volume AD of DMTCS Proceedings (2005) 157-166 Discr. Math. Theor. Comput. Sci. (2005)
16. F. Giroire, Directions to use probabilistic algorithms for cardinality for DNA analysis, in: Journées Ouvertes Biologie Informatique Mathématiques, JOBIM 2006, pp. 3-5, July 2006
17. F. Giroire, Réseaux, algorithmique et analyse combinatoire de grands ensembles, Ph.D. Thesis, Université Paris VI, November 2006
18. G. Iannaccone, C. Diot, I. Graham, N. McKeown, Monitoring very high speed links, in: ACM SIGCOMM Internet Measurement Workshop, San Francisco, CA, November 2001
19. Knuth D.E. The Art of Computer Programming, Volume~3: Sorting and Searching (1973), Addison-Wesley
20. Whang K.-Y., Zanden B.T.V., and Taylor H.M. A linear-time probabilistic counting algorithm for database applications. TODS 15 2 (1990) 208-229