Power-law based estimation of set similarity join size

Proceedings of the VLDB Endowment

Volumn 2, Issue 1, 2009, Pages 658-669

  Lee, Hongrae ^a   Ng, Raymond T ^a   Shim, Kyuseok ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b Seoul National University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

COUNTING TECHNIQUES; EFFICIENT ESTIMATION; EXPERIMENTAL EVALUATION; INCLUSION-EXCLUSION; NOVEL TECHNIQUES; POWER LAW RELATIONSHIP; SET-SIMILARITY JOINS; SUCCINCT REPRESENTATION;

EID: 77957718350     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/1687627.1687702     Document Type: Article

References (30)

1. R. Agrawal and R. Srikant. Fast Algorithms for Mining Association Rules. In Proc. VLDB, pages 487-499, 1994.
2. A. Arasu, V. Ganti, and R. Kaushik. Efficient Exact Set-Similarity Joins. In Proc. VLDB, pages 918-929, 2006.
3. R. J. Bayardo, Y. Ma, and R. Srikant. Scaling up all pairs similarity search. In Proc. WWW, pages 131-140, 2007.
4. M. Boley and H. Grosskreutz. A Randomized Approach for Approximating the Number of Frequent Sets. In Proc. ICDM, pages 43-52, 2008.
5. A. Z. Broder. On the Resemblance and Containment of Documents. In Proc. SEQUENCES, pages 21-29, 1997.
6. A. Ceglar and J. F. Roddick. Association mining. ACM Computing Surveys, 38(2), 2006.
7. S. Chaudhuri, V. Ganti, and R. Kaushik. A Primitive Operator for Similarity Joins in Data Cleaning. In Proc. ICDE, page 5, 2006.
8. Z. Chen, F. Korn, N. Koudas, and S. Muthukrishnan. Selectivity Estimation For Boolean Queries. In Proc. PODS, pages 216-225, 2000.
9. K.-T. Chuang, J.-L. Huang, and M.-S. Chen. Power-law relationship and self-similarity in the itemset support distribution: analysis and applications. The VLDB Journal, 17(5):1121-1141, 2008.
10. A. Clauset, C. R. Shalizi, and M. E. J. Newman. Power-law distributions in empirical data. SIAM Review, (to appear) 2009.
11. E. Cohen. Size-estimation framework with applications to transitive closure and reachability. Journal of Computer and System Sciences, 55(3):441-453, 1997.
12. T. Dasu, T. Johnson, S. Muthukrishnan, and V. Shkapenyuk. Mining database structure; or, how to build a data quality browser. In Proc. SIGMOD, pages 240-251, 2002.
13. C. Faloutsos, B. Seeger, A. Traina, and J. Caetano Traina. Spatial join selectivity using power laws. In SIGMOD, pages 177-188, 2000.
14. Ferenc Bodon. Apriori implementation of ferenc bodon. http://www.cs.bme.hu/bodon/en/apriori/.
15. A. Fuxman, E. Fazli, and R. J. Miller. Conquer: Efficient management of inconsistent databases. In Proc. SIGMOD, pages 155-166, 2005.
16. M. Hadjieleftheriou, A. Chandel, N. Koudas, and D. Srivastava. Fast Indexes and Algorithms for Set Similarity Selection Queries. In Proc. ICDE, pages 267-276, 2008.
17. M. Hadjieleftheriou, X. Yu, N. Koudas, and D. Srivastava. Hashed Samples: Selectivity Estimators For Set Similarity Selection Queries. In Proc. VLDB, pages 201-212, 2008.
18. Hans D. Mittelmann. Decision tree for optimization software. http://plato.asu.edu/sub/nonlsq.html.
19. M. A. Herna'ndez and S. J. Stolfo. The merge/purge problem for large databases. In Proc. SIGMOD, pages 127-138, 1995.
20. IBM. Quest synthetic data generation code. http://www.almaden.ibm.com/cs/disciplines/iis/.
21. P. G. Ipeirotis, E. Agichtein, P. Jain, and L. Gravano. To search or to crawl?: towards a query optimizer for text-centric tasks. In Proc. SIGMOD, pages 265-276, 2006.
22. R. Jin, S. McCallen, Y. Breitbart, D. Fuhry, and D. Wang. Estimating the Number of Frequent Itemsets in a Large Database. In Proc. EDBT, pages 505-516, 2009.
23. C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. Society for Industrial Mathematics, 1987.
24. H. Lee, R. T. Ng, and K. Shim. Extending Q-Grams to Estimate Selectivity of String Matching with Edit Distance. In Proc. VLDB, pages 195-206, 2007.
25. H. Lee, R. T. Ng, and K. Shim. Approximate Substring Selectivity Estimation. In Proc. EDBT, pages 827-838, 2009.
26. A. Metwally, D. Agrawal, and E. E. Abbadi. DETECTIVES: DETEcting Coalition hiT Inflation attacks in adVertising nEtworks Streams. In Proc. WWW, pages 241-250, 2007.
27. T. Pitoura and P. Triantafillou. Self-join size estimation in large-scale distributed data systems. In ICDE, pages 764-773, 2008.
28. M. Sahami and T. Heilman. A Web-based Kernel Function for Measuring the Similarity of Short Text Snippets. In Proc. WWW, pages 377-386, 2006.
29. S. Sarawagi and A. Kirpal. Efficient set joins on similarity predicates. In Proc. SIGMOD, pages 743-754, 2004.
30. G. K. Zipf. Human Behavior and the Principle of Least Effort. Addison-Wesley, 1947.