Scaling up top-K cosine similarity search

Data and Knowledge Engineering

Volumn 70, Issue 1, 2011, Pages 60-83

  Zhu, Shiwei ^a   Wu, Junjie ^a   Xiong, Hui ^b   Xia, Guoping ^a  

^a BEIHANG UNIVERSITY   (China)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Cosine similarity; Diagonal traversal strategy; Max first traversal strategy; Similarity search

Indexed keywords

APPLICATION DOMAINS; CORRELATED PAIRS; COSINE MEASURE; COSINE SIMILARITY; DATA ALGORITHM; DATA SETS; DIAGONAL TRAVERSAL STRATEGY; K-COSINE; MAX-FIRST TRAVERSAL STRATEGY; MONOTONE PROPERTIES; SCALING-UP; SIMILARITY SEARCH; SIMILARITY THRESHOLD; UPPER BOUND;

ALGORITHMS;

COST BENEFIT ANALYSIS;

EID: 78649795821     PISSN: 0169023X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.datak.2010.08.004     Document Type: Article

References (44)

1. R. Agrawal, T. Imielinski, and A. Swami Mining association rules between sets of items in large databases Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data 1993 207 216 Washington, DC
2. C. Alexander Market Models: A Guide to Financial Data Analysis 2001 John Wiley & Sons
3. A. Arasu, V. Ganti, and R. Kaushik Efficient exact set-similarity joins Proceedings of the 32th International Conference on Very Large Data Bases 2006
4. A. Awekar, N.F. Samatova, and P. Breimyer Incremental all pairs similarity search for varying similarity thresholds Proceedings of the 3rd Workshop on Social Network Mining and Analysis 2009
5. R.J. Bayardo, Y. Ma, and R. Srikant Scaling up all pairs similarity search Proceedings of the 16th International Conference on World Wide Web 2007 131 140
6. J. Blanchard, F. Guillet, R. Gras, and H. Briand Using information-theoretic measures to assess association rule interestingness Proceedings of the Fifth IEEE International Conference on Data Mining 2005 66 73 Houston, TX
7. S. Brin, R. Motwani, and C. Silverstein Beyond market basket: generalizing association rules to correlations Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data 1997 265 276 Tucson, AZ
8. S. Chaudhuri, V. Ganti, and R. Kaushik Efficient set joins on similarity predicates Proceedings of the ACM SIGMOD International Conference on Management of Data 2004
9. S. Chaudhuri, V. Ganti, and R. Kaushik A primitive operator for similarity joins in data cleaning Proceedings of the 22th International Conference on Data Engineering 2006
10. P. Cohen, J. Cohen, S.G. West, and L.S. Aiken Applied Multiple Regression/Correlation Analysis for the Behavioral Science 3rd edn. 2002 Lawrence Erlbaum
11. R. Fagin, R. Kumar, and D. Sivakumar Efficient similarity search and classification via rank aggregation Proceedings of the ACM SIGMOD International Conference on Management of Data 2003
12. A. Gionis, P. Indyk, and R. Motwani Similarity search in high dimensions via hashing Proceedings of the 25th International Conference on Very Large Data Bases 1999
13. J. Han, H. Cheng, D. Xin, and X.F. Yan Frequent pattern mining: current status and future directions Data Mining and Knowledge Discovery 15 1 2007 55 86
14. J. Han, and M. Kamber Data Mining: Concepts and Techniques 2nd edn 2005 Morgan Kaufmann
15. J. Han, J. Pei, and Y. Yin Mining frequent patterns without candidate generation Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data 2000 1 12 Dallas, TX
16. Z. He, X. Xu, and S. Deng Mining top-k strongly correlated item pairs without minimum correlation threshold International Journal of Knowledge-Based and Intelligent Engineering Systems 10 2 2006 57 69
17. R.J. Hilderman, and H.J. Hamilton Knowledge Discovery and Measures of Interest 2001 Kluwer Academic Publishers Norwell, MA, USA
18. I.F. Ilyas, V. Markl, P.J. Haas, P. Brown, and A. Aboulnaga Cords: automatic discovery of correlations and soft functional dependencies Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data 2004 647 658
19. P. Indyk, and R. Motwani Approximate nearest neighbors: towards removing the curse of dimensionality Proceedings of the 30th Symposium on the Theory of Computing 1998
20. J. Keller, M. Popescu, and J.A. Mitchell Taxonomy-based soft similarity measures in bioinformatics Proceedings of the IEEE International Conference on Fuzzy Systems July 2004 23 30
21. J. Ko, L. Si, E. Nyberg, and T. Mitamura Probabilistic models for answer-ranking in multilingual question-answering ACM Transactions on Information Systems 28 3 2010 1 37
22. H. Kpckeö, and E. Rahm Frameworks for entity matching: a comparison Data & Knowledge Engineering 69 2 2010 197 210
23. W.P. Kuo, T.-K. Jensen, A.J. Butte, L. Ohno-Machado, and I.S. Kohane Analysis of matched mrna measurements from two different microarray technologies Bioinformatics 18 2002 405C412
24. Y.-K. Lee, W.-Y. Kim, Y.D. Cai, and J. Han Comine: efficient mining of correlated patterns Proceedings of the Third IEEE International Conference on Data Mining 2003 581 584 Melbourne, FL
25. E. Omiecinski Alternative interestmeasures formining associations IEEE Transactions on Knowledge and Data Engineering 15 2003 57 69
26. G. Piatetsky-Shapiro Notes of aaai'91 workshop knowledge discovery in databases Proceedings of AAAI'91 Knowledge Discovery in Databases Workshop, Anaheim, CA 1991 AAAI/MIT Press
27. G. Pirró, and D. Talia Ufome: an ontology mapping system with strategy prediction capabilities Data & Knowledge Engineering 69 5 2010 444 471
28. M. Sahami, and T. Heilman A web-based kernel function for measuring the similarity of short text snippets Proceedings of the 15th International Conference on World Wide Web 2006 377 386
29. G. Solskinnsbakk, and J.A. Gulla Combining ontological profiles with context in information retrieval Data & Knowledge Engineering 69 3 2010 251 260
30. H.V. Storch, and F.W. Zwiers Statistical Analysis in Climate Research 2002 Cambridge University Press
31. A. Tagarelli, and S. Greco Semantic clustering of xml documents ACM Transactions on Information Systems 28 1 2010 1 56
32. P.-N. Tan, V. Kumar, and J. Srivastava Selecting the right interestingness measure for association patterns Proceedings of the 2002 ACM SIGKDD International Conference on Management of Data 2002 32 41 Edmonton, Canada
33. Pang-Ning Tan, Michael Steinbach, and Vipin Kumar Introduction to Data Mining 2005 Addison-Wesley
34. T. Wu, Y. Chen, and J. Han Association mining in large databases: a re-examination of its measures Proceedings of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases 2007 621 628
35. C. Xiao, W. Wang, X. Lin, and J.X. Yu Efficient similarity joins for near duplicate detection Proceedings of the 17th International World Wide Web Conference 2008
36. H. Xiong, M. Brodie, and S. Ma Top-cop: mining top-k strongly correlated pairs in large databases Proceedings of the Sixth IEEE International Conference on Data Mining 2006
37. H. Xiong, X. He, C.H.Q. Ding, Y. Zhang, V. Kumar, and S.R. Holbrook Identification of functional modules in protein complexes via hyperclique pattern discovery Proceedings of the 10th Pacific Symposium on Biocomputing 2005 221 232
38. H. Xiong, S. Shekhar, P.-N. Tan, and V. Kumar Exploiting a support-based upper bound of pearson's correlation coefficient for efficiently identifying strongly correlated pairs Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2004 334 343
39. H. Xiong, P.-N. Tan, and V. Kumar Hyperclique pattern discovery Data Mining and Knowledge Discovery Journal 13 2 2006 219 242
40. H. Xiong, W.J. Zhou, M. Brodie, and S. Ma Top-k ? correlation computation Informs Journal on Computing 20 4 2008 539 552
41. M.J. Zaki Scalable algorithms for association mining IEEE Transactions on Knowledge and Data Engineering 12 2000 372 390
42. J. Zhang, and J. Feigenbaum Finding highly correlated pairs efficiently with powerful pruning Proceedings of the 15th ACM Conference on Information and Knowledge Management 2006 152 161
43. Y. Zhao, and G. Karypis Criterion functions for document clustering: experiments and analysis Machine Learning 55 3 June 2004 311 331
44. W.J. Zhou, and H. Xiong Volatile correlation computation: a checkpoint view Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2008 848 856