Efficient and accurate nearest neighbor and closest pair search in high-dimensional space

ACM Transactions on Database Systems

Volumn 35, Issue 3, 2010, Pages

  Tao, Yufei ^a   Yi, Ke ^b   Sheng, Cheng ^c   Kalnis, Panos ^c  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^c KING ABDULLAH UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Saudi Arabia)

Author keywords

Algorithms; Experimentation; Theory

Indexed keywords

ACCESS METHODS; APPROXIMATE METHODS; DATA SET SIZE; EFFECTIVE INDEX; HIGH DIMENSIONAL SPACES; HIGH DIMENSIONALITY; HIGH-DIMENSIONAL; LINEAR SPACES; LOCALITY SENSITIVE HASHING; LOW-DIMENSIONAL SPACES; NEAREST NEIGHBOR SEARCHES; NEAREST NEIGHBORS; ORDER OF MAGNITUDE; ORDERS OF MAGNITUDE; QUADRATIC COMPLEXITY; QUERY COSTS; QUERY DISTRIBUTIONS; QUERY RESULTS; RELATIONAL DATABASE; RUNNING TIME; SUBSTANTIAL REDUCTION;

APPROXIMATION THEORY; DATABASE SYSTEMS; PROBLEM SOLVING;

EXPERIMENTS;

EID: 77955220172     PISSN: 03625915     EISSN: 15574644     Source Type: Journal    
DOI: 10.1145/1806907.1806912     Document Type: Article

References (47)

1. ANDONI, A. AND INDYK, P. 2006. Near-Optimal hashing algorithms for approximate nearest neighbor in high dimensions. In Proceedings of the Annual IEEE Symposium on Foundations of Computer Science (FOCS'06). 459-468
2. ARYA, S., MOUNT, D. M., NETANYAHU, N. S., SILVERMAN, R., AND WU, A. Y. 1998. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM 45, 6, 891-923
3. ATHITSOS, V.,POTAMIAS, M.,PAPAPETROU, P., AND KOLLIOS, G. 2008. Nearest neighbor retrieval using distance-based hashing. In Proceedings of the International Conference on Data Engineering (ICDE'08). 327-336
4. BAWA, M.,CONDIE, T., AND GANESAN, P. 2005. Lsh forest: Self-Tuning indexes for similarity search In Proceedings of the International World Wide Web Conference (WWW'05). 651-660
5. BECKMANN, N., KRIEGEL, H., SCHNEIDER, R., AND SEEGER, B. 1990. The R*-tree: An efficient and robust access method for points and rectangles. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 322-331
6. BENNETT, K. P.,FAYYAD, U., AND GEIGER, D. 1999. Density-Based indexing for approximate nearestneighbor queries. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 233-243
7. BERCHTOLD, S., BOHM, C., JAGADISH, H. V.,KRIEGEL, H.-P., AND SANDER, J. 2000. Independent quantization: An index compression technique for high-dimensional data spaces. In Proceedings of the International Conference on Data Engineering (ICDE'00). 577-588
8. BERCHTOLD, S., KEIM, D. A., KRIEGEL, H.-P., AND SEIDL, T. 2000. Indexing the solution space: A new technique for nearest neighbor search in high-dimensional space. IEEE Trans. Knowl. Data Engin. 12, 1, 45-57
9. BEYER, K. S., GOLDSTEIN, J., RAMAKRISHNAN, R., AND SHAFT, U. 1999. When is "nearest neighbor" meaningful? In Proceedings of the International Conference on Database Theory (ICDT'99). 217-235
10. BOHM, C. 2000. A cost model for query processing in high dimensional data spaces. ACM Trans Datab. Syst. 25, 2, 129-178
11. BREUNIG, M. M., KRIEGEL, H.-P., NG, R. T., AND SANDER, J. 2000. Lof: Identifying density-based local outliers. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 93-104
12. CHARIKAR, M. 2002. Similarity estimation techniques from rounding algorithms. In Proceedings of the Annual ACM Symposium on Theory of Computing (STOC'02)
13. CHAUDHURI, S. AND GRAVANO, L. 1999. Evaluating top-k selection queries. In Proceedings of the International Conference on Very Large Databases (VLDB'99). 397-410
14. CHEN, C.-M. AND LING, Y. 2002. A sampling-based estimator for top-k query. In Proceedings of the Interational Conference on Data Engineering (ICDE'02). 617-627
15. CIACCIA, P. AND PATELLA, M. 2000. Pac nearest neighbor queries: Approximate and controlled search in high-dimensional and metric spaces. In Proceedings of the International Conference on Data Engineering (ICDE'00). 244-255
16. CORRAL, A.,MANOLOPOULOS, Y.,THEODORIDIS, Y., AND VASSILAKOPOULOS,M. 2000. Closest pair queries in spatial databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 189-200
17. DATAR, M., IMMORLICA, N., INDYK, P., AND MIRROKNI, V. S. 2004. Locality-Sensitive hashing scheme based on p-stable distributions. In Proceedings of the Annual ACMSymposium on Computational Geometry (SoCG'04). 253-262
18. FAGIN, R., KUMAR, R., AND SIVAKUMAR, D. 2003. Efficient similarity search and classification via rank aggregation. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 301-312
19. FAGIN, R.,LOTEM, A., AND NAOR,M. 2001. Optimal aggregation algorithms for middleware. In Proceedings of the ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'01). 102-113
20. FERHATOSMANOGLU, H., TUNCEL, E., AGRAWAL, D., AND ABBADI, A. E. 2001. Approximate nearest neighbor searching in multimedia databases. In Proceedings of the International Conference on Data Engineering (ICDE'01). 503-511
21. FERRAGINA, P. AND GROSSI, R. 1999. The String B-tree: A new data structure for string search in external memory and its applications. J. ACM 46, 2, 236-280
22. GAEDE, V. AND GUNTHER, O. 1998. Multidimensional access methods. ACM Comput. Surv. 30, 2, 170-231
23. GIONIS, A., INDYK, P., AND MOTWANI, R. 1999. Similarity search in high dimensions via hashing In Proceedings of the International Conference on Very Large Databases (VLDB'99). 518-529
24. GOLDSTEIN, J. AND RAMAKRISHNAN, R. 2000. Contrast plots and p-sphere trees: Space vs. time in nearest neighbour searches. In Proceedings of the International Conference on Very Large Databases (VLDB'00). 429-440
25. GUTTMAN, A. 1984. R-trees: A dynamic index structure for spatial searching. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 47-57
26. HAR-PELED, S. 2001. A replacement for voronoi diagrams of near linear size. In Proceedings of the Annual IEEE Symposium on Foundations of Computer Science (FOCS'01). 94-103
27. HJALTASON, G. R. AND SAMET, H. 1998. Incremental distance join algorithms for spatial databases In Proceedings of the ACM SIGMOD International Conference on Management of Data. 237-248
28. HJALTASON, G. R. AND SAMET, H. 1999. Distance browsing in spatial databases. ACMTrans. Datab Syst. 24, 2, 265-318
29. HOULE, M. E. AND SAKUMA, J. 2005. Fast approximate similarity search in extremely high- dimensional data sets. In Proceedings of the International Conference on Data Engineering (ICDE'05) 619-630
30. INDYK, P., LEWENSTEIN, M., LIPSKY, O., AND PORAT, E. 2004. Closest pair problems in very high dimensions. In Proceedings of the International Colloquium on Automata, Languages and Programming (ICALP'04). 782-792
31. INDYK, P. AND MOTWANI, R. 1998. Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of the Annual ACM Symposium on Theory of Computing (STOC'98). 604-613
32. JAGADISH, H. V., OOI, B. C., TAN, K.-L., YU, C., AND ZHANG, R. 2005. idistance: An adaptive b+- tree based indexing method for nearest neighbor search. ACM Trans. Datab. Syst. 30, 2, 364-397
33. KLEINBERG, J. M. 1997. Two algorithms for nearest-neighbor search in high dimensions. In Proceedings of the Annual ACM Symposium on Theory of Computing (STOC'97). 599-608
34. KORN, F., PAGEL, B.-U., AND FALOUTSOS, C. 2001. On the 'dimensionality curse' and the 'selfsimilarity blessing'. IEEE Trans. Knowl. Data Engin. 13, 1, 96-111
35. KOUDAS, N., OOI, B. C., SHEN, H. T., AND TUNG, A. K. H. 2004. Ldc: Enabling search by partial distance in a hyper-dimensional space. In Proceedings of the International Conference on Data Engineering (ICDE'04). 6-17
36. KRAUTHGAMER, R. AND LEE, J. R. 2004. Navigating nets: Simple algorithms for proximity search. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'04). 798-807
37. LENHOF, H.-P. AND SMID, M. 1992. Enumerating the k closest pairs optimally. In Proceedings of the Annual IEEE Symposium on Foundations of Computer Science (FOCS'92). 380-386
38. LI, C., CHANG, E. Y., GARCIA-MOLINA, H., AND WIEDERHOLD, G. 2002. Clustering for approximate similarity search in high-dimensional spaces. IEEE Trans. Knowl. Data Engin. 14, 4, 792-808
39. LIN, K.-I., JAGADISH, H. V., AND FALOUTSOS, C. 1994. The tv-tree: An index structure for highdimensional data. VLDB J. 3, 4, 517-542
40. LOPEZ, M. A. AND LIAO, S. 2000. Finding k-closest-pairs efficiently for high dimensional data. In Proceedings of the Canadian Conference on Computational Geometry (CCCG'00). 197-204
41. LV, Q., JOSEPHSON, W.,WANG, Z.,CHARIKAR, M., AND LI, K. 2007. Multi-Probe lsh: Efficient indexing for high-dimensional similarity search. In Proceedings of the International Conference on Very Large Databases (VLDB'07). 950-961
42. PANIGRAHY, R. 2006. Entropy based nearest neighbor search in high dimensions. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'06). 1186-1195
43. ROUSSOPOULOS, N., KELLEY, S., AND VINCENT, F. 1995. Nearest neighbor queries. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 71-79
44. SHAMOS, M. I. AND HOEY, D. 1975. Closest-point problems. In Proceedings of the Annual IEEE Symposium on Foundations of Computer Science (FOCS'75). 151-162
45. TAO, Y.,YI, K., SHENG, C., AND KALNIS, P. 2009. Quality and efficiency in high dimensional nearest neighbor search. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 563-576
46. WEBER, R., SCHEK, H.-J., AND BLOTT, S. 1998. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In Proceedings of the International Conference on Very Large Databases (VLDB'98). 194-205
47. WONG, R. C.-W., TAO, Y., FU, A. W.-C., AND XIAO, X. 2007. On efficient spatial matching. In Proceedings of the International Conference on Very Large Databases (VLDB'07). 579-590.