Scalable nearest neighbor algorithms for high dimensional data

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 36, Issue 11, 2014, Pages 2227-2240

  Muja, Marius ^a   Lowe, David G ^b  

^a BitLit Media Inc   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

algorithm configuration; approximate search; big data; Nearest neighbor search

Indexed keywords

ALGORITHM CONFIGURATIONS; APPROXIMATE SEARCH; HIGH DIMENSIONAL DATA; NEAREST NEIGHBOR ALGORITHM; NEAREST NEIGHBOR SEARCH;

BIG DATA;

EID: 84906339516     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2014.2321376     Document Type: Article

References (65)

1. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
2. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
3. J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proc. IEEE 9th Int. Conf. Comput. Vis., 2003, pp. 1470-1477.
4. J. Hays and A. A. Efros, "Scene completion using millions of photographs," ACM Trans. Graph., vol. 26, p. 4, 2007.
5. G. Shakhnarovich, P. Viola, and T. Darrell, "Fast pose estimation with parameter-sensitive hashing," in Proc. IEEE 9th Int. Conf. Comput. Vis., 2003, pp. 750-757.
6. A. C. Berg, T. L. Berg, and J. Malik, "Shape matching and object recognition using low distortion correspondences," in Proc. IEEE CS Conf. Comput. Vis. Pattern Recog., 2005, vol. 1, pp. 26-33.
7. A. Torralba, R. Fergus, and W.T. Freeman, "80 million tiny images: A large data set for nonparametric object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 11, pp. 1958-1970, Nov. 2008.
8. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei Fei, "ImageNet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2009, pp. 248-255.
9. J. L. Bentley, "Multidimensional binary search trees used for associative searching," Commun. ACM, vol. 18, no. 9, pp. 509-517, 1975.
10. J. H. Friedman, J. L. Bentley, and R. A. Finkel, "An algorithm for finding best matches in logarithmic expected time," ACM Trans. Math. Softw., vol. 3, no. 3, pp. 209-226, 1977.
11. S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, "An optimal algorithm for approximate nearest neighbor searching in fixed dimensions," J. ACM, vol. 45, no. 6, pp. 891-923, 1998.
12. J. S. Beis and D. G. Lowe, "Shape indexing using approximate nearest-neighbour search in high-dimensional spaces," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 1997, pp. 1000-1006.
13. C. Silpa-Anan and R. Hartley, "Optimised KD-trees for fast image descriptor matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
14. M. Muja and D.G. Lowe, "Fast approximate nearest neighbors with automatic algorithm configuration," in Proc. Int. Conf. Computer Vis. Theory Appl., 2009, pp. 331-340.
15. R. F. Sproull, "Refinements to nearest-neighbor searching in kdimensional trees," Algorithmica, vol. 6, no. 1, pp. 579-589, 1991.
16. S. Dasgupta and Y. Freund, "Random projection trees and low dimensional manifolds," in Proc. 40th Annu. ACM Symp. Theory Comput., 2008, pp. 537-546.
17. Y. Jia, J. Wang, G. Zeng, H. Zha, and X. S. Hua, "Optimizing kdtrees for scalable visual descriptor indexing," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3392-3399.
18. K. Fukunaga and P. M. Narendra, "A branch and bound algorithm for computing k-nearest neighbors," IEEE Trans. Comput., vol. C-24, no. 7, pp. 750-753, Jul. 1975.
19. S. Brin, "Near neighbor search in large metric spaces," in Proc. 21th Int. Conf. Very Large Data Bases, 1995, pp. 574-584.
20. A. W. Moore, "The anchors hierarchy: Using the triangle inequality to survive high dimensional data," in Proc. 16th Conf. Uncertainity Artif. Intell., 2000, pp. 397-405.
21. P. N. Yianilos, "Data structures and algorithms for nearest neighbor search in general metric spaces," in Proc. ACM-SIAM Symp. Discrete Algorithms, 1993, pp. 311-321.
22. A. Beygelzimer, S. Kakade, and J. Langford, "Cover trees for nearest neighbor," in Proc. 23rd Int. Conf. Mach. Learning, 2006, pp. 97-104.
23. T. Liu, A. Moore, A. Gray, K. Yang," An investigation of practical approximate nearest neighbor algorithms," presented at the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 2004.
24. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 2161-2168.
25. B. Leibe, K. Mikolajczyk, and B. Schiele, "Efficient clustering and matching for object class recognition," in Proc. British Mach. Vis. Conf., 2006, pp. 789-798.
26. G. Schindler, M. Brown, and R. Szeliski, "City-Scale location recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-7.
27. H. Jegou, M. Douze, and C. Schmid, "Product quantization for nearest neighbor search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 1-15, Jan. 2010.
28. A. Babenko and V. Lempitsky, "The inverted multi-index," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 3069-3076.
29. A. Andoni and P. Indyk, "Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions," Commun. ACM, vol. 51, no. 1, pp. 117-122, 2008.
30. Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li, "Multiprobe LSH: Efficient indexing for high-dimensional similarity search," in Proc. Int. Conf. Very Large Data Bases, 2007, pp. 950-961.
31. M. Bawa, T. Condie, and P. Ganesan, "LSH forest: Self-tuning indexes for similarity search," in Proc. 14th Int. Conf. World Wide Web, 2005, pp. 651-660.
32. Y. Weiss, A. Torralba, and R. Fergus, "Spectral hashing," in Proc. Adv. Neural Inf. Process. Syst., 2008, p. 6.
33. P. Jain, B. Kulis, and K. Grauman, "Fast image search for learned metrics," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
34. B. Kulis and K. Grauman, "Kernelized locality-sensitive hashing for scalable image search," in Proc. IEEE 12th Int. Conf. Comput. Vis., 2009, pp. 2130-2137.
35. B. Kulis and T. Darrell, "Learning to hash with binary reconstructive embeddings," in Proc. 23rd Adv. Neural Inf. Process. Syst., 2009, vol. 22, pp. 1042-1050.
36. M. Raginsky and S. Lazebnik, "Locality-sensitive binary codes from shift-invariant kernels," in Proc. Adv. Neural Inf. Process. Syst., 2009, vol. 22, pp. 1509-1517.
37. J. Wang, S. Kumar, and S. F. Chang, "Semi-supervised hashing for scalable image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3424-3431.
38. J. He, W. Liu, and S. F. Chang, "Scalable similarity search with optimized kernel hashing," in Proc. Int. Conf. Knowledge Discovery Data Mining, 2010, pp. 1129-1138.
39. H. Xu, J. Wang, Z. Li, G. Zeng, S. Li, and N. Yu, "Complementary hashing for approximate nearest neighbor search," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 1631-1638.
40. T. B. Sebastian and B. B. Kimia, "Metric-based shape retrieval in large databases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2002, vol. 3, pp. 291-296.
41. K. Hajebi, Y. Abbasi-Yadkori, H. Shahbazi, and H. Zhang, "Fast approximate nearest-neighbor search with k-nearest neighbor graph," in Proc. 22nd Int. Joint Conf. Artif. Intell., 2011, pp. 1312-1317.
42. J. Wang, J. Wang, G. Zeng, Z. Tu, R. Gan, and S. Li, "Scalable k-NN graph construction for visual descriptors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 1106-1113.
43. J. A. Nelder and R. Mead, "A simplex method for function minimization," Comput. J., vol. 7, no. 4, pp. 308-313, 1965.
44. F. Hutter, "Automated configuration of algorithms for solving hard computational problems," Ph.D. dissertation, Comput. Sci. Dept., Univ. British Columbia, Vancouver, BC, Canada, 2009.
45. F. Hutter, H. H. Hoos, and K. Leyton-Brown, "ParamILS: An automatic algorithm configuration framework," J. Artif. Intell. Res., vol. 36, pp. 267-306, 2009.
46. J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, 2012.
47. M. Muja, "Scalable nearest neighbour methods for high dimensional data," Ph.D. dissertation, Comput. Sci. Dept., Univ. British Columbia, Vancouver, BC, Canada, 2013.
48. D. Arthur and S. Vassilvitskii, "K-Means++: The advantages of careful seeding," in Proc. Symp. Discrete Algorithms, 2007, pp. 1027-1035.
49. M. Calonder, V. Lepetit, C. Strecha, and P. Fua, "BRIEF: Binary robust independent elementary features," in Proc. 11th Eur. Conf. Comput. Vis., 2010, pp. 778-792.
50. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in Proc. IEEE Int. Conf. Comput. Vis., Barcelona, Spain, 2011, pp. 2564-2571.
51. S. Leutenegger, M. Chli, and R. Siegwart, "BRISK: Binary robust invariant scalable keypoints," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 2548-2555.
52. M. Muja and D. G. Lowe, "Fast matching of binary features," in Proc. 9th Conf. Comput. Robot Vis., 2012, pp. 404-410.
53. S. Winder and M. Brown, "Learning local image descriptors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
54. K. Mikolajczyk and J. Matas, "Improving descriptors for fast tree matching by optimal linear projection," in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1-8.
55. J. Hays and A. A. Efros, "IM2GPS: Estimating geographic information from a single image," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
56. A. Halevy, P. Norvig, and F. Pereira, "The unreasonable effectiveness of data," IEEE Intell. Syst., vol. 24, no. 2, pp. 8-12, Mar./Apr. 2009.
57. A. Torralba, R. Fergus, and Y. Weiss, "Small codes and large image databases for recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
58. M. Aly, M. Munich, and P. Perona, "Distributed Kd-trees for retrieval from very large image collections," presented at the British Mach. Vis. Conf., DuDundee, U.K., 2011.
59. M. Muja and D. G. Lowe, "FLANN: Fast library for approximate nearest neighbors," [Online]. Available: http://www.cs.ubc.ca/research/flann
60. M. Cummins and P. Newman, "Highly scalable appearance-only SLAM-FAB-MAP 2.0," presented at the Robotics: Science and Systems Conf., vol. 5, Seattle, Washington, USA, 2009.
61. M. Havlena, A. Torii, M. Jancosek, and T. Pajdla, "Automatic reconstruction of Mars artifacts," in Proc. Eur. Planet. Sci. Congress, 2009, p. 280.
62. M. Havlena, A. Torii, J. Knopp, and T. Pajdla, "Randomized structure from motion based on atomic 3D models from camera triplets," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2009, pp. 2874-2881.
63. M. Quigley, K. Conley, B. Gerkey, J. Faust, T. B. Foote, J. Leibs, R. Wheeler, and A.Y. Ng, "ROS: An open-source robot operating system," in Proc. ICRA Open-Source Softw. Workshop, 2009.
64. P. Turcot and D.G. Lowe, "Better matching with fewer features: The selection of useful features in large database recognition problems," in Proc. Comput. Vis. Workshops, 2009, pp. 2109-2116.
65. G. Bradski and A. Kaehler, Learning OpenCV: Comput. Vision with the OpenCV Library. Sebastopol, CA, USA: O'Reilly Media, 2008