Is the k-NN classifier in high dimensions affected by the curse of dimensionality?

Computers and Mathematics with Applications

Volumn 65, Issue 10, 2013, Pages 1427-1437

  Pestov, Vladimir ^a  

^a UNIVERSITY OF OTTAWA   (Canada)

Author keywords

Approximate k NN classifier; Borel dimensionality reduction; Nearest neighbour search; The curse of dimensionality

Indexed keywords

CURSE OF DIMENSIONALITY; DIMENSIONALITY REDUCTION; HIGH DIMENSIONAL SPACES; K-NEAREST NEIGHBOURS; K-NN CLASSIFIER; NEAREST NEIGHBOUR SEARCH; NEAREST-NEIGHBOUR CLASSIFIER; STATISTICAL LEARNING;

ALGORITHMS; MATHEMATICAL MODELS;

LEARNING ALGORITHMS;

EID: 84878544778     PISSN: 08981221     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.camwa.2012.09.011     Document Type: Article

References (38)

1. A. Zakai, and Y. Ritov Consistency and localizability J. Mach. Learn. Res. 10 2009 827 856
2. H. Samet Foundations of Multidimensional and Metric Data Structures 2005 Morgan Kaufmann Publishers Inc. San Francisco, CA
3. P. Zezula, G. Amato, V. Dohnal, and M. Batko Similarity Search. The Metric Space Approach 2006 Springer Science + Business Media New York
4. P. Indyk Nearest neighbours in high-dimensional spaces J.E. Goodman, J. O'Rourke, Handbook of Discrete and Computational Geometry 2004 Chapman and Hall, CRC Boca Raton, London, New York, Washington, DC 877 892
5. V. Pestov Indexability, concentration, and VC theory J. Discrete Algorithms 13 2012 2 18 10.1016/j.jda.2011.10.002
6. P. Indyk, R. Motwani, Approximate nearest neighbours: towards removing the curse of dimensionality, in: Proc. 30th ACM Symp. Theory of Computing, 1998, pp. 604-613.
7. E. Kushilevitz, R. Ostrovsky, and Y. Rabani Efficient search for approximate nearest neighbor in high dimensional spaces SIAM J. Comput. 30 2000 457 474
8. C. Stone Consistent nonparametric regression Ann. Statist. 5 1977 595 645
9. G.F. Hughes On the mean accuracy of statistical pattern recognizers IEEE Trans. Inform. Theory 14 1968 55 63
10. A.S. Kechris Classical Descriptive Set Theory 1995 Springer-Verlag
11. Luc Devroye, László Györfi, and Gábor Lugosi A Probabilistic Theory of Pattern Recognition 1996 Springer-Verlag New York
12. F. Cérou, and A. Guyader Nearest neighbor classification in infinite dimension ESAIM Probab. Stat. 10 2006 340 355 (Pubitemid 46054960)
13. D. Preiss Gaussian measures and the density theorem Comment. Math. Univ. Carolin. 22 1981 181 193
14. L. Devroye On the almost everywhere convergence of nonparametric regression function estimates Ann. Statist. 9 1981 1310 1319
15. D. Preiss Dimension of metrics and differentiation of measures General Topology and its Relations to Modern Analysis and Algebra, V Prague, 1981 Sigma Ser. Pure Math. vol. 3 1983 Heldermann Berlin 565 568
16. P. Assouad, and T. Quentin de Gromard Recouvrements, derivation des mesures et dimensions Rev. Mat. Iberoam. 22 2006 893 953
17. E. Chávez, G. Navarro, R. Baeza-Yates, and J.L. Marroquín Searching in metric spaces ACM Comput. Surv. 33 2001 273 321 (Pubitemid 33768485)
18. P. Ciaccia, M. Patella, P. Zezula, M-tree: an efficient access method for similarity search in metric spaces, in: Proc. 23rd Int. Conf. on Very Large Data Bases, VLDB'97, Athens, Greece, 1997, pp. 426-435.
19. V. Pestov, and A. Stojmirović Indexing schemes for similarity search: an illustrated paradigm Fund. Inform. 70 2006 367 385 (Pubitemid 43760897)
20. R. Weber, H.-J. Schek, S. Blott, A quantatitive analysis and performance study for similarity-search methods in high-dimensional spaces, in: Proceedings of the 24-th VLDB Conference, New York, 1998, pp. 194-205.
21. P.B. Miltersen, Cell probe complexity - a survey, in: 19th Conference on the Foundations of Software Technology and Theoretical Computer Science, FSTTCS, Advances in Data Structures Workshop, 1999.
22. O. Barkol, Y. Rabani, Tighter lower bounds for nearest neighbor search and related problems in the cell probe model, in: Proc. 32nd ACM Symp. on the Theory of Computing, 2000, pp. 388-396.
23. M. Pǎtrascu, M. Thorup, Higher lower bounds for near-neighbor and further rich problems, in: Proc. 47th IEEE Symp. on Foundations of Computer Science, 2006, pp. 646-654. (Pubitemid 351175552)
24. I. Volnyansky, V. Pestov, Curse of dimensionality in pivot-based indexes, in: Proc. 2nd Int. Workshop on Similarity Search and Applications, SISAP 2009, Prague, Czech Republic, 2009, pp. 39-46.
25. V. Pestov Lower bounds on performance of metric tree indexing schemes for exact similarity search in high dimensions Algorithmica 2012 21 10.1007/s00453-012-9638-2
26. S.S. Vempala The Random Projection Method DIMACS Series in Discrete Mathematics and Theoretical Computer Science vol. 65 2004 Amer. Math. Soc. Providence, RI
27. T.M. Cover, and P.E. Hart Nearest neighbour pattern classification IEEE Trans. Inform. Theory 13 1967 21 27
28. Rk do not cut all subsets of k + 2 points Adv. Math. 31 1979 305 308
29. M. Vidyasagar Learning and Generalization, with Applications to Neural Networks second ed. 2003 Springer-Verlag
30. K. Beyer, J. Goldstein, R. Ramakrishnan, U. Shaft, When is nearest neighbor meaningful? in: Proc. 7-th Intern. Conf. on Database Theory, ICDT-99, Jerusalem, 1999, pp. 217-235.
31. V. Pestov, Intrinsic dimension of a dataset: what properties does one expect? in: Proc. 20th Int. Joint Conf. on Neural Networks, Orlando, FL, 2007, pp. 1775-1780.
32. V. Pestov An axiomatic approach to intrinsic dimension of a dataset Neural Netw. 21 2-3 2008 204 213
33. M. Gromov Metric Structures for Riemannian and Non-Riemannian Spaces Progress in Mathematics vol. 152 1999 Birkhäuser Verlag
34. K.Q. Weinberger, F. Sha, and L.K. Saul Convex optimizations for distance metric learning and pattern classification IEEE Signal Process. Mag. 27 2010 146 158
35. C. Hegde, M.A. Davenport, M.B. Wakin, R.G. Baraniuk, Efficient machine learning using random projections, in: NIPS Workshop on Efficient Machine Learning, December 2007, p. 2.
36. R. Fraiman, A. Justel, and M. Svarc Pattern recognition via projection-based kNN rules Comput. Statist. Data Anal. 54 2010 1390 1403
37. H. Duan, V. Pestov, V. Singla, A consistent classifier based on approximate nearest neighbours (in preparation).
38. T. Crilly The emergence of topological dimension theory (with the assistance of Dale Johnson) History of Topology 1999 North-Holland Amsterdam 1 24