Non-parametric detection of meaningless distances in high dimensional data

Statistics and Computing

Volumn 22, Issue 2, 2012, Pages 375-385

  Kabán, Ata ^a  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

Chebyshev bound; Curse of dimensionality; Distance concentration; High dimensional data; Nearest neighbour; Statistical test

Indexed keywords

EID: 81955161252     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11222-011-9229-0     Document Type: Article

References (14)

1. Aggarwal, C. C., Hinneburg, A., Keim, D. A.: On the surprising behavior of distance metrics in high dimensional space. In: Proc. ICDT, pp. 420-434 (2001).
2. Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is nearest neighbor meaningful? In: Proc. ICDT, pp. 217-235 (1999).
3. Clarke, R., Ressom, H. W., Wang, A., Xuan, J., Liu, M. C., Gehan, E. A., Wang, Y.: The properties of high-dimensional data spaces: Implications for exploring gene and protein expression data. Nat. Rev., Cancer 8, 37-49 (2008).
4. Durrant, R. J., Kabán, A.: When is 'nearest neighbour' meaningful: a converse theorem and implications. J. Complex. 25(4), 385-397 (2009).
5. François, D., Wertz, V., Verleysen, M.: The concentration of fractional distances. IEEE Trans. Knowl. Data Eng. 19(7), 873-886 (2007).
6. Giannella, C.: New instability results for high dimensional nearest neighbor search. Inf. Process. Lett. 109(19), 1109-1113 (2009).
7. Hsu, C.-M., Chen, M.-S.: On the design and applicability of distance functions in high-dimensional data space. IEEE Transactions on Knowledge and Data Engineering 21(4) (2009).
8. Kabán, A.: On the distance concentration awareness of certain data reduction techniques. Pattern Recognit. 44(2), 265-277 (2011).
9. Kontorovich, L.: Measure concentration of strongly mixing processes with applications. Ph. D. thesis, School of Computer Science, Carnegie Mellon University (2007).
10. Kowalczyk, A.: Classification of anti-learnable biological and synthetic data. In: Proc. PKDD, pp. 176-187 (2007).
11. Pramanik, S., Li, J.: Fast approximate search algorithm for nearest neighbor queries in high dimensions. In: Proc. ICDE, p. 251 (1999).
12. Radovanović, M., Nanopoulos, A., Ivanović, M.: Hubs in space: popular nearest neighbors in high-dimensional data. J. Mach. Learn. Res. 11, 2487-2531 (2010).
13. Saw, J. G., Yang, M. C. K., Mo, T. C.: Chebyshev inequality with estimated mean and variance. Am. Stat. 38(2), 130-132 (1984).
14. Shim, J., Sohn, I., Kim, S., Lee, J.-W., Green, P., Hwang, C.: Selecting marker genes for cancer classification using supervised weighted kernel clustering and the support vector machine. Comput. Stat. Data Anal. 53(5), 1736-1742 (2009).