Efficient kNN classification with different numbers of nearest neighbors

IEEE Transactions on Neural Networks and Learning Systems

Volumn 29, Issue 5, 2018, Pages 1774-1785

  Zhang, Shichao ^a   Li, Xuelong ^b   Zong, Ming ^a   Zhu, Xiaofeng ^a   Wang, Ruili ^c  

^a GUANGXI NORMAL UNIVERSITY   (China)

^b XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

^c MASSEY UNIVERSITY   (New Zealand)

Author keywords

Decision tree; K nearest neighbor (KNN) classification; Sparse coding

Indexed keywords

COSTS; DATA MINING; DECISION TREES; SAMPLING; TESTING;

CLASSIFICATION ACCURACY; CLASSIFICATION METHODS; CLASSIFICATION PERFORMANCE; CLASSIFICATION TASKS; CROSS-VALIDATION METHODS; K-NEAREST NEIGHBOR METHOD; K-NN CLASSIFICATIONS; SPARSE RECONSTRUCTION;

NEAREST NEIGHBOR SEARCH;

EID: 85018487453     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2017.2673241     Document Type: Article

References (68)

1. S. Zhang, "Shell-neighbor method and its application in missing data imputation, " Appl. Intell., vol. 35, no. 1, pp. 123-133, 2011.
2. L. Shao, L. Liu, and X. Li, "Feature learning for image classification via multiobjective genetic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 7, pp. 1359-1371, Jul. 2014.
3. D. Tao, X. Tang, X. Li, and X. Wu, "Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1088-1099, Jul. 2006.
4. J. Yu, X. Gao, D. Tao, X. Li, and K. Zhang, "A unified learning framework for single image super-resolution, " IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 4, pp. 780-792, Apr. 2014.
5. Q. Zhu, L. Shao, X. Li, and L. Wang, "Targeting accurate object extraction from an image: A comprehensive study of natural image matting, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 2, pp. 185-207, Feb. 2015.
6. K. Huang, D. Tao, Y. Yuan, X. Li, and T. Tan, "Biologically inspired features for scene classification in video surveillance, " IEEE Trans. Syst., Man, Cybern., Part B, vol. 41, no. 1, pp. 307-313, Feb. 2011.
7. S. Zhang, "Nearest neighbor selection for iteratively KNN imputation, " J. Syst. Softw., vol. 85, no. 11, pp. 2541-2552, 2012.
8. T. Wang, Z. Qin, S. Zhang, and C. Zhang, "Cost-sensitive classification with inadequate labeled data, " Inf. Syst., vol. 37, no. 5, pp. 508-516, 2012.
9. X. Zhu, X. Li, and S. Zhang, "Block-row sparse multiview multilabel learning for image classification, " IEEE Trans. Cybern., vol. 46, no. 2, pp. 450-461, Feb. 2015.
10. X. Wu et al., "Top 10 algorithms in data mining, " Knowl. Inf. Syst., vol. 14, no. 1, pp. 1-37, 2008.
11. K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with multiscale similarity learning, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 10, pp. 1648-1659, Oct. 2013.
12. D. Landgrebe, "A survey of decision tree classifier methodology, " IEEE Trans. Syst., Man Cybern., vol. 21, no. 3, pp. 660-674, May 1991.
13. H. Liu, X. Li, and S. Zhang, "Learning instance correlation functions for multi-label classification, " IEEE Trans. Cybern., vol. 47, no. 2, pp. 499-510, Feb. 2017.
14. S. Zhang, "Parimputation: From imputation and null-imputation to partially imputation, " IEEE Intell. Inform. Bull., vol. 9, no. 1, pp. 32-38, Jan. 2008.
15. G. Góra and A. Wojna, "RIONA: A classifier combining rule induction and k-NN method with automated selection of optimal neighbourhood, " in Proc. ECML, 2002, pp. 111-123.
16. B. Li, Y. W. Chen, and Y. Q. Chen, "The nearest neighbor algorithm of local probability centers, " IEEE Trans. Syst., Man B, vol. 38, no. 1, pp. 141-154, Feb. 2008.
17. X. Zhu, H.-I. Suk, and D. Shen, "Multi-modality canonical feature selection for Alzheimer's disease diagnosis, " in Proc. MICCAI, 2014, pp. 162-169.
18. J. Wang, P. Neskovic, and L. N. Cooper, "Neighborhood size selection in the k-nearest-neighbor rule using statistical confidence, " Pattern Recognit., vol. 39, no. 3, pp. 417-423, 2006.
19. U. Lall and A. Sharma, "A nearest neighbor bootstrap for resampling hydrologic time series, " Water Resour. Res., vol. 32, no. 3, pp. 679-693, 1996.
20. D. Cheng, S. Zhang, Z. Deng, Y. Zhu, and M. Zong, "KNN algorithm with data-driven k value, " in Proc. ADMA, 2014, pp. 499-512.
21. X. Zhu, S. Zhang, Z. Jin, Z. Zhang, and Z. Xu, "Missing value estimation for mixed-attribute data sets, " IEEE Trans. Knowl. Data Eng., vol. 23, no. 1, pp. 110-121, Jan. 2011.
22. H. A. Fayed and A. F. Atiya, "A novel template reduction approach for the K-nearest neighbor method, " IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 890-896, May 2009.
23. X. Zhu, Z. Huang, H. T. Shen, and X. Zhao, "Linear cross-modal hashing for efficient multimedia search, " in Proc. ACM MM, 2013, pp. 143-152.
24. H. Wang, "Nearest neighbors by neighborhood counting, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 942-953, Jun. 2006.
25. Q. Liu and C. Liu, "A novel locally linear KNN method with applications to visual recognition, " IEEE Trans. Neural Netw. Learn. Syst., to be published.
26. X. Zhu, S. Zhang, J. Zhang, and C. Zhang, "Cost-sensitive imputing missing values with ordering, " in Proc. AAAI, 2007, pp. 1922-1923.
27. J. Hou, H. Gao, Q. Xia, and N. Qi, "Feature combination and the kNN framework in object classification, " IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1368-1378, Jun. 2016.
28. S. Zhang, X. Li, M. Zong, X. Zhu, and D. Cheng, "Learning k for kNN classification, " ACM Trans. Intell. Syst. Technol., vol. 8, no. 3, pp. 1-19, 2017.
29. D. Tao, J. Cheng, X. Gao, X. Li, and C. Deng, "Robust sparse coding for mobile image labeling on the cloud, " IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 1, pp. 62-72, Jan. 2017.
30. S. Zhang, M. Zong, K. Sun, Y. Liu, and D. Cheng, "Efficient kNN algorithm based on graph sparse reconstruction, " in Proc. ADMA, 2014, pp. 356-369.
31. X. Zhu, L. Zhang, and Z. Huang, "A sparse embedding and least variance encoding approach to hashing, " IEEE Trans. Image Process., vol. 23, no. 9, pp. 3737-3750, Sep. 2014.
32. B. Li, S. Yu, and Q. Lu. (2003). "An improved k-nearest neighbor algorithm for text categorization." [Online]. Available: https://arxiv.org/abs/cs/0306099.
33. F. Sahigara, D. Ballabio, R. Todeschini, and V. Consonni, "Assessing the validity of QSARS for ready biodegradability of chemicals: An applicability domain perspective, " Current Comput. Aided Drug Design, vol. 10, no. 2, pp. 137-147, 2013.
34. X. Zhu, Z. Huang, Y. Yang, H. T. Shen, C. Xu, and J. Luo, "Selftaught dimensionality reduction on the high-dimensional small-sized data, " Pattern Recognit., vol. 46, no. 1, pp. 215-229, 2013.
35. E. Blanzieri and F. Melgani, "Nearest neighbor classification of remote sensing images with the maximal margin principle, " IEEE Trans. Geosci. Remote Sens., vol. 46, no. 6, pp. 1804-1811, Jun. 2008.
36. X. Zhu, X. Li, S. Zhang, C. Ju, and X. Wu, "Robust joint graph sparse coding for unsupervised spectral feature selection, " IEEE Trans. Neural Netw. Learn. Syst., to be published.
37. H. Liu and S. Zhang, "Noisy data elimination using mutual k-nearest neighbor for classification mining, " J. Syst. Softw., vol. 85, no. 5, pp. 1067-1074, 2012.
38. K. S. Ni and T. Q. Nguyen, "An adaptable-nearest neighbors algorithm for MMSE image interpolation, " IEEE Trans. Image Process., vol. 18, no. 9, pp. 1976-1987, Mar. 2009.
39. Y. Pang, Z. Ji, P. Jing, and X. Li, "Ranking graph embedding for learning to rerank, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 8, pp. 1292-1303, Aug. 2013.
40. T. Mary-Huard and S. Robin, "Tailored aggregation for classification, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 2098-2105, Nov. 2009.
41. Y. Qin, S. Zhang, X. Zhu, J. Zhang, and C. Zhang, "Semi-parametric optimization for missing data imputation, " Appl. Intell., vol. 27, no. 1, pp. 79-88, 2007.
42. C. Zhang, Y. Qin, X. Zhu, and J. Zhang, "Clustering-based missing value imputation for data preprocessing, " in Proc. IEEE Int. Conf., Aug. 2006, pp. 1081-1086.
43. X. Zhu, H.-I. Suk, and D. Shen, "A novel multi-relation regularization method for regression and classification in ad diagnosis, " in Proc. MICCAI, 2014, pp. 401-408.
44. Y. Song, J. Huang, D. Zhou, H. Zha, and C. L. Giles, "IKNN: Informative K-nearest neighbor pattern classification, " in Knowledge Discovery in Databases. Berlin, Germany: Springer, 2007, pp. 248-264.
45. P. Vincent and Y. Bengio, "K-local hyperplane and convex distance nearest neighbor algorithms, " in Proc. NIPS, 2001, pp. 985-992.
46. Z. Don, W. Liang, Y. Wu, M. Pei, and Y. Jia, "Nonnegative correlation coding for image classification, " Sci. China Inf. Sci., vol. 59, no. 1, pp. 1-14, 2016.
47. D.-Y. Liu, H.-L. Chen, B. Yang, X.-E. Lv, L.-N. Li, and J. Liu, "Design of an enhanced fuzzy k-nearest neighbor classifier based computer aided diagnostic system for thyroid disease, " J. Med. Syst., vol. 36, no. 5, pp. 3243-3254, 2012.
48. J. Gou, T. Xiong, and Y. Kuang, "A novel weighted voting for k-nearest neighbor rule, " J. Comput., vol. 6, no. 5, pp. 833-840, 2011.
49. V. Premachandran and R. Kakarala, "Consensus of k-NNs for robust neighborhood selection on graph-based manifolds, " in Proc. CVPR, Jun. 2013, pp. 1594-1601.
50. G. Guo, H. Wang, H. D. Bell, Y. Bi, and K. Greer, "KNN model-based approach in classification, " in On The Move to Meaningful Internet Systems: CoopIS, DOA, and ODBASE (Lecture Notes in Computer Science), vol. 2888, R. Meersman, Z. Tari, and D. C. Schmidt, Eds. Berlin, Germany: Springer, 2003, pp. 986-996.
51. S. Manocha and M. Girolami, "An empirical analysis of the probabilistic k-nearest neighbour classifier, " Pattern Recognit. Lett., vol. 28, no. 13, pp. 1818-1824, 2007.
52. S. Sun and R. Huang, "An adaptive k-nearest neighbor algorithm, " in Proc. FSKD, vol. 1. 2010, pp. 91-94.
53. X. Zhu, H.-I. Suk, and D. Shen, "Matrix-similarity based loss function and feature selection for Alzheimer's disease diagnosis, " in Proc. CVPR, 2014, pp. 3089-3096.
54. J. Zhang, M. Wang, S. Zhang, X. Li, and X. Wu, "Spatiochromatic context modeling for color saliency analysis, " IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1177-1189, Jun. 2016.
55. X. Li, Y. Pang, and Y. Yuan, "L1-norm-based 2DPCA, " IEEE Trans. Syst., Man, Cybern. B, vol. 40, no. 4, pp. 1170-1175, Aug. 2010.
56. X. Zhu, Z. Huang, H. Cheng, J. Cui, and H. T. Shen, "Sparse hashing for fast multimedia search, " ACM Trans. Inf. Syst., vol. 31, no. 2, p. 9, 2013.
57. X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang, "Face recognition using Laplacianfaces, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, pp. 328-340, Mar. 2005.
58. X. He and P. Niyogi, "Locality preserving projections, " in Proc. Neural Inf. Process. Syst., vol. 16. 2004, p. 153.
59. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
60. H. Li and J. Sun, "Majority voting combination of multiple case-based reasoning for financial distress prediction, " Expert Syst. Appl., vol. 36, no. 3, pp. 4363-4373, 2009.
61. A. Bahety, "Extension and Evaluation of ID3-Decision Tree Algorithm, " Entropy, vol. 2, no. 1, p. 1, 2014.
62. Z. H. Zhou and Y. Yu, "Ensembling local learners throughmultimodal perturbation, " IEEE Trans. Syst. Man B, vol. 35, no. 4, pp. 725-735, Apr. 2005.
63. Z. H. Zhou, Ensemble Methods: Foundations and Algorithms. London, U.K.: Chapman & Hall, 2012.
64. Z. Deng, X. Zhu, D. Cheng, M. Zong, and S. Zhang, "Efficient kNN classification algorithm for big data, " Neurocomputing, vol. 195, pp. 143-148, Jun. 2016.
65. K. Tsuda, M. Kawanabe, and K.-R. Müller, "Clustering with the fisher score, " in Proc. NIPS, 2002, pp. 729-736.
66. X. Zhu, H.-I. Suk, and D. Shen, "A novel matrix-similarity based loss function for joint regression and classification in AD diagnosis, " NeuroImage, vol. 100, pp. 91-105, Oct. 2014.
67. Y. Pang, Y. Yuan, and X. Li, "Effective Feature Extraction in High- Dimensional Space, " IEEE Trans. Syst., Man B, vol. 38, no. 6, pp. 1652-1656, Dec. 2008.
68. S. Zhang and X. Wu, "Large scale data mining based on data partitioning, " Appl. Artif. Intell., vol. 15, no. 2, pp. 129-139, 2001.