Fuzzy nearest neighbor algorithms: Taxonomy, experimental analysis and prospects

Information Sciences

Volumn 260, Issue , 2014, Pages 98-119

  Derrac, Joaquín ^a,b   García, Salvador ^c   Herrera, Francisco ^b,d  

^a CARDIFF UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF GRANADA   (Spain)

^c UNIVERSITY OF JAÉN   (Spain)

^d KING ABDULAZIZ UNIVERSITY   (Saudi Arabia)

Author keywords

Classification; Experimental framework; Fuzzy nearest neighbor; Nearest neighbor; Taxonomy

Indexed keywords

EXPERIMENTAL FRAMEWORK; FUZZY NEAREST NEIGHBOR ALGORITHM; K-NEAREST NEIGHBOR CLASSIFIER; MACHINE LEARNING TECHNIQUES; NEAREST NEIGHBOR ALGORITHM; NEAREST NEIGHBOR CLASSIFICATION; NEAREST NEIGHBOR CLASSIFIER; NEAREST NEIGHBORS;

ALGORITHMS; CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS; ROUGH SET THEORY; TAXONOMIES;

FUZZY SETS;

EID: 84891736888     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2013.10.038     Document Type: Article

References (110)

1. D.W. Aha, D. Kibler, and M.K. Albert Instance-based learning algorithms Machine Learning 6 1991 37 66
2. J. Alcalá-Fdez, A. Fernández, J. Luengo, J. Derrac, S. García, L. Sánchez, and F. Herrera Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, Journal of Multiple-Valued Logic and Soft Computing 17 2011
3. J. Alcalá-Fdez, L. Sánchez, S. García, M.J. del Jesus, S. Ventura, J.M. Garrell, J. Otero, C. Romero, J. Bacardit, V.M. Rivas, J.C. Fernández, and F. Herrera KEEL: a software tool to assess evolutionary algorithms for data mining problems Soft Computing 13 2008 307 318
4. E. Alpaydin Introduction to Machine Learning second ed. 2010 The MIT Press
5. A. Andoni, and P. Indyk Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions Communications of the ACM 51 2008
6. M. Arif, M.U. Akram, F.A. Afsar, and F.A.A. Minhas Pruned fuzzy k-nearest neighbor classifier for beat classification Journal of Biomedical Science and Engineering 3 2010
7. K. Atanassov Intuitionistic fuzzy sets Fuzzy Sets and Systems 20 1986 87 96
8. J. Bedzek, and P. Castelaz Prototype classification and feature selection with fuzzy sets IEEE Transactions on Systems, Man, and Cybernetics 7 1977 87 92
9. J. Bedzek, and S.K. Chuah Generalized k-nearest neighbor rules Fuzzy Sets and Systems 18 1986 237 256
10. A. Ben-David A lot of randomness is hiding in accuracy Engineering Applications of Artificial Intelligence 20 2007 875 885
11. M. Béreau, and B. Dubuisson A fuzzy extended k-nearest neighbors rule Fuzzy Sets and Systems 44 1991 17 32
12. H. Bian, L. Mazlack, Fuzzy-rough nearest neighbor classification approach, in: Proceedings of the 22th International Conference of the North American Fuzzy Information Processing Society (NAFIPS'03), Chicago, Illinois, USA, July 24-26, pp. 500-505.
13. D. Cabello, S. Barro, J. Salceda, R. Ruiz, and J. Mira Fuzzy k-nearest neighbor classifiers for ventricular arrhytmia detection International Journal of Biomedical Computing 27 1991 77 93
14. O. Castillo, P. Melin, E. Ramírez, and J. Soria Hybrid intelligent system for cardiac arrhythmia classification with fuzzy k-nearest neighbors and neural networks combined with a fuzzy system Expert Systems with Applications 39 2012 2947 2955
15. H.L. Chen, B. Yang, G. Wang, J. Liu, X. Xu, S. Wang, and D. Liu A novel bankruptcy prediction model based on an adaptive fuzzy k-nearest neighbor method Knowledge Based Systems 24 2011 1348 1359
16. Y. Chen, E.K. Garcia, M.R. Gupta, A. Rahimi, and L. Cazzanti Similarity-based classification: concepts and algorithms Journal of Machine Learning Research 10 2009 747 776
17. M.A. Chikh, M. Saidi, and N. Settouti Diagnosis of diabetes diseases using an artificial immune recognition system2 (AIRS2) with fuzzy k-nearest neighbor Journal of medical systems 36 2012 2721 2729
18. T.W. Chua, K. Leman, N.T. Pham, Human action recognition via sum-rule fusion of fuzzy k-nearest neighbor classifiers, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), Taipei, Taiwan, June 27-30, pp. 484-489.
19. T.W. Chua, W.W. Tan, A new fuzzy rule-based initialization method for k-nearest neighbor classifier, in: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE'09), Jeju island, Korea, August 20-24, pp. 415-420.
20. F. Chung-Hoon, C. Hwang, An interval type-2 fuzzy k-nearest neighbor, in: Proceedings of the 12th IEEE International Conference on Fuzzy Systems (IEEE FUZZY'03), St. Louis, Missouri, USA, May 25-28, pp. 802-807.
21. J. Cohen A coefficient of agreement for nominal scales Educational and Psychological Measurement 20 1960 37 46
22. C. Cornelis, R. Jensen, G. Hurtado, and D. Slezak Attribute selection with fuzzy decision reducts Information Sciences 180 2010 209 224
23. T.M. Cover, and P.E. Hart Nearest neighbor pattern classification IEEE Transactions on Information Theory 13 1967 21 27
24. M. De Cock, C. Cornelis, and E.E. Kerre Fuzzy rough sets: the forgotten step IEEE Transactions on Fuzzy Systems 15 2007 121 130
25. T. Denoeux A k-nearest neighbor classification rule based on Dempster-Shafer theory IEEE Transactions on Systems, Man, and Cybernetics 25 1995 804 813
26. J. Derrac, C. Cornelis, S. García, and F. Herrera Enhancing evolutionary instance selection algorithms by means of fuzzy rough set based feature selection Information Sciences 186 2012 73 92
27. S. Destercke A k-nearest neighbours method based on imprecise probabilities Soft Computing 16 2012 833 844
28. E. Fix, J. Hodges, Discriminatory analysis, nonparametric discrimination: Consistency properties, Technical Report 4, USAF School of Aviation Medicine, 1951.
29. A. Frank, A. Asuncion, UCI Machine Learning Repository, 2010.
30. M. Friedman The use of ranks to avoid the assumption of normality implicit in the analysis of variance Journal of the American Statistical Association 32 1937 674 701
31. S. García, J. Derrac, J.R. Cano, and F. Herrera Prototype selection for nearest neighbor classification: taxonomy and empirical study IEEE Transactions on Pattern Analysis and Machine Intelligence 34 2012 417 435
32. S. García, A. Fernández, J. Luengo, and F. Herrera A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability Soft Computing 13 2009 959 977
33. S. García, A. Fernández, J. Luengo, and F. Herrera Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power Information Sciences 180 2010 2044 2064
34. S. García, and F. Herrera An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons Journal of Machine Learning Research 9 2008 2677 2694
35. N. Gayar, F. Schwenker, G. Palm, A study of the robustness of kNN classifiers trained using soft labels, in: Proceedings of the Second IAPR Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR 2006), Ulm, Germany, August 31-September 2, pp. 67-80.
36. V. Gesu, G. Bosco, Combining one class fuzzy kNN's, in: 7th International Workshop on Fuzzy Logic and Applications (WILF 2007), Camogli, Italy, July 7-10, pp. 152-160.
37. P. Grother, G.T. Candela, and J.L. Blue Fast implementations of nearest-neighbor classifiers Pattern Recognition 30 1997 459 465
38. S. Hadjitodorov An intuitionistic fuzzy sets application to the k-NN method Notes on Intuitionistic Fuzzy Sets 1 1995 66 69
39. S. Hadjitodorov An intuitionistic fuzzy version of the nearest prototype classification method, based on a moving-of-pattern procedure International Journal of General Systems 30 2000 155 165
40. J.H. Han, Y.K. Kim, A fuzzy k-NN algorithm using weights from the variance of membership values, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE CVPR'99), Ft. Collins, Colorado, USA, June 23-25, pp. 394-399.
41. P.E. Hart The condensed nearest neighbor rule IEEE Transactions on Information Theory 18 1968 515 516
42. L. Hartert, M.S. Mouchaweh, and P. Billaudel A semi-supervised dynamic version of fuzzy k-nearest neighbours to monitor evolving systems Evolving Systems 1 2010 3 15
43. T. Hastie, and R. Tibshirani Discriminant adaptive nearest neighbor classification IEEE Transactions on Pattern Analysis and Machine Intelligence 18 1996 607 616
44. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning: Data Mining, Inference and Prediction seciond ed. 2009 Springer New York, Berlin, Heidelberg
45. M. Hayat, and A. Khan Discriminating outer membrane proteins with fuzzy k-nearest neighbor algorithms based on the general form of chou's PseAAC Protein and Peptide Letters 19 2012 411 421
46. X. Hu, and C. Xie Improving fuzzy k-NN by using genetic algorithm Journal of Computational Information Systems 1 2005 203 213
47. Z. Huan, N. Yi, H. Nie, Web document classification based on fuzzy k-NN algorithm, in: Proceedings of the 2009 International Conference on Computational Intelligence and Security (CIS 2009), Beijing, China, December 11-14, pp. 193-196.
48. Y. Huang, and Y. Li Prediction of protein subcellular locations using fuzzy k-NN method Bioinformatics 20 2004 21 28
49. E. Hüllermeier Possibilistic instance-based learning Artificial Intelligence 148 2003 335 383
50. H. Ishibuchi, T. Nakashima, Evolution of fuzzy nearest neighbor neural networks, in: Proceedings of the 1997 IEEE International Conference on Evolutionary Computation (IEEE CEC'97), Indianapolis, USA, April 13-16, pp. 673-678.
51. M.Z. Jahromi, E. Parvinnia, and R. John A method of learning weighted similarity function to improve the performance of nearest neighbor Information Sciences 179 2009 2964 2973
52. R. Jensen, C. Cornelis, Fuzzy-rough instance selection, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2010), Barcelona, Spain, July 18-23, pp. 1776-1782.
53. R. Jensen, and C. Cornelis Fuzzy-rough nearest neighbour classification Transactions on Rough Sets 13 2011 56 72
54. J.Y. Jiang, S.C. Tsai, and S.J. Lee FSKNN: multi-label text categorization based on fuzzy similarity and k nearest neighbors Expert Systems with Applications 39 2012 2813 2821
55. K. Josien, and T.W. Liao Integrated use of Fuzzy C-Means and fuzzy kNN for gt part family and machine cell formation International Journal of Production Research 38 2000 3513 3536
56. A. Jóẃik A learning scheme for a fuzzy k-NN rule Pattern Recognition Letters 1 1983 287 289
57. X.M. Kang, X.P. Liu, J.H. Zhai, M.Y. Zhai, Instances selection for NN with fuzzy rough technique, in: Proceedings of the International Conference on Machine Learning and Cybernetics (ICMLC 2011), Guilin, China, July 10-13, pp. 1097-1100.
58. N. Karnik, J. Mendel, and Q. Liang Type-2 fuzzy logic systems IEEE Transactions on Fuzzy Systems 7 1999 643 658
59. J.M. Keller, M.R. Gray, and J.A. Givens A fuzzy k-nearest neighbor algorithm IEEE Transactions on Systems, Man, and Cybernetics 15 1985 580 585
60. A.K. Ghosh On optimum choice of k in nearest neighbor classification Computational Statistics & Data Analysis 50 2006 3113 3123
61. S. Kim, J. Sim, J. Lee, Fuzzy k-nearest neighbor method for protein secondary structure prediction and its parallel implementation, in: Proceedings of the International Conference on Intelligent Computing (ICIC 2006), Kunming, China, August 16-19, pp. 444-463.
62. V. Kissiov, and S. Hadjitodorov A fuzzy version of the k-NN method Fuzzy Sets and Systems 49 1992 323 329
63. L.I. Kuncheva An intuitionistic fuzzy k-nearest neighbors rule Notes on Intuitionistic Fuzzy Sets 1 1995 56 60
64. M. Kuske, R. Rubio, A.C. Romain, J. Nicolas, and S. Marco Fuzzy k-NN applied to moulds detection Sensors and Actuators B 106 2004 52 60
65. K. Leszczynski, S. Cosby, R. Bissett, D. Provost, S. Boyko, S. Loose, and E. Mvilongo Application of a fuzzy pattern classifier to decision making in portal verification of radiotherapy Physics in Medicine and Biology 44 1999 253 269
66. S. Liang, C. Li, A fast and scalable fuzzy-rough nearest neighbor algorithm, in: Proceedings of the IEEE Global Congress on Intelligent Systems (GCIS'09), Xiamen, China, May 19-21, pp. 311-314.
67. T. Liao, and D. Li Two manufacturing applications of the fuzzy k-NN algorithm Fuzzy Sets and Systems 92 1997 289 303
68. T. Liao, D. Li, and Y. Li Detection of welding flaws from radiographic images with fuzzy clustering methods Fuzzy Sets and Systems 108 1999 145 158
69. T. Liao, D. Li, and Y. Li Extraction of welds from radiographic images using fuzzy classifiers Information Sciences 126 2000 21 40
70. Y.C. Liaw, M.L. Leou, and C.M. Wu Fast exact k nearest neighbors search using an orthogonal search tree Pattern Recognition 43 2010 2351 2358
71. F.O.S.S. Lisboa, M. Nicoletti, and A. Ramer A version of the NGE model suitable for fuzzy domains Journal of Intelligent & Fuzzy Systems 18 2007 1 17
72. R.J. Little, and D. Rubin Statistical Analysis with Missing Data second ed. 2002 John Wiley and Sons
73. D.Y. Liu, H.L. Chen, B. Yang, X.E. Lv, L.N. Li, and J. Liu Design of an enhanced fuzzy k-nearest neighbor classifier based computer aided diagnostic system for thyroid disease Journal of medical systems 36 2012 3243 3254
74. J. McNames A fast nearest-neighbor algorithm based on a principal axis search tree IEEE Transactions on Pattern Analysis and Machine Intelligence 23 2001 964 976
75. H.B. Mitchell, and P.A. Schaefer A soft k-nearest neighbor voting scheme International Journal of Intelligent Systems 16 2001 459 468
76. T. Nakashima, H. Ishibuchi, Learning of fuzzy reference sets in nearest neighbor classification, in: Proceedings of the 18th International Conference of the North American Fuzzy Information Processing Society (NAFIPS'99), New York, USA, June 10-12, pp. 357-360.
77. R. Nock, M. Sebban, and D. Bernard A simple locally adaptive nearest neighbor rule with application to pollution forecasting International Journal of Pattern Recognition and Artificial Intelligence 17 2003 1369 1382
78. R. Ostermark A fuzzy vector valued kNN-algorithm for automatic outlaier detection Applied Soft Computing 9 2009 1263 1272
79. R. Paredes, and E. Vidal Learning weighted metrics to minimize nearest-neighbor classification error IEEE Transactions on Pattern Analysis and Machine Intelligence 28 2006 1100 1110
80. V. Petridis, and V.G. Kaburlasos Finknn: a fuzzy interval number k-nearest neighbor classifier for prediction of sugar production from populations of samples Journal of Machine Learning Research 4 2004 17 37
81. T.D. Pham, An optimally wieghted fuzzy k-NN algorithm, in: Proceedings of the Third International Conference on Advances in Pattern Recognition, Part I (ICAPR 2005), Bath, UK, August 22-25, pp. 239-247.
82. Y. Qu, C. Shang, Q. Shen, N.M. Parthaláin, W. Wu, Kernel-based fuzzy-rough nearest neighbour classification, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), Taipei, Taiwan, June 27-30, pp. 1523-1529.
83. C. Raptis, C. Siettos, C. Kiranoudis, and G. Bafas Classification of aged wine distillates using fuzzy and neural network systems Journal of Food Engineering 46 2000 267 275
84. S. Roh, T. Ahn, and W. Pedrycz The refinement of models with the aid of the fuzzy k-nearest neighbors approach IEEE Transactions on Instrumentation and Measurement 59 2010 604 615
85. J.L.A. Rosa, N.F.F. Ebecken, Data mining for data classification based on the kNN-fuzzy method supported by genetic algorithm, in: Proceedings of the 5th International Conference on High Performance Computing for Computational Science (VECPAR'02), Porto, Portugal, June 26-28, Lecture Notes in Computer Science, vol. 2565, pp. 126-133.
86. M. Sarkar Fuzzy-rough nearest neighbor algorithms in classification Fuzzy Sets and Systems 158 2012 2134 2152
87. J. Shaffer Modified sequentially rejective multiple test procedures Journal of American Statistical Association 81 1986 826 831
88. D.J. Sheskin Handbook of Parametric and Nonparametric Statistical Procedures fifth ed. 2011 Chapman & Hall/CRC
89. S. Singh A single nearest neighbour fuzzy approach for pattern recognition International Journal of Pattern Recognition and Artificial Intelligence 13 1998 49 54
90. M. Stone Cross-validatory choice and assessment of statistical predictions Journal of the Royal Statistical Society, Series B (Methodological) 36 1974 111 147
91. I. Triguero, J. Derrac, S. García, and F. Herrera A taxonomy and experimental study on prototype generation for nearest neighbor classification IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42 2012 86 100
92. D.C. Tu, J.H. Zhao, M.H. Liu, J. Shen, and F. Yu Preliminary study on quantification of duck color based on fuzzy k nearest neighbor method Applied Mechanics and Materials 39 2010 210 215
93. T.Q. Tung, and D. Lee A method to improve protein subcellular localization prediction by integrating various biological data sources BMC Bioinformatics 10 2009 1 10
94. C.J. Veenman, and M.J.T. Reinders The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier IEEE Transactions on Pattern Analysis and Machine Intelligence 27 2005 1417 1429
95. J. Wang, P. Neskovic, and L. Cooper Improving nearest neighbor rule with a simple adaptive distance measure Pattern Recognition Letters 28 2007 207 213
96. D. Wettschereck, D.W. Aha, and T. Mohri A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms Artificial Intelligence Review 11 1997 273 314
97. D. Wilson, and T. Martinez Improved heterogeneous distance functions Journal of Artificial Intelligence Research 6 1997 1 34
98. D.L. Wilson Asymptotic properties of nearest neighbor rules using edited data IEEE Transactions on System, Man and Cybernetics 2 1972 408 421
99. D.R. Wilson, and T.R. Martinez An integrated instance-based learning algorithm Computational Intelligence 16 2000 1 28
100. I.H. Witten, E. Frank, and M.A. Hall Data Mining: Practical Machine Learning Tools and Techniques third ed. Morgan Kaufmann Series in Data Management Systems 2011 Morgan Kaufmann
101. X. Wu, and V. Kumar The Top Ten Algorithms in Data Mining Data Mining and Knowledge Discovery 2009 Chapman & Hall/CRC
102. R.R. Yager Using fuzzy methods to model nearest neighbor rules IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics 32 2002 512 525
103. M. Yang, and C. Chen On strong consistency of the fuzzy generalized nearest neighbor rule Fuzzy Sets and Systems 60 1993 273 281
104. M. Yang, and C. Chen Convergence rate of the fuzzy generalized nearest neighbor rule Computers & Mathematics with Applications 27 1994 1 8
105. M. Yang, and C. Cheng On the edited fuzzy k-nearest neighbor rule IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics 28 1998 461 466
106. S. Yu, S. Backer, and P. Scheunders Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery Pattern Recognition Letters 23 2002 183 190
107. L. Zadeh Fuzzy sets Information and Control 8 1965 338 353
108. L. Zadeh Fuzzy sets as a basis for a theory of possibility Fuzzy Sets and Systems 1 1978 3 28
109. J.H. Zhai, N. Li, M.Y. Zhai, The condensed fuzzy k-nearest neighbor rule based on sample fuzzy entropy, in: Proceedings of the 2011 International Conference on Machine Learning and Cybernetics (ICMLC'11), Guilin, China, July 10-13, pp. 282-286.
110. H. Zhu, O. Basir, A k-NN associated fuzzy evidential reasoning classifier with adaptive neighbor selection, in: Proceedings of the Third IEEE International Conference on Data Mining (ICDM 2003), Melbourne, Florida, USA, December 19-22, pp. 709-712.