A taxonomy and experimental study on prototype generation for nearest neighbor classification

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 42, Issue 1, 2012, Pages 86-100

  Triguero, Isaac ^a   Derrac, Joaquín ^a   García, Salvador ^b   Herrera, Francisco ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b UNIVERSITY OF JAÉN   (Spain)

Author keywords

Classification; learning vector quantization (LVQ); nearest neighbor (NN); prototype generation (PG); taxonomy

Indexed keywords

CLASSIFICATION ACCURACY; CLASSIFICATION RULES; DATA SETS; EXPERIMENTAL STUDIES; FORMER PROCESS; LARGE DATASETS; LEARNING VECTOR QUANTIZATION; MAIN CHARACTERISTICS; NEAREST NEIGHBOR CLASSIFICATION; NEAREST NEIGHBOR RULE; NEAREST NEIGHBORS; NOISE SENSITIVITY; PROTOTYPE GENERATION (PG); PROTOTYPE SELECTION; REDUCTION TECHNIQUES; STORAGE REQUIREMENTS; THEORETICAL POINTS; TIME RESPONSE; TRAINING DATA;

CLASSIFICATION (OF INFORMATION); PATTERN RECOGNITION; SEMANTICS; STATISTICAL TESTS; TAXONOMIES; VECTOR QUANTIZATION;

DATA REDUCTION;

EID: 84655176618     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2010.2103939     Document Type: Article

References (84)

1. T. M. Cover and P. E. Hart, "Nearest neighbor pattern classification, " IEEE Trans. Inf. Theory, vol. IT-13, no. 1, pp. 21-27, Jan. 1967.
2. E. Alpaydin, Introduction to Machine Learning, 2nd ed. Cambridge, MA: MIT Press, 2010.
3. V. Cherkassky and F. Mulier, Learning From Data: Concepts, Theory and Methods, 2nd ed. New York: Interscience, 2007.
4. I. Kononenko and M. Kukar, Machine Learning and Data Mining: Introduction to Principles and Algorithms. West Sussex: Horwood, 2007.
5. E. K. Garcia, S. Feldman, M. R. Gupta, and S. Srivastava, "Completely lazy learning, " IEEE Trans. Knowl. Data Eng., vol. 22, no. 9, pp. 1274-1285, Sep. 2010.
6. A. N. Papadopoulos and Y. Manolopoulos, Nearest Neighbor Search: A Database Perspective. New York: Springer-Verlag, 2004.
7. G. Shakhnarovich, T. Darrell, and P. Indyk, Eds., Nearest-Neighbor Methods in Learning and Vision: Theory and Practice. Cambridge, MA: MIT Press, 2006.
8. X. Wu and V. Kumar, Eds., The Top Ten Algorithms in Data Mining. (Chapman & Hall/CRC Data Mining and Knowledge Discovery Series). Boca Raton, FL: CRC, 2009.
9. A. Shintemirov, W. Tang, and Q. Wu, "Power transformer fault classification based on dissolved gas analysis by implementing bootstrap and genetic programming, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 39, no. 1, pp. 69-79, Jan. 2009.
10. P. G. Espejo, S. Ventura, and F. Herrera, "A survey on the application of genetic programming to classification, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 40, no. 2, pp. 121-144, Mar. 2009.
11. S. Magnussen, R. McRoberts, and E. Tomppo, "Model-basedmean square error estimators for k-nearest neighbour predictions and applications using remotely sensed data for forest inventories, " Remote Sens. Environ., vol. 113, no. 3, pp. 476-488, 2009.
12. M. Govindarajan and R. Chandrasekaran, "Evaluation of k-nearest neighbor classifier performance for direct marketing, " Expert Syst. Appl., vol. 37, no. 1, pp. 253-258, 2009.
13. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
14. Y. Chen, E. Garcia, M. Gupta, A. Rahimi, and L. Cazzanti, "Similaritybased classification: Concepts and algorithms, " J. Mach. Learning Res., vol. 10, pp. 747-776, 2009.
15. K. Weinberger and L. Saul, "Distance metric learning for large margin nearest neighbor lassification, " J. Mach. Learning Res., vol. 10, pp. 207-244, 2009.
16. F. Fernández and P. Isasi, "Local feature weighting in nearest prototype classification, " IEEE Trans. Neural Netw., vol. 19, no. 1, pp. 40-53, Jan. 2008. (Pubitemid 351279224)
17. E. Pekalska and R. P. Duin, "Beyond traditional kernels: Classification in two dissimilarity-based representation spaces, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 38, no. 6, pp. 729-744, Nov. 2008.
18. P. Cunningham, "A taxonomy of similarity mechanisms for case-based reasoning, " IEEE Trans. Knowl. Data Eng., vol. 21, no. 11, pp. 1532-1543, Nov. 2009.
19. B. Li, Y. W. Chen, and Y. Chen, "The nearest neighbor algorithm of local probability centers, " IEEE Trans. Syst., Man, Cybern. B. Cybern., vol. 38, no. 1, pp. 141-154, Feb. 2008. (Pubitemid 351285021)
20. C.-M. Hsu and M.-S. Chen, "On the design and applicability of distance functions in high-dimensional data space, " IEEE Trans. Knowl. Data Eng., vol. 21, no. 4, pp. 523-536, Apr. 2009.
21. D. R. Wilson and T. R. Martinez, "Reduction techniques for instancebased learning algorithms, " Mach. Learning, vol. 38, no. 3, pp. 257-286, 2000. (Pubitemid 30572450)
22. S. W. Kim and J. Oomenn, "A brief taxonomy and ranking of creative prototype reduction schemes, " Pattern Anal. Appl., vol. 6, pp. 232-244, 2003.
23. H. Brighton and C. Mellish, "Advances in instance selection for instancebased learning algorithms, " Data Mining Knowl. Discov., vol. 6, no. 2, pp. 153-172, 2002. (Pubitemid 37113870)
24. E. Marchiori, "Class conditional nearest neighbor for large margin instance selection, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 364-370, Feb. 2010.
25. N. García-Pedrajas, "Constructing ensembles of classifiers by means of weighted instance selection, " IEEE Trans. Neural Netw., vol. 20, no. 2, pp. 258-277, Feb. 2009.
26. E. Pekalska, R. P. W. Duin, and P. Paclík, "Prototype selection for dissimilarity-based classifiers, " Pattern Recognit., vol. 39, no. 2, pp. 189-208, 2006. (Pubitemid 41586089)
27. M. Lozano, J. M. Sotoca, J. S. Sánchez, F. Pla, E. Pekalska, and R. P. W. Duin, "Experimental study on prototype optimisation algorithms for prototype-based classification in vector spaces, " Pattern Recognit., vol. 39, no. 10, pp. 1827-1838, 2006. (Pubitemid 43947094)
28. H. A. Fayed, S. R. Hashem, and A. F. Atiya, "Self-generating prototypes for pattern classification, " Pattern Recognit., vol. 40, no. 5, pp. 1498-1509, 2007. (Pubitemid 46123389)
29. W. Lam, C. K. Keung, and D. Liu, "Discovering useful concept prototypes for classification based on filtering and abstraction, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 8, pp. 1075-1090, Aug. 2002.
30. J. S. Sánchez, "High training set size reduction by space partitioning and prototype abstraction, " Pattern Recognit., vol. 37, no. 7, pp. 1561-1564, 2004. (Pubitemid 38653386)
31. R. Xu and D. Wunsch, "Survey of clustering algorithms, " IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 645-678, May 2005. (Pubitemid 40718010)
32. C.-L. Chang, "Finding prototypes for nearest neighbor classifiers, " IEEE Trans.comput., vol. C-23, no. 11, pp. 1179-1184, Nov. 1974.
33. T. Kohonen, "The self organizing map, " Proc. IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 1990.
34. C. H. Chen and A. Józwik, "A sample set condensation algorithm for the class sensitive artificial neural network, " Pattern Recognit. Lett., vol. 17, no. 8, pp. 819-823, Jul. 1996. (Pubitemid 126384186)
35. R. Kulkarni and G. Venayagamoorthy, "Particle swarm optimization in wireless-sensor networks: A brief survey, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev. to be published. DOI: 10. 1109/TSMCC. 2010. 2054080.
36. F. Fernández and P. Isasi, "Evolutionary design of nearest prototype classifiers, " J. Heurist., vol. 10, no. 4, pp. 431-454, 2004.
37. L. Nanni and A. Lumini, "Particle swarm optimization for prototype reduction, " Neurocomputing, vol. 72, no. 4-6, pp. 1092-1097, 2008.
38. J. Alcalá-Fdez, L. Sánchez, S. García, M. J. del Jesus, S. Ventura, J. M. Garrell, J. Otero, C. Romero, J. Bacardit, V. M. Rivas, J. C. Fernández, and F. Herrera, "KEEL: A software tool to assess evolutionary algorithms for data mining problems, " Soft Comput., vol. 13, no. 3, pp. 307-318, 2008.
39. J. Koplowitz and T. Brown, "On the relation of performance to editing in nearest neighbor rules, " Pattern Recognit., vol. 13, pp. 251-255, 1981. (Pubitemid 11413647)
40. S. Geva and J. Sitte, "Adaptive nearest neighbor pattern classification, " IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 318-322, Mar. 1991.
41. Q. Xie, C. A. Laszlo, and R. K. Ward, "Vector quantization technique for nonparametric classifier design, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 12, pp. 1326-1330, Dec. 1993.
42. Y. Hamamoto, S. Uchimura, and S. Tomita, "A bootstrap technique for nearest neighbor classifier design, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 1, pp. 73-79, Jan. 1997. (Pubitemid 127762882)
43. R. Odorico, "Learning vector quantization with training count (LVQTC), " Neural Netw., vol. 10, no. 6, pp. 1083-1088, 1997. (Pubitemid 27431589)
44. C. Decaestecker, "Finding prototypes for neares neghbour classification by means of gradient descent and deterministic annealing, " Pattern Recognit., vol. 30, no. 2, pp. 281-288, 1997. (Pubitemid 127407618)
45. T. Bezdek, J. C . and Reichherzer, G. Lim, and Y. Attikiouzel, "Multiple prototype classifier design, " IEEE Trans. Syst., Man Cybern. C, Appl. Rev., vol. 28, no. 1, pp. 67-79, Feb. 1998.
46. R. Mollineda, F. Ferri, and E. Vidal, "A merge-based condensing strategy for multiple prototype classifiers, " IEEE Trans. Syst., Man Cybern. B, Cybern., vol. 32, no. 5, pp. 662-668, Oct. 2002.
47. J. S. Sánchez, R. Barandela, A. I. Marqués, R. Alejo, and J. Badenas, "Analysis of new techniques to obtain quality training sets, " Pattern Recognit. Lett., vol. 24, no. 7, pp. 1015-1022, 2003. (Pubitemid 36049311)
48. S. W. Kim and J. Oomenn, "Enhancing prototype reduction schemes with lvq3-type algorithms, " Pattern Recognit., vol. 36, pp. 1083-1093, 2003.
49. C.-W. Yen, C.-N. Young, and M. L. Nagurka, "A vector quantization method for nearest neighbor classifier design, " Pattern Recognit. Lett., vol. 25, no. 6, pp. 725-731, 2004. (Pubitemid 38459484)
50. J. Li, M. T. Manry, C. Yu, and D. R. Wilson, "Prototype classifier design with pruning, " Int. J. Artif. Intell. Tools, vol. 14, no. 1-2, pp. 261-280, 2005.
51. T. Raicharoen and C. Lursinsap, "A divide-and-conquer approach to the pairwise opposite class-nearest neighbor (POC-NN) algorithm, " Pattern Recognit. Lett., vol. 26, no. 10, pp. 1554-1567, 2005. (Pubitemid 40744195)
52. A. Cervantes, I. M. Galván, and P. Isasi, "AMPSO: A new particle swarm method for nearest neighborhood classification, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 5, pp. 1082-1091, Oct. 2009.
53. U. Garain, "Prototype reduction using an artificial immune model, " Pattern Anal. Appl., vol. 11, no. 3-4, pp. 353-363, 2008.
54. D. L. Wilson, "Asymptotic properties of nearest neighbor rules using edited data, " IEEE Trans. Syst., Man Cybern., vol. SMC-2, no. 3, pp. 408-421, Jul. 1972.
55. P. E. Hart, "The condensed nearest neighbor rule, " IEEE Trans. Inf. Theory, vol. IT-18, no. 3, pp. 515-516, May 1968.
56. S. García, J.-R. Cano, E. Bernadó-Mansilla, and F. Herrera, "Diagnose of effective evolutionary prototype selection using an overlapping measure, " Int. J. Pattern Recognit. Artif. Intell., vol. 28, no. 8, pp. 1527-1548, 2009.
57. S. García, J. R. Cano, and F. Herrera, "A memetic algorithm for evolutionary prototype selection: A scaling up approach, " Pattern Recognit., vol. 41, no. 8, pp. 2693-2709, 2008.
58. H. A. Fayed and A. F. Atiya, "A novel template reduction approach for the k-nearest neighbor method, " IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 890-896, May 2009.
59. J. Derrac, S. García, and F. Herrera, "IFS-CoCo: Instance and feature selection based on cooperative coevolution with nearest neighbor rule, " Pattern Recognit., vol. 43, no. 6, pp. 2082-2105, 2010.
60. P. Domingos, "Unifying instance-based and rule-based induction, " Mach. Learning, vol. 24, no. 2, pp. 141-168, 1996. (Pubitemid 126724383)
61. O. Luaces and A. Bahamonde, "Inflating examples to obtain rules, " Int. J. Intell. Syst., vol. 18, pp. 1113-1143, 2003.
62. H. A. Fayed, S. R. Hashem, and A. F. Atiya, "Hyperspherical prototypes for pattern classification, " Int. J. Pattern Recognit. Artif. Intell., vol. 23, no. 8, pp. 1549-1575, 2009.
63. D. Wettschereck, D. W. Aha, and T. Mohri, "A reviewand empirical evaluation of feature weightingmethods for a class of lazy learning algorithms, " Artif. Intell. Rev., vol. 11, no. 1-5, pp. 273-314, 1997. (Pubitemid 127508242)
64. R. Paredes and E. Vidal, "Learning weighted metrics to minimize nearestneighbor classification error, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1100-1110, Jul. 2006.
65. M. Z. Jahromi, E. Parvinnia, and R. John, "A method of learning weighted similarity function to improve the performance of nearest neighbor, " Inf. Sci., vol. 179, no. 17, pp. 2964-2973, 2009.
66. C. Vallejo, J. Troyano, and F. Ortega, "InstanceRank: Bringing order to datasets, " Pattern Recognit. Lett., vol. 31, no. 2, pp. 133-142, 2010.
67. C. G. Atkeson, A. W. Moore, and S. Schaal, "Locally weighted learning, " Artif. Intell. Rev., vol. 11, pp. 11-73, 1997. (Pubitemid 127508233)
68. D. R. Wilson and T. R. Martinez, "Improved heterogeneous distance functions, " J. Artif. Intell. Res., vol. 6, no. 1, pp. 1-34, 1997. (Pubitemid 127667584)
69. R. D. Short and K. Fukunaga, "Optimal distance measure for nearest neighbor classification, " IEEE Trans. Inf. Theory, vol. IT-27, no. 5, pp. 622-627, Sep. 1981. (Pubitemid 12446969)
70. C. Gagné and M. Parizeau, "Coevolution of nearest neighbor classifiers, " Int. J. Pattern Recognit. Artif. Intell., vol. 21, no. 5, pp. 921-946, 2007. (Pubitemid 47229590)
71. J. Wang, P. Neskovic, and L. Cooper, "Improving nearest neighbor rule with a simple adaptive distance measure, " Pattern Recognit. Lett., vol. 28, no. 2, pp. 207-213, 2007. (Pubitemid 44716403)
72. Y. Sun, A. K. C. Wong, and M. S. Kamel, "Classification of imbalanced data: A review., " Int. J. Pattern Recognit. Artif. Intell., vol. 23, no. 4, pp. 687-719, 2009.
73. H. He and E. A. Garcia, "Learning from imbalanced data, " IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263-1284, Sep. 2009.
74. S. García and F. Herrera, "Evolutionary under-sampling for classification with imbalanced data sets: Proposals and taxonomy, " Evol.comput., vol. 17, no. 3, pp. 275-306, 2009.
75. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "Smote: Synthetic minority over-sampling technique, " J. Artif. Intell. Res., vol. 16, pp. 321-357, 2002. (Pubitemid 43057176)
76. N. V. Chawla, D. A. Cieslak, L. O. Hall, and A. Joshi, "Automatically countering imbalance and its empirical relationship to cost, " Data Mining Knowl. Discov., vol. 17, no. 2, pp. 225-252, 2008.
77. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed. San Francisco, CA: Morgan Kaufmann, 2005.
78. A. Ben-David, "A lot of randomness is hiding in accuracy, " Eng. Appl. Artif. Intell., vol. 20, pp. 875-885, 2007. (Pubitemid 47419312)
79. A. Asuncion and D. Newman. (2007). UCI machine learning repository. [Online]. Available: http://www.ics. uci.edu/mlearn/MLRepository.html.
80. S. García, A. Fernández, J. Luengo, and F. Herrera, "A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability, " Soft Comput., vol. 13, no. 10, pp. 959-977, 2009.
81. D. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, 2nd ed. London, U. K. : Chapman & Hall, 2006.
82. J. Demšar, "Statistical comparisons of classifiers over multiple data sets, " J. Mach. Learning Res., vol. 7, pp. 1-30, 2006. (Pubitemid 43022939)
83. S. García and F. Herrera, "An extension on "statistical comparisons of classifiers overmultiple data sets" for all pairwise comparisons, " J. Mach. Learning Res., vol. 9, pp. 2677-2694, 2008.
84. S. García, A. Fernández, J. Luengo, and F. Herrera, "Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power, " Inf. Sci., vol. 180, pp. 2044-2064, 2010.