Selecting critical patterns based on local geometrical and statistical information

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 33, Issue 6, 2011, Pages 1189-1201

  Li, Yuhua ^a   Maguire, Liam ^a  

^a UNIVERSITY OF ULSTER   (United Kingdom)

Author keywords

border pattern; data reduction; edge pattern; Pattern selection

Indexed keywords

BENCH-MARK PROBLEMS; BORDER PATTERNS; CLASS BOUNDARY; DECISION SURFACES; EDGE PATTERN; EDGE PATTERNS; INPUT SPACE; MULTI-LAYER PERCEPTRONS; NEAREST NEIGHBORS; PATTERN SELECTION; RADIAL BASIS FUNCTIONS; REDUCED DATA; SPATIAL INFORMATIONS; STATE-OF-THE-ART APPROACH; STATISTICAL INFORMATION; TANGENT HYPERPLANE; TRAINING DATA SETS;

PATTERN RECOGNITION SYSTEMS; RADIAL BASIS FUNCTION NETWORKS; STATISTICS;

DATA REDUCTION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; METHODOLOGY; PROBABILITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; PROBABILITY;

EID: 79955475773     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2010.188     Document Type: Article

References (70)

1. ftp://ftp.dice.ucl.ac.be/pub/neural-nets/ELENA/databases/ REAL/phoneme/, 2009.
2. S. Ajoka, S. Tsuge, M. Shishibori, and K. Kita, "Fast Multidimensional Nearest Neighbor Search Algorithm Using Priority Queue," Electrical Eng. in Japan, vol. 164, no. 3, pp. 69-77, 2008.
3. F. Angiulli, "Fast Nearest Neighbor Condensation for Large Data Sets Classification," IEEE Trans. Knowledge and Data Eng., vol. 19, no. 11, pp. 1450-1464, Nov. 2007. (Pubitemid 47573790)
4. F. Angiulli, "Condensed Nearest Neighbor Data Domain Description," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1746-1758, Oct. 2007.
5. R. Barandela, F.J. Ferri, and J.S. Sanchez, "Decision Boundary Preserving Prototype Selection for Nearest Neighbour Classification," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 19, no. 6, pp. 787-806, Sept. 2005. (Pubitemid 41374104)
6. J.L. Bentley and J.H. Friedman, "Data-Structures for Range Searching," ACM Computing Surveys, vol. 11, no. 4, pp. 397-409, 1979.
7. J.C. Bezdek and L.I. Kuncheva, "Nearest Prototype Classifier Designs: An Experimental Study," Int'l J. Intelligent Systems, vol. 16, no. 12, pp. 1445-1473, Dec. 2001. (Pubitemid 33110647)
8. D.S. Broomhead and D. Lowe, "Multivariable Function Interpolation and Adaptive Networks," Complex Systems, vol. 2, pp. 321-335, 1988.
9. C.J.C. Burges, "A Tutorial on Support Vector Machines for Pattern Recognition," Data Mining and Knowledge Discovery, vol. 2, pp. 121-167, 1998. (Pubitemid 128695475)
10. C. Cachin, "Pedagogical Pattern Selection-Strategies," Neural Networks, vol. 7, no. 1, pp. 175-181, 1994.
11. G.C. Cawley and N.L.C. Talbot, "Efficient Leave-One-Out Cross-Validation of Kernel Fisher Discriminant Classifiers," Pattern Recognition, vol. 36, pp. 2585-2592, 2003. (Pubitemid 37178388)
12. V. Cerverón and F.J. Ferri, "Another Move Toward the Minimum Consistent Subset: A Tabu Search Approach to the Condensed Nearest Neighbor Rule," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 31, no. 3, pp. 408-413, June 2001. (Pubitemid 32676845)
13. S.H. Cha and S.N. Srihari, "A Fast Nearest Neighbor Search Algorithm by Filtration," Pattern Recognition, vol.35, no. 2, pp. 515-525, Feb. 2002. (Pubitemid 34034625)
14. C.C. Chang and C.J. Lin, LIBSVM: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
15. D. Chaudhuri, C.A. Murthy, and B.B. Chaudhuri, "Finding a Subset of Representative Points in a Data Set," IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 9, pp. 1416-1424, Sept. 1994.
16. Y.X. Chen, X. Dang, H.X. Peng, and H.L. Bart, "Outlier Detection with the Kernelized Spatial Depth Function," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 288-305, Feb. 2009.
17. S.H. Choi and P. Rockett, "The Training of Neural Classifiers with Condensed Data sets," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 32, no. 2, pp. 202-206, Apr. 2002. (Pubitemid 34249378)
18. B.V. Dasarathy, "Minimal Consistent Set (MCS) Identification for Optimal Nearest Neighbor Decision Systems Design," IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 3, pp. 511-517, Mar. 1994.
19. J. Demsar, "Statistical Comparisons of Classifiers over Multiple Data Sets," J. Machine Learning Research, vol. 7, pp. 1-30, Jan. 2006. (Pubitemid 43022939)
20. G.M. Foody, "The Significance of Border Training Patterns in Classification by a Feedforward Neural Network Using Back Propagation Learning," Int'l J. Remote Sensing, vol. 20, no. 18, pp. 3549-3562, Dec. 1999. (Pubitemid 30037566)
21. J.H. Friedman, J.L. Bentley, and R.A. Finkel, "An Algorithm for Finding Best Matches in Logarithmic Expected Time," ACM Trans. Math. Software, vol. 3, no. 3, pp. 209-226, Sept. 1977.
22. K. Fukunaga, Introduction to Statistical Pattern Recognition, second ed. Morgan Kaufmann, 1990.
23. K. Fukunage and P.M. Narendra, "A Branch and Bound Algorithm for Computing K-Nearest Neighbors," IEEE Trans. Computers, vol. 24, no. 7, pp. 750-753, July 1975.
24. K. Fukunaga and L.D Hostetle, "Optimization of k Nearest-Neighbor Density Estimates," IEEE Trans. Information Theory, vol. 19, no. 3, pp. 320-326, May 1973.
25. A.K. Ghosh, "On Nearest Neighbor Classification Using Adaptive Choice of k," J. Computational and Graphical Statistics, vol. 16, no. 2, pp. 482-502, June 2007.
26. A.K. Ghosh, "On Optimum Choice of k in Nearest Neighbour Classification," Computational Statistical & Data Analysis, vol. 50, pp. 3113-3123, 2006. (Pubitemid 43729105)
27. G. Guo and J.S. Zhang, "Reducing Examples to Accelerate Support Vector Regression," Pattern Recognition Lectters, vol. 28, no. 16, pp. 2173-2183, 2007. (Pubitemid 47538772)
28. P. Hart, "The Condensed Nearest Neighbor Rule," IEEE Trans. Information Theory, vol. 14, no. 3, pp. 515-516, May 1968.
29. T.K. Ho and M. Basu, "Complexity Measures of Supervised Classification Problems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 289-300, Mar. 2002. (Pubitemid 35289347)
30. S.W. Kim and B.J. Oommen, "A Brief Taxonomy and Ranking of Creative Prototype Reduction Schemes," Pattern Analysis and Applications J., vol. 6, no. 3, pp. 232-244, 2004.
31. S.W. Kim and B.J. Oommen, "Enhancing Prototype Reduction Schemes with LVQ3-Type Algorithms," Pattern Recognition, vol. 36, no. 5, pp. 1083-1093, 2003. (Pubitemid 36175216)
32. M.A. Kramer and J.A. Leonard, "Diagnosis Using Backpropaga-tion Neural Networks-Analysis and Criticism," Computers & Chemical Eng., vol. 14, no. 12, pp. 1323-1338, 1990.
33. J. Li, M.T. Manry, C. Yu, and D.R. Wilson, "Prototype Classifier Design with Pruning," Int'l J. Artificial Intelligence Tools, vol. 14, nos. 1/2, pp. 261-280, 2005.
34. Y.H. Li, M.J. Pont, and N.B. Jones, "Improving the Performance of Radial Basis Function Classifiers in Condition Monitoring and Fault Diagnosis Applications Where 'Unknown' Faults May Occur," Pattern Recognition Letters, vol. 23, no. 5, pp. 569-577, Mar. 2002. (Pubitemid 34262287)
35. Y.H. Li, M.J. Pont, N.B. Jones, and J.A. Twiddle, "Applying MLP and RBF Classifiers in Embedded Condition Monitoring and Fault Diagnosis Systems," Trans. Inst. of Measurement and Control, vol. 23, no. 5, pp. 315-343, 2001. (Pubitemid 34080558)
36. T. Lin and H.B. Zha, "Riemannian Manifold Learning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 796-809, May 2008. (Pubitemid 351486749)
37. M.T. Lozano, J.M. Sotoca, J.S. Sánchez, F. Pla, E. Pekalska, and R.P.W. Duin, "Experimental Study on Prototype Optimisation Algorithms for Prototype-Based Classification in Vector Spaces," Pattern Recognition, vol. 39, no. 10, pp. 1827-1838, 2006. (Pubitemid 43947094)
38. A. Lyhyaoui, M. Martinez, I. Mora, M. Vazquez, J.L. Sancho, and A.R. Figueiras-Vidal, "Sample Selection via Clustering to Construct Support Vector-Like Classifiers," IEEE Trans. Neural Networks, vol. 10, no. 6, pp. 1474-1481, Nov. 1999. (Pubitemid 30536579)
39. E. Marchiori, "Hit Miss Networks with Applications to Instance Selection," J. Machine Learning Research, vol. 9, pp. 997-1017, 2008.
40. E. Marchiori, "Class Conditional Nearest Neighbor for Large Margin Instance Selection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 2, pp. 364-370, Feb. 2010.
41. M. Markou and S. Singh, "A Neural Network-Based Novelty Detector for Image Sequence Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1664-1677, Oct. 2006. (Pubitemid 46414200)
42. K.G. Mehrotra, C.K. Mohan, and S. Ranka, "Bounds on the Number of Samples Needed for Neural Learning," IEEE Trans. Neural Networks, vol. 2, no. 6, pp. 548-558, Nov. 1991. (Pubitemid 23563889)
43. N. Mekuz and J.K. Tsotsos, "Parameterless Isomap with Adaptive Neighborhood Selection," Proc. DAGM Symp., K. Franke, et al., eds., pp. 364-373, 2006. (Pubitemid 44608882)
44. D.J. Newman, S. Hettich, C.L. Blake, and C.J. Merz, UCI Repository of Machine Learning Databases, Dept. of Information and Computer Science, http://www.ics.uci.edu/~mlearn/MLRepository.html, Univ. of California, 1998.
45. R. Paredes and E. Vidal, "Learning Prototypes and Distances: A Prototype Reduction Technique Based on Nearest Neighbor Error Minimization," Pattern Recognition, vol. 39, no. 2, pp. 180-188, 2006. (Pubitemid 41586088)
46. E. Pekalska, R.P.W. Duin, and P. Paclik, "Prototype Selection for Dissimilarity-Based Classifiers," Pattern Recognition, vol. 39, no. 2, pp. 189-208, Feb. 2006. (Pubitemid 41586089)
47. R.T. Peres and C.E. Pedreira, "Generalized Risk Zone: Selecting Observations for Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 7, pp. 1331-1337, July 2009.
48. D. Prokhorov, "IJCNN 2001 Neural Network Competition," Slide presentation at Int'l Joint Conf. Neural Networks, http:// www.geocities.com/ ijcnn/nnc-ijcnn01.pdf, 2001.
49. T. Raicharoen and C. Lursinsap, "A Divide-and-Conquer Approach to the Pairwise Opposite Class-Nearest Neighbor (POC-NN) Algorithm," Pattern Recognition Letters, vol. 26, no. 10, pp. 1554-1567, July 2005. (Pubitemid 40744195)
50. G. Ratsch, T. Onoda, and K.R. Muller, "Soft Margins for AdaBoost," Machine Learning, vol. 42, no. 2, pp.287-320, 2001. (Pubitemid 32188795)
51. D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Representations by Back-Propagating Errors," Nature, vol. 323, pp. 533-536, Oct. 1986. (Pubitemid 16025374)
52. H. Samet, "K-Nearest Neighbor Finding Using MaxNearestDist," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp.243-252, Feb. 2008.
53. O. Samko, A.D. Marshall, and P.L. Rosin, "Selection of the Optimal Parameter Value for the Isomap Algorithm," Pattern Recognition Letters, vol. 27, no. 9, pp. 968-979, June 2006.
54. J.S. Sánchez and A.I. Marqués, "An LVQ-Based Adaptive Algorithm for Learning from Very Small Codebooks," Neurocom-puting, vol. 69, nos. 7-9, pp. 922-927, 2006. (Pubitemid 43230405)
55. J.S. Sánchez, "High Training Set Size Reduction by Space Partitioning and Prototype Abstraction," Pattern Recognition, vol. 37, no. 7, pp. 1561-1564, 2004. (Pubitemid 38653386)
56. J.S. Sánchez, F. Pla, and F.J. Ferri, "Prototype Selection for the Nearest Neighbour Rule through Proximity Graphs," Pattern Recognition Letters, vol. 18, no. 6, pp. 507-513, 1997. (Pubitemid 127424198)
57. J.L. Sancho, W.E. Pierson, B. Ulug, A.R. Figueiras-Vidal, and S.C. Ahalt, "Class Separability Estimation and Incremental Learning Using Boundary Methods," Neurocomputing, vol. 35, pp. 3-26, Nov. 2000.
58. L.K. Saul and S.T. Roweis, "Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds," J. Machine Learning Research, vol. 4, no. 2, pp. 119-155, Feb. 2004.
59. B. Schölkopf, R.C. Williamson, A.J. Smola, J.S. Taylor, and J.C. Platt, "Support Vector Method for Novelty Detection," Advances in Neural Information Processing Systems, vol. 12, pp. 582-588, MIT Press, 2000.
60. H. Shin and S. Cho, "Fast Pattern Selection for Support Vector Classifiers," Proc. Conf. Advances in Knowledge Discovery and Data Mining, vol. 2637, pp. 376-387, 2003. (Pubitemid 36557439)
61. H. Shin and S. Cho, "Neighborhood Property-Based Pattern Selection for Support Vector Machines," Neural Computation, vol. 19, no. 3, pp. 816-855, Mar. 2007.
62. T. Tambouratzis, "Counter-Clustering for Training Pattern Selection," Computer J., vol. 43, no. 3, pp. 177-190, 2000.
63. D.M.J. Tax and R.P.W. Duin, "Support Vector Data Description," Machine Learning, vol. 54, pp.45-66, 2004.
64. J.B. Tenenbaum, V.de. Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, no. 5500, pp. 2319-2323, Dec. 2000. (Pubitemid 32041577)
65. V.N. Vapnik, The Nature of Statistical Learning Theory. Springer-Verlag, 1995.
66. K.R. Varshney and A.S. Willsky, "Classification Using Geometric Level Sets," J. Machine Learning Research, vol. 11, pp. 491-516, Feb. 2010.
67. C.J. Veenman and M.J.T. Reinders, "The Nearest Subclass Classifier: A Compromise between the Nearest Mean and Nearest Neighbor Classifier," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 9, pp. 1417- 1429, Sept. 2005.
68. G.H. Wen, L.J. Jiang, and J. Wen, "Using Locally Estimated Geodesic Distance to Optimize Neighborhood Graph for Isometric Data Embedding," Pattern Recognition, vol. 41, no. 7, pp. 2226-2236, July 2008.
69. D.R. Wilson and T.R. Martinez, "Reduction Techniques for Instance-Based Learning Algorithms," Machine Learning, vol. 38, no. 3, pp. 257-286, Mar. 2000. (Pubitemid 30572450)
70. H. Zhang and G. Sun, "Optimal Reference Subset Selection for Nearest Neighbor Classification by Tabu Search," Pattern Recognition, vol. 35, no. 7, pp. 1481-1490, 2002. (Pubitemid 34440048)