Noise reduction for instance-based learning with a local maximal margin approach

Journal of Intelligent Information Systems

Volumn 35, Issue 2, 2010, Pages 301-331

  Segata, Nicola ^a   Blanzieri, Enrico ^a   Delany, Sarah Jane ^b   Cunningham, Pádraig ^c  

^a UNIVERSITY OF TRENTO   (Italy)

^b DUBLIN INSTITUTE OF TECHNOLOGY   (Ireland)

^c UNIVERSITY COLLEGE DUBLIN   (Ireland)

Author keywords

Editing techniques; k NN; Locality; Noise reduction; SVM

Indexed keywords

EDITING TECHNIQUES; K-NN; LOCALITY; NOISE REDUCTIONS; SVM;

ACOUSTIC NOISE MEASUREMENT; CLASSIFICATION (OF INFORMATION); SUPPORT VECTOR MACHINES;

DATA REDUCTION;

EID: 77956266577     PISSN: 09259902     EISSN: 15737675     Source Type: Journal    
DOI: 10.1007/s10844-009-0101-z     Document Type: Article

References (86)

1. DW Aha D Kibler MK Albert 1991 Instance-based learning algorithms Machine Learning 6 1 37 66
2. F Angiulli 2007 Fast nearest neighbor condensation for large data sets classification IEEE Transactions on Knowledge and Data Engineering 19 11 1450 1464 10.1109/TKDE.2007.190645
3. Asuncion, A.; & Newman, D. J. (2007). UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. http://www.ics.uci.edu/~mlearn/MLRepository.html.
4. Bakir, G. H.; Bottou, L.; & Weston, J. (2005). Breaking SVM complexity with cross-training. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (vol. 17, pp. 81-88). Cambridge: MIT.
5. Bello-Tomás, J. J.; González-Calero, P. A.; & Díaz-Agudo, B. (2004). JColibri: An object-oriented framework for building CBR systems. In Advances in case-based reasoning, 7th European conference, (ECCBR 2004), LNCS (vol. 3155, pp. 32-46). New York: Springer.
6. Beygelzimer, A.; Kakade, S.; & Langford, J. (2006). Cover trees for nearest neighbor. In 23rd international conference on machine learning (pp. 97-104).
7. Blanzieri, E.; & Bryl, A. (2007). Evaluation of the highest probability SVM nearest neighbor classifier with variable relative error cost. In Fourth conference on email and anti-spam, (CEAS 07). Mountain View, California.
8. Blanzieri, E.; & Melgani, F. (2006). An adaptive SVM nearest neighbor classifier for remotely sensed imagery. In IEEE international conference on geoscience and remote sensing symposium, (IGARSS 06) (pp. 3931-3934).
9. E Blanzieri F Melgani 2008 Nearest neighbor classification of remote sensing images with the maximal margin principle IEEE Transactions on Geoscience and Remote Sensing 46 6 1804 1811 10.1109/TGRS.2008.916090
10. Bottou, L.; & Lin, C. (2007). Support vector machine solvers. In Large scale kernel machines (pp. 1-28).
11. L Bottou V Vapnik 1992 Local learning algorithms Neural Computation 4 6 888 900 10.1162/neco.1992.4.6.888
12. Bottou, L.; Cortes, C.; Denker, J.; Drucker, H.; Guyon, I.; Jackel L.; et al. (1994). Comparison of classifier methods: A case study in handwritten digit recognition. In 12th IAPR international conference on pattern recognition (vol. 2).
13. H Brighton C Mellish 2002 Advances in instance selection for instance-based learning algorithms Data Mining and Knowledge Discovery 6 2 153 172 1027.68673 10.1023/A:1014043630878 1917935
14. Brodley, C. E. (1993). Addressing the selective superiority problem: Automatic algorithm/model class selection. In 10th international machine learning conference (ICML) (pp. 17-24). Amherst, MA.
15. Cabailero, Y.; Bello, R.; Garcia, M. M.; Pizano, Y.; Joseph, S.; & Lezcano, Y. (2005). Using rough sets to edit training set in k-NN Method. In 5th international conference on intelligent systems design and applications, (ISDA 05) (pp. 456-461).
16. Cameron-Jones, R. M. (1995). Instance selection by encoding length heuristic with random mutation hill climbing. In 8th Australian joint conference on artificial intelligence (pp. 99-106).
17. Cao, G.; Shiu, S.; & Wang, X. (2001). A fuzzy-rough approach for case base maintenance. In D. Aha & I. Watson (Eds.), Case-based reasoning research and development: 4th international conference on case-based reasoning (ICCBR 01), LNAI (vol. 2080, pp. 118-130).
18. Z Cataltepe YS Abu-mostafa M Magdon-ismail 1999 No free lunch for early stopping Neural Computation 11 995 1009 10.1162/089976699300016557
19. Chang, C. C.; & Lin, C. J. (2001). LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm.
20. CL Chang 1974 Finding prototypes for nearest neighbor classifiers IEEE Transactions on Computers C-23 11 1179 1184 10.1109/T-C.1974.223827
21. Chou, C. H.; Kuo, B. H.; & Chang, F. (2006). The generalized condensed nearest neighbor rule as a data reduction method. In 18th international conference on Pattern recognition (ICPR 06) (pp. 556-559). Washington, DC: IEEE Computer Society.
22. C Cortes V Vapnik 1995 Support-vector networks Machine Learning 20 3 273 297 0831.68098
23. Cunningham, P.; Doyle, D.; & Loughrey, J. (2003). An evaluation of the usefulness of case-based explanation. In 5th international conference on case-base reasoning (ICCBR 03) (pp. 122-130). New York: Springer.
24. SJ Delany D Bridge 2006 Textual case-based reasoning for spam filtering: A comparison of feature-based and feature-free approaches Artificial Intelligence Review 26 1-2 75 87 10.1007/s10462-007-9041-6
25. Delany, S. J.; & Cunningham, P. (2004). An analysis of case-based editing in a spam filtering system. In P. Funk & P. González-Calero (Eds.), Advances in case-based reasoning, 7th European conference on case-based reasoning (ECCBR 2004), LNAI (vol. 3155, pp. 128-141). New York: Springer.
26. J Demsar 2006 Statistical comparisons of classifiers over multiple data sets Journal of Machine Learning Research 7 1 30 2274360
27. B Díaz-Agudo P González-Calero J Recio-García A Sánchez 2007 Building CBR systems with jCOLIBRI Journal Science of Computer Programming 69 1-3 68 75 1129.68080 10.1016/j.scico.2007.02.004 (special issue on Experimental Software and Toolkits)
28. O Dunn 1961 Multiple comparisons among means Journal of the American Statistical Association 56 52 64 0103.37001 10.2307/2282330 124952
29. M Friedman 1937 The use of ranks to avoid the assumption of normality implicit in the analysis of variance Journal of the American Statistical Association 32 675 701 10.2307/2279372
30. M Friedman 1940 A comparison of alternative tests of significance for the problem of m rankings The Annals of Mathematical Statistics 11 86 92 0063.01455 10.1214/aoms/1177731944
31. A Gamberger N Lavrac S Dzeroski 2000 Noise detection and elimination in data preprocessing: Experiments in medical domains Applied Artificial Intelligence 14 2 205 223 10.1080/088395100117124
32. G Gates 1972 The reduced nearest neighbor rule IEEE Transactions on Information Theory 18 3 431 433 10.1109/TIT.1972.1054809
33. MG Genton 2001 Classes of kernels for machine learning: A statistics perspective Journal of Machine Learning Research 2 299 312 10.1162/ 15324430260185646
34. A Guttman 1984 R-trees: A dynamic index structure for spatial searching ACM Sigmod Record 14 2 47 57 10.1145/971697.602266
35. Hao, X.; Zhang, C.; Xu, H.; Tao, X.; Wang, S.; & Hu, Y. (2008). An improved condensing algorithm. In 7th IEEE/ACIS international conference on computer and information science, (ICIS 08) (pp. 316-321).
36. P Hart 1968 The condensed nearest neighbor rule IEEE Transactions on Information Theory 14 3 515 516 10.1109/TIT.1968.1054155
37. C Hsu C Lin 2002 A comparison of methods for multiclass support vector machines IEEE Transactions on Neural Networks 13 2 415 425 10.1109/72.991427
38. D Huang TWS Chow 2005 Enhancing density-based data reduction using entropy Neural Computation 18 2 470 495 10.1162/089976606775093927
39. Jiang, Y.; & Zhou, Z. (2004). Editing training data for knn classifiers with neural network ensemble. In F. Yin, J. Wang, & Guo C. (Eds.), Advances in neural networks (ISNN 2004),LNCS (vol. 3173, pp. 356-361). New York: Springer.
40. S Knerr L Personnaz G Dreyfus J Fogelman A Agresti M Ajiz, et al. 1990 Single-layer learning revisited: A stepwise procedure for building and training a neural network Optimization Methods and Software 1 23 34
41. J Koplowitz TA Brown 1981 On the relation of performance to editing in nearest neighbor rules Pattern Recognition 13 3 251 255 10.1016/0031-3203(81) 90102-3
42. Kressel, U.; et al. (1999). Pairwise classification and support vector machines. In Advances in kernel methods: support vector learning (pp. 255-268).
43. Leake, D. B. (1996). CBR in context: The present and future. In D. B. Leake (Ed.), Case based reasoning: Experiences, lessons, and future directions (pp. 3-30). Cambridge: MIT.
44. Y Lee O Mangasarian 2001 SSVM: A smooth support vector machine for classification Computational Optimization and Applications 20 1 5 22 1017.90105 10.1023/A:1011215321374 1849977
45. Li, R. L.; & Hu, J. F. (2003). Noise reduction to text categorization based on density for KNN. In International conference on machine learning and cybernetics (vol. 5, pp. 3119-3124).
46. HT Lin CJ Lin R Weng 2007 A note on Platt's probabilistic outputs for support vector machines Machine Learning 68 3 267 276 10.1007/s10994-007-5018-6
47. AC Lorena A Carvalho 2004 Evaluation of noise reduction techniques in the splice junction recognition problem Genetics and Molecular Biology 27 665 672 10.1590/S1415-47572004000400031
48. DG Lowe 1995 Similarity metric learning for a variable-kernel classifier Neural Computation 7 1 72 85 10.1162/neco.1995.7.1.72
49. A Malossini E Blanzieri RT Ng 2006 Detecting potential labeling errors in microarrays by data perturbation Bioinformatics 22 17 2114 2121 10.1093/bioinformatics/btl346
50. McKenna, E.; & Smyth, B. (2000). Competence-guided case-base editing techniques. In 5th European workshop on advances in case-based reasoning (ECCBR 00) (pp. 186-197). London: Springer.
51. P Mitra CA Murthy SK Pal 2002 Density-based multiscale data condensation IEEE Transactions on Pattern Analysis and Machine Intelligence 24 6 734 747 10.1109/TPAMI.2002.1008381
52. C Nugent D Doyle P Cunningham 2008 Gaining insight through case-based explanation Journal of Intelligent Information Systems 32 3 267 295 10.1007/s10844-008-0069-0
53. Osuna, E.; Freund, R.; & Girosi, F. (1997). Support vector machines: Training and applications. Tech. rep. Cambridge: Massachusetts Institute of Technology.
54. R Pan Q Yang SJ Pan 2007 Mining competent case bases for case-based reasoning Artificial Intelligence 171 16-17 1039 1068 1168.68487 10.1016/j.artint.2007.04.018 2375986
55. J Park K Im C Shin S Park 2004 MBNR: Case-based reasoning with local feature weighting by neural network Applied Intelligence 21 3 265 276 1101.68781 10.1023/B:APIN.0000043559.83167.3d
56. Pawlak, Z. (1992). Rough sets: Theoretical aspects of reasoning about data. Norwell: Kluwer.
57. Pechenizkiy, M.; Tsymbal, A.; Puuronen, S.; & Pechenizkiy, O. (2006). Class noise and supervised learning in medical domains: The effect of feature extraction. In 19th IEEE symposium on computer-based medical systems (CBMS 06) (pp. 708-713). Washington, DC: IEEE Computer Society.
58. J Platt N Cristianini J Shawe-Taylor 2000 Large margin DAGs for multiclass classification Advances in Neural Information Processing Systems 12 3 547 553
59. Platt, J. C. (1999a). Fast training of support vector machines using sequential minimal optimization (pp. 185-208). Cambridge: MIT.
60. Platt, J. C. (1999b). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In P. J. Bartlett, B. Schölkopf, D. Schuurmans, & A. J. Smola (Eds.), Advances in large margin classifiers (pp. 61-74). Boston: MIT.
61. J Quinlan 1986 The effect of noise on concept learning Machine learning: An artificial intelligence approach 2 149 166
62. G Ritter H Woodruff S Lowry T Isenhour 1975 An algorithm for a selective nearest neighbor decision rule IEEE Transactions on Information Theory 21 6 665 669 0323.68023 10.1109/TIT.1975.1055464
63. Roth-Berghofer, T. (2004). Explanations and case-based reasoning: Foundational issues. In P. Funk & P. A. González-Calero (Eds.), Advances in case-based reasoning, 7th European conference on case-based reasoning, (ECCBR 04), LNCS (vol. 3155, pp. 389-403). New York: Springer.
64. Salamó, M.; & Golobardes, E. (2001). Rough sets reduction techniques for case-based reasoning. In D. W. Aha & I. Watson (Eds.), Case-based reasoning research and development, 4th international conference on case-based reasoning, (ICCBR 01), LNCS (vol. 2080, pp. 467-482). New York: Springer.
65. Salamó, M.; & Golobardes, E. (2002). Deleting and building sort out techniques for case base maintenance. In S. Craw & A. D. Preece (Eds.), Advances in case-based reasoning, 6th European conference on case-based reasoning, (ECCBR 02), LNCS (vol. 2416, pp. 365-379). New York: Springer.
66. Salamó, M.; & Golobardes, E. (2004). Global, local and mixed rough sets case base maintenance techniques. In 6th Catalan conference on artificial intelligence (pp. 127-134). Amsterdam: IOS.
67. JS Sánchez R Barandela AI Marqués R Alejo J Badenas 2003 Analysis of new techniques to obtain quality training sets Pattern Recognition Letters 24 7 1015 1022 10.1016/S0167-8655(02)00225-8
68. Schölkopf, B.; & Smola, A. J. (2001). Learning with kernels: Support vector machines, regularization, optimization, and beyond (adaptive computation and machine learning). Cambridge: MIT.
69. Segata, N. (2009). FaLKM-lib v1.0: A library for fast local kernel machines. Tech. Rep. DISI-09-025, DISI, University of Trento. Software, Available at http://disi.unitn.it/~segata/FaLKM-lib.
70. Segata, N.; & Blanzieri, E. (2007). Operators for transforming kernels into quasi-local kernels that improve SVM accuracy. Tech. Rep. DISI-08-009, DISI, University of Trento.
71. Segata, N.; & Blanzieri, E. (2009a). Empirical assessment of classification accuracy of local SVM. In The 18th annual Belgian-Dutch conference on machine learning (Benelearn 2009) (pp. 47-55).
72. Segata, N.; & Blanzieri, E. (2009b). Fast local support vector machines for large datasets. In 6th international conference on machine learning and data mining (MLDM 09), LNCS (vol. 5632, pp. 295-310). New York: Springer.
73. Segata, N.; Blanzieri, E.; & Cunningham, P. (2009). A scalable noise reduction technique for large case-based systems. In 8th international conference on case-based reasoning (ICCBR 09), LNCS (vol. 5650, pp. 328-342). New York: Springer.
74. Smyth, B.; & Keane, M. (1995). Remembering to forget: A competence preserving case deletion policy for CBR system. In C. Mellish (Ed.), 14th international joint conference on artificial intelligence, (IJCAI 95) (pp. 337-382). San Francisco: Morgan Kaufmann.
75. Sriperumbudur, B. K.; & Lanckriet, G. (2007). Nearest neighbor prototyping for sparse and scalable support vector machines. Tech. rep.; Dept. of ECE, UCSD.
76. Tang, S.; & Chen, S. P. (2008a). An effective data preprocessing mechanism of ultrasound image recognition. In 2nd international conference on bioinformatics and biomedical engineering, (ICBBE 08) (pp. 2708-2711).
77. Tang, S.; & Chen, S. P. (2008b). Data cleansing based on mathematic morphology. In 2nd international conference on bioinformatics and biomedical engineering, (ICBBE 08) (pp. 755-758).
78. I Tomek 1976 An experiment with the edited nearest-neighbor rule IEEE Transactions on Systems, Man and Cybernetics 6 6 448 452 0332.68081 10.1109/TSMC.1976.4309523 428853
79. V Vapnik 1993 Principles of risk minimization for learning theory Advances in Neural Information Processing Systems 4 831 838
80. Vapnik, V. (1999). The nature of statistical learning theory (information science and statistics). New York: Springer.
81. Wess, S.; Althoff, K.; & Derwand, G. (1994). Using kd trees to improve the retrieval step in case-based reasoning. In Topics in case-based reasoning: 1st European workshop (EWCBR 93): Selected papers (p. 167). New York: Springer.
82. F Wilcoxon 1945 Individual comparisons by ranking methods Biometrics 1 6 80 83 10.2307/3001968
83. DL Wilson 1972 Asymptotic properties of nearest neighbor rules using edited data IEEE Transactions on Systems, Man and Cybernetics 2 3 408 421 0276.62060 10.1109/TSMC.1972.4309137
84. Wilson, D. R.; & Martinez, T. R. (1997). Instance pruning techniques. In 14th international conference on machine learning (ICML 97) (pp. 403-411).
85. DR Wilson TR Martinez 2000 Reduction techniques for instance-based learning algorithms Machine Learning 38 3 257 286 0954.68126 10.1023/A:1007626913721
86. Zhang, J. (1992). Selecting typical instances in instance-based learning. In 9th international workshop on Machine learning (ML 92) (pp. 470-479). San Francisco: Morgan Kaufmann.