Towards incremental classifier fusion

Intelligent Data Analysis

Volumn 14, Issue 1, 2010, Pages 3-30

  Sannen, Davy ^a   Lughofer, Edwin ^b   Van Brussel, Hendrik ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b JOHANNES KEPLER UNIVERSITY LINZ   (Austria)

Author keywords

Classifier fusion; Ensemble learning; Incremental learning; On line visual quality inspection

Indexed keywords

EID: 77649089043     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-2010-0406     Document Type: Article

References (69)

1. P.P. Angelov, E. Lughofer and X. Zhou Evolving fuzzy classifiers using different model architectures, Fuzzy Sets and Systems 159(23) (2008), 3160-3182.
2. L. Breiman, Pasting small votes for classification in large databases and on-line, Machine Learning 36(1-bb-2) (1999), 85-103.
3. L. Breiman, J. Friedman, C.J. Stone and R.A. Olshen, Classification and Regression Trees, Chapman and Hall, Boca Raton, 1993.
4. S.B. Cho and J.H. Kim, Combining multiple neural networks by fuzzy integral for robust classification, IEEE Transactions on Systems, Man, and Cybernetics 25(2) (1995), 380-384.
5. C. Cortes, L.D. Jackel, S.A. Solla, V. Vapnik and J.S. Denker, Learning Curves: Asymptotic Values and Rate of Convergence, in: Advances in Neural Information Processing Systems, J.D. Cowan, G. Tesauro and J. Alspector, eds, Morgan Kaufmann, 6 (1994), pp. 327-334.
6. S.L. Crawford, Extensions to the CART algorithm, International Journal of Man-Machine Studies 31 (1989), 197-217.
7. A.P. Dempster, A generalization of Bayesian inference, Journal of the Royal Statistical Society, Series B 30 (1968), 205-247.
8. C.P. Diehl and G. Cauwenberghs, SVM incremental learning, adaptation and optimization, In Proceedings of the International Joint Conference on Neural Networks 4 (2003), 2685-2690.
9. T.G. Dietterich, Ensemble methods in machine learning, Lecture Notes in Computer Science 1857 (2000), 1-15.
10. D. Dubois and H. Prade, Fuzzy Sets and Systems: Theory and Applications, Academic Press, New York, 1980.
11. R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis, John Wiley & Sons, New York, 1973.
12. R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification - Second Edition, Wiley-Interscience, Southern Gate, Chichester, West Sussex PO 19 8SQ, England, 2000.
13. J. Feng, An intelligent decision support system based on machine learning and dynamic track of psychological evaluation criterion, in: Intelligent Decision and Policy Making Support Systems, J. Kacpzryk, ed., Springer, Berlin Heidelberg, 2008, pp. 141-157.
14. D. P. Filev and F. Tseng, Novelty detection based machine health prognostics, In Proc. of the 2006 International Symposium on Evolving Fuzzy Systems, Lake District, UK, 2006, pp. 193-199.
15. Y. Freund and R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55 (1997), 119-139.
16. G. Fung and O. L. Mangasarian, Incremental support vector machine classification, In In Second SIAM International Conference on Data Mining, Arlington, VA, USA, 2002, pp. 247-260.
17. P.D. Gader, M.A. Mohamed and J.M. Keller, Fusion of handwritten word classifiers, Pattern Recognition Letters 17(6) (1996), 577-584.
18. R.M. Gray, Vector quantization, IEEE ASSP Magazine 1(2) (1984), 4-29.
19. L. Hansen and P. Salamon, Neural network ensembles, IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10) (1990), 993-1001.
20. T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer Verlag, New York, Berlin, Heidelberg, Germany, 2001.
21. Y.S. Huang and C.Y. Suen, A method of combining multiple experts for the recognition of unconstrained handwritten numerals, IEEE Transactions on Pattern Analysis and Machine Intelligence 17(1) (1995), 90-94.
22. R. Jacobs, M. Jordan, S.J. Nowlan and G.E. Hinton, Adaptive mixtures of local experts, Neural Computation 3 (1991), 79-87.
23. N. Kerdprasop and K. Kerdprasop, Data partitioning for incremental data mining, In Proceedings of the 1st International Forum on Information and Computer Science (IFICT 2003), Hamamatsu, Japan, 2003, pp. 114-118.
24. J.V. Kittler, M. Hatef, R.P.W. Duin and J. Matas, On combining classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3) (1998), 226-239.
25. A. Krogh and J. Vedelsby, Neural network ensembles, cross validation, and active learning, in: Proceedings of the 7th Conference on Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky and T. Leen, eds, volume 7, MIT Press, 1995, pp. 231-238.
26. L.I. Kuncheva, Using measures of similarity and inclusion for multiple classifier fusion by decision templates, Fuzzy Sets and Systems 122(3) (2001), 401-407.
27. L.I. Kuncheva, A theoretical study on six classifier fusion strategies, IEEE Transactions on Pattern Analysis and Machine Intelligence 24(2) (2002), 281-286.
28. L.I. Kuncheva, Classifier ensembles for changing environments, in: Multiple Classifier Systems, F. Roli, J. Kittler and T. Windeatt, eds, volume 3077 of Lecture Notes in Computer Science, Springer, 2004, pp. 1-15.
29. L.I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, Wiley, 2004.
30. L.I. Kuncheva, J.C. Bezdek and R.P.W. Duin, Decision templates for multiple classifier fusion: An experimental comparison, Pattern Recognition 34(2) (2001), 299-314.
31. L.I. Kuncheva, C.J. Whitaker, C.A. Shipp and R.P.W. Duin, Limits on the majority vote accuracy in classifier fusion, Pattern Analysis & Applications 6(1) (2003), 22-31.
32. X. Li, L. Wang and E. Sung, Multilabel SVM active learning for image classification, In Image Processing, 2004. ICIP '04. 2004 International Conference on, vol. 4, pages 2207-2010, 2004.
33. C.P. Lim and R.F. Harrison, Online pattern classification with multiple neural network systems: An experimental study, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 33(2) (2003), 235-247.
34. E. Lughofer, Evolving vector quantization for classification of on-line data streams, In Proc. of the Conference on Computational Intelligence for Modelling, Control and Automation (CIMCA 2008), Vienna, 2008, pp. 780-786.
35. E. Lughofer, Extensions of vector quantization for incremental clustering, Pattern Recognition 41(3) (2008), 995-1011.
36. E. Lughofer, P. Angelov and X. Zhou, Evolving single- and multi-model fuzzy classifiers with FLEXFIS-Class, In Proceedings of FUZZ-IEEE 2007, London, UK, 2007, pp. 363-368.
37. E. Lughofer and C. Guardiola, On-line fault detection with data-driven evolving fuzzy models, Journal of Control and Intelligent Systems 36(4) (2008), 307-317.
38. E. Lughofer, S. Raiser and R. Richter, Object recognition in deviation images - a comparison of methods, In Technical Report FLLL-TR-06-1, 2006.
39. E. Lughofer, J. E. Smith, P. Caleb-Solly, M.A. Tahir, C. Eitzinger, D. Sannen and M. Nuttin, On human-machine interaction during on-line image classifier training, Proc. of the International Conference on Computational Intelligence for Modelling, Control and Automation (CIMCA), Vienna, 2008, pp. 995-1001.
40. T. M. Mitchell, Machine Learning, McGraw-Hill Education, United Kingdom, 1997.
41. S. Mukherjee, P. Tamayo, S. Rogers, R.M. Rifkin, A. Engle, C. Campbell, T.R. Golub and J.P. Mesirov, Estimating Dataset Size Requirements for Classifying DNA Microarray Data, Journal of Computational Biology 10(2) (2003), 119-142.
42. N.C. Oza, Online Ensemble Learning, PhD thesis, University of California, USA, 2001.
43. S. Pang, S. Ozawa and N. Kasabov, Incremental linear discriminant analysis for classification of data streams, IEEE Transaction on Systems, Men and Cybernetics - part B: Cybernetics 35(5) (2005), 905-914.
44. R. Polikar, Ensemble based systems in decision making, IEEE Circuits and Systems Magazine 6(3) (2006), 21-45.
45. R. Polikar, L. Upda, S.S. Upda and V. Honavar, Learn++: An incremental learning algorithm for supervised neural networks, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 31(4) (2001), 497-508.
46. S.J. Qin, W. Li and H.H. Yue, Recursive PCA for adaptive process monitoring, Journal of Process Control 10 (2000), 471-486.
47. J. R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers Inc, USA, 1993.
48. G.L. Rogova, Combining the results of several neural network classifiers, Neural Networks 7(5) (1994), 777-781.
49. J.A. Roubos, M. Setnes and J. Abonyi, Learning fuzzy classification rules from data, Information Sciences-Informatics and Computer Science: An International Journal 150 (2003), 77-93.
50. S. Rüuping, Incremental learning with support vector machines, In Proceedings of the 1st IEEE International Conference on Data Mining (ICDM'01), San Jose, CA, USA, 2001, pp. 641-645.
51. D. Ruta and B. Gabrys, An overview of classifier fusion methods, Computing and Information Systems Journal 7(1) (2000), 1-10.
52. D. Sannen, M. Nuttin, J. Smith, M.A. Tahir, P. Caleb-Solly, E. Lughofer and C. Eitzinger, An on-line interactive self-adaptive image classification framework, in: Computer Vision Systems, A. Gasteratos, M. Vincze and J.K. Tsotsos, eds, volume 5008 of LNCS, Springer, 2008, pp. 171-180.
53. D. Sannen, H. Van Brussel and M. Nuttin, Classifier fusion using discounted Dempster-Shafer combination, in: Machine Learning and Data Mining in Pattern Recognition, 5th International Conference, MLDM 2007, Poster Proceedings, P. Perner, ed., IBaI publishing, 2007, pp. 216-230.
54. G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976.
55. M. Song and H. Wang, Incremental estimation of gaussian mixture models for online data stream clustering, In Proc. of International Conference on Bioinformatics and its Applications, Fort Lauderdale, Florida, USA, 2004.
56. M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Statistical Society 36 (1974), 111-147.
57. R.S. Sutton and S.D. Whitehead, Online learning with random representations, In Proceedings of the 10th International Conference on Machine Learning, San Mateo, CA, USA, Morgan Kaufmann, 1993, pp. 314-321.
58. P. Utgoff, Incremental induction of decision trees, Machine Learning 4 (1989), 161-186.
59. G. Valentini and F. Masulli, Ensembles of learning machines, in: Neural Nets: 13th Italian Workshop on Neural Nets, Vietri sul Mare, Italy, 2002, Revised Papers, M. Marinaro and R. Tagliaferri, eds, volume 2486 of Lecture Notes in Computer Science, Springer, 2002, pp. 3-22.
60. V.N. Vapnik, S. Golowich and A. Smola, Support vector method for function approximation, regression estimation and signal processing, in: Advances in Neural Information Processing Systems, M. Mozer, M. Jordan and T. Petsche, eds, Cambridge, MA, MIT Press, 9 (1997), pp. 281-287.
61. V. Vapnik and O. Chapelle, Bounds On Error Expectation for Support Vector Machines, Neural Computation 12(9) (2000), 2013-2036.
62. K. Veropoulos, Machine Learning Approaches to Medical Decision Making, PhD thesis, Department of Computer Science, University of Bristol.
63. H. Wang, W. Fan, P.S. Yu and J. Han, Mining concept-drifting data streams using ensemble classifiers, In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, 2003, pp. 226-235. ACM.
64. P.D. Wasserman, Advanced Methods in Neural Computing, John Wiley & Sons, Inc., New York, 1993.
65. T. Wetter, Medical decision support systems, In Lecture Notes in Computer Science, eds, Medical Data Analysis, Springer, Berlin / Heidelberg, 2000, pp. 458-466.
66. G. Widmer and M. Kubat, Learning in the presence of concept drift and hidden contexts, Machine Learning 23(1) (1996), 69-101.
67. D.H. Wolpert, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation 1(1) (1997), 67-82.
68. K. Woods, W.P. Kegelmeyer and K. Bowyer, Combination of multiple classifiers using local accuracy estimates, IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4) (1997), 405-410.
69. L. Xu, A. Krzyźak and C.Y. Suen, Methods of combining multiple classifiers and their applications to handwriting recognition, IEEE Transactions on Systems, Man, and Cybernetics 22(3) (1992), 418-435.