Hybrid active learning for reducing the annotation effort of operators in classification systems

Pattern Recognition

Volumn 45, Issue 2, 2012, Pages 884-896

  Lughofer, Edwin ^a  

^a JOHANNES KEPLER UNIVERSITY LINZ   (Austria)

Author keywords

Active learning; Certainty based selection; On line update of classifiers; Reduction of annotation effort; Unsupervised selection criteria

Indexed keywords

ACTIVE LEARNING; CERTAINTY-BASED SELECTION; CLASSIFICATION ACCURACY; CLASSIFICATION SYSTEM; CLUSTER CENTERS; DATA SAMPLE; DATA-DRIVEN; DISTRIBUTION CHARACTERISTICS; INPUT SAMPLE; NEW STRATEGY; OFF-LINE TRAINING; ON-LINE CLASSIFICATION; ON-LINE QUALITY CONTROL; ONLINE TRAINING; PREDICTIVE POWER; REAL WORLD DATA; SAMPLE SELECTION; SECOND PHASE; SELECTION CRITERIA; TRAINING PROCESS; TRAINING SAMPLE; UCI REPOSITORY;

LEARNING ALGORITHMS; QUALITY CONTROL; VIRTUAL REALITY;

PERSONNEL TRAINING;

EID: 80052961340     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.08.009     Document Type: Article

References (56)

1. C. Eitzinger, W. Heidl, E. Lughofer, S. Raiser, J. Smith, M. Tahir, D. Sannen, and H. van Brussel Assessment of the influence of adaptive components in trainable surface inspection systems Machine Vision and Applications 21 5 2010 613 626
2. M. Carey, E. Parris, and H. Lloyd-Thomas A comparison of features for speech, music discrimination Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 1999 1999 IEEE Press Phoenix, USA 149 152
3. J. Bergstra, N. Casagrande, D. Erhan, D. Eck, and B. Kegl Aggregate features and ADA-boost for music classification Machine Learning 65 2006 473 484 (Pubitemid 44836054)
4. E. Lughofer On-line evolving image classifiers and their application to surface inspection Image and Vision Computing 28 7 2010 1065 1079
5. C. Diehl, G. Cauwenberghs, SVM incremental learning, adaptation and optimization, in: Proceedings of the International Joint Conference on Neural Networks, vol. 4, Boston, 2003, pp. 26852690.
6. A. Bouchachia, Incremental induction of classification fuzzy rules, in: IEEE Workshop on Evolving and Self-Developing Intelligent Systems (ESDIS) 2009, Nashville, USA, 2009, pp. 3239.
7. E. Lughofer, J.E. Smith, P. Caleb-Solly, M. Tahir, C. Eitzinger, D. Sannen, and M. Nuttin On humanmachine interaction during on-line image classifier training IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 39 5 2009 960 971
8. M. Costabile, D. Fogli, P. Mussion, and A. Piccinno Visual interactive systems for end-user development: a model-based design methodology IEEE Transactions on Systems, Man and Cybernetics, Part A: Cybernetics 37 6 2007 1029 1046 (Pubitemid 350130841)
9. J. Zhou, H. Peng, and C.Y. Suen Data-driven decomposition for multi-class classification Pattern Recognition 41 1 2008 67 76 (Pubitemid 47299107)
10. H. He, and E. Garcia Learning from imbalanced data IEEE Transactions on Knowledge and Data Engineering 21 9 2009 1263 1284
11. D. Cohn, L. Atlas, and R. Ladner Improving generalization with active learning Machine Learning 15 2 1994 201 221
12. D. Mackay Information-based objective functions for active data selection Neural Computation 4 4 1992 305 318
13. A. Fernald, and P. Kuhl Acoustic determinants of infant preference for motherese speech Infant Behavior and Development 10 3 1987 279 293
14. D. Lewis, J. Catlett, Heterogeneous uncertainty sampling for supervised learning, in: Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, New Jersey, 1994, pp. 148156.
15. C. Thompson, M. Califf, R. Mooney, Active learning for natural language parsing and information extraction, in: Proceedings of Sixteenth International Conference on Machine Learning, Bled, Slovenia, 1999, pp. 406414.
16. I. Dagan, S. Engelson, Committee-based sampling for training probabilistic classifier, in: Proceedings of Twelfth International Conference on Machine Learning, 1995, pp. 150157.
17. I. Muslea, Active Learning with Multiple Views, Ph.D. Thesis, University of Southern California, 2000.
18. J. Hühn, and E. Hüllermeier FR3: a fuzzy rule learner for inducing reliable classifiers IEEE Transactions on Fuzzy Systems 17 1 2009 138 149
19. J. Cheng, and K. Wang Active learning for image retrieval with co-SVM Pattern Recognition 40 2006 330 334
20. G. Tur, D.-H. Tür, and R. Schapire Combining active and semi-supervised learning for spoken language understanding Speech Communication 45 2 2005 171 186 (Pubitemid 40220192)
21. E. Lughofer, C. Guardiola, Applying evolving fuzzy models with adaptive local error bars to on-line fault detection, in: Proceedings of Genetic and Evolving Fuzzy Systems 2008, Witten-Bommerholz, Germany, 2008, pp. 3540.
22. K. Fukumizu Statistical active learning in multilayer perceptrons IEEE Transactions on Neural Networks 11 1 2000 17 26 (Pubitemid 32132884)
23. S. Tong, and D. Koller Support vector machine active learning with application to text classification Journal of Machine Learning Research 2 2001 45 66
24. D. Cohn, Z. Ghahramani, and M. Jordan Active learning with statistical models Journal of Artificial Intelligence Research 4 1 1996 129 145 (Pubitemid 126646151)
25. N. Roy, A. Mccallum, Toward optimal active learning through sampling estimation of error reduction, in: In Proceedings of the Eighteenth International Conference on Machine Learning, Morgan Kaufmann, 2001, pp. 441448.
26. H. Raghavan, O. Madani, and R. Jones Active learning with feedback on both features and instances Journal of Machine Learning Research 7 2006 1655 1686 (Pubitemid 44223089)
27. W. Hu, W. Hu, N. Xi, and S. Maybank Unsupervised active learning based on hierarchical graph-theoretic clustering IEEE Transactions on Systems Man and Cybernetics - Part B: Cybernetics 39 5 2009 1147 1161
28. E. Lughofer Extensions of vector quantization for incremental clustering Pattern Recognition 41 3 2008 995 1011 (Pubitemid 47632671)
29. J. Hartigan Clustering Algorithms 1975 John Wiley & Sons New York, USA
30. J. Bezdek Pattern Recognition with Fuzzy Objective Function Algorithms 1981 Kluwer Academic/Plenum Publishers USA
31. D. Sannen, E. Lughofer, H.V. Brussel, Increasing on-line classification performance using incremental classifier fusion, in: Proceedings of the International Conference on Adaptive and Intelligent Systems (ICAIS'09), Klagenfurt, Austria, 2009, pp. 101107.
32. P. Utgoff Incremental induction of decision trees Machine Learning 4 2 1989 161 186
33. P. Domingos, G. Hulten, Catching up with the data: research issues in mining data streams, in: Proceedings of the Workshop on Research Issues in Data Mining and Knowledge Discovery, Santa Barbara, CA, 2001.
34. H. Mohammed, J. Leander, M. Marbach, R. Polikar, Can adaboost.m1 learn incrementally? A comparison to learn under different combination rules, in: Proceedings of the Artificial Neural Networks Conference 2006, Athens, Greece, 2006, pp. 172179.
35. R. Polikar, L. Upda, S. Upda, and V. Honavar Learn: An incremental learning algorithm for supervised neural networks IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 31 4 2001 497 508 (Pubitemid 34191829)
36. N.K. Kasabov Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 31 6 2001 902 918
37. N.K. Kasabov, and Q. Song DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction IEEE Transactions on Fuzzy Systems 10 2 2002 144 154 (Pubitemid 34554860)
38. E. Lughofer FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models IEEE Transactions on Fuzzy Systems 16 6 2008 1393 1410
39. E. Lughofer, Evolving vector quantization for classification of on-line data streams, in: Proceedings of the Conference on Computational Intelligence for Modelling, Control and Automation (CIMCA 2008), Vienna, Austria, 2008, pp. 780786.
40. R. Gray Vector quantization IEEE ASSP Magazine 1 2 1984 4 29
41. E. Lughofer Towards robust evolving fuzzy systems P. Angelov, D. Filev, N. Kasabov, Evolving Intelligent Systems: Methodology and Applications 2010 John Wiley & Sons New York 87 126
42. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning: Data Mining, Inference and Prediction second ed. 2009 Springer New York, Berlin, Heidelberg
43. G. Widmer, and M. Kubat Learning in the presence of concept drift and hidden contexts Machine Learning 23 1 1996 69 101 (Pubitemid 126737384)
44. E. Lughofer, P. Angelov, X. Zhou, Evolving single- and multi-model fuzzy classifiers with FLEXFIS-Class, in: Proceedings of FUZZ-IEEE 2007, London, UK, 2007, pp. 363368.
45. T. Takagi, and M. Sugeno Fuzzy identification of systems and its applications to modeling and control IEEE Transactions on Systems, Man and Cybernetics 15 1 1985 116 132
46. N. Draper, and H. Smith Applied Regression Analysis. Probability and Mathematical Statistics 1981 John Wiley & Sons New York
47. L. Wang, and J. Mendel Fuzzy basis functions universal approximation and orthogonal least-squares learning IEEE Transactions on Neural Networks 3 5 1992 807 814 (Pubitemid 23555771)
48. E. Lughofer Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications 2011 Springer Berlin, Heidelberg
49. S. Raiser, E. Lughofer, C. Eitzinger, and J. Smith Impact of object extraction methods on classification performance in surface inspection systems Machine Vision and Applications 21 5 2010 627 641
50. UCI machine learning repository: center for machine learning and intelligent systems (19872011). 〈 http://archive.ics.uci.edu/ml/datasets/ 〉.
51. A. Jain, and R. Dubes Algorithms for Clustering Data 1988 Prentice Hall Upper Saddle River, New Jersey
52. L. Breiman, J. Friedman, C. Stone, and R. Olshen Classification and Regression Trees 1993 Chapman and Hall Boca Raton
53. R. Duda, P. Hart, D. Stork, Pattern Classification, second ed. Wiley-Interscience (John Wiley & Sons), Southern Gate, Chichester, West Sussex, England, 2000.
54. V. Vapnik Statistical Learning Theory 1998 Wiley and Sons New York
55. B. Schölkopf, and A. Smola Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond 2002 MIT Press London, England
56. C.-W. Hsu, C.-C. Chang, C.-J. Lin, A practical guide to support vector classification, 2006.