Transductive active learning - A new semi-supervised learning approach based on iteratively refined generative models to capture structure in data

Information Sciences

Volumn 293, Issue , 2015, Pages 275-298

  Reitmaier, Tobias ^a   Calma, Adrian ^a   Sick, Bernhard ^a  

^a UNIVERSITY OF KASSEL   (Germany)

Author keywords

Generative modeling; Pool based active learning; Semi supervised learning; Separate components model; Shared components model; Transductive learning

Indexed keywords

CLUSTERING ALGORITHMS; ITERATIVE METHODS; LAKES; LEARNING SYSTEMS;

ACTIVE LEARNING; ACTIVE-LEARNING PROCESS; CLASSIFICATION PERFORMANCE; GENERATIVE MODEL; PROBABILISTIC CLASSIFIERS; SEMI-SUPERVISED; SHARED COMPONENTS; TRANSDUCTIVE LEARNING;

SEMI-SUPERVISED LEARNING;

EID: 84922594422     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2014.09.009     Document Type: Article

References (65)

1. N. Abe, H. Mamitsuka, Query learning strategies using boosting and bagging, in: Proceedings of the Fifteenth International Conference on Machine Learning (ICML '98), Madison, WI, 1998, pp. 1-9.
2. D. Angluin Queries and concept learning Machine Learn. 2 4 1988 319 342
3. A. Asuncion, D. Newman, UCI Machine Learning Repository, < http://archive.ics.uci.edu/ml/ > (04/14/2014).
4. L. Atlas, D. Cohn, R. Ladner, M.A. El-Sharkawi, and R.J. Marks II Training connectionist networks with queries and selective sampling Advances in Neural Information Processing Systems 2 1990 Morgan Kaufmann Denver, CO 566 573
5. C.M. Bishop Neural Networks for Pattern Recognition 1995 Oxford University Press Oxford
6. C.M. Bishop Pattern Recognition and Machine Learning 2006 Springer New York, NY
7. A. Blum, T. Mitchell, Combining labeled and unlabeled data with co-training, in: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, New York, NY, 1998, pp. 92-100.
8. U. Brefeld, T. Scheffer, Co-EM support vector learning, in: Proceedings of the Twenty-First International Conference on Machine Learning (ICML '04), Banff, AB, 2004, pp. 121-128.
9. O. Buchtala, A. Hofmann, B. Sick, Fast and efficient training of RBF networks, in: Proceedings of the 13th International Conference on Artificial Neural Networks (ICANN'03), Istanbul, Turkey, 2003, pp. 43-51.
10. O. Buchtala, P. Neumann, B. Sick, A strategy for an efficient training of radial basis function networks for classification applications, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN'03), Portland, OR, 2003, pp. 1025-1030.
11. C.G. Cawlay, Baseline methods for active learning, in: JMLR: Workshop and Conference Proceedings 16, Sardinia, Italy, 2011, pp. 47-57.
12. N. Cebron, and M.R. Berthold Active learning for object classification: from exploration to exploitation Data Min. Knowl. Disc. 18 2 2009 283 299
13. C.-C. Chang, and C.-J. Lin LIBSVM: a library for support vector machines ACM Trans. Intell. Syst. Technol. 2 2011 27:1 27:27
14. D.A. Cohn Neural network exploration using optimal experiment design Neural Netw. 9 6 1996 1071 1083
15. C. Constantinopoulos, and A. Likas Semi-supervised and active learning with the probabilistic RBF classifier Neurocomputing 71 13-15 2008 2489 2498
16. C. Constantinopoulos, and A.C. Likas Efficient training algorithms for the probabilistic RBF network G.A. Vouros, T. Panayiotopoulos, Third Helenic Conference on Artificial Intelligence (SETN '04) Lecture Notes in Computer Science vol. 3025 2004 Springer 183 190
17. C. Constantinopoulos, and A.C. Likas An incremental training method for the probabilistic RBF network IEEE Trans. Neural Netw. 17 4 2006 966 974
18. C. Constantinopoulos, and A.C. Likas Unsupervised learning of Gaussian mixtures based on variational component splitting IEEE Trans. Neural Netw. 18 3 2007 745 755
19. M. Culver, D. Kun, S. Scott, Active learning to maximize area under the ROC curve, in: Proceedings of the Sixth International Conference on Data Mining (ICDM '06), Hong Kong, China, 2006, pp. 149-158.
20. C.K. Dagli, S. Rajaram, T.S. Huang, Utilizing information theoretic diversity for SVM active learning, in: Proceedings of the 18th International Conference on Pattern Recognition (ICPR '06), Hong Kong, China, 2006, pp. 506-511.
21. V. de Sa Learning classification with unlabeled data Adv. Neural Inform. Process. Syst. 6 1994 112 119
22. P. Donmez, J.G. Carbonell, P.N. Bennett, Dual strategy active learning, in: Proceedings of the 18th European Conference on Machine Learning (ECML '07), Warsaw, Poland, 2007, pp. 116-127.
23. R.O. Duda, P.E. Hart, and D.G. Stork Pattern Classification 2001 John Wiley & Sons Chichester, NY
24. D. Fisch, E. Kalkowski, and B. Sick Knowledge fusion for probabilistic generative classifiers with data mining applications IEEE Trans. Knowl. Data Eng. 26 3 2014 652 666
25. D. Fisch, E. Kalkowski, B. Sick, S.J. Ovaska, In your interest - objective interestingness measures for a generative classifier, in: Proceedings of the third International Conference on Agents and Artificial Intelligence (ICAART '11), Rome, Italy, 2011, pp. 414-423.
26. D. Fisch, B. Sick, Training of radial basis function classifiers with resilient propagation and variational Bayesian inference, in: International Joint Conference on Neural Networks (IJCNN '09), Atlanta, GA, 2009, pp. 838-847.
27. C. Fraley, and A.E. Raftery How many clusters? Which clustering method? Answers via model-based cluster analysis Comput. J. 41 8 1998 578 588
28. J. Grim, Maximum-likelihood design of layered neural networks, in: Proceedings of the 13th International Conference on Pattern Recognition, Wien, Austria, 1996, pp. 85-89.
29. Y. Guo, R. Greiner, Optimistic active learning using mutual information, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI '07), Hyderabad, India, 2007, pp. 823-829.
30. I. Guyon, G. Cawley, G. Dror, V. Lemaire, Results of the active learning challenge, in: JMLR: Workshop and Conference Proceedings 16, Sardinia, Italy, 2011, pp. 19-45.
31. T. Hastie, and R. Tibshirani Discriminant analysis by Gaussian mixtures J. Roy. Stat. Soc. Ser. B 58 1 1996 155 176
32. S.C. Hoi, R. Jin, M.R. Lyu, Large-scale text categorization by batch mode active learning, in: Proceedings of the 15th International World Wide Web conference (WWW '06), Edinburgh, Scotland, UK, 2006, pp. 633-642.
33. J. Jun, I. Horace, Active learning with SVM, in: J. Ramón, R. Dopico, J. Dorado, A. Pazos (Eds.), Encyclopedia of Artificial Intelligence, vol. 3, IGI Global, Hershey, PA, 2009, pp. 1-7.
34. S.S. Keerthi, and C.-J. Lin Asymptotic behaviors of support vector machines with Gaussian kernel Neural Comput. 15 7 2003 1667 1689
35. D.D. Lewis, W.A. Gale, A sequential algorithm for training text classifiers, in: Proceedings of the Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '94), Dublin, Ireland, 1994, pp. 3-12.
36. K. Li, W. Zhang, X. Ma, Z. Cao, C. Zhang, A novel semi-supervised SVM based on tri-training, in: Proceedings of the 2008 Second International Symposium on Intelligent Information Technology Application (IITA '08), Maputo, Mosambique, 2008, pp. 47-51.
37. M.-W. Mak, and S.-Y. Kung Estimation of elliptical basis function parameters by the EM algorithm with application to speaker verification IEEE Trans. Neural Netw. 11 4 2000 961 969
38. G.J. McLachlan, and D. Peel Finite Mixture Models 2000 Wiley Series in Probability and Statistics New York, NY
39. P. Mitra, C.A. Murthy, and S.K. Pal A probabilistic active support vector learning algorithm IEEE Trans. Pattern Anal. Machine Intell. 26 3 2004 413 418
40. J. Moody, and C.J. Darken Fast learning in networks of locally-tuned processing units Neural Comput. 1 2 1989 281 294
41. I. Muslea, S. Minton, C.A. Knoblock, Selective sampling with redundant views, in: Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence (AAAI '00), Austin, TX, 2000, pp. 621-626.
42. I. Muslea, S. Minton, C.A. Knoblock, Active + semi-supervised learning = robust multi-view learning, in: Proceedings of the Nineteenth International Conference on Machine Learning (ICML '02), Sydney, Australia, 2002, pp. 435-442.
43. H.T. Nguyen, A. Smeulders, Active learning using pre-clustering, in: Proceedings of the Twenty-First International Conference on Machine learning (ICML '04), Banff, AB, 2004, pp. 623-630.
44. K. Nigam, R. Ghani, Analyzing the effectiveness and applicability of co-training, in: Proceedings of the Ninth International Conference on Information and Knowledge Management (CIKM '00), McLean, VA, 2000, pp. 86-93.
45. D. Pierce, C. Cardie, Limitations of co-training for natural language learning from large datasets, in: Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing (EMNLP '01), Pittsburgh, PA, 2001, pp. 1-9.
46. T. Reitmaier, B. Sick, Active classifier training with the 3DS strategy, in: IEEE Symposium on Computational Intelligence and Data Mining (CIDM '11), France, Paris, 2011, pp. 88-95.
47. T. Reitmaier, and B. Sick Let us know your decision: pool-based active training of a generative classifier with the selection strategy 4DS Inform. Sci. 230 2013 106 131
48. B. Ripley, Pattern recognition and neural networks, < http://www.stats.ox.ac.uk/pub/PRNN/ > (04/14/2014).
49. H.J. Scudder Probability of error of some adaptive pattern recognition machines Proc. IEEE Trans. Inform. Theory 1956 363 371
50. B. Settles, Active learning literature survey, Computer Sciences Technical Report 1648, University of Wisconsin, Department of Computer Science, 2009.
51. B. Settles, From theories to queries: active learning in practice, in: JMLR: Workshop and Conference Proceedings 16, Sardinia, Italy, 2011, pp. 1-18.
52. B. Settles, M. Craven, An analysis of active learning strategies for sequence labeling tasks, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP '08), Honolulu, HI, 2008, pp. 1070-1079.
53. H.S. Seung, M. Opper, H. Sompolinsky, Query by committee, in: Proceedings of the Fifth Annual ACM Conference on Computational Learning Theory (COLT '92), Pittsburgh, PA, 1992, pp. 287-294.
54. V. Sindhwan, P. Melville, R.D. Lawrence, Uncertainty sampling and transductive experimental design for active dual supervision, in: Proceedings of the 26th International Conference on Machine Learning (ICML '09), vol. 382, Montreal, Quebec, CA, 2009, pp. 953-960.
55. K. Small, and D. Roth Margin-based active learning for structured predictions Int. J. Machine Learn. Cybernet. 1 1-4 2010 413 424
56. R.L. Streit, and T.E. Luginbuhl Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Netw. 5 5 1994 764 783
57. M.K. Titsias, A. Likas, A probabilistic RBF network for classification, in: Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN '00), vol. 4, Como, Italy, 2000, pp. 238-243.
58. M.K. Titsias, and A.C. Likas Shared kernel models for class conditional density estimation IEEE Trans. Neural Netw. 12 5 2001 987 997
59. S. Tong, and D. Koller Support vector machine active learning with applications to text classification J. Machine Learn. Res. 2 2002 45 66
60. UCL, UCL/MLG Elena Database, < http://mlg.info.ucl.ac.be/index.php?page=Elena > (04/14/2014).
61. X.-Z. Wang, L.-C. Dong, and J.-H. Yan Maximum ambiguity-based sample selection in fuzzy decision tree induction IEEE Trans. Knowl. Data Eng. 24 8 2012 1491 1505
62. R. Xu, and D. Wunsch Survey of clustering algorithms IEEE Trans. Neural Netw. 16 3 2005 645 678
63. K. Yu, J. Bi, V. Tresp, Active learning via transductive experimental design, in: Proceedings of the 23rd International Conference on Machine Learning (ICML '06), Pittsburgh, PA, 2006, pp. 1081-1088.
64. Y. Zhou, S. Goldman, Democratic co-learning, in: Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI '04), Boca Raton, FL, 2004, pp. 594-602.
65. Z.-H. Zhou, and M. Li Tri-training: exploiting unlabeled data using three classifiers IEEE Trans. Knowl. Data Eng. 17 11 2005 1529 1541