A method for stopping active learning based on stabilizing predictions and the need for user-adjustable stopping

CoNLL 2009 - Proceedings of the Thirteenth Conference on Computational Natural Language Learning

Volumn , Issue , 2009, Pages 39-47

  Bloodgood, Michael ^a   Vijay Shanker, K ^b  

^a Whiting School of Engineering   (United States)

^b Institute University of Delaware Newark   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

ACTIVE LEARNING; ANNOTATION PERFORMANCE; STOPPING METHODS; ACTIVE SUPPORTS; PERFORMANCE TRADEOFF;

LEARNING SYSTEMS; ARTIFICIAL INTELLIGENCE;

EID: 84858247890     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/1596374.1596384     Document Type: Conference Paper

References (20)

1. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555-596.
2. Jason Baldridge and Miles Osborne. 2008. Active learning and logarithmic opinion pools for hpsg parse selection. Nat. Lang. Eng., 14(2):191-222.
3. Michael Bloodgood and K. Vijay-Shanker. 2009. Taking into account the differences between actively and passively acquired data: The case of active learning with support vector machines for imbalanced datasets. In NAACL.
4. Colin Campbell, Nello Cristianini, and Alex J. Smola. 2000. Query learning with large margin classifiers. In ICML'00: Proceedings of the Seventeenth International Conference on Machine Learning, pages 111-118, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
5. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37-46.
6. Gordon Cormack and Thomas Lynam. 2005. Trec 2005 spam track overview. In TREC-14.
7. Susan Dumais, John Platt, David Heckerman, and Mehran Sahami. 1998. Inductive learning algorithms and representations for text categorization. In CIKM'98: Proceedings of the seventh international conference on Information and knowledge management, pages 148-155, New York, NY, USA. ACM.
8. Seyda Ertekin, Jian Huang, Léon Bottou, and C. Lee Giles. 2007. Learning on the border: active learning in imbalanced data classification. In Mário J. Silva, Alberto H. F. Laender, Ricardo A. Baeza-Yates, Deborah L. McGuinness, Bjørn Olstad, Øystein Haug Olsen, and André O. Falcão, editors, Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, CIKM 2007, Lisbon, Portugal, November 6-10, 2007, pages 127-136. ACM.
9. Thorsten Joachims. 1998. Text categorization with suport vector machines: Learning with many relevant features. In ECML, pages 137-142.
10. Thorsten Joachims. 1999. Making large-scale SVM learning practical. In Advances in Kernel Methods - Support Vector Learning, pages 169-184.
11. Florian Laws and Hinrich Schütze. 2008. Stopping criteria for active learning of named entity recognition. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 465-472, Manchester, UK, August. Coling 2008 Organizing Committee.
12. Ki-Joong Lee, Young-Sook Hwang, Seonho Kim, and Hae-Chang Rim. 2004. Biomedical named entity recognition using two-phase model based on svms. Journal of Biomedical Informatics, 37(6):436-447.
13. AndrewMcCallum and Kamal Nigam. 1998. A comparison of eventmodels for naive bayes text classification. In Proceedings of AAAI-98, Workshop on Learning for Text Categorization.
14. Greg Schohn and David Cohn. 2000. Less is more: Active learning with support vector machines. In Proc. 17th International Conf. on Machine Learning, pages 839-846.Morgan Kaufmann, San Francisco, CA.
15. D. Sculley. 2007. Online active learning methods for fast label-efficient spam filtering. In Conference on Email and Anti-Spam (CEAS), Mountain View, CA, USA.
16. Simon Tong and Daphne Koller. 2002. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research (JMLR), 2:45-66.
17. Andreas Vlachos. 2008. A stopping criterion for active learning. Computer Speech and Language, 22(3):295-312.
18. Jingbo Zhu and Eduard Hovy. 2007. Active learning for word sense disambiguation with methods for addressing the class imbalance problem. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 783-790.
19. Jingbo Zhu, Huizhen Wang, and Eduard Hovy. 2008a. Learning a stopping criterion for active learning for word sense disambiguation and text classification. In IJCNLP.
20. Jingbo Zhu, Huizhen Wang, and Eduard Hovy. 2008b. Multi-criteria-based strategy to stop active learning for data annotation. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 1129-1136, Manchester, UK, August. Coling 2008 Organizing Committee.