Assessing the costs of machine-assisted corpus annotation through a user study

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 3318-3324

  Ringger, Eric ^a   Carmen, Marc ^a   Haertel, Robbie ^a   Seppi, Kevin ^a   Lonsdale, Deryle ^a   McClanahan, Peter ^a   Carroll, James ^a   Ellison, Noel ^a  

^a 3361 TMCB   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BUDGET CONTROL; COSTS;

ACTIVE LEARNING; CORPUS ANNOTATIONS; EXPERIMENTAL PARAMETERS; EXPERT ANNOTATIONS; INFERENTIAL STATISTICS; SKILL LEVELS; USER STUDY; VARYING BACKGROUND;

COST ESTIMATING;

EID: 84859987891     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. Engelson, S. and Dagan, I. (1996). "Minimizing manual annotation cost in supervised training from corpora." ACL. Pp. 319-326.
2. Hwa, R. (2000). "Sample selection for statistical grammar induction". In Proceedings of EMNLP/VLC-2000. Pp. 45-52.
3. Lewis, D., and Catlett, J. (1994). "Heterogeneous uncertainty sampling for supervised learning." ICML.
4. Lewis, D., and Gale, W. (1995). "A sequential algorithm for training text classifiers: Corrigendum and additional data." SIGIR Forum, 29(2), Pp. 13-19.
5. Ngai, G. and Yarowsky, D. (2000) "Rule Writing or An-notation: Cost-efficient Resource Usage for Base Noun Phrase Chunking." ACL. Pp. 117-125.
6. Osborne, M., & Baldridge, J. (2004). "Ensemble-based AL for Parse Selection". HLT-NAACL 2004, Boston, Massachusetts, USA. Pp. 89-96.
7. Ratnaparkhi, A. (1996). "A Maximum Entropy Model for Part-Of-Speech Tagging." EMNLP.
8. Ringger, E., McClanahan, P., Haertel, R., Busby, G., Carmen, M., Carroll, J., Seppi, K., & Lonsdale, D. (2007). "Active Learning for Part-of-Speech Tagging: Accelerating Corpus Annotation". ACL Linguistic Annotation Workshop (LAW) 2007. Prague, Czech Republic.
9. Seung, H., Opper, M., & Sompolinsky, H. (1992). "Query by committee". COLT. Pp. 287-294.
10. Thrun S., and Moeller, K. (1992). "Active exploration in dynamic environments." NIPS.
11. Tomanek, K., Wermter, J., & Hahn, U. (2007). "An Approach to Text Corpus Construction which Cuts Annotation Costs and Maintains Reusability of An-notated Data." EMNLP. Pp. 486-495.
12. Toutanova, K. & Manning, C. (2000). "Enriching the Knowledge Sources Used in a Maximum Entropy Part-of-Speech Tagger." EMNLP. Pp. 63-70.
13. Toutanova, K., Klein, D., Manning, C., and Singer, Y. (2003). "Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network." HLT-NAACL. Pp. 252-259.