Annotation of nuggets and relevance in GALE distillation evaluation

Proceedings of the 6th International Conference on Language Resources and Evaluation, LREC 2008

Volumn , Issue , 2008, Pages 3578-3584

  Babko Malaya, Olga ^a  

^a BAE SYSTEMS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DISTILLATION;

AMOUNT OF INFORMATION; BAE SYSTEMS; COMPLEX QUESTIONS; END USERS; EVALUATION METHODOLOGIES; NON-REDUNDANT; PHASE 2; STATE OF THE ART;

SEARCH ENGINES;

EID: 85037337108     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (7)

1. Lin, J. and D. Demner-Fushman, 2005. Automatically evaluating answers to definition questions. In Proceedings of the Human Language Technology Conference (HLT-EMNLP 2005).
2. Lin J. and P. Zhang. 2007. Deconstructing Nuggets: The Stability and Reliability of Complex Question Answering Evaluation. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pages 327-334, Amsterdam, the Netherlands
3. Marton, G. and A. Radul, 2006. Nuggeteer: automatic nugget-based evaluation using description and judgments. In Proceedings of NAACL-HLT 2006
4. Voorhees, E. 2003. Overview of the TREC 2003 question answering track. In Proceedings of TREC 2003.
5. Shamber L. 1994. Relevance and information behavior. In Annual Review of Information Science and Technology. 29:3-48.
6. Zhou, L. N. Kwon, and E. H. Hovy. 2007. A Semi-Automated Evaluation Scheme: Automated Nuggetization for Manual Annotation. In Proceedings of the Human Language Technology/North American Association of Computational Linguistics conference (HLT-NAACL 2007). Rochester, NY
7. White J. V., D. Hunter, and J. D. Goldstein, 2008. "Statistical Evaluation of Information Distillation Systems", submitted to LREC 2008