New metrics for meaningful evaluation of informally structured speech retrieval

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7224 LNCS, Issue , 2012, Pages 170-181

  Eskevich, Maria ^a   Magdy, Walid ^a   Jones, Gareth J F ^a  

^a DUBLIN CITY UNIVERSITY   (Ireland)

Author keywords

evaluation metrics; informally structured speech; Speech retrieval

Indexed keywords

CONTENT SEGMENTATION; EVALUATION METRICS; RETRIEVAL PERFORMANCE; SEARCH TASKS; SPEECH DATA; SPEECH RETRIEVAL; TEST COLLECTION;

ARTIFICIAL INTELLIGENCE;

INFORMATION RETRIEVAL;

EID: 84860186165     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-28997-2_15     Document Type: Conference Paper

References (12)

1. B̈uttcher, S., Clarke, C.L.A., Cormack, G.V.: Information Retrieval: Implementing and Evaluating Search Engines. MIT Press (2010)
2. Carletta, J.: Unleashing the killer corpus: experiences in creating the multieverything AMI Meeting Corpus. Language Resources and Evaluation Journal 41(2), 181-190 (2007)
3. Choi, F.Y.Y.: Advances in domain independent linear text segmentation. In: Proc. of the 1st NAACL Conference, Seattle, Washington, USA, pp. 26-33 (2000)
4. Garofolo, J.S., Auzanne, C.G.P., Voorhees, E.M.: The TREC Spoken Document Retrieval Track: A Success Story. In: Proc. of RIAO 2000, Paris, France, pp. 1-20 (2000)
5. Hearst, M.A.: TextTiling: Segmenting Text into Multi-paragraph Subtopic Passages. Computational Linguistics 23(1), 33-64 (1997) (Pubitemid 127458657)
6. Hiemstra, D.: Using Language Models for Information Retrieval. Ph.D. thesis, Center of Telematics and Information Technology, AE Enschede, The Netherlands (2000)
7. Kamps, J., Pehcevski, J., Kazai, G., Lalmas, M., Robertson, S.: INEX 2007 Evaluation Measures. In: Fuhr, N., Kamps, J., Lalmas, M., Trotman, A. (eds.) INEX 2007. LNCS, vol. 4862, pp. 24-33. Springer, Heidelberg (2008)
8. Kekalainen, J., Jarvelin, K.: Using graded relevance assessments in ir evaluation. Journal of the American Society for Information Science and Technology 53(13), 1120-1129 (2002) (Pubitemid 35359181)
9. Liu, B., Oard, D.W.: One-sided measures for evaluating ranked retrieval effectiveness with spontaneous conversational speech. In: Proc. of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA (2006)
10. Pecina, P., Hoffmannov́a, P., Jones, G.J.F., Zhang, Y., Oard, D.W.: Overview of the CLEF-2007 Cross-Language Speech Retrieval Track. In: Peters, C., Jijkoun, V., Mandl, T., M̈uller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 674-686. Springer, Heidelberg (2008)
11. Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130-137 (1980)
12. Renals, S., Hain, T., Boulard, H.: Recognition and interpretation of meetings: The AMI and AMIDA projects. In: Proc. of the IEEE Workshop ASRU (2007)