Building an information retrieval test collection for spontaneous conversational speech

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2004, Pages 41-48

  Oard, Douglas W ^a   Soergel, Dagobert ^a   Doermann, David ^a   Huang, Xiaoli ^a   Murray, G Craig ^a   Wang, Jianqiang ^a   Ramabhadran, Bhuvana ^b   Franz, Martin ^b   Gustman, Samuel ^c   Mayfield, James ^d   Kharevych, Liliya ^e   Strassel, Stephanie ^f  

^a UNIVERSITY OF MARYLAND   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c Survivors of the Shoah   (United States)

^d JOHNS HOPKINS UNIVERSITY APPLIED PHYSICS LABORATORY   (United States)

^e CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^f UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Automatic Speech Recognition; Oral History; Search Guided Relevance Assessment

Indexed keywords

AUTOMATIC SPEECH RECOGNITION; INFORMATION SEARCH; SEARCH-GUIDED RELEVANCE ASSESSMENT; TEST COLLECTIONS;

ENGINEERING RESEARCH; MATHEMATICAL MODELS; QUERY LANGUAGES; RELIABILITY; SEARCH ENGINES; SPEECH RECOGNITION;

INFORMATION RETRIEVAL;

EID: 8644291021     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. EU-US working group on spoken-word audio collections, 2003. http://www.dcs.shef.ac.uk/spandh/projects/swag/.
2. C. Buckley and E. Voorhees. Evaluating evaluation measure stability. In SIGIR 2000, pages 33-40, 2000.
3. William Byrne et al. Automated recognition of spontaneous speech for access to multilingual oral history archives. IEEE Transactions on Speech and Audio Processing, 12(4), 2004.
4. J. Carletta. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249-254, 1996.
5. S. F. Chen and J. Goodman. An empirical study of smoothing techniques for language modeling. In Computer Speech and Language, 1999.
6. C. Cieri et al. Corpora for topic detection and tracking. In Topic Detection and Tracking: Event-Based Information Organization. Kluwer Academic, Boston, 2002.
7. C. Cleverdon. The Cranfield tests on index language devices. ASLIB Proceedings, 19(6):173-194, 1967.
8. G.V. Cormack et al. Efficient construction of large test collections. In SIGIR '98, pages 282-289, 1998.
9. Martin Franz et al. Ad hoc and multilingual information retrieval at IBM. In TREC-7, 1998.
10. J.S. Garofolo et al. The TREC spoken document retrieval track: A success story. In TREC-8, 1999.
11. J. Godfrey et al. SWITCHBOARD: telephone speech corpus for research and development. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 517-520, 1992.
12. Samuel Gustman et al. Supporting access to large digital oral history archives. In The Second Joint Conference on Digital Libraries, pages 18-27, 2002.
13. Xiaoli Huang and Dagobert Soergel. Relevance judges' understanding of topical relevance types: An explication of an enriched concept of topical relevance. In Annual Meeting of the American Society for Information Science and Technology, 2004. to appear.
14. R.V. Katter. The influence of scale form on relevance judgment. Information Storage and Retrieval, 4(1):1-11, 1968.
15. B. Ramabhadran et al. Towards automatic transcription of large spoken archives - English ASR for the MALACH project. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, 2003.
16. S. E. Robertson et al. Okapi at TREC-3. In TREC-3, pages 109-126, 1994.
17. T. Rong et al. Towards the identification of the optimal number of relevance categories. Journal of the American Society for Information Science, 50(3):254-264, 1998.
18. L. Schamber. Variations in relevance and information behavior. In Annual Review of Information Science and Technology, volume 29, pages 3-48. 2000.
19. K. Sparck-Jones and C.J. van Rijsbergen. Information retrieval test collections. Journal of Documentation, 32(1):59-72, 1976.
20. E.M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing and Management, 36:697-716, 2000.
21. S. Whittaker et al. Play it again: A study of the factors underlying search browsing behavior. In CHI '98, pages 247-248, 1998.
22. Y. Yang and X. Liu. A re-examination of text categorization methods. In SIGIR '99, 1999.