Towards Minimal Test Collections for evaluation of Audio Music Similarity and Retrieval

WWW'12 - Proceedings of the 21st Annual Conference on World Wide Web Companion

Volumn , Issue , 2012, Pages 917-923

  Urbano, Julián ^a   Schedl, Markus ^b  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

^b JOHANNES KEPLER UNIVERSITY LINZ   (Austria)

Author keywords

Evaluation; Music information retrieval; Relevance judgments; Test collections

Indexed keywords

AUDIO MUSIC; DEGREE OF CONFIDENCE; EVALUATION; MUSIC INFORMATION RETRIEVAL; RELEVANCE JUDGMENT; TEST COLLECTION;

INFORMATION RETRIEVAL SYSTEMS; INSECTICIDES;

WORLD WIDE WEB;

EID: 84861054565     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2187980.2188223     Document Type: Conference Paper

References (9)

1. B. Carterette. Low-Cost and Robust Evaluation of Information Retrieval Systems. Ph.D. dissertation, Department of Computer Science, University of Massachusetts Amherst, 2008.
2. B. Carterette. Robust Test Collections for Retrieval Evaluation. In International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 55-62, 2007.
3. B. Carterette, J. Allan, and R. Sitaraman. Minimal Test Collections for Retrieval Evaluation. In International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 268-275, 2006.
4. J.S. Downie. The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future. Computer Music Journal. 28(2): 12-23, 2004.
5. J.S. Downie, A.F. Ehmann, M. Bay, and M.C. Jones. The Music Information Retrieval Evaluation eXchange: Some Observations and Insights. In Advances in Music Information Retrieval, W.R. Zbigniew and A.A. Wieczorkowska, eds. Springer. 2010, 93-115.
6. J. Urbano. Information Retrieval Meta-Evaluation: Challenges and Opportunities in the Music Domain. In International Society for Music Information Retrieval Conference, pages 609-614, 2011.
7. J. Urbano, D. Martín, M. Marrero, and J. Morato. Audio Music Similarity and Retrieval: Evaluation Power and Stability. In International Society for Music Information Retrieval Conference, pages 597-602, 2011.
8. E.M. Voorhees. Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. Information Processing and Management. 36(5): 697-716, 2000.
9. E.M. Voorhees and D.K. Harman. TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.