Evaluation of algorithms using games: The case of music tagging

Proceedings of the 10th International Society for Music Information Retrieval Conference, ISMIR 2009

Volumn , Issue , 2009, Pages 387-392

  Law, Edith ^a   West, Kris ^b   Mandel, Michael ^c   Bay, Mert ^b   Stephen Downie, J ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b IMIRSEL/UIUC ^*  (United States)

^c Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EVALUATION METRICS; GROUND TRUTH; HUMAN COMPUTATION GAMES; NOVEL ALGORITHM;

INFORMATION RETRIEVAL;

ALGORITHMS;

EID: 84873647698     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (8)

1. Doug Turnbull, Luke Barrington, David Torres, and Gert Lanckriet. Towards musical query-by-semanticdescription using the cal500 data set. In SIGIR, pages 439-446, 2007.
2. Michael I. Mandel and Daniel P.W. Ellis. A web-based game for collecting music metadata. Journal of New Music Research, 37(2):151-165, 2008.
3. Edith Law. The problem of accuracy as an evaluation criterion. ICML Workshop on Evaluation Methods in Machine Learning, 2008.
4. Douglas Turnbull, Ruoran Liu, Luke Barrington, and Gert Lanckriet. A game-based approach for collecting semantic annotations of music. In ISMIR, pages 535- 538, 2007.
5. J. Stephen Downie. The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research. Acoustical Science and Technology, 29(4):247-255, 2008.
6. Edith Law and Luis von Ahn. Input-agreement: A new mechanism for data collection using human computation games. CHI, pages 1197-1206, 2009.
7. Keith Rijsbergen. Information Retrieval. Butterworth- Heinemann, Newton, MA, USA, 1979.
8. Jean-Julien Aucouturier. Ten experiments on the modelling of polyphonic timbre. PhD thesis, University of Paris 6, France, June 2006.