Crowdsourcing music similarity judgments using mechanical turk

Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR 2010

Volumn , Issue , 2010, Pages 183-188

  Lee, Jin Ha ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AMAZON MECHANICAL TURKS; AUDIO MUSIC; CROWDSOURCING; GROUND TRUTH DATA; HUMAN JUDGMENTS; MECHANICAL TURKS; MUSIC INFORMATION RETRIEVAL; MUSIC SIMILARITY; TIME-CONSUMING TASKS;

INFORMATION RETRIEVAL;

EID: 84873598993     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. O. Alonso and S. Mizzaro: "Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment, " Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation, pp. 15-16, 2009.
2. J-J. Aucouturier and F. Pachet: "Music Similarity Measures: What's the Use?" Proceedings of the 3rd International Society of Music Information Retrieval (ISMIR) Conference, pp. 157-163, 2002.
3. D. Axe; "Geeks Spot Fossett?" Wired, Retrieved from http://www.wired.com/dangerroom/2007/09/geeks-spot-foss/comment-page-3/2007.
4. A. Berenzweig, B. Logan, D. P. W. Ellis, and B. Whitman: "A Large-Scale Evaluation of Acoustic and Subjective Music Similarity Measures, " Proceedings of the 4th ISMIR, pp. 99-105, 2003.
5. D. P. W. Ellis, B. Whitman, A. Berenzweig, and S. Lawrence: "The Quest For Ground Truth in Musical Artist Similarity, " Proceedings of the 3rd ISMIR Conference, pp. 170-177, 2002.
6. P. G. Ipeirotis: "Demographics of Mechanical Turk, " CeDER Working Papers, Retrieved from http://hdl.handle.net/2451/29585, 2010.
7. M. C. Jones, J. S. Downie, and A. F. Ehmann: "Human Similarity Judgments: Implications for the Design of Formal Evaluations, " Proceedings of the 8th ISMIR, pp. 539-542, 2007.
8. A. Kittur, E. H. Chi, and B. Suh: "Crowdsourcing User Studies With Mechanical Turk, " CHI 2008 Proceedings, pp. 453-456, 2008.
9. MIREX 2009 Wiki, Retrieved from http://musicir.org/mirex/2009/2009.
10. A. Novello and M. F. McKinney: "Assessment of Perceptual Music Similarity, " Proceedings of the 8th ISMIR, pp. 111-112, 2007.
11. E. Schubert and C. Stevens: "The Effect of Implied Harmony, Contour and Musical Expertise on Judgments of Similarity of Familiar Melodies, " Journal of New Music Research, Vol. 35, No. 2, pp. 161-174, 2006.
12. R. Snow, B. O'Connor, D. Jurafsky, and A. Y. Ng: "Cheap and Fast - But is it Good? Evaluating Non- Expert Annotations for Natural Language Tasks, " Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 254- 263, 2008.
13. R. Timmers: "Predicting the Similarity Between Expressive Performances of Music From Measurements of Tempo and Dynamics, " Journal of the Acoustical Society of America, Vol. 117, No. 1, pp. 391-399, 2005.