User-centered Measures Vs. System effectiveness in finding similar songs

Proceedings of the 13th International Society for Music Information Retrieval Conference, ISMIR 2012

Volumn , Issue , 2012, Pages 331-336

  Hu, Xiao ^a   Kando, Noriko ^b  

^a UNIVERSITY OF HONG KONG   (Hong Kong)

^b NATIONAL INSTITUTE OF INFORMATICS   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

MUSIC INFORMATION RETRIEVAL; MUSIC RETRIEVAL SYSTEMS; SYSTEM EFFECTIVENESS; USER EVALUATIONS; USER-CENTERED MEASURES; WEAK CORRELATION;

INFORMATION RETRIEVAL;

EID: 84873422267     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. J. Allan, B. Carterette and J. Lewis: "When will information retrieval be 'good enough'? User effectiveness as a function of retrieval accuracy," Proceedings of the ACM SIGIR Conference on Information Retrieval, pp. 433-440, 2005.
2. A. Al-Maskari, M. Sanderson, P. Clough and E. Airio: "The good and the bad system: Does the test collection predict users' effectiveness?," Proc. of the ACM SIGIR Conference, pp. 59-66, 2005.
3. C. W. Cleverdon and E. M. Keen: "Factors determining the performance of indexing systems," Vol. 1: Design. Vol 2: Results. Cranfield, U.K: Aslib Cranfield Research Project, 1966.
4. S. J. Cunningham, D. Bainbridge and A. Falconer: "'More of an art than a science': supporting the creation of playlists and mixes," Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR), 2006.
5. J. S. Downie: "The Music Information Retrieval Evaluation eXchange (2005-2007): A window into music information retrieval research," Acoustical Science and Technology, 29(4), pp. 247-255. 2008.
6. W. Hersh, A. Turpin, S. Price, B. Chan, D. Kraemer, L. Sacherek, and D. Olson: "Do batch and user evaluations give the same results?," Proc. of the ACM SIGIR Conference, pp. 17-24, 2000.
7. K. Hoashi, S. Hamawaki, H. Ishizaki, Y. Takishima, and J. Katto: "Usability evaluation of visualization interfaces for content-based music retrieval systems," Proc. of ISMIR, 2009.
8. X. Hu, and J. Liu: "Evaluation of music information retrieval: towards a user-centered approach," Proceedings of the 4th Workshop on HumanComputer Interaction and Information Retrieval (HCIR), 2010.
9. S. Pauws, and B. Eggen: "PATS: Realization and evaluation of an automatic playlist generator," Proc. of ISMIR, 2002.
10. S. Pauws, and S. van de Wijdeven: "User evaluation of a new interactive playlist generation concept," Proc. of ISMIR, 2005.
11. K. Spärck Jones: Information Retrieval Experiment, London, Butterworths. 1981.
12. A. Turpin and W. Hersh: "Why batch and user evaluations do not give the same results," Proc. of the ACM SIGIR Conference, pp. 225-231, 2001.
13. A. Turpin and F. Scholer: "User performance versus precision measures for simple search tasks," Proc. of the ACM SIGIR Conference, pp. 11-18, 2006.
14. F. Vignoli and S. Pauws: "A music retrieval system based on user driven similarity and its evaluation," Proc. of ISMIR, 2005.
15. E. M. Voorhees and D. K. Harman: TREC: Experiments in Information Retrieval Evaluation, MIT Press, 2005.