Performance evaluation of relevance feedback methods

CIVR 2008 - Proceedings of the International Conference on Content-based Image and Video Retrieval

Volumn , Issue , 2008, Pages 239-248

  Huiskes, Mark J ^a   Lew, Michael S ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Algorithms; Experimentation; Human factors; Measurement; Performance

Indexed keywords

CONTENT BASED RETRIEVAL; CONTROL THEORY; HUMAN ENGINEERING; IMAGE RETRIEVAL; INFORMATION RETRIEVAL;

ALGORITHMS; EXPERIMENTATION; HUMAN FACTORS; MEASUREMENT; PERFORMANCE;

FEEDBACK;

EID: 57549109868     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1386352.1386387     Document Type: Conference Paper

References (50)

1. L. Armitage and P. Enser. Analysis of user need in image archives. Journal of Information Science, 23(4):287-299, 1997.
2. Y. Chen, X. Zhou, and T. Huang. One-class SVM for learning in image retrieval. Proc. IEEE ICIP 2001. Thessaloniki, Greece, 1:34-37, 2001.
3. G. Ciocca and R. Schettini. A relevance feedback mechanism for content-based image retrieval. Information Processing and Management, 35(5):605-632, 1999.
4. P. Clough, H. Mueller, and M. Sanderson. The clef cross language image retrieval track (imageclef) 2004. In Fifth Workshop of the Cross-Language Evaluation Forum (CLEF 2004), LNCS, New York, NY, USA, 2004. ACM Press.
5. I. Cox, M. Miller, T. Minka, and T. Papathomas. The Bayesian image retrieval system, PicHunter: Theory, implementation, and psychophysical experiments. IEEE Trans. Image Processing, 9(1):20-37, 2000.
6. R. Datta, J. Li, and J. Z. Wang. Content-based image retrieval: approaches and trends of the new age. In MIR'05: Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, pages 253-262, New York, NY, USA, 2005. ACM Press.
7. A. Dimai. Assessment of effectiveness of content based image retrieval systems. In VISUAL'99: Proceedings of the Third International Conference on Visual Information and Information Systems, pages 525-532, London, UK, 1999. Springer-Verlag.
8. A. Dong and B. Bhanu. A new semi-supervised em algorithm for image retrieval. cvpr, 02:662, 2003.
9. P. Enser. Query analysis in a visual information retrieval context. Journal of Document and Text Management, 01(1):662, 1993.
10. M. Everingham, A. Zisserman, C. K. I. Williams, and L. Van Gool. The PASCAL Visual Object Classes Challenge 2006 (VOC2006) Results, http: //www.pascalnetwork.org/challenges/VOC/voc2006/results.pdf, 2006.
11. C. Fluhr, P.-A. Moellic, and P. Hede. Usage-oriented multimedia information retrieval technological evaluation. In MIR'06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 301-306, New York, NY, USA, 2006. ACM Press.
12. P. H. Gosselin and M. Cord. Semantic kernel updating for content-based image retrieval. In ISMSE'04: Proceedings of the IEEE Sixth International Symposium on Multimedia Software Engineering (ISMSE'04), pages 537-544, Washington, DC, USA, 2004. IEEE Computer Society.
13. Y. Hastie, R. Tibshirani, and J. Friedman. Elements of Statistical Learning - Datamining, Inference, and Prediction. Springer, 2001.
14. J. He, H. Tong, M. Li, H.-J. Zhang, and C. Zhang. Mean version space: a new active learning method for content-based image retrieval. In MIR'04: Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, pages 15-22, New York, NY, USA, 2004. ACM Press.
15. X. He, W.-Y. Ma, O. King, M. Li, and H. Zhang. Learning and inferring a semantic space from user's relevance feedback for image retrieval. In MULTIMEDIA'02: Proceedings of the tenth ACM international conference on Multimedia, pages 343-346, New York, NY, USA, 2002. ACM.
16. D. R. Heisterkamp and J. Peng. Kernel vector approximation files for relevance feedback retrieval in large image databases. Multimedia Tools Appl., 26(2): 175-189, 2005.
17. C. Hoi and M. Lyu. Biased support vector machine for relevance feedback in image retrieval. In Proc. Intl. Joint Conf. on Neural Networks, Budapest. Hungary, pages 3189-3194, 2004.
18. C.-H. Hoi and M. R. Lyu. A novel log-based relevance feedback technique in content-based image retrieval. In MULTIMEDIA'04: Proceedings of the 12th annual A CM international conference on Multimedia, pages 24-31, New York, NY, USA, 2004. ACM Press.
19. D. P. Huijsmans and N. Sebe. How to complete performance graphs in content-based image retrieval: Add generality and normalize scope. IEEE Trans. Pattern Anal. Mach. Intell., 27(2):245-251, 2005.
20. M. J. Huiskes. Aspect-based relevance learning for image retrieval. In W. Leow, editor, Proceedings of CIVR05, LNCS 3568, pages 639-649. Springer, 2005.
21. M. J. Huiskes. Image searching and browsing by active aspect-based relevance learning. In Proceedings of CIVR06, LNCS 4071, pages 211-220. Springer, 2006.
22. C. H. C. Leung and H. H.-S. Ip. Benchmarking for content-based visual information search. In VISUAL'00: Proceedings of the 4th International Conference on Advances in Visual Information Systems, pages 442-456, London, UK, 2000, Springer-Verlag.
23. M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl., 2(1):1-19, 2006.
24. Y.-Y. Lin, T.-L. Liu, and H.-T. Chen. Semantic manifold learning for image retrieval. In MULTIMEDIA'05: Proceedings of the 13th annual ACM international conference on Multimedia, pages 249-258, New York, NY, USA, 2005. ACM Press.
25. J. Luo and M. A. Nascimento. Content-based sub-image retrieval using relevance feedback. In MMDB'04: Proceedings of the 2nd ACM international workshop on Multimedia databases, pages 2-9, New York, NY, USA, 2004. ACM Press.
26. S. Marchand-Maillet. The Benchathlon Network. http://www.benchathlon.net, 2005.
27. S. Marchand-Maillet and M. Worring. Benchmarking image and video retrieval: an overview. In MIR'06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval., pages 297-300, New York, NY, USA, 2006. ACM Press.
28. G. Marchionini. Human performance measures for video retrieval. In MIR'06: Proceedings of the 8th A CM international workshop on Multimedia information retrieval, pages 307-312, New York, NY, USA, 2006. ACM Press.
29. M. Markkula and E. Sormunen. Searching for photos - journalists' practices in pictorial ir, 1998.
30. H. Müller, S. Marchand-Maillet, and T. Pun. The truth about corel - evaluation in image retrieval. In CIVR'02: Proceedings of the International Conference on Image and Video Retrieval, pages 38-49, London, UK, 2002. Springer-Verlag.
31. H. Müller, W. Müller, D. M. Squire, S. Marchand-Maillet, and T. Pun. Performance evaluation in content-based image retrieval: overview and proposals. Pattern Recogn. Lett., 22(5):593-601, 2001.
32. M. Naphade, J. R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis. Large-scale concept ontology for multimedia. IEEE MultiMedia, 13(3):86-91, 2006.
33. J. Peng. Multi-class relevance feedback content-based image retrieval. Comput. Vis. Image Underst., 90(1):42-67, 2003.
34. V. Raghavan, P. Bollmann, and G. S. Jung. A critical investigation of recall and precision as measures of retrieval system performance. ACM Trans. Inf. Syst., 7(3):205-229, 1989.
35. Y. Rui, T. Huang, M. Ortega, and S. Mehrotra. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. Circuits and Systems for Video Technology, 8(5):644-655, 1998.
36. L. Schamber, M. Eisenberg, and M. S. Nilan. A re-examination of relevance: toward a dynamic, situational definition. Inf. Process. Manage., 26(6):755-776, 1990.
37. A. F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and trecvid. In MIR'06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 321-330, New York, NY, USA, 2006. ACM Press.
38. A. Smeulders, M. Worring, S. Santini, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):20-37, 2000.
39. J. Smith, M. Naphade, A. Natsev, and J. Tesic. Multimedia research challenges for industry. Proceedings of the International Conference on Image and Video Retrieval (CIVR 2005), Singapore., Lecture Notes in Computer Science(3568):28-37, 2005.
40. C. G. M. Snoek, M. Worring, J. C. van Gemert, J.-M. Geusebroek, and A. W. M. Smeulders. The challenge problem for automated detection of 101 semantic concepts in multimedia, In MULTIMEDIA'06: Proceedings of the 14th annual ACM international conference on Multimedia, pages 421-430, New York, NY, USA, 2006. ACM Press.
41. K. Sparck Jones and C. van Rijsbergen. Report on the need for and provision of an ideal information retrieval test collection. British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge, 1975.
42. D. M. Squire, W. Müller, H. Müller, and T. Pun. Content-based query of image databases: inspirations from text retrieval. Pattern Recogn. Lett., 21(13-14):1193-1198, 2000.
43. K. Tieu and P. Viola. Boosting image retrieval. International Journal of Computer Vision, 56(1/2):17-36, 2004.
44. S. Tong and E. Chang. Support vector machine active learning for image retrieval. In MULTIMEDIA'01: Proceedings of the ninth ACM international conference on Multimedia, pages 107-118, New York, NY, USA, 2001. ACM Press.
45. E. Voorhees. The philosophy of information retrieval evaluation. In Proceedings of the The Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, pages 355-370, 2001.
46. T. Westerveld. Trecvid as a re-usable test-collection for video retrieval. In Proceedings of the Multimedia Information Retrieval Workshop 2005, Salvador, Brazil, August 2005.
47. Y. Wu and A. Zhang. A feature re-weighting approach for relevance feedback in image retrieval, 2002.
48. T. Yoshizawa and H. Schweitzer. Long-term learning of semantic grouping from relevance-feedback. In MIR'04: Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, pages 165-172, New York, NY, USA, 2004. ACM Press.
49. H. Zhang, C. Zheng, M. Li, and Z. Su. Relevance feedback and learning in content-based image search. WWW: Internet and web information systems, 6:131-155, 2003.
50. X. Zhou and T. Huang. Relevance feedback in image retrieval: a comprehensive review. ACM Multimedia Systems Journal, 8(6):536-544, April 2003.