Gaze-based relevance feedback for realizing region-based image retrieval

IEEE Transactions on Multimedia

Volumn 16, Issue 2, 2014, Pages 440-454

  Papadopoulos, Georgios Th ^a   Apostolakis, Konstantinos C ^a   Daras, Petros ^a  

^a INFORMATION TECHNOLOGIES INSTITUTE   (Greece)

Author keywords

Gaze analysis; gaze tracking; image retrieval; relevance feedback

Indexed keywords

EXPERIMENTAL EVALUATION; GAZE ANALYSIS; GAZE-TRACKING; ITERATIVE ESTIMATION; REGION-BASED IMAGE RETRIEVAL; RELEVANCE ASSESSMENTS; RELEVANCE FEEDBACK; SPATIAL CHARACTERISTICS;

ESTIMATION; IMAGE PROCESSING; ITERATIVE METHODS;

IMAGE RETRIEVAL;

EID: 84893367343     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2013.2291535     Document Type: Article

References (59)

1. R. Datta, D. Joshi, J. Li, and J. Wang, "Image retrieval: Ideas, influences, and trends of the new age," ACM Comput. Surveys (CSUR), vol. 40, no. 2, pp. 5-5, 2008.
2. A. Hanjalic, R. Lienhart,W. Ma, and J. Smith, "The holy grail ofmultimedia information retrieval: So close or yet so far away?," Proc. IEEE, vol. 96, no. 4, pp. 541-547, 2008.
3. X. Zhou and T. Huang, "Relevance feedback in image retrieval:Acomprehensive review,"Multimedia Syst., vol. 8, no. 6, pp. 536-544, 2003.
4. D. Tao, X. Tang, X. Li, and X.Wu, "Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1088-1099, 2006.
5. X. Tian, D. Tao, X.-S. Hua, and X. Wu, "Active reranking for web image search," IEEE Trans. Image Process., vol. 19, no. 3, pp. 805-820, 2010.
6. D. Liu, K. Hua, K. Vu, and N. Yu, "Fast query point movement techniques for large cbir systems," IEEE Trans. Knowl. Data Eng., vol. 21, no. 5, pp. 729-743, 2009.
7. D. Tao, X. Tang, X. Li, and Y. Rui, "Direct kernel biased discriminant analysis: a new content-based image retrieval relevance feedback algorithm," IEEE Trans. Multimedia, vol. 8, no. 4, pp. 716-727, 2006. (Pubitemid 44144157)
8. X. Tian, D. Tao, and Y. Rui, "Sparse transfer learning for interactive video search reranking," ACM Trans. Multimedia Comput., Commun., Applicat. (TOMCCAP), vol. 8, no. 3, pp. 26-26, 2012.
9. W. Jiang, G. Er, Q. Dai, and J. Gu, "Similarity-based online feature selection in content-based image retrieval," IEEE Trans. Image Process., vol. 15, no. 3, pp. 702-712, 2006. (Pubitemid 43257568)
10. C. Li and C. Hsu, "Image retrieval with relevance feedback based on graph-theoretic region correspondence estimation," IEEE Trans. Multimedia, vol. 10, no. 3, pp. 447-456, 2008. (Pubitemid 351459507)
11. D. Djordjevic and E. Izquierdo, "An object-and user-driven system for semantic-based image annotation and retrieval," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 3, pp. 313-323, 2007. (Pubitemid 46393329)
12. G. Aggarwal, T. Ashwin, and S. Ghosal, "An image retrieval system with automatic query modification," IEEE Trans. Multimedia, vol. 4, no. 2, pp. 201-214, 2002.
13. M. Kherfi and D. Ziou, "Relevance feedback for cbir: a new approach based on probabilistic feature weighting with positive and negative examples," IEEE Trans. Image Process., vol. 15, no. 4, pp. 1017-1030, 2006.
14. D. Kelly and J. Teevan, "Implicit feedback for inferring user preference: a bibliography," in Proc. ACM SIGIR Forum, 2003, vol. 37, no. 2, pp. 18-28.
15. E. Cheng, F. Jing, L. Zhang, and H. Jin, "Scalable relevance feedback using click-through data for web image retrieval," in Proc. 14th Annu. ACM Int. Conf. Multimedia, 2006, pp. 173-176. (Pubitemid 47113529)
16. J. Wang, E. Pohlmeyer, B. Hanna, Y. Jiang, P. Sajda, and S. Chang, "Brain state decoding for rapid image retrieval," in Proc. 17th ACM Int. Conf. Multimedia, 2009, pp. 945-954.
17. L. Kozma, A. Klami, and S. Kaski, "Gazir: Gaze-based zooming interface for image retrieval," in Proc. 2009 ACM Int. Conf. Multimodal Interfaces, 2009, pp. 305-312.
18. J. Sang, C. Xu, and D. Lu, "Learn to personalized image search from the photo sharing websites," IEEE Trans. Multimedia, vol. 14, no. 4, pp. 963-974, 2012.
19. S. Liu, P. Cui, H. Luan, W. Zhu, S. Yang, and Q. Tian, "Social visual image ranking for web image search," in Advances inMultimediaModeling. New York, NY, USA: Springer, 2013, pp. 239-249.
20. A. Klami, C. Saunders, T. De Campos, and S. Kaski, "Can relevance of images be inferred from eye movements?," in Proc. 1st ACM Int. Conf. Multimedia Information Retrieval, 2008, pp. 134-140.
21. A. Faro, D. Giordano, C. Pino, and C. Spampinato, "Visual attention for implicit relevance feedback in a content based image retrieval," in Proc. 2010 ACM Symp. Eye-Tracking Research & Applicat., 2010, pp. 73-76.
22. S. Hajimirza, M. Proulx, and E. Izquierdo, "Reading users' minds from their eyes: A method for implicit image annotation," IEEE Trans. Multimedia, vol. 14, no. 3, pp. 805-815, 2012.
23. D. Hardoon and K. Pasupa, "Image ranking with implicit feedback from eye movements," in Proc. 2010 ACM Symp. Eye-Tracking Research & Applicat., 2010, pp. 291-298.
24. A. Klami, "Inferring task-relevant image regions from gaze data," in Proc. IEEE Workshop Mach. Learning for Signal Processing, 2010.
25. F. Jing, M. Li, H.-J. Zhang, and B. Zhang, "Relevance feedback in region-based image retrieval," IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 5, pp. 672-681, 2004.
26. L. Terissi and J. Gómez, "3D head pose and facial expression tracking using a single camera," J. Universal Comput. Sci., vol. 16, no. 6, pp. 903-920, 2010.
27. J. Ahlberg, Candide-3-an Updated Parameterized Face, 2001, Rep.No. LiTH-ISY.
28. D. DeMenthon and L. Davis, "Model-based object pose in 25 lines of code," Int. J. Comput. Vision, vol. 15, no. 1, pp. 123-141, 1995.
29. B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. 7th Int. Joint Conf. Artificial Intell., 1981.
30. N. Otsu, "A threshold selection method from gray-level histograms," Automatica, vol. 11, no. 285-296, pp. 23-27, 1975.
31. J. Zhu and J. Yang, "Subpixel eye gaze tracking," in Proc. 5th IEEE Int. Conf. Automatic Face and Gesture Recognition, 2002, pp. 124-129.
32. V. Mezaris, I. Kompatsiaris, and M. Strintzis, "Still image segmentation tools for object-based multimedia applications," Int. J. Pattern Recognit. Artif. Intell., vol. 18, no. 04, pp. 701-725, 2004.
33. G. T. Papadopoulos, C. Saathoff, H. J. Escalante, V. Mezaris, I. Kompatsiaris, andM. G. Strintzis, "A comparative study of object-level spatial context techniques for semantic image analysis," Comput. Vision Image Understand., vol. 115, no. 9, pp. 1288-1307, Sep. 2011.
34. K. V. D. Sande, T. Gevers, and C. Snoek, "Evaluating color descriptors for object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1582-1596, 2010.
35. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Proc. Workshop Statist. Learning in Comput. Vision, ECCV, 2004, vol. 1, p. 22.
36. K. Rayner, "Eye movements in reading and information processing: 20 years of research," Psychol. Bull., vol. 124, no. 3, p. 372, 1998.
37. E. Kowler, "Eye movements: The past 25 years," Vision Res., vol. 51, no. 13, pp. 1457-1483, 2011.
38. D. Salvucci and J.Goldberg, "Identifying fixations and saccades in eyetracking protocols," in Proc. 2000 ACM Symp. Eye Tracking Research & Applicat., 2000, pp. 71-78.
39. H. Widdel, "Operational problems in analysing eye movements," in Proc. 2nd Eur. Conf. Eye Movements, 1984, vol. 22, pp. 21-29.
40. Y. Zhang, H. Fu, Z. Liang, Z. Chi, and D. Feng, "Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system," in Proc. 2010 ACM Symp. Eye-Tracking Res. & Applicat., 2010, pp. 37-40.
41. R. Harris, A Primer of Multivariate Statistics. Mahwah, NJ, USA: Lawrence Erlbaum, 2001.
42. M. Hall, "Correlation-based feature selection for machine learning," Ph.D. dissertation, Univ. Waikato, Hamilton, New Zealand, 1999.
43. N. Johnson, S. Kotz, and N. Balakrishnan, Distributions in Statistics: Continuous Univariate Distributions. New York, NY, USA: Houghton Mifflin, 1970, vol. 2.
44. T. Mitchell et al., Machine Learning. BurrRidge, IL, USA: McGraw-Hill, 1997, vol. 45.
45. V.Vapnik, The Nature of Statistical Learning Theory. NewYork,NY, USA: Springer, 1999.
46. D. Tax andR.Duin, "Using two-class classifiers for multiclass classification," in Proc. 16th IEEE Int. Conf. Pattern Recognition, 2002, vol. 2, pp. 124-127. (Pubitemid 44844696)
47. T. Ohno, N. Mukawa, and A. Yoshikawa, "Freegaze: a gaze tracking system for everyday gaze interaction," in Proc. 2002 ACM Symp. Eye Tracking Research & Applicat., 2002, pp. 125-132.
48. T. Ishikawa, S. Baker, I. Matthews, and T. Kanade, Passive Driver Gaze Tracking With Active Appearance Models. Pittsburgh, PA, USA: Carnegie Mellon Univ., Robotics Inst., 2004.
49. E. Pogalin, A. Redert, I. Patras, and E. Hendriks, "Gaze tracking by using factorized likelihoods particle filtering and stereo vision," in Proc. 3rd IEEE Symp. 3D Data Processing, Visualization, and Transmission, 2006, pp. 57-64.
50. J. Chen, Y. Tong, W. Gray, and Q. Ji, "A robust 3D eye gaze tracking system using noise reduction," in Proc. 2008 ACM Symp. Eye Tracking Res. & Applicat., 2008, pp. 189-196.
51. E. Lee and K. Park, "A robust eye gaze tracking method based on a virtual eyeball model," Mach. Vision Applicat., vol. 20, no. 5, pp. 319-337, 2009.
52. R. Kohavi et al., "A study of cross-validation and bootstrap for accuracy estimation and model selection," in Proc. Int. Joint Conf. Artificial Intell., Mahwah, NJ, USA: Lawrence Erlbaum, 1995, vol. 14, pp. 1137-1145.
53. J. O'Connor and E. Robertson, Student's t-test, MacTutor History of Mathematics Archive. St. Andrews, U.K.: Univ. St. Andrews.
54. N. Vasconcelos and A. Lippman, "Learning from user feedback in image retrieval systems," Adv. Neural Inf. Process. Syst., vol. 12, pp. 977-983, 1999.
55. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, 2012.
56. L. Zhang, L.Wang, andW. Lin, "Generalized biased discriminant analysis for content-based image retrieval," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 42, no. 1, pp. 282-290, 2012.
57. J.-H. Su,W.-J. Huang, P. S. Yu, and V. S. Tseng, "Efficient relevance feedback for content-based image retrieval by mining user navigation patterns," IEEE Trans. Knowl. Data Eng., vol. 23, no. 3, pp. 360-372, 2011.
58. A. Grigorova, F. D. Natale, C. Dagli, and T. Huang, "Content-based image retrieval by feature adaptation and relevance feedback," IEEE Trans. Multimedia, vol. 9, no. 6, pp. 1183-1192, 2007. (Pubitemid 47460399)
59. Y. Ishikawa, R. Subramanya, and C. Faloutsos, "Mindreader: Querying databases through multiple examples," Sch. Comput. Sci., Carnegie Mellon Univ., p. 551, 1998.