Information-theoretic model comparison unifies saliency metrics

Proceedings of the National Academy of Sciences of the United States of America

Volumn 112, Issue 52, 2015, Pages 16054-16059

  Kümmerer, Matthias ^a   Wallis, Thomas S A ^a   Bethge, Matthias ^a,b,c  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b Bernstein Center for Computational Neuroscience   (Germany)

^c MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

Eye movements; Likelihood; Point processes; Probabilistic modeling; Visual attention

Indexed keywords

ARTICLE; COMPUTER PROGRAM; EYE MOVEMENT; HUMAN; INFORMATION; MEDICAL LITERATURE; PREDICTION; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; THEORETICAL MODEL; ALGORITHM; BIOLOGICAL MODEL; BIOLOGY; COMPARATIVE STUDY; COMPUTER SIMULATION; EYE FIXATION; PATTERN RECOGNITION; PHOTOSTIMULATION; PHYSIOLOGY; PROCEDURES; REPRODUCIBILITY; STATISTICAL MODEL;

ALGORITHMS; COMPUTATIONAL BIOLOGY; COMPUTER SIMULATION; EYE MOVEMENTS; FIXATION, OCULAR; HUMANS; MODELS, BIOLOGICAL; MODELS, STATISTICAL; PATTERN RECOGNITION, VISUAL; PHOTIC STIMULATION; REPRODUCIBILITY OF RESULTS;

EID: 84952683316     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1510393112     Document Type: Article

References (49)

1. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254-1259.
2. Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185-207.
3. Borji A, Sihite DN, Itti L (2013) Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans Image Processing 22(1):55-69.
4. Borji A, Tavakoli HR, Sihite DN, Itti L (2013) Analysis of scores, datasets, and models in visual saliency prediction. Proceedings of the 2013 IEEE International Conference on Computer Vision (IEEE Computer Society, Washington, DC), pp 921-928.
5. Zhang J, Sclaroff S (2013) Saliency detection: A Boolean map approach. Proceedings of the 2013 IEEE International Conference on Computer Vision (IEEE Computer Society, Washington, DC), pp 153-160.
6. Bruce ND, Wloka C, Frosst N, Rahman S, Tsotsos JK (2015) On computational modeling of visual saliency: Examining what's right, and what's left. Vision Res 116(2):92-112.
7. Riche N, Duvinage M, Mancas M, Gosselin B, Dutoit T (2013) Saliency and human fixations: State-of-the-art and study of comparison metrics. Proceedings of the 2013 IEEE International Conference on Computer Vision (IEEE Computer Society, Washington, DC), pp 1153-1160.
8. Wilming N, Betz T, Kietzmann TC, König P (2011) Measures and limits of models of fixation selection. PLoS One 6(9):e24038.
9. Shannon CE, Weaver W (1949) The Mathematical Theory of Communication (Univ of Illinois Press, Urbana, IL).
10. Barthelmé S, Trukenbrod H, Engbert R, Wichmann F (2013) Modeling fixation locations using spatial point processes. J Vis 13(12):1-34.
11. Kümmerer M, Theis L, Bethge M (2015) Deep Gaze I: Boosting saliency prediction with feature maps trained on ImageNet. arXiv:1411.1045.
12. Engbert R, Trukenbrod HA, Barthelmé S, Wichmann FA (2015) Spatial statistics and attentional dynamics in scene viewing. J Vis 15(1):14.
13. Bernardo JM (1979) Reference posterior distributions for Bayesian inference. J R Stat Soc Ser B Methodol 41(2):113-147.
14. Tatler BW, Vincent BT (2008) Systematic tendencies in scene viewing. J Eye Mov Res 2(2):1-18.
15. Tatler BW, Vincent BT (2009) The prominence of behavioural biases in eye guidance. Vis Cogn 17(6-7):1029-1054.
16. Tatler BW, Hayhoe MM, Land MF, Ballard DH (2011) Eye guidance in natural vision: Reinterpreting salience. J Vis 11(5):5.
17. Ehinger KA, Hidalgo-Sotelo B, Torralba A, Oliva A (2009) Modeling search for people in 900 scenes: A combined source model of eye guidance. Vis Cogn 17(6-7):945-978.
18. Dorr M, Martinetz T, Gegenfurtner KR, Barth E (2010) Variability of eye movements when viewing dynamic natural scenes. J Vis 10(10):28.
19. Vincent BT, Baddeley RJ, Correani A, Troscianko T, Leonards U (2009) Do we look at lights? Using mixture modelling to distinguish between low- and high-level factors in natural image viewing. Vis Cogn 17(6-7):856-879.
20. Najemnik J, Geisler WS (2009) Simple summation rule for optimal fixation selection in visual search. Vision Res 49(10):1286-1294.
21. Najemnik J, Geisler WS (2005) Optimal eye movement strategies in visual search. Nature 434(7031):387-391.
22. Morvan C, Maloney LT (2012) Human visual search does not maximize the postsaccadic probability of identifying targets. PLoS Comput Biol 8(2):e1002342.
23. Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. Proceedings of the 2009 IEEE International Conference on Computer Vision (IEEE Computer Society, Washington DC), pp 2106-2113.
24. Kienzle W, Franz MO, Schölkopf B, Wichmann FA (2009) Center-surround patterns emerge as optimal predictors for human saccade targets. J Vis 9(5):1-15.
25. Judd T, Durand F, Torralba A A Benchmark of Computational Models of Saliency to Predict Human Fixations. Cambridge, MA: MIT Computer Science and Artificial Intelligence Laboratory; 2012. Report No.: MIT-CSAIL-TR-2012-001.
26. Torralba A, Oliva A, Castelhano MS, Henderson JM (2006) Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychol Rev 113(4):766-786.
27. Harel J, Koch C, Perona P (2006) Graph-based visual saliency. Advances in Neural Information Processing Systems 2006, pp 545-552.
28. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) SUN: A Bayesian framework for saliency using natural statistics. J Vis 8(7):1-20.
29. Kienzle W, Wichmann FA, Schölkopf B, Franz MO (2007) A Nonparametric Approach to Bottom-Up Visual Saliency, eds Schölkopf B, Platt J, Hoffman T (MIT Press, Cambridge, MA), pp 689-696.
30. Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Washington, DC), pp 1-8.
31. Bruce N, Tsotsos J (2009) Saliency, attention, and visual search: An information theoretic approach. J Vis 9(3):1-24.
32. Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Washington, DC), pp 2376-2383.
33. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915-1926.
34. Erdem E, Erdem A (2013) Visual saliency estimation by nonlinearly integrating features using region covariances. J Vis 13(4):11-11.
35. Riche N, et al. (2013) RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis. Signal Process Image Commun 28(6):642-658.
36. Zhang J, Sclaroff S (2015) Exploiting surroundedness for saliency detection: A Boolean map approach. IEEE Trans Pattern Anal Mach Intell, in press.
37. Vig E, Dorr M, Cox D (2014) Large-scale optimization of hierarchical features for saliency prediction in natural images. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society, Washington, DC), pp 2798-2805.
38. Jones E, Oliphant T, Peterson P, others (2001) SciPy: Open source scientific tools for Python. Available at www.scipy.org/. Accessed November 24, 2014.
39. Tatler BW, Baddeley RJ, Gilchrist ID (2005) Visual correlates of fixation selection: Effects of scale and time. Vision Res 45(5):643-659.
40. Itti L, Baldi PF (2005) Bayesian surprise attracts human attention. Advances in Neural Information Processing Systems 2005, pp 547-554.
41. Itti L, Baldi P (2005) A Principled Approach to Detecting Surprising Events in Video A principled approach to detecting surprising events in video. Proceedings of the 2005 IEEE Conference on Computer Vision and Patter Recognition (IEEE Computer Society, Washington, DC) Vol 1, pp 631-637.
42. Itti L (2005) Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis Cogn 12(6):1093-1123.
43. Baldi P, Itti L (2005) Attention: Bits versus wows. Proceedings of the 2005 IEEE International Conference on Neural Networks and Brain, Vol 1, pp PL-56-PL-61.
44. Baldi P, Itti L (2010) Of bits and wows: A Bayesian theory of surprise with applications to attention. Neural Netw 23(5):649-666.
45. Wang W, Wang Y, Huang Q, Gao W (2010) Measuring visual saliency by site entropy rate. Proceedings of the 2010 IEEE Conferernce on Computer Vision and Pattern Recognition (IEEE Computer Society, Washington, DC), pp 2368-2375.
46. Rajashekar U, Cormack LK, Bovik AC (2004) Point-of-gaze analysis reveals visual search strategies. SPIE Proceedings Vol. 5292: Human Vision and Electronic Imaging IX (International Society for Optics and Photonics, Bellingham WA), pp 296-306.
47. Le Meur O, Le Callet P, Barba D (2007) Predicting visual fixations on video based on low-level visual features. Vision Res 47(19):2483-2498.
48. Le Meur O, Baccino T (2013) Methods for comparing scanpaths and saliency maps: Strengths and weaknesses. Behav Res Methods 45(1):251-266.
49. Engbert R, Trukenbrod HA, Barthelmé S, Wichmann FA (2014) Spatial statistics and attentional dynamics in scene viewing. arXiv:1405.3270v2.