Bayesian models of object perception

Current Opinion in Neurobiology

Volumn 13, Issue 2, 2003, Pages 150-158

  Kersten, Daniel ^a   Yuille, Alan ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATION; BAYES THEOREM; BODY MOVEMENT; DEPTH PERCEPTION; EVALUATION; FEEDBACK SYSTEM; HUMAN; ILLUMINATION; NORMAL DISTRIBUTION; NUCLEAR MAGNETIC RESONANCE IMAGING; PRIORITY JOURNAL; RETINA IMAGE; REVIEW; VISUAL CORTEX; VISUAL INFORMATION; VISUAL STIMULATION; VISUAL SYSTEM;

EID: 0038396223     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0959-4388(03)00042-4     Document Type: Review

References (65)

1. Helmholtz H: Handbuch der physiologischen optik. Leipzig: Voss L; 1867. [Translation of title: Handbook of Physiological Optics.].
2. Grenander U: Elements of Pattern Theory. Baltimore: John Hopkins University Press; 1996.
3. Knill DC, Richards W: Perception as Bayesian Inference. Edited by Knill DC, Richards W. Cambridge: Cambridge University Press; 1996.
4. Mamassian P, Landy MS, Maloney LT: Bayesian modelling of visual perception In Probabilistic Models of the Brain: Perception and Neural Function. Edited by Rao RPN, Olshausen BA, Lewicki MS. Cambridge, MA: MIT Press; 2002: 13-36.
5. Rao RPN, Olshausen BA, Lewicki MS: Probabilistic Models of the Brain: Perception and Neural Function. Edited by Rajesh PN, Rao R, Bruno A, Olshausen B, Michael S, Lewicki M. Cambridge, Mass: MIT Press; 2002.
6. Bayes T. Essay towards solving a problem in the doctrine of chances. Philosop. Trans. Roy Soc. 53:1763;370-418.
7. Barlow H.B. A method of determining the overall quantum efficiency of visual discriminations. J. Physiol. (Lond). 160:1962;155-168.
8. Liu Z., Knill D.C., Kersten D. Object classification for human and ideal observers. Vision Res. 35:1995;549-568.
9. Eckstein M.P., Thomas J.P., Palmer J., Shimozaki S.S. A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays. Percept. Psychophys. 62:2000;425-451.
10. Portilla J., Simoncelli E.P. A parametric texture model based on joint statistics of complex wavelet coefficients. Intl. J. Comput. Vision. 40:2000;9-71.
11. Simoncelli E.P., Olshausen B.A. Natural image statistics and neural representation. Annu. Rev. Neurosci. 24:2001;1193-1216 This is an excellent review of generic natural image statistics with a primary emphasis on early neural coding. Although the basic theme is related to the theme of this review, it does not focus on statistics for inference of specific object properties.
12. Zhu S.C., Wu Y., Mumford D. Minimax entropy principle and its applications to texture modeling. Neural. Comput. 9:1997;1627-1660.
13. Judd D.B., MacAdam D.L., Wyszecki G. Spectral distribution of typical daylight as a function of correlated color temperature. J. Opt. Soc. Am. 54:1964;1031-1040.
14. Golz J., MacLeod D.I. Influence of scene statistics on colour constancy. Nature. 415:2002;637-640 We see objects as having constant surface color despite the fact that the retinal input is a function of both surface color and illumination. A reddish scene under white light can give rise to the same average stimulation as a neutral scene under red light. Using a spectral database of natural images, the authors showed that the correlation between redness and luminance could be used to disambiguate the scene from the lighting color, and further, that the weight given to the estimate of the illuminant was statistically optimal.
15. Fine I., MacLeod D.I.A. Visual segmentation based on the luminance and chromaticity statistics of natural scenes [abstract]. J. Vision. 1(3):2001;63a. URL: http://journalofvision.org/1/3/63, DOI 10.1167/1.3.63. Differences in luminance between two points in a natural scene are statistically associated with differences in chromaticity. The authors incorporated this information into a Bayesian model to distinguish whether two points are on the same surface or not, and showed that it predicted human performance. (This research will appear as an article in the Journal of the Optical Society of America A, Special Issue on Bayesian and Statistical Approaches to Vision to appear in 2003.).
16. Oliva A., Schyns P.G. Diagnostic colors mediate scene recognition. Cognit. Psychol. 41:2000;176-210.
17. Torralba A, Sinha P: Statistical context priming for object detection. In Proceedings of the International Conference on Computer Vision, ICCVO1; Vancouver, Canada: IEEE Computer Society; 2001:763-770.
18. Geisler W.S., Perry J.S., Super B.J., Gallogly D.P. Edge co-occurrence in natural images predicts contour grouping performance. Vision Res. 41:2001;711-724 The authors used spatial filtering to extract local edge fragments from natural images. They measured statistics on the distance between the element centers, the orientation difference between the elements, and the direction of the second element relative to the orientation of first (reference) element, and showed that a model derived from the measured statistics could quantitatively predict human contour detection performance.
19. Elder J.H., Goldberg R.M. Ecological statistics of gestalt laws for the perceptual organization of contours. J. Vision. 2:2002;324-353. URL: http://journalofvision.org/2/4/5/, DOI 10.1167/2.4.5. Statistics of contours are measured from hand-segmented images and used to put probability distributions on the proximity, continuation, and luminance of contour elements. The relative effectiveness of these cues for grouping contour segments into extended contours is evaluated.
20. Zhu S.C. Embedding gestalt laws in Markov random fields. IEEE T Pattern Anal. 21:1999;1170-1187.
21. Konishi S.M., Yuille A.L., Coughlan J.M., Zhu S.C. Statistical edge detection: learning and evaluating edge cues. IEEE Pattern Anal. 25:2003;57-74 This paper measures the statistical effectiveness of different visual features for detecting edges and determines the optimal way to combine them. The statistics of feature responses are measured using image databases where human observers have determined the ground truth positions of the edges. [Also see: Konishi SM, Yuille AL, Coughlan JM, Zhu SC: Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues. In Proceedings Computer Vision and Pattern Recognition CVPR'99; Fort Collins, Colorado: 1999.].
22. Chen HF, Belhumeur PN, Jacobs DW: In search of illumination invariants. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2000, 1:254-261.
23. Troje N.F. Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. J. Vision. 2:2002;371-387.
24. Atick J.J., Griffin P.A., Redlich A.N. Statistical approach to shape from shading: reconstruction of three-dimensional face surfaces from single two-dimensional images. Neural. Comput. 8:1996;1321-1340.
25. Vetter T., Troje N.F. Separation of texture and shape in images of faces for image coding and synthesis. J. Opt. Soc. Am. A. 14:1997;2152-2161.
26. Leopold D.A., O'Toole A.J., Vetter T., Blanz V. Prototype-referenced shape encoding revealed by high-level aftereffects. Nat. Neurosci. 4:2001;89-94.
27. Howe C.Q., Purves D. Range image statistics can explain the anomalous perception of length. Proc. Natl. Acad Sci. USA. 99:2002;13184-13188 Why does a vertical line appear longer than the same line drawn horizontally? Although there have been several recent studies of the statistics of image intensities and contours, this paper takes the next step to directly compare image statistics on line length with measurements of the causes in the originating 3D scene. The authors found that the average ratio of the 3D physical length to the projected image length from real-world measurements shows the same pattern as perceived length. This result is in the spirit of the 1930s and '40s work of Egon Brunswik.
28. Fiser J., Aslin R.N. Statistical learning of new visual feature combinations by infants. Proc. Natl. Acad Sci. USA. 99:2002;15822-15826.
29. Geisler W.S., Diehl R.L. Bayesian natural selection and the evolution of perceptual systems. Philos. Trans. R Soc. Lond. B Biol. Sci. 357:2002;419-448.
30. Bülthoff H.H., Yuille A. Bayesian models for seeing surfaces and depth. Comment Theor. Biol. 2:1991;283-314.
31. Mamassian P., Goutcher R. Prior knowledge on the illumination position. Cognition. 81:2001;B1-9.
32. Weiss Y., Simoncelli E.P., Adelson E.H. Motion illusions as optimal percepts. Nat. Neurosci. 5:2002;598-604 This paper shows that even an ideal observer 'sees' illusions. The 'aperture problem' is a classic problem in motion perception - how do you combine locally ambiguous motion measurements into a single velocity for an object? The authors devise an ideal observer whose prior knowledge is that motions tend to be slow, and integrates local measurements according to their reliabilities. With this model (using a single free parameter per experiment), the authors account for a wide range of motion phenomena and results in human perception that are usually thought to reflect different neural mechanisms.
33. Feldman J. Bayesian contour integration. Percept. Psychophys. 63:2001;1171-1182.
34. Kersten D: High-level vision as statistical inference. In The New Cognitive Neurosciences 2nd Edition. Edited by Gazzaniga MS. Cambridge: MIT Press; 1999:353-363.
35. Freeman W.T. The generic viewpoint assumption in a framework for visual perception. Nature. 368:1994;542-545.
36. Brainard D.H., Freeman W.T. Bayesian color constancy. J. Opt. Soc. Am. A. 14:1997;1393-1411.
37. Bloj M.G., Kersten D., Hurlbert A.C. Perception of three-dimensional shape influences colour perception through mutual illumination. Nature. 402:1999;877-879.
38. Yuille AL, Bülthoff HH: Bayesian decision theory and psychophysics. In Perception as Bayesian Inference. Edited by Knill DC, Richards W. Cambridge: Cambridge University Press; 1996:123-161.
39. Koenderink J.J., van Doorn A.J., Kappers A.M., Todd J.T. Ambiguity and the 'mental eye' in pictorial relief. Perception. 30:2001;431-448.
40. Clark JJ, Yuille AL: Data Fusion for Sensory Information Processing. Boston: Kluwer Academic Publishers; 1990.
41. Jacobs R.A. What determines visual cue reliability? Trends Cogn. Sci. 6:2002;345-350 Combining information according to reliability raises the obvious question of the title, where does the information come from? This paper provides a nice overview of recent work on optimal cue combination in human vision together with a discussion of what determines the weighting for cue integration. The article also discusses integration in the context of Kalman filtering, which is a special case of Bayesian estimation.
42. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 415:2002;429-433 Object size can be perceived both visually and by touch. When the information from vision and touch disagree, vision usually dominates. The traditional interpretation that this is because 'vision is the dominant sense' is not particularly informative. The authors showed that when one takes into account the reliability of the sensory measurements, information from vision and touch are integrated optimally. Visual dominance occurs when the reliability of the visual estimation is greater than that of the haptic one.
43. Landy M.S., Maloney L.T., Johnston E.B., Young M. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res. 35:1995;389-412.
44. Yuille A., Grzywacz N. A computational theory for the perception of coherent visual motion. Nature. 333:1988;71-74.
45. Landy M.S., Kojima H. Ideal cue combination for localizing texture-defined edges. J. Opt. Soc. Am. A. 18:2001;2307-2320.
46. Saunders J.A., Knill D.C. Perception of 3D surface orientation from skew symmetry. Vision Res. 41:2001;3163-3183 The projection of a symmetric flat object has a distorted or 'skewed' symmetry that provides partial information about the object's orientation in depth. The authors show that human observers integrate symmetry information with stereo information weighted according to the reliability of the source.
47. Mamassian P., Landy M.S. Interaction of visual prior constraints. Vision Res. 41:2001;2653-2668 The authors and others had previously shown that human observers interpret shape assuming prior knowledge constraints that both the viewpoint and the light source tend to be above the object. The authors manipulated the reliability of each of the two constraints by changing the contrast of different parts of the stimuli. The results suggested that prior knowledge constraints behave like depth cues in integration: that is, cues with more reliable information have higher weight attributed to their corresponding prior knowledge constraint.
48. Bülthoff H.H., Mallot H.A. Integration of depth modules: stereo and shading. J. Opt. Soc. Am. A. 5:1988;1749-1758.
49. Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference 2nd Edn. San Mateo: Morgan Kaufmann Publishers; 1988.
50. Lorenceau J., Shiffrar M. The influence of terminators on motion integration across space. Vision Res. 32:1992;263-273.
51. McDermott J., Weiss Y., Adelson E.H. Beyond junctions: nonlocal form constraints on motion interpretation. Perception. 30:2001;905-923 The interpretation of the translating diamond (described by [50]) shown in (Figure 4d, depends on which local motion measurements are integrated. These authors show how non-local form information related to surface occlusions determines the selection of motion elements to be integrated. The web-based interactive demonstrations are highly recommended: http://www.perceptionweb.com/perc0801/square.html.
52. Murray S.O., Kersten D., Olshausen B.A., Schrater P., Woods D.L. Shape perception reduces activity in human primary visual cortex. Proc. Natl. Acad. Sci. USA. 99:2002;15164-15169 Earlier fMRI work by several groups (see [61]) has shown that the human lateral occipital complex (LOC) increases its activity during object perception. The authors use fMRI to show that when local visual information is perceptually organized into whole objects, activity in V1 decreases over the same period that activity in higher, LOC areas increases. The authors interpret the activity changes in terms of high-level hypotheses that compete to explain away the incoming retinal data.
53. Mamassian P., Knill D.C., Kersten D. The perception of cast shadows. Trends Cogn. Sci. 2:1998;288-295.
54. Blake A., Bülthoff H.H. Does the brain know the physics of specular reflection? Nature. 343:1990;165-169.
55. Tu Z., Zhu S.-C. Image segmentation by data-driven Markov chain. IEEE T. Pattern Anal. 24:2002;657-673 A Bayesian model for image segmentation that uses low-level visual features to rapidly search through high-level interpretations. This model yields good segmentations when tested on a database of a hundred natural images and compared with ground-truth estimates by human observers.
56. Burgi P.Y., Yuille A.L., Grzywacz N.M. Probabilistic motion estimation based on temporal coherence. Neural Comput. 12:2000;1839-1867.
57. Lee T.S., Yang C.F., Romero R.D., Mumford D. Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency. Nat. Neurosci. 5:2002;589-597 Electrode recording of cells in V1 and V2 of macaque monkeys respond to the apparently high-level task of detecting stimuli that pop-out from shape-from-shading. These responses changed with the animal's behavioral adaptation to contingencies, which suggests dependence on experience and utility.
58. Pouget A., Dayan P., Zemel R. Information processing with population codes. Nat. Rev. Neurosci. 1:2000;125-132.
59. Mumford D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66:1992;241-251.
60. Bullier J. Integrated model of visual processing. Brain Res. Brain Res. Rev. 36:2001;96-107 This paper reviews the evidence that feedback connections are used to enable high-level visual areas to influence perception by interacting with low-level areas V1 and V2. In particular, the cooling of areas such as motion area MT (middle temporal cortex) is shown to affect the response of neurons in lower-areas.
61. Grill-Spector K., Kourtzi Z., Kanwisher N. The lateral occipital complex and its role in object recognition. Vision Res. 41:2001;1409-1422.
62. Tu Z, Zhu S-C: Parsing images into region and curve processes. In Proceedings of the 7th European Conference on Computer Vision April 29,2002; Copenhagen, Denmark: LNCS2352. Edited by Heyden A, Sparr G, Nielsen M, Johnasen P. Berlin, Heidelberg: Springer-Verlag; 2002:393. URL: http://link.springer.de/link/service/series/0558/tocs/t2352.htm#toc2352 This paper uses a feedforward and feedback Bayesian model to parse images into regions and curves by requiring these alternative explanations to compete to explain the data. The curves can 'explain away' image features, which might adversely affect the quality of the image segmentations by distorting the estimates of the regions.
63. Belhumeur PN, Kriegman DJ, Yuille A: The bas-relief ambiguity. In IEEE Conference on Computer vision and Pattern Recognition. Puerto Rico: IEEE Computer Society;1997: 1060-1066.
64. Sinha P, Adelson E: Recovering reflectance and illumination in a world of painted polyhedra. In Proceedings of Fourth International Conference on Computer Vision; Berlin: IEEE Computer Society Press;1993:156-163.
65. Kersten D, Schrater PR: Pattern inference theory: a probabilistic approach to vision. In Perception and the Physical World. Edited by Mausfeld R, Heyer D: Chichester: John Wiley & Sons, Ltd.; 2002: 191-228.