Merging the senses into a robust percept

Trends in Cognitive Sciences

Volumn 8, Issue 4, 2004, Pages 162-169

  Ernst, Marc O ^a   Bülthoff, Heinrich H ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; INFORMATION ANALYSIS; KNOWLEDGE ENGINEERING; ROBUSTNESS (CONTROL SYSTEMS); SENSORY PERCEPTION;

ROBUST PERCEPT; SENSORY SIGNALS;

COGNITIVE SYSTEMS;

AUDITORY SYSTEM; BAYES THEOREM; BRAIN; EYE DOMINANCE; HUMAN; INFORMATION; LEARNING; MAXIMUM LIKELIHOOD METHOD; MOTOR PERFORMANCE; NERVE CELL; NORMAL DISTRIBUTION; PERCEPTION; RELIABILITY; REVIEW; SENSORY STIMULATION; SENSORY SYSTEM; TACTILE STIMULATION; VISION;

BRAIN; CUES; FIELD DEPENDENCE-INDEPENDENCE; HUMANS; MENTAL PROCESSES; MODELS, PSYCHOLOGICAL; PERCEPTION; SPATIAL BEHAVIOR; VISUAL PERCEPTION;

EID: 1842576869     PISSN: 13646613     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tics.2004.02.002     Document Type: Review

References (63)

1. Kersten D., et al. Illusory motion from shadows. Nature. 379:1996;31.
2. Bülthoff H.H. Shape from X: psychophysics and computation. Landy M., Movshon A. Computational Models of Visual Processing. 1991;305-330 MIT Press.
3. Blake A., Bülthoff H.H. Does the brain know the physics of specular reflection? Nature. 343:1990;165-168.
4. Bülthoff H.H., Mallot H.A. Integration of depth modules: stereo and shading. J. Opt. Soc. Am. A. 5:1988;1749-1758.
5. Stein B.E., Meredith M.A. The Merging of the Senses. 1993;MIT Press.
6. Newell F.N., et al. Viewpoint dependence in visual and haptic object recognition. Psychol. Sci. 12:2001;37-42.
7. Newell F.N., et al. Cross-modal perception of actively explored objects. O'Modhrain S., et al. Eurohaptics 2003 Conference Proceedings. 2003;291-299.
8. Sinha P., Poggio T. The role of learning in 3-D form perception. Nature. 384:1996;460-463.
9. Rock I., Victor J. Vision and touch: an experimentally created conflict between the two senses. Science. 143:1964;594-596.
10. Shams L., et al. What you see is what you hear. Nature. 408:2000;788.
11. Welch, R.B. and Warren, D.H. (1986) Intersensory interactions. In Handbook of Perception and Human Performance (Boff, K.R., et al., eds.), pp. 25.1-25.36, Wiley.
12. Spence C., Squire S. Multisensory integration: maintaining the perception of synchrony. Curr. Biol. 13:2003;519-521.
13. Cochran W.G. Problems arising in the analysis of a series of similar experiments. J. R. Stat. Soc. 4:1937;102-118.
14. Kalman R.E., Bucy R.S. New results in linear filtering and prediction problems. J. Basic Eng. Ser. D. 83:1961;95-108.
15. Goodwin G.C., Sin K.S. Adaptive Filtering Prediction and Control. 1984;Prentice Hall.
16. Oruç I., et al. Weighted linear cue combination with possibly correlated error. Vis. Res. 43:2003;2451-2468.
17. Johnston E.B., et al. Integration of stereopsis and motion shape cues. Vis. Res. 34:1994;2259-2275.
18. Young M.J., et al. A perturbation analysis of depth perception from combinations of texture and motion cues. Vis. Res. 33:1993;2685-2696.
19. Backus B.T., et al. Horizontal and vertical disparity, eye position, and stereoscopic slant perception. Vis. Res. 39:1999;1143-1170.
20. Landy M.S., Kojima H. Ideal cue combination for localizing texture defined edges. J. Opt. Soc. Am. A Opt Image Sci. Vis. 18:2001;2307-2320.
21. Brenner E., van Damme W.J.M. Perceived distance, shape and size. Vis. Res. 39:1999;975-986.
22. Robles-De-La-Torre G., Hayward V. Force can overcome object geometry in the perception of shape through active touch. Nature. 412:2001;445-448.
23. Drewing K., Ernst M.O. Cue integration in the haptic perception of virtual shapes. Granum E., et al. Presence 2003 Conference Proceedings. 2003;39.
24. Gharamani Z., et al. Morasso P.G., Sanguineti V. Self-organization, Computational Maps, and Motor Control. 1997;117-147 Elsevier.
25. van Beers R.J., et al. Integration of proprioceptive and visual position information: an experimentally supported model. J. Neurophysiol. 81:1999;1355-1364.
26. Jacobs R.A. What determines visual cue reliability. Trends Cogn. Sci. 6:2002;345-350.
27. Jacobs R.A. Optimal integration of texture and motion cues to depth. Vis. Res. 39:1999;3621-3629.
28. Perotti V.J., et al. The perception of surface curvature from optical motion. Percept. Psychophys. 60:1998;377-388.
29. Ernst M.O., Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 415:2002;429-433.
30. Gepshtein S., Banks M.S. Viewing geometry determines how vision and haptics combine in size perception. Curr. Biol. 13:2003;483-488.
31. Alais D., Burr D. The ventriloquist effect results from near optimal crossmodal integration. Curr. Biol. 14:2004;257-262.
32. Knill D.C., Saunders J.A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vis. Res. 43:2003;2539-2558.
33. Anastasio T.J., et al. Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Comput. 12:2000;1165-1187.
34. Zemel R.S., Dayan P. Combining probabilistic population codes. Int. J. Comput. Artif. Intell. 15:1997;1114-1119.
35. Deneve S., et al. Reading population codes: a neural implementation of ideal observers. Nat. Neurosci. 2:1999;740-745.
36. Pouget A., et al. Information processing with population codes. Nat. Rev. Neurosci. 1:2000;125-132.
37. Barber M.J., et al. Neural representation of probabilistic information. Neural Comput. 15:2003;1843-1864.
38. Pouget A., et al. Computation and inference with population codes. Annu. Rev. Neurosci. 26:2003;381-410.
39. Adelson E.H., Bergen J.R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A. 2:1985;284-299.
40. Ohzawa I., et al. Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science. 249:1990;1037-1041.
41. Landy M.S., et al. Measurement and modeling of depth cue combination: in defense of weak fusion. Vis. Res. 35:1995;389-412.
42. Banks M.S., Backus B.T. Extra-retinal and perspective cues cause the small range of the induced effect. Vis. Res. 38:1998;187-194.
43. Hillis J.M., et al. Combining sensory information: mandatory fusion within, but not between, senses. Science. 298:2002;1627-1630.
44. Ernst M.O., et al. Touch can change visual slant perception. Nat. Neurosci. 3:2000;69-73.
45. Epstein W., Morgan C.L. Adaptation to uniocular image magnification: modification of the disparity-depth relationship. Am. J. Psychol. 83:1970;322-329.
46. Adams W.J., et al. Adaptation to three-dimensional distortions in human vision. Nat. Neurosci. 4:2001;1063-1064.
47. De Gelder B., Bertelson P. Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7:2003;460-467.
48. Atkins J.E., et al. Experience-dependent visual cue recalibration based on discrepancies between visual and haptic percepts. Vis. Res. 43:2003;2603-2613.
49. Atkins J.E., et al. Experience-dependent visual cue integration based on consistencies between visual and haptic percepts. Vis. Res. 41:2001;449-461.
50. Jacobs R.A., Fine I. Experience-dependent integration of texture and motion cues to depth. Vis. Res. 39:1999;4062-4075.
51. Ittleson W.H. Visual Space Perception. 1960;Springer.
52. Knill D.C., et al. Implications of a Bayesian formulation of visual information for processing for psychophysics. Knill D.C., Richards W. Perception as Bayesian Inference. 1996;239-286 Cambridge University Press.
53. Massaro D.W., Friedman D. Models of integration given multiple sources of information. Psychol. Rev. 97:1990;225-252.
54. Clark J.J., Yuille A.L. Data Fusion for Sensory Information Processing Systems. 1990;Kluwer.
55. Howard I.P., Rogers B.J. Interactions between depth cues. Howard I.P., Rogers B.J. Seeing in Depth. Vol. 2:2002;469-493 Porteous.
56. Yuille A.L., Bülthoff H.H. Bayesian theory and psychophysics. Knill D., Richards W. Perception as Bayesian Inference. 1996;123-161 Cambridge University Press.
57. Knill D., Richards W. Perception as Bayesian Inference. 1996;Cambridge University Press.
58. Mamassian P., et al. Bayesian modelling of visual perception. Rao P.N., et al. Probabilistic Models of the Brain. 2002;13-36 MIT Press.
59. Kersten D., Yuille A.L. Bayesian models of object perception. Curr. Opin. Neurobiol. 13:2003;150-158.
60. Schrater P.R., Kersten D. How optimal depth cue integration depends on the task. Int. J. Comput. Vision. 40:2000;71-89.
61. Trommershäuser J., et al. Statistical decision theory and the selection of rapid, goal-directed movements. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20:2003;1419-1433.
62. Triesch J., et al. Fast temporal dynamics of visual cue integration. Perception. 31:2002;421-434.
63. Kersten D. High-level vision as statistical inference. Gazzaniga M.S. The New Cognitive Neurosciences. 1999;353-363 MIT Press.