Sight and sound out of synch: Fragmentation and renormalisation of audiovisual integration and subjective timing

Cortex

Volumn 49, Issue 10, 2013, Pages 2875-2887

  Freeman, Elliot D ^a   Ipser, Alberta ^a   Palmbaha, Austra ^a   Paunoiu, Diana ^a   Brown, Peter ^b   Lambert, Christian ^c   Leff, Alex ^d   Driver, Jon ^d  

^a CITY UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c ST GEORGE'S UNIVERSITY OF LONDON   (United Kingdom)

^d UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Audiovisual integration; Illusions; Individual differences; Psychophysics; Sensory timing

Indexed keywords

ADULT; ARTICLE; BRAIN DAMAGE; CONTROLLED STUDY; DIFFUSION TENSOR IMAGING; FUNCTIONAL NEUROIMAGING; HEARING; HUMAN; LIP READING; NERVE CONDUCTION; NUCLEAR MAGNETIC RESONANCE IMAGING; PSYCHOPHYSIOLOGY; SENSORIMOTOR INTEGRATION; SIMULATION; STIMULUS; VISUAL INFORMATION; VOICE;

AUDIOVISUAL INTEGRATION; ILLUSIONS; INDIVIDUAL DIFFERENCES; PSYCHOPHYSICS; SENSORY TIMING;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AGED; AGING; ALGORITHMS; ATTENTION; AUDITORY PERCEPTION; BASAL GANGLIA; COGNITION DISORDERS; COMPUTER SIMULATION; DIFFUSION TENSOR IMAGING; FEMALE; HUMANS; ILLUSIONS; IMAGE PROCESSING, COMPUTER-ASSISTED; INTELLIGENCE TESTS; MAGNETIC RESONANCE IMAGING; MALE; MIDDLE AGED; MYASTHENIA GRAVIS; PHOTIC STIMULATION; PONS; PSYCHOMETRICS; SPACE PERCEPTION; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84889682288     PISSN: 00109452     EISSN: 19738102     Source Type: Journal    
DOI: 10.1016/j.cortex.2013.03.006     Document Type: Article

References (74)

1. Alais D., Carlile S. Synchronizing to real events: Subjective audiovisual alignment scales with perceived auditory depth and speed of sound. Proceedings of the National Academy of Sciences of the United States of America 2005, 102(6):2244-2247.
2. Arnold D.H., Clifford C.W., Wenderoth P. Asynchronous processing in vision: Color leads motion. Current Biology 2001, 11(8):596-600.
3. Arnold D.H., Johnston A., Nishida S. Timing sight and sound. Vision Research 2005, 45(10):1275-1284.
4. Aschersleben G., Prinz W. Synchronizing actions with events: The role of sensory information. Perception & Psychophysics 1995, 57(3):305-317.
5. Bertelson P., Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics 1981, 29(6):578-584.
6. Bertini C., Leo F., Avenanti A., Làdavas E. Independent mechanisms for ventriloquism and multisensory integration as revealed by theta-burst stimulation. The European Journal of Neuroscience 2010, 31(10):1791-1799.
7. Bushara K.O., Grafman J., Hallett M. Neural correlates of auditory-visual stimulus onset asynchrony detection. The Journal of Neuroscience 2001, 21(1):300-304.
8. Cappe C., Rouiller E.M., Barone P. Multisensory anatomical pathways. Hearing Research 2009, 258(1-2):28-36.
9. Calvert G.A. Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex 2001, 11(12):1110-1123.
10. Crawford J.R., Howell D.C. Comparing an individual's test score against norms derived from small samples. The Clinical Neuropsychologist 1998, 12(4):482-486.
11. Dennett D., Kinsbourne M. Time and the observer: The where and when of consciousness in the brain. Behavioral and Brain Sciences 1995, 15(1992):1-35.
12. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 2008, 57(1):11-23.
13. Freeman E., Driver J. Direction of visual apparent motion driven solely by timing of a static sound. Current Biology 2008, 18(16):1262-1266.
14. Fujisaki W., Shimojo S., Kashino M., Nishida S. Recalibration of audiovisual simultaneity. Nature Neuroscience 2004, 7(7):773-778.
15. García-Pérez M.A., Alcalá-Quintana R. On the discrepant results in synchrony judgment and temporal-order judgment tasks: A quantitative model. Psychonomic Bulletin & Review 2012, 19(5):820-846.
16. Ghazanfar A.A., Schroeder C.E. Is neocortex essentially multisensory?. Trends in Cognitive Sciences 2006, 10(6):278-285.
17. Halliday A., Mingay R. On the resolution of small time intervals and the effect of conduction delays on the judgement of simultaneity. Quarterly Journal of Experimental Psychology 1964, 16(1):37-41.
18. Halverson H.E., Freeman J.H. Medial auditory thalamic input to the lateral pontine nuclei is necessary for auditory eyeblink conditioning. Neurobiology of Learning and Memory 2010, 93(1):92-98.
19. Hamilton R.H., Shenton J.T., Coslett H.B. An acquired deficit of audiovisual speech processing. Brain and Language 2006, 98(1):66-73.
20. Hanson J.V.M., Heron J., Whitaker D. Recalibration of perceived time across sensory modalities. Experimental Brain Research 2008, 185(2):347-352.
21. Harris L.R., Harrar V., Jaekl P., Kopinska A. Mechanisms of simultaneity constancy. Issues of Space and Time in Perception and Action 2008, 232-253. Cambridge University Press. R. Nijhawan (Ed.).
22. Heron J., McGraw P.V., Horoshenkov K.V. Adaptation minimizes distance-related audiovisual delays. Journal of Vision 2007, 7(13):1-8.
23. Howard I.P., Templeton W.B. Human Spatial Orientation 1966, John Wiley & Sons, Oxford.
24. Ivry R.B., Spencer R.M.C. The neural representation of time. Current Opinion in Neurobiology 2004, 14(2):225-232.
25. Keesey J.C. Does myasthenia gravis affect the brain?. Journal of the Neurological Sciences 1999, 170(2):77-89.
26. Keetels M., Vroomen J. Perception of synchrony between the senses. The Neural Bases of Multisensory Processes 2012, 1-27. CRC Press, Boca Raton (FL). M.M. Murray, M.T. Wallace (Eds.).
27. King A.J. Multisensory integration: Strategies for synchronization. Current Biology 2005, 15(9):R336-R339.
28. Kolomiets B.P., Deniau J.M., Mailly P., Menetrey A., Glowinski J., Thierry A.M. Segregation and convergence of information flow through the cortico-subthalamic pathways. The Journal of Neuroscience 2001, 21(15):5764-5772.
29. Kopinska A., Harris L.R. Simultaneity constancy. Perception 2004, 33(9):1049-1060.
30. Lambert C., Zrinzo L., Nagy Z., Lutti A., Hariz M., Foltynie T., et al. Confirmation of functional zones within the human subthalamic nucleus: Patterns of connectivity and sub-parcellation using diffusion weighted imaging. NeuroImage 2012, 60(1):83-94.
31. Macaluso E., Driver J. Multisensory spatial interactions: A window onto functional integration in the human brain. Trends in Neurosciences 2005, 28(5):264-271.
32. Martin B., Giersch A., Huron C., Van Wassenhove V. Temporal event structure and timing in schizophrenia: Preserved binding in a longer "now". Neuropsychologia 2013, 51(2):358-371.
33. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264(5588):746-748.
34. Meredith M.A., Nemitz J.W., Stein B.E. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. The Journal of Neuroscience 1987, 7(10):3215-3229.
35. Miller L.M., D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. The Journal of Neuroscience 2005, 25(25):5884-5893.
36. Miyazaki M., Yamamoto S., Uchida S., Kitazawa S. Bayesian calibration of simultaneity in tactile temporal order judgment. Nature Neuroscience 2006, 9(7):875-877.
37. Morein-Zamir S., Soto-Faraco S., Kingstone A. Auditory capture of vision: Examining temporal ventriloquism. Cognitive Brain Research 2003, 17(1):154-163.
38. Moutoussis K., Zeki S. A direct demonstration of perceptual asynchrony in vision. Proceedings of the Royal Society B: Biological Sciences 1997, 264(1380):393-399.
39. Munhall K.G., Gribble P., Sacco L., Ward M. Temporal constraints on the McGurk effect. Perception & Psychophysics 1996, 58(3):351-362.
40. Roach N.W., Heron J., Whitaker D., McGraw P.V. Asynchrony adaptation reveals neural population code for audiovisual timing. Proceedings of the Royal Society B: Biological Sciences 2011, 278(1710):1314-1322.
41. Robertson I.H. The Test of Everyday Attention 2006, Harcourt Assessment, London.
42. Scharnowski F., Rees G., Walsh V. Time and the brain: Neurorelativity. Trends in Cognitive Sciences 2013, 17(2):51-52.
43. Schneider K.A., Bavelier D. Components of visual prior entry. Cognitive Psychology 2003, 47(4):333-366.
44. Sekuler R., Sekuler A., Lau R. Sound alters visual motion perception. Nature 1997, 385(23rd January):308.
45. Sherwin I., Efron R. Temporal ordering deficits following anterior temporal lobectomy. Brain and Language 1980, 11(1):195-203.
46. Soto-Faraco S., Alsius A. Conscious access to the unisensory components of a cross-modal illusion. NeuroReport 2007, 18(4):347-350.
47. Soto-Faraco S., Alsius A. Deconstructing the McGurk-MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance 2009, 35(2):580-587.
48. Spence C., Ngo M.K. Does attention or multisensory integration explain the crossmodal facilitation of masked visual target identification?. The New Handbook of Multisensory Processing 2012, 345-358. MIT Press, Cambridge, Mass. B.E. Stein (Ed.).
49. Spence C., Parise C. Prior-entry: A review. Consciousness and Cognition 2010, 19(1):364-379.
50. Spence C., Shore D.I., Klein R.M. Multisensory prior entry. Journal of Experimental Psychology: General 2001, 130(4):799-832.
51. Spence C., Squire S. Multisensory integration: Maintaining the perception of synchrony. Current Biology 2003, 13(13):R519-R521.
52. Spreen O., Benton A.L. Neurosensory Centre Comprehensive Examination for Aphasia 1969, Neuropsychology Laboratory, University of Victoria, Victoria, British Columbia.
53. Sternberg S., Knoll R.L. The perception of temporal order: Fundamental issues and a general model. Attention and Performance IV 1973, 629-685. S. Kornblum (Ed.).
54. Stevenson R.A., Altieri N.A., Kim S., Pisoni D.B., James T.W. Neural processing of asynchronous audiovisual speech perception. NeuroImage 2010, 49(4):3308-3318.
55. Stone J.V., Hunkin N.M., Porrill J., Wood R., Keeler V., Beanland M., et al. When is now? Perception of simultaneity. Proceedings of the Royal Society B: Biological Sciences 2001, 268(1462):31-38.
56. Sugita Y., Suzuki Y. Implicit estimation of sound-arrival time. Nature 2003, 421(6926):911.
57. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America 1954, 26(2):212-215.
58. Teki S., Grube M., Griffiths T.D. A unified model of time perception accounts for duration-based and beat-based timing mechanisms. Frontiers in Integrative Neuroscience 2011, 5(January):90.
59. Wechsler Abbreviated Scale of Intelligence 1999, The Psychological Corporation, Harcourt Brace & Co., San Antonio.
60. Treisman M. Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". Psychological Monographs 1963, 77(13):1-31.
61. van Eijk R.L.J., Kohlrausch A., Juola J.F., van De Par S. Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type. Perception & Psychophysics 2008, 70(6):955-968.
62. van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 2007, 45(3):598-607.
63. Vatakis A., Ghazanfar A.A., Spence C. Facilitation of multisensory integration by the "unity effect" reveals that speech is special. Journal of Vision 2008, 8(9):1-11.
64. Vatakis A., Spence C. Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli. Perception & Psychophysics 2007, 69(5):744-756.
65. Vroomen J., De Gelder B. Temporal ventriloquism: Sound modulates the flash-lag effect. Journal of Experimental Psychology: Human Perception and Performance 2004, 30(3):513-518.
66. Vroomen J., Keetels M. Perception of intersensory synchrony: A tutorial review. Attention, Perception, & Psychophysics 2010, 72(4):871.
67. Vroomen J., Stekelenburg J.J. Perception of intersensory synchrony in audiovisual speech: Not that special. Cognition 2011, 118(1):75-83.
68. Warrington E.K., James M. The Visual Object and Space Perception Battery 1991, Thames Valley Test Company, Bury St Edmunds.
69. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 1980, 88(3):638-667.
70. Wichmann F.A., Hill N.J. The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics 2001, 63(8):1293-1313.
71. Wiener M., Matell M.S., Coslett H.B. Multiple mechanisms for temporal processing. Frontiers in Integrative Neuroscience 2011, 5(July):1-3.
72. Witkin H.A., Wapner S., Leventhal T. Sound localization with conflicting visual and auditory cues. Journal of Experimental Psychology 1952, 43(1):58-67.
73. Yamamoto S., Miyazaki M., Iwano T., Kitazawa S. Bayesian calibration of simultaneity in audiovisual temporal order judgments. PloS ONE 2012, 7(7):e40379.
74. Zeki S., Bartels A. The asynchrony of consciousness. Proceedings of the Royal Society B: Biological Sciences 1998, 265(1405):1583-1585.