Asynchrony adaptation reveals neural population code for audio-visual timing

Proceedings of the Royal Society B: Biological Sciences

Volumn 278, Issue 1710, 2011, Pages 1314-1322

  Roach, Neil W ^a   Heron, James ^b   Whitaker, David ^b   McGraw, Paul V ^a  

^a UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^b UNIVERSITY OF BRADFORD   (United Kingdom)

Author keywords

Auditory visual timing; Multisensory; Population coding

Indexed keywords

ALGORITHM; BRAIN; DEVELOPMENTAL BIOLOGY; NEUROLOGY; NUMERICAL MODEL; PERCEPTION; PHYSIOLOGICAL RESPONSE; POPULATION STRUCTURE; SAMPLING; SIGNAL; TEMPORAL VARIATION;

EID: 79953327019     PISSN: 09628452     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rspb.2010.1737     Document Type: Article

References (53)

1. King, A. J. & Palmer, A. R. 1985 Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp. Brain Res. 60, 492-500. (doi:10.1007/BF00236934)
2. Spence, C. & Squire, S. 2003 Multisensory integration: maintaining the perception of synchrony. Curr. Biol. 13, R519-R521. (doi:10.1016/S0960-9822(03)00445-7)
3. Fujisaki, W., Shimojo, S., Kashino, M. & Nishida, S. 2004 Recalibration of audiovisual simultaneity. Nat. Neurosci. 7, 773-778. (doi:10.1038/nn1268)
4. Hanson, J. V. M., Heron, J. & Whitaker, D. 2008 Recalibration of perceived time across sensory modalities. Exp. Brain Res. 185, 347-352. (doi:10.1007/s00221-008-1282-3)
5. Harrar, V. & Harris, L. R. 2005 Simultaneity constancy: detecting events with touch and vision. Exp. Brain Res. 166, 465-473. (doi:10.1007/s00221-005-2386-7)
6. Harrar, V. & Harris, L. R. 2008 The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity. Exp. Brain. Res. 186, 517-524. (doi:10.1007/s00221-007-1253-0)
7. Heron, J., Roach, N.W., Whitaker, D. & Hanson, J. V. M. 2010 Attention regulates the plasticity of multisensory timing. Eur. J. Neurosci. 31, 1755-1762. (doi:10.1111/j.1460-9568.2010.07194.x)
8. Heron, J., Whitaker, D., McGraw, P. V. & Horoshenkov, K. V. 2007 Adaptation minimizes distance-related audiovisual delays. J. Vision 7, 1-8.
9. Keetels, M. & Vroomen, J. 2007 No effect of auditoryvisual spatial disparity on temporal recalibration. Exp. Brain Res. 182, 559-565. (doi:10.1007/s00221-007-1012-2)
10. Keetels, M. & Vroomen, J. 2008 Temporal recalibration to tactile-visual asynchronous stimuli. Neurosci. Lett. 430, 130-134. (doi:10.1016/j.neulet.2007.10.044)
11. Vatakis, A., Navarra, J., Soto-Faraco, S. & Spence, C. 2007 Temporal recalibration during asynchronous audiovisual speech perception. Exp. Brain Res. 181, 173-181. (doi:10.1007/s00221-007-0918-z)
12. Vroomen, J., Keetels, M., de Gelder, B. & Bertelson, P. 2004 Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cogn. Brain Res. 22, 32-35. (doi:10.1016/j.cogbrainres.2004.07.003)
13. Ulrich, R. 1987 Threshold models of temporal-order judgments evaluated by a ternary response task. Percept. Psychophys. 42, 224-239.
14. Sternberg, S. & Knoll, R. L. 1973 The perception of temporal order: fundamental issues and a general model. In Attention and performance IV (ed. S. Kornblum), pp. 629-685. New York, NY: Academic Press.
15. Gibbon, J. & Rutschmann, R. 1969 Temporal order judgement and reaction time. Science 165, 413-415. (doi:10.1126/science.165.3891.413)
16. Rutschmann, R. 1966 Perception of temporal order and relative visual latency. Science 152, 1099-1101. (doi:10.1126/science.152.3725.1099)
17. Arden, G. B. & Weale, R. A. 1954 Variations of the latent period of vision. Proc. R. Soc. Lond. B 142, 258-267. (doi:10.1098/rspb.1954.0025)
18. Titchener, E. B. 1908 Lectures on the elementary psychology of feeling and attention. New York, NY: Macmillan.
19. Moutoussis, K. & Zeki, S. 1997 A direct demonstration of perceptual asynchrony in vision. Proc. R. Soc. Lond. B 264, 393-399. (doi:10.1098/rspb.1997.0056)
20. Bald, L., Berrien, F. K., Price, J. B. & Sprague, R. O. 1942 Errors in perceiving the temporal order of auditory and visual stimuli. J. Appl. Psychol. 26, 382-388. (doi:10.1037/h0059216)
21. Navarra, J., Hartcher-O'Brien, J., Piazza, E. & Spence, C. 2009 Adaptation to audiovisual asynchrony modulates the speeded detection of sound. Proc. Natl Acad. Sci. USA 106, 9169-9173. (doi:10.1073/pnas.0810486106)
22. Di Luca, M., Machulla, T. & Ernst, M. O. 2009 Recalibration of multisensory simultaneity: cross-modal transfer coincides with a change in perceptual latency. J. Vision 9, 1-16. (doi:10.1167/9.12.7)
23. Pouget, A., Dayan, P. & Zemel, R. 2000 Information processing with population codes. Nat. Rev. Neurosci. 1, 125-132. (doi:10.1038/35039062)
24. Averbeck, B. B., Latham, P. E. & Pouget, A. 2006 Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358-366. (doi:10.1038/nrn1888)
25. Jazayeri, M. 2008 Probablistic sensory recoding. Curr. Opin. Neurobiol. 18, 1-7. (doi:10.1016/j.conb.2008.09.004)
26. Deas, R. W., Roach, N. W. & McGraw, P. V. 2008 Distortions of perceived auditory and visual space following adaptation to motion. Exp. Brain Res. 191, 473-485. (doi:10.1007/s00221-008-1543-1)
27. Stevens, S. S. 1953 On the brightness of lights and loudness of sounds. Science 118, 576.
28. Jazayeri, M. & Movshon, J. A. 2006 Optimal representation of sensory information by neural populations. Nat. Neurosci. 9, 690-696. (doi:10.1038/nn1691)
29. Seriès, P., Stocker, A. A. & Simoncelli, E. P. 2009 Is the homunculus 'aware' of sensory adaptation? Neural. Comp. 21, 3271-3304. (doi:10.1162/neco.2009.09-08-869)
30. Efron, B. & Tibshirani, R. J. 1993 An introduction to the bootstrap. New York, NY: Chapman & Hall.
31. Eagleman, D., Cui, M. & Stetson, C. 2009 A neural model for temporal order judgments and their active recalibration: a common mechanism for space and time? J. Vision 9, 2. (doi:10.1167/9.8.2)
32. Gibson, J. J. & Radner, M. 1937 Adaptation, aftereffect and contrast in the perception of tilted lines. I. Quantitative studies. J. Exp. Psychol. 20, 453-467. (doi:10.1037/ h0059826)
33. Levison, E. & Sekuler, R. 1976 Adaptation alters perceived direction of motion. Vision Res. 16, 770-781.
34. Jin, D. Z., Dragoi, V., Sur, M. & Seung, H. S. 2005 Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex. J. Neurophysiol. 94, 4038-4050. (doi:10.1152/jn.00571.2004)
35. Kohn, A. & Movshon, J. A. 2004 Adaptation changes the direction tuning of macaque MT neurons. Nat. Neurosci. 7, 764-772. (doi:10.1038/nn1267)
36. Seung, H. S. & Sompolinsky, H. 1993 Simple models for reading neuronal population codes. Proc. Natl Acad. Sci. USA 90, 10749-10753. (doi:10.1073/pnas.90.22.10749)
37. Deneve, S., Latham, P. & Pouget, A. 1999 Optimal decoding of noisy neuronal populations using recurrent networks. Nat. Neurosci. 2, 740-745. (doi:10.1038/11205)
38. Stevens, S. S. & Greenbaum, H. B. 1966 Regression effect in psychophysical judgement. Percept. Psychophys. 1, 439-446.
39. Teghtsoonian, R. & Teghtsoonian, M. 1978 Range and regression effects in magnitude scaling. Percept. Psychophys. 24, 305-314.
40. Schwartz, O., Hsu, A. & Dayan, P. 2007 Space and time in visual context. Nat. Rev. Neurosci. 8, 522-535. (doi:10.1038/nrn2155)
41. Loe, P. R. & Benevento, L. A. 1969 Auditory-visual interaction in single units in the orbito-insular cortex of the cat. Elecroencephalogr. Clin. Neurophys. 26, 395-398.
42. Benevento, L. A., Fallon, J., Davis, B. J. & Rezak, M. 1977 Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp. Neurol. 57, 849-872. (doi:10.1016/0014-4886(77)90112-1)
43. Meredith, M. A., Nemitz, J. W. & Stein, B. E. 1987 Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, 3215-3229.
44. Wallace, M. T., Wilkinson, L. K. & Stein, B. E. 1996 Representation and integration of multiple sensory inputs in primate superior colliculus. J. Neurophysiol. 76, 1246-1266.
45. Takahashi, K., Saiki, J. & Watanabe, K. 2008 Realignment of temporal simultaneity between vision and touch. Neuroreport 19, 319-322. (doi:10.1097/WNR.0b013e3282f4f039)
46. Haggard, P., Clark, S. & Kalogeras, J. 2002 Voluntary action and conscious awareness. Nat. Neurosci. 5, 382-385. (doi:10.1038/nn827)
47. Stetson, C., Xu, C., Montague, P. R. & Eaglement, D. M. 2006 Motor-sensory recalibration leads to an illusory reversal of action and sensation. Neuron 51, 651-659. (doi:10.1016/j.neuron.2006.08.006)
48. Kennedy, J. S., Buchner, M. J. & Rushton, S. K. 2009 Adaptation to sensory-motor temporal misalignment: instrumental or perceptual learning? Quart. J. Exp. Psychol. 62, 453-469. (doi:10.1080/17470210801985235)
49. Heron, J., Hanson, J. V. M. & Whitaker, D. 2009 Effect before cause: supramodal recalibration of sensorimotor timing. PLoS ONE 4, e7681. (doi:10.1371/journal.pone.0007681)
50. Kohn, A. 2007 Visual adaptation: physiology, mechanisms, and functional benefits. J. Neurophysiol. 97, 3155-3164. (doi:10.1152/jn.00086.2007)
51. Coltheart, M. 1971 Visual feature-analyzers and aftereffects of tilt and curvature. Psychol. Rev. 78, 114-121. (doi:10.1037/h0030639)
52. Clifford, C. W., Wenderoth, P. & Spehar, B. A. 2000 A functional angle on some after-effects in cortical vision. Proc. R. Soc. Lond. B 267, 1705-1710. (doi:10.1098/ rspb.2000.1198)
53. Wilson, H. R. & Humanski, R. 1993 Spatial frequency adaptation and contrast gain control. Vision Res. 33, 1133-1149. (doi:10.1016/0042-6989(93)90248-U)