Rhythmicity and cross-modal temporal cues facilitate detection

Neuropsychologia

Volumn 63, Issue 1, 2014, Pages 43-50

  ten Oever, Sanne ^a   Schroeder, Charles E ^b,c   Poeppel, David ^d   van Atteveldt, Nienke ^a,e   Zion Golumbic, Elana ^b,c,f  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b Department of Medicine   (United States)

^c NATHAN S KLINE INSTITUTE FOR PSYCHIATRIC RESEARCH   (United States)

^d NEW YORK UNIVERSITY   (United States)

^e VU UNIVERSITY AMSTERDAM   (Netherlands)

^f BAR ILAN UNIVERSITY   (Israel)

Author keywords

Audiovisual integration; Detection; Rhythmicity; Temporal prediction

Indexed keywords

ADULT; ANTICIPATION; ARTICLE; AUDIOVISUAL INTEGRATION; AUDITORY STIMULATION; AUDITORY THRESHOLD; CROSS MODAL TEMPORAL CUE; FEMALE; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; MALE; NORMAL HUMAN; PERCEPTIVE DISCRIMINATION; PERIODICITY; PREDICTIVE VALUE; PSYCHOPHYSIOLOGY; RESPONSE TIME; RHYTHMICITY; SENSORY STIMULATION; TASK PERFORMANCE; VISUAL STIMULATION; ASSOCIATION; PSYCHOMETRY; SIGNAL NOISE RATIO; VISION; YOUNG ADULT; COGNITION; HEARING; MENTAL FUNCTION; PERCEPTION; PREDICTION; SOUND DETECTION; STIMULUS RESPONSE;

ACOUSTIC STIMULATION; ADULT; AUDITORY THRESHOLD; CUES; FEMALE; HUMANS; MALE; PERIODICITY; PSYCHOMETRICS; SIGNAL-TO-NOISE RATIO; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84906832198     PISSN: 00283932     EISSN: 18733514     Source Type: Journal    
DOI: 10.1016/j.neuropsychologia.2014.08.008     Document Type: Article

References (89)

1. Anstis S., Geier J., Hudak M. Afterimages from unseen stimuli. i-Perception 2012, 3(8):499-502.
2. Bar M. Predictions in the Brain: Using Oour Past to Generate a Future 2011, OUP, USA.
3. Barnes R., Jones M.R. Expectancy, attention, and time. Cognitive Psychology 2000, 41(3):254-311.
4. Beauchamp M.S., Lee K.E., Argall B.D., Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41(5):809-823.
5. Besle J., Schevon C.A., Mehta A.D., Lakatos P., Goodman R.R., McKhann G.M., et al. Tuning of the human neocortex to the temporal dynamics of attended events. The Journal of Neuroscience 2011, 31(9):3176-3185.
6. Capizzi M., Sanabria D., Correa Á. Dissociating controlled from automatic processing in temporal preparation. Cognition 2012, 123(2):293-302.
7. Chandrasekaran C., Trubanova A., Stillittano S., Caplier A., Ghazanfar A.A. The natural statistics of audiovisual speech. PLoS Computational Biology 2009, 5(7):e1000436.
8. Correa A. Enhancing behavioral performance by visual temporal orienting. Attention and Time 2010, 357-370.
9. Correa Á., Lupiáñez J., Milliken B., Tudela P. Endogenous temporal orienting of attention in detection and discrimination tasks. Attention, Perception, & Psychophysics 2004, 66(2):264-278.
10. Correa Á., Nobre A.C. Neural modulation by regularity and passage of time. Journal of Neurophysiology 2008, 100(3):1649-1655.
11. Coull J.T., Nobre A.C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. The Journal of Neuroscience 1998, 18(18):7426-7435.
12. Cravo A.M., Rohenkohl G., Wyart V., Nobre A.C. Endogenous modulation of low frequency oscillations by temporal expectations. Journal of Neurophysiology 2011, 106(6):2964-2972.
13. Cravo A.M., Rohenkohl G., Wyart V., Nobre A.C. Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex. The Journal of Neuroscience 2013, 33(9):4002-4010.
14. Daikhin L., Ahissar M. Responses to deviants are modulated by subthreshold variability of the standard. Psychophysiology 2011, 49(1):31-42.
15. Dubois M., Poeppel D., Pelli D.G. Seeing and hearing a word: combining eye and ear is more efficient than combining the parts of a word. PloS One 2013, 8(5):e64803.
16. Eisler H. Experiments on subjective duration 1868-1975: A collection of power function exponents. Psychological Bulletin 1976, 83(6):1154.
17. Ellis R.J., Jones M.R. Rhythmic context modulates foreperiod effects. Attention, Perception, & Psychophysics 2010, 72(8):2274-2288.
18. Fender D., Julesz B. Extension of Panum's fusional area in binocularly stabilized vision. JOSA 1967, 57(6):819-826.
19. Florentine M., Buus S., Geng W. Psychometric functions for gap detection in a yes-no procedure. The Journal of the Acoustical Society of America 1999, 106:3512.
20. Florentine M., Fastl H., Buus S. Temporal integration in normal hearing, cochlear impairment, and impairment simulated by masking. The Journal of the Acoustical Society of America 1988, 84:195.
21. Friston K. Prediction, perception and agency. International Journal of Psychophysiology 2011, 83(2):248-252.
22. Fujisaki W., Shimojo S., Kashino M., Nishida S.y. Recalibration of audiovisual simultaneity. Nature Neuroscience 2004, 7(7):773-778.
23. Gescheider G. The classical psychophysical methods. Psychophysics:Tthe Fundamentals 1997, (Chapter 3:), L. E. Associates, Hillsdale, NJ, (3rd ed.).
24. Giraud A.L., Poeppel D. Cortical oscillations and speech processing: emerging computational principles and operations. Nature Neuroscience 2012, 15(4):511-517.
25. Grant K.W., Seitz P.F. The use of visible speech cues for improving auditory detection of spoken sentences. The Journal of the Acoustical Society of America 2000, 108:1197.
26. Green D.M. Maximum-likelihood procedures and the inattentive observer. The Journal of the Acoustical Society of America 1995, 97:3749.
27. Green D.M., Swets J.A. Signal detection theory and psychophysics 1966, Vol. 1974. Wiley, New York.
28. Greenberg S., Carvey H., Hitchcock L., Chang S. Temporal properties of spontaneous speech-a syllable-centric perspective. Journal of Phonetics 2003, 31(3):465-485.
29. Griffin I.C., Miniussi C., Nobre A.C. Orienting attention in time. Frontiers in Bioscience 2001, 6:660-671.
30. Jones M.R., Johnston H.M., Puente J. Effects of auditory pattern structure on anticipatory and reactive attending. Cognitive Psychology 2006, 53(1):59-96.
31. Jones M.R., Moynihan H., MacKenzie N., Puente J. Temporal aspects of stimulus-driven attending in dynamic arrays. Psychological Science 2002, 13(4):313-319.
32. Kaernbach C. Simple adaptive testing with the weighted up-down method. Perception, & Psychophysics 1991, 49(3):227-229.
33. Kayser C., Logothetis N.K. Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Frontiers in Integrative Neuroscience 2009, 3.
34. Lakatos P., Chen C.M., O'Connell M.N., Mills A., Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 2007, 53(2):279-292.
35. Lakatos P., Karmos G., Mehta A.D., Ulbert I., Schroeder C.E. Entrainment of neuronal oscillations as a mechanism of attentional selection. Science 2008, 320(5872):110-113.
36. Lakatos P., Musacchia G., O'Connell M., Falchier A., Javitt D., Schroeder C. The spectrotemporal filter mechanism of auditory selective attention. Neuron 2013, 77(4):750-761.
37. Lakatos P., O'Connell M.N., Barczak A., Mills A., Javitt D.C., Schroeder C.E. The leading sense: supramodal control of neurophysiological context by attention. Neuron 2009, 64(3):419-430.
38. Landy M.S., Maloney L.T., Johnston E.B., Young M. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research 1995, 35(3):389-412.
39. Lange K., Röder B. Orienting attention to points in time improves stimulus processing both within and across modalities. Journal of Cognitive Neuroscience 2006, 18(5):715-729.
40. Large E.W., Jones M.R. The dynamics of attending: how people track time-varying events. Psychological Review 1999, 106(1):119.
41. Lindström R., Paavilainen P., Kujala T., Tervaniemi M. Processing of audiovisual associations in the human brain: dependency on expectations and rule complexity. Frontiers in Psychology 2012, 3.
42. Los S.A., Van der Burg E. Sound speeds vision through preparation, not integration. Journal of Experimental Psychology: Human Perception and Performance 2013, 39:1612-1624.
43. Lovelace C.T., Stein B.E., Wallace M.T. An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cognitive Brain Research 2003, 17(2):447-453.
44. Luo H., Liu Z., Poeppel D. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biology 2010, 8(8):e1000445.
45. Luo H., Poeppel D. Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron 2007, 54(6):1001-1010.
46. Lütkenhöner B. Auditory signal detection appears to depend on temporal integration of subthreshold activity in auditory cortex. Brain Research 2011, 1385:206-216.
47. Mathewson K.E., Fabiani M., Gratton G., Beck D.M., Lleras A. Rescuing stimuli from invisibility: Inducing a momentary release from visual masking with pre-target entrainment. Cognition 2010, 115(1):186-191.
48. Miller J., van der Ham F., Sanders A.F. Overlapping stage models and the additive factor method. Acta Psychologica 1995, 90:11-28.
49. Minelli A., Marzi C.A., Girelli M. Lateralized readiness potential elicited by undetected visual stimuli. Experimental Brain Research 2007, 179(4):683-690.
50. Morey R.D. Confidence intervals from normalized data: a correction to Cousineau (2005). Reason 2008, 9(45.4):49.46.
51. Munhall K., Vatikiotis-Bateson E. Spatial and temporal constraints on audiovisual speech perception. The Handbook of Multisensory Processing 2004, 177-188. The MIT Press, Cambridge, Massachusetts. S.C. Calvert, G.A. Stein B. (Eds.).
52. Nachmias J. On the psychometric function for contrast detection. Vision Research 1981, 21(2):215-223.
53. Niemi P., Näätänen R. Foreperiod and simple reaction time. Psychological Bulletin 1981, 89(1):133.
54. Nobre A., Correa A., Coull J. The hazards of time. Current Opinion in Neurobiology 2007, 17(4):465-470.
55. Nobre K., Coull J.T. Attention and time 2010, Oxford University Press, Oxford, UK.
56. Oruç I., Maloney L.T., Landy M.S. Weighted linear cue combination with possibly correlated error. Vision Research 2003, 43(23):2451-2468.
57. Palmer S.E. Vision Science: Photons to Phenomenology 1999, The MIT press, Cambridge, MA.
58. Plank T., Rosengarth K., Song W., Ellermeier W., Greenlee M.W. Neural correlates of audio-visual object recognition: effects of implicit spatial congruency. Human Brain Mapping 2012, 33(4):797-811.
59. Ratlifl F., Comsweet T., Ross J., Holt J., Johnstone J., Ratliff F., et al. Hysteresis in the perception of motion direction as evidence for neural cooperativity. Nature 1986, 324:20.
60. Rohenkohl G., Cravo A.M., Wyart V., Nobre A.C. Temporal expectation improves the quality of sensory information. The Journal of Neuroscience 2012, 32(24):8424-8428.
61. Schroeder C.E., Lakatos P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends in Neurosciences 2009, 32(1):9-18.
62. Schroeder C.E., Lakatos P., Kajikawa Y., Partan S., Puce A. Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 2008, 12(3):106-113.
63. Schroeder C.E., Wilson D.A., Radman T., Scharfman H., Lakatos P. Dynamics of active sensing and perceptual selection. Current Opinion in Neurobiology 2010, 20(2):172-176.
64. Schubotz R.I. Prediction of external events with our motor system: towards a new framework. Trends in Cognitive Sciences 2007, 11(5):211-218.
65. Schwartz J.L., Berthommier F., Savariaux C. Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 2004, 93(2):B69-B78.
66. Sternberg S. Separate modifiability, mental modules, and the use of pure and composite measures to reveal them. Acta Psychologica 2001, 106:147-246.
67. Summerfield C., Egner T. Expectation (and attention) in visual cognition. Trends in Cognitive Sciences 2009, 13(9):403-409.
68. Summerfield C., Egner T., Greene M., Koechlin E., Mangels J., Hirsch J. Predictive codes for forthcoming perception in the frontal cortex. Science 2006, 314(5803):1311-1314.
69. Ten Oever S., Sack A., Wheat K.L., Bien N., Van Atteveldt N. Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs. Frontiers in Psychology 2013, 4.
70. Thorne J.D., De Vos M., Viola F.C., Debener S. Cross-modal phase reset predicts auditory task performance in humans. The Journal of Neuroscience 2011, 31(10):3853-3861.
71. Thorne J.D., Debener S. Irrelevant visual stimuli improve auditory task performance. NeuroReport 2008, 19(5):553.
72. Treutwein B. Adaptive psychophysical procedures. Vision Research 1995, 35(17):2503-2522.
73. Trommershauser J., Kording K., Landy M.S. Sensory Cue Integration 2011, Oxford University Press, New York.
74. Van Atteveldt N., Formisano E., Blomert L., Goebel R. The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cerebral Cortex 2007, 17(4):962-974.
75. Van Atteveldt N., Formisano E., Goebel R., Blomert L. Integration of letters and speech sounds in the human brain. Neuron 2004, 43(2):271-282.
76. Van Atteveldt N., Murray M.M., Thut G., Schroeder C.E. Multisensory integration: flexible use of general operations. Neuron 2014, 81(6):1240-1253.
77. Van Atteveldt, N., Musacchia, G., Sehatpour, P., Zion Golumbic, E. M., Lakatos, P., Gaspar, P., et al. (2011). A combined EEG-fMRI investigation of "rhythmic" versus "vigilant" processing II: functional neuro-anatomy and slow dynamics, In Proceedings of the International Conference on Cognitive Neuroscience (ICON XI), Mallorca, Spain.
78. Van Wassenhove V., Grant K.W., Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 2005, 102(4):1181.
79. Van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 2007, 45(3):598-607.
80. Vroomen J., Keetels M., de Gelder B., Bertelson P. Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cognitive Brain Research 2004, 22(1):32-35.
81. Wallace M., Wilkinson L., Stein B. Representation and integration of multiple sensory inputs in primate superior colliculus. Journal of Neurophysiology 1996, 76(2):1246-1266.
82. Welch R.B., Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 1980, 88(3):638.
83. Woodman G.F., Kang M.-S., Thompson K., Schall J.D. The effect of visual search efficiency on response preparation. Psychological Science 2008, 19:128-136.
84. Yrttiaho S., Tiitinen H., Alku P., Miettinen I., May P.J. Temporal integration of vowel periodicity in the auditory cortex. The Journal of the Acoustical Society of America 2010, 128(1):224-234.
85. Yuille A.L., Bülthoff H.H. Bayesian decision theory and psychophysics. Perception as Bayesian inference 1996, 123-161. Cambridge University Press. D.C. Knill, W. Richards (Eds.).
86. Zampini M., Shore D.I., Spence C. Audiovisual temporal order judgments. Experimental Brain Research 2003, 152(2):198-210.
87. Zchaluk K., Foster D.H. Model-free estimation of the psychometric function. Attention, Perception, & Psychophysics 2009, 71(6):1414-1425.
88. Zion Golumbic E.M., Cogan G.B., Schroeder C.E., Poeppel D. Visual input enhances selective speech envelope tracking in auditory cortex at a "Cocktail Party". The Journal of Neuroscience 2013, 33(4):1417-1426.
89. Zion Golumbic, E. M., Poeppel, D., & Schroeder, C. E. (2012). Temporal context in speech processing and attentional stream selection: a behavioral and neural perspective. Brain and Language.