Temporal modulations in speech and music

Neuroscience and Biobehavioral Reviews

Volumn 81, Issue , 2017, Pages 181-187

  Ding, Nai ^a,b,d   Patel, Aniruddh D ^c,f   Chen, Lin ^a,d   Butler, Henry ^c   Luo, Cheng ^a   Poeppel, David ^d,e  

^a ZHEJIANG UNIVERSITY   (China)

^b ZHEJIANG UNIVERSITY OF FINANCE AND ECONOMICS   (China)

^c TUFTS UNIVERSITY   (United States)

^d NEW YORK UNIVERSITY   (United States)

^e MAX PLANCK INSTITUTE FOR EMPIRICAL AESTHETICS   (Germany)

^f CANADIAN INSTITUTE FOR ADVANCED RESEARCH   (Canada)

Author keywords

Modulation spectrum; Music; Rhythm; Speech; Temporal modulations

Indexed keywords

CULTURAL DIVERSITY; HUMAN; LANGUAGE; LINGUISTICS; MODULATION; MUSIC; PRIORITY JOURNAL; REVIEW; SOUND INTENSITY; SPECTROSCOPY; SPEECH; SPEECH ANALYSIS; HEARING; PERIODICITY; SOUND DETECTION; SPEECH PERCEPTION; TIME FACTOR;

AUDITORY PERCEPTION; HUMANS; LANGUAGE; MUSIC; PERIODICITY; SOUND SPECTROGRAPHY; SPEECH ACOUSTICS; SPEECH PERCEPTION; TIME FACTORS;

EID: 85013391518     PISSN: 01497634     EISSN: 18737528     Source Type: Journal    
DOI: 10.1016/j.neubiorev.2017.02.011     Document Type: Review

References (69)

1. Barton, B., Venezia, J.H., Saberi, K., Hickok, G., Brewer, A.A., Orthogonal acoustic dimensions define auditory field maps in human cortex. Proc. Natl. Acad. Sci. 109 (2012), 20738–20743.
2. Brown, S., Jordania, J., Universals in the world's musics. Psychol. Music 41 (2013), 229–248.
3. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., Ghazanfar, A.A., The natural statistics of audiovisual speech. PLoS Comput. Biol., 5, 2009, e1000436.
4. Chi, T., Gao, Y., Guyton, M.C., Ru, P., Shamma, S., Spectro-temporal modulation transfer functions and speech intelligibility. J. Acoust. Soc. Am. 106 (1999), 2719–2732.
5. Chi, T., Ru, P., Shamma, S.A., Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 118 (2005), 887–906.
6. Dau, T., Kollmeier, B., Kohlrausch, A., Modeling auditory processing of amplitude modulation. I. Detection and masking with narrow-band carriers. J. Acoust. Soc. Am. 102 (1997), 2892–2905.
7. Dau, T., Kollmeier, B., Kohlrausch, A., Modeling auditory processing of amplitude modulation. II. Spectral and temporal integration. J. Acoust. Soc. Am. 102 (1997), 2906–2919.
8. De Coensel, B., Botteldooren, D., Muer, T.D., 1/f noise in rural and urban soundscapes. Acta Acust.United Acust. 89 (2003), 287–295.
9. Delgutte, B., Hammond, B., Cariani, P., Neural coding of the temporal envelope of speech: relation to modulation transfer functions. Palmer, A., Reese, A., Summerfield, A., Meddis, R., (eds.) Psychophysical and Physiological Advances in Hearing, 1998, Whurr Publishing, London, 595–603.
10. Di Liberto, G.M., O'Sullivan, J.A., Lalor, E.C., Low-Frequency cortical entrainment to speech reflects phoneme-Level processing. Curr. Biol. 25 (2015), 2457–2465.
11. Ding, N., Simon, J.Z., Cortical entrainment to continuous speech: functional roles and interpretations. Front. Hum. Neurosci., 8, 2014, 10.3389/fnhum.2014.00311.
12. Ding, N., Melloni, L., Zhang, H., Tian, X., Poeppel, D., Cortical tracking of hierarchical linguistic structures in connected speech. Nat. Neurosci. 19 (2016), 158–164.
13. Doelling, K.B., Poeppel, D., Cortical entrainment to music and its modulation by expertise. Proc. Natl. Acad. Sci. 112 (2015), E6233–E6242.
14. Drullman, R., Festen, J.M., Plomp, R., Effect of reducing slow temporal modulations on speech reception. J. Acoust. Soc. Am. 95 (1994), 2670–2680.
15. Elliott, T., Theunissen, F., The modulation transfer function for speech intelligibility. PLoS Comput. Biol., 5, 2009.
16. Falk, T.H., Chan, W.-Y., Shein, F., Characterization of atypical vocal source excitation, temporal dynamics and prosody for objective measurement of dysarthric word intelligibility. Speech Commun. 54 (2012), 622–631.
17. Farbood, M.M., Marcus, G., Poeppel, D., Temporal dynamics and the identification of musical key. J. Exp. Psychol. Hum. Percept. Perform. 39 (2013), 911–918.
18. Ghazanfar, A.A., Takahashi, D.Y., Mathur, N., Fitch, W.T., Cineradiography of monkey lip-smacking reveals putative precursors of speech dynamics. Curr. Biol., 22, 2012, 2012.
19. Ghitza, O., The theta-syllable: a unit of speech information defined by cortical function. Front. Psychol., 4, 2013.
20. Ghitza, O., Giraud, A.-L., Poeppel, D., Neuronal oscillations and speech perception: critical-band temporal envelopes are the essence. Front. Hum. Neurosci., 2013, 6.
21. Gill, P., Zhang, J., Woolley, S.M., Fremouw, T., Theunissen, F.E., Sound representation methods for spectro-temporal receptive field estimation. J. Comput. Neurosci. 21 (2006), 5–20.
22. Giraud, A.-L., Poeppel, D., Cortical oscillations and speech processing: emerging computational principles and operations. Nat. Neurosci. 15 (2012), 511–517.
23. Gordon, J.W., The perceptual attack time of musical tones. J. Acoust. Soc. Am. 82 (1987), 88–105.
24. Goswami, U., Leong, V., Speech rhythm and temporal structure: converging perspectives?. Lab. Phonol. 4 (2013), 67–92.
25. Greenberg, S., Carvey, H., Hitchcock, L., Chang, S., Temporal properties of spontaneous speech—a syllable-centric perspective. J. Phon. 31 (2003), 465–485.
26. Greenberg, S., Speaking in shorthand—a syllable-centric perspective for understanding pronunciation variation. Speech Commun. 29 (1999), 159–176.
27. Hämäläinen, J.A., Rupp, A., Soltész, F., Szücs, D., Goswami, U., Reduced phase locking to slow amplitude modulation in adults with dyslexia: an MEG study. Neuroimage 59 (2012), 2952–2961.
28. Henry, M.J., Obleser, J., Frequency modulation entrains slow neural oscillations and optimizes human listening behavior. Proc. Natl. Acad. Sci. 109 (2012), 20095–20100.
29. Houtgast, T., Steeneken, H.J., A review of the MTF concept in room acoustics and its use for estimating speech intelligibility in auditoria. J. Acoust. Soc. Am. 77 (1985), 1069–1077.
30. Joris, P.X., Schreiner, C.E., Rees, A., Neural processing of amplitude-modulated sounds. Physiol. Rev. 84 (2004), 541–577.
31. Jun, S.-A., Prosodic typology. Jun, S.-A., (eds.) Prosodic Typology: The Phonology of Intonation and Phrasing, 2005, Oxford University Press, Oxford, UK, 430–458.
32. Kerlin, J.R., Shahin, A.J., Miller, L.M., Attentional gain control of ongoing cortical speech representations in a cocktail party. J. Neurosci. 30 (2010), 620–628.
33. Kraus, N., Slater, J., Music and language: relations and disconnections. Celesia, G.G., Hickok, G., (eds.) The Human Auditory System, 2015, Fundamental Organization and Clinical Disorders.
34. Krause, J.C., Braida, L.D., Acoustic properties of naturally produced clear speech at normal speaking rates. J. Acoust. Soc. Am. 115 (2004), 362–378.
35. Large, E.W., Palmer, C., Perceiving temporal regularity in music. Cogn. Sci. 26 (2002), 1–37.
36. Levitin, D.J., Chordia, P., Menon, V., Musical rhythm spectra from Bach to Joplin obey a 1/f power law. Proc. Natl. Acad. Sci. 109 (2012), 3716–3720.
37. London, J., Hearing in Time. 2012, Oxford University Press, New York.
38. Luo, H., Poeppel, D., Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron 54 (2007), 1001–1010.
39. McKinney, M.F., Moelants, D., Davies, M.E., Klapuri, A., Evaluation of audio beat tracking and music tempo extraction algorithms. J. New Music Res. 36 (2007), 1–16.
40. Noorden, L.V., Moelants, D., Resonance in the perception of musical pulse. J. New Music Res. 28 (1999), 43–66.
41. Norman-Haignere, S., Kanwisher, N.G., McDermott, J.H., Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron 88 (2015), 1281–1296.
42. Nozaradan, S., Peretz, I., Missal, M., Mouraux, A., Tagging the neuronal entrainment to beat and meter. J. Neurosci. 31 (2011), 10234–10240.
43. Nozaradan, S., Peretz, I., Mouraux, A., Selective neuronal entrainment to the beat and meter embedded in a musical rhythm. J. Neurosci. 32 (2012), 17572–17581.
44. Overath, T., McDermott, J.H., Zarate, J.M., Poeppel, D., The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nat. Neurosci. 18 (2015), 903–911.
45. Patel, A.D., Iversen, J.R., Acoustic and perceptual comparison of speech and drum sounds in the north indian tabla tradition: an empirical study of sound symbolism. Proceedings of the 15th International Congress of Phonetic Sciences, Barcelona, 2003, 925–928.
46. Patel, A.D., Music, Language, and the Brain. 2008, Oxford University Press, New York, NY.
47. Payton, K.L., Braida, L.D., A method to determine the speech transmission index from speech waveforms. J. Acoust. Soc. Am. 106 (1999), 3637–3648.
48. Pellegrino, F., Coupé C., Marsico, E., Across-language perspective on speech information rate. Language 87 (2011), 539–558.
49. Poeppel, D., Idsardi, W.J., Wassenhove, V.v., Speech perception at the interface of neurobiology and linguistics. Phil. Trans. R. Soc. B: Biol. Sci. 363 (2008), 1071–1086.
50. Ruspantini, I., Saarinen, T., Belardinelli, P., Jalava, A., Parviainen, T., Kujala, J., Salmelin, R., Corticomuscular coherence is tuned to the spontaneous rhythmicity of speech at 2–3 Hz. J. Neurosci. 32 (2012), 3786–3790.
51. Santoro, R., Moerel, M., Martino, F.D., Goebel, R., Ugurbil, K., Yacoub, E., Formisano, E., Encoding of natural sounds at multiple spectral and temporal resolutions in the human auditory cortex. PLoS Comput. Biol., 10, 2014, e1003412.
52. Scheirer, E.D., Tempo and beat analysis of acoustic musical signals. J. Acoust. Soc. Am. 103 (1998), 588–601.
53. Schneider, D.M., Woolley, S., Sparse and background-invariant coding of vocalizations in auditory scenes. Neuron 79 (2013), 141–152.
54. Schroeder, C.E., Lakatos, P., Kajikawa, Y., Partan, S., Puce, A., Neuronal oscillations and visual amplification of speech. Trends Cogn. Sci. 12 (2008), 106–113.
55. Shamma, S., On the role of space and time in auditory processing. Trends Cogn. Sci. 5 (2001), 340–348.
56. Shannon, R.V., Zeng, F.-G., Kamath, V., Wygonski, J., Ekelid, M., Speech recognition with primarily temporal cues. Science 270 (1995), 303–304.
57. Singh, N.C., Theunissen, F.E., Modulation spectra of natural sounds and ethological theories of auditory processing. J. Acoust. Soc. Am., 114, 2003.
58. Steinschneider, M., Nourski, K.V., Fishman, Y.I., Representation of speech in human auditory cortex: is it special?. Hear. Res., 2013, 57–73.
59. Stevens, K.N., Toward a model for lexical access based on acoustic landmarks and distinctive features. J. Acoust. Soc. Am. 111 (2002), 1872–1891.
60. Theunissen, F.E., David, S.V., Singh, N.C., Hsu, A., Vinje, W.E., Gallant, J.L., Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Netw. Comput. Neural Syst. 12 (2001), 289–316.
61. Todd, N.P.M., The auditory primal sketch: a multiscale model of rhythmic grouping. J. New Music Res. 23 (1994), 25–70.
62. Turk, A., Shattuck-Hufnagel, S., What is speech rhythm? A commentary on Arvaniti and Rodriquez, Krivokapić and Goswami and Leong. Lab. Phonol. 4 (2013), 93–118.
63. Turner, F., Pöppel, E., The neural lyre: poetic meter, the brain, and time. Poetry 142 (1983), 277–309.
64. Van Gulik, R.H., The Lore of the Chinese Lute. 1969, Sophia University.
65. van der Wel, R.P., Sternad, D., Rosenbaum, D.A., Moving the arm at different rates: slow movements are avoided. J. Mot. Behav. 42:1 (2009), 29–36.
66. Voss, R.F., Clarke, J., 1/f ‘noise’ in music and speech. Nature 258 (1975), 317–318.
67. Yang, X., Wang, K., Shamma, S.A., Auditory representations of acoustic signals. IEEE Trans. Inf. Theory 38 (1992), 824–839.
68. Zeng, F.-G., Nie, K., Liu, S., Stickney, G., Rio, E.D., Kong, Y.-Y., Chen, H., On the dichotomy in auditory perception between temporal envelope and fine structure cues (L). J. Acoust. Soc. Am., 116, 2004, 1351.
69. Zion Golumbic, E.M., Ding, N., Bickel, S., Lakatos, P., Schevon, C.A., McKhann, G.M., Goodman, R.R., Emerson, R., Mehta, A.D., Simon, J.Z., Poeppel, D., Schroeder, C.E., Mechanisms underlying selective neuronal tracking of attended speech at a cocktail party. Neuron 77 (2013), 980–991.