Seeing the talker’s face improves free recall of speech for young adults with normal hearing but not older adults with hearing loss

Journal of Speech, Language, and Hearing Research

Volumn 59, Issue 3, 2016, Pages 590-599

  Rudner, Mary ^a   Mishra, Sushmit ^a   Stenfelt, Stefan ^a   Lunner, Thomas ^a,b   Rönnberg, Jerker ^a  

^a LINKÖPING UNIVERSITY   (Sweden)

^b OTICON A S   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; AGED; AGING; ANALYSIS OF VARIANCE; AUDITORY STIMULATION; FACIAL RECOGNITION; FEMALE; HEARING IMPAIRMENT; HEARING TEST; HUMAN; MALE; MIDDLE AGED; NEUROPSYCHOLOGICAL TEST; NOISE; PSYCHOLOGY; READING; RECALL; SHORT TERM MEMORY; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; YOUNG ADULT;

ACOUSTIC STIMULATION; ADULT; AGED; AGING; ANALYSIS OF VARIANCE; FACIAL RECOGNITION; FEMALE; HEARING LOSS; HEARING TESTS; HUMANS; MALE; MEMORY, SHORT-TERM; MENTAL RECALL; MIDDLE AGED; NEUROPSYCHOLOGICAL TESTS; NOISE; READING; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; YOUNG ADULT;

EID: 84976285493     PISSN: 10924388     EISSN: None     Source Type: Journal    
DOI: 10.1044/2015_JSLHR-H-15-0014     Document Type: Article

References (69)

1. Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 839–843.
2. Andersson, U., & Lyxell, B. (1998). Phonological deterioration in adults with an acquired severe hearing impairment. Scandinavian Audiology, 49(Suppl.), 93–100.
3. Baddeley, A. (2012). Working memory, theories models and controversy. The Annual Review of Psychology, 63, 12.1–12.29.
4. Besle, J., Fort, A., Delpuech, C., & Giard, M. H. (2004). Bimodal speech: Early suppressive visual effects in human auditory cortex. European Journal of Neuroscience, 20(8), 2225–2234. doi:10.1111/j.1460-9568.2004.03670.x
5. Besser, J., Koelewijn, T., Zekveld, A. A., Kramer, S. E., & Festen, J. M. (2013). How linguistic closure and verbal working memory relate to speech recognition in noise—A review. Trends in Amplification, 17(2), 75–93. doi:10.1177/1084713813495459
6. Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound. Cambridge, MA: MIT Press.
7. Buchan, J. N., & Munhall, K. G. (2012). The effect of a concurrent working memory task and temporal offsets on the integration of auditory and visual speech information. Seeing and Perceiving, 25(1), 87–106. doi:10.1163/187847611X620937
8. Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport, 10, 2619–2623.
9. Campanella, S., & Belin, P. (2007). Integrating face and voice in person perception. Trends in Cognitive Sciences, 11, 535–543. doi:10.1016/j.tics.2007.10.001
10. Classon, E., Rudner, M., & Rönnberg, J. (2013). Working memory compensates for hearing related phonological processing deficit. Journal of Communication Disorders, 46, 17–29. doi:10.1016/ j.jcomdis.2012.10.001
11. Costa, A., Strijkers, K., Martin, C., & Thierry, G.(2009)The time course of word retrieval revealed by event-related brain potentials during overt speechProceedings of the National Academy of Sciences, USA106(50)214422144610.1073/ pnas.0908921106
12. Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104, 268–294. doi:10.1037/0096-3445.104.3.268
13. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behaviour, 19, 450–466.
14. Ezzatian, P., Li, L., Pichora-Fuller, K., & Schneider, B. A. (2015). Delayed stream segregation in older adults: More than just informational masking. Ear and Hearing, 36, 482–484.
15. Fernández, L. M., Visser, M., Ventura-Campos, N., Ávila, C., & Soto-Faraco, S. (2015). Top-down attention regulates the neural expression of audiovisual integration. NeuroImage, 119, 272–285.
16. Foo, C., Rudner, M., Rönnberg, J., & Lunner, T. (2007). Recognition of speech in noise with new hearing instrument compression release settings requires explicit cognitive storage and processing capacity. Journal of the American Academy of Audiology, 18, 618–631.
17. Forster, K. I., & Forster, J. C. (2003). DMDX: A windows display program with millisecond accuracy. Behavior Research Methods, Instruments, & Computers, 35, 116–124. doi:10.3758/ BF03195503
18. Frtusova, J. B., Winneke, A. H., & Phillips, N. A. (2013). ERP evidence that auditory–visual speech facilitates working memory in younger and older adults. Psychology and Aging, 28, 481–494. doi:10.1037/a0031243
19. Gosselin, P. A., & Gagné, J. P. (2011). Older adults expend more listening effort than young adults recognizing audiovisual speech in noise. International Journal of Audiology, 50, 786–792. doi:10.3109/14992027.2011.599870
20. Grant, K. W., Walden, B. E., & Seitz, P. F. (1998). Auditory– visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory–visual integration. Journal of the Acoustical Society of America, 103, 2677–2690. doi:10.1121/1.422788
21. Grimault, N., Micheyl, C., Carlyon, R. P., Arthaud, P., & Collet, L. (2001). Perceptual auditory stream segregation of sequences of complex sounds in subjects with normal and impaired hearing. British Journal of Audiology, 35, 173–182.
22. Grimm, G., Herzke, T., Berg, D., & Hohmann, V. (2006). The Master Hearing Aid: A PC-based platform for algorithm development and evaluation. Acta Acustica united with Acustica, 92, 618–628.
23. Hasher, L., & Zacks, R. T. (1988). Working memory, comprehension, and aging: A review and a new view. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 22, pp. 193–225). San Diego, CA: Academic Press.
24. Holube, I., Fredelake, S., Vlaming, M., & Kollmeier, B. (2010). Development and analysis of an International Speech Test Signal (ISTS). International Journal of Audiology, 49, 891–903. doi:10.3109/14992027.2010.506889
25. Johansson, M. S. K., & Arlinger, S. (2003). Prevalence of hearing impairment in population in Sweden. International Journal of Audiology, 42(1), 18–28. doi:10.3109/14992020309056081
26. Keidser, G., Best, V., Freeston, K., & Boyce, A. (2015). Cognitive spare capacity: Evaluation data and its association with comprehension of dynamic conversations. Frontiers in Psychology, 6(597). doi:10.3389/fpsyg.2015.00597
27. Lazard, D. S., Lee, H. J., Gaebler, M., Kell, C. A., Truy, E., & Giraud, A. L. (2010). Phonological processing in post-lingual deafness and cochlear implant outcome. NeuroImage, 49, 3443–3451.
28. Levitt, H. (1971). Transformed up-down methods in psychoacoustics. The Journal of the Acoustical Society of America, 49, 467–477. doi:10.1121/1.1912375
29. Lin, F. R., Ferrucci, L., An, Y., Goh, J. O., Doshi, J., Metter, E. J.,Resnick, S. M. (2014). Association of hearing impairment with brain volume changes in older adults. NeuroImage, 90, 84–92. doi:10.1016/j.neuroimage.2013.12.059
30. Lin, F. R., Thorpe, R., Gordon-Salant, S., & Ferrucci, L. (2011). Hearing loss prevalence and risk factors among older adults in the United States. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences, 66(5), 582–590. doi:10.1093/gerona/glr002
31. Lin, F. R., Yaffe, K., Xia, J., Xue, Q. L., Harris, T. B., Purchase-Helzner, E., Simonsick, E. M. (2013). Hearing loss and cognitive decline in older adults. JAMA Internal Medicine, 173(4), 293–299. doi:10.1001/jamainternmed.2013.1868
32. Lunner, T. (2003). Cognitive function in relation to hearing aid use. International Journal of Audiology, 42(Suppl. 1), S49–S58.
33. McCoy, S. L., Tun, P. A., Cox, L. C., Colangelo, M., Stewart, R. A., & Wingfield, A. (2005). Hearing loss and perceptual effort: Downstream effects on older adults’ memory for speech. Quarterly Journal of Experimental Psychology, 58(1), 22–33.
34. Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M. (2013a). Seeing the talker’s face supports executive processing of speech in steady state noise. Frontiers in Systems Neuroscience, 7(96). doi:10.3389/fnsys.2013.00096
35. Mishra, S., Lunner, T., Stenfelt, S., Rönnberg, J., & Rudner, M.(2013b)Visual information can hinder working memory processing of speechJournal of Speech, Language, and Hearing Research561120113210.1044/1092-4388(2012/12-0033)
36. Mishra, S., Stenfelt, S., Lunner, T., Rönnberg, J., & Rudner, M. (2014). Cognitive spare capacity in older adults with hearing loss. Frontiers in Aging Neuroscience, 6(96). doi:10.3389/ fnagi.2014.00096
37. Moore, B. C., & Glasberg, B. R. (1998). Use of a loudness model for hearing-aid fitting: I. Linear hearing aids. British Journal of Audiology, 32, 317–335.
38. Moradi, S., Lidestam, B., & Rönnberg, R. (2013). Gated audiovisual speech identification in silence versus noise: Effect on time and accuracy. Frontiers in Psychology, 4(359). doi:10.3389/ fpsyg.2013.00359
39. Murdock, B. B. (1974). Human memory: Theory and data. Mahwah, NY: Erlbaum.
40. Murphy, D. R., Craik, F. I. M., Li, K. Z. H., & Schneider, B. A. (2000). Comparing the effects of aging and background noise on short-term memory performance. Psychology and Aging, 15, 323–334. doi:10.1037/0882-7974.15.2.323
41. Ng, E. H., Rudner, M., Lunner, T., Pedersen, M. S., & Rönnberg, J. (2013). Effects of noise and working memory capacity on memory processing of speech for hearing aid users. Intenational Journal of Audiology, 52, 433–441. doi:10.3109/14992027.2013
42. Peelle, J. E., & Sommers, M. S. (2015). Prediction and constraint in audiovisual speech perception. Cortex, 68, 169–181. doi:10.1016/ j.cortex.2015.03.006
43. Peelle, J. E., Troiani, V., Grossman, M., & Wingfield, A. (2011). Hearing loss in older adults affects neural systems supporting speech comprehension. Journal of Neuroscience, 31, 12638–12643. doi:10.1523/JNEUROSCI.2559-11.2011
44. Pichora-Fuller, M. K, Schneider, B. A., & Daneman, M. (1995). How young and old adults listen to and remember speech in noise. The Journal of the Acoustical Society of America, 97, 593–608.
45. Picou, E. M., & Ricketts, T. A. (2014). The effect of changing the secondary task in dual-task paradigms for measuring listening effort. Ear and Hearing, 35, 611–622. doi:10.1097/ AUD.0000000000000055
46. Picou, E. M., Ricketts, T. A., & Hornsby, B. W. (2013). How hearing aids, background noise, and visual cues influence objective listening effort. Ear and Hearing, 34, e52–e64.
47. Piolino, P., Coste, C., Martinelli, P., Macé, A. L., Quinette, P., Guillery-Girard, B., & Belleville, S. (2010). Reduced specificity of autobiographical memory and aging: Do the executive and feature binding functions of working memory have a role? Neuropsychologia, 48, 429–440. doi:10.1016/j.neuropsychologia. 2009.09.035
48. Rabbitt, P. (1990). Mild hearing loss can cause apparent memory failures which increase with age and reduce with IQ. Acta Otolaryngologica, 476(Suppl.), 167–175.
49. Rönnberg, J. (1995). Perceptual compensation in the deaf and blind: Myth or reality. In R. A. Dixon & L. Bäckman (Eds.), Compensating for psychological deficits and declines: Managing losses and promoting gains. 251–274. Mahwah, NJ: Erlbaum.
50. Rönnberg, J., Arlinger, S., Lyxell, B., & Kinnefors, C. (1989). Visual evoked potentials: Relation to adult speechreading and cognitive function. Journal of Speech, Language, and Hearing Research, 32, 725–735.
51. Rönnberg, J., Danielsson, H., Rudner, M., Arlinger, S., Sternäng, O., Wahlin, Å., & Nilsson, L.-G. (2011). Hearing loss is negatively related to episodic and semantic long-term memory but not to short-term memory. Journal of Speech, Language, and Hearing Research, 54, 705–726. doi:10.1044/1092-4388(2010/09-0088)
52. Rönnberg, J., Hygge, S., Keidser, G., & Rudner, M. (2014). The effect of functional hearing loss and age on long- and short-term visuospatial memory: Evidence from the UK Biobank resource. Frontiers in Aging Neuroscience, 6(326). doi:10.3389/ fnagi.2014.00326
53. Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B.. (2013). The ease of language understanding (ELU) model: Theoretical, empirical, and clinical advances. Frontiers in Systems Neuroscience. doi:10.3389/ fnsys.2013.00031
54. Rose, N. S., Buchsbaum, B. R., & Craik, F. I. (2014). Short-term retention of a single word relies on retrieval from long-term memory when both rehearsal and refreshing are disrupted. Memory and Cognition, 42, 689–700. doi:10.3758/s13421-014-0398-x
55. Rudner, M., Fransson, P., Ingvar, M., Nyberg, L., & Rönnberg, J. (2007). Neural representation of binding lexical signs and words in the episodic buffer of working memory. Neuropsychologia, 45, 2258–2276.
56. Rudner, M., & Lunner, T. (2014). Cognitive spare capacity and speech communication: A narrative overview. BioMed Research International. doi:10.1155/ 2014/869726
57. Rudner, M., & Rönnberg, J. (2008). The role of the episodic buffer in working memory for language processing. Cognitive Processing, 9, 19–28. doi:10.1007/s10339-007-0183-x
58. Rudner, M., Rönnberg, J., & Lunner, T. (2011). Working memory supports listening in noise for persons with hearing impairment. Journal of the American Academy of Audiology, 22, 156–167. doi:10.3766/jaaa.22.3.4
59. Schweinberger, S. R, Kloth, N., & Robertson, D. M. (2011). Hearing facial identities: Brain correlates of face—voice integration in person identification. Cortex, 47, 1026–1037. doi:10.1016/ j.cortex.2010.11.011
60. Sommers, M. S., Tye-Murray, N., & Spehar, B. (2005). Auditory– visual speech perception and auditory–visual enhancement in normal-hearing younger and older adults. Ear and Hearing, 26(3), 263–275. doi:10.1097/00003446-200506000-00003
61. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.
62. Tye-Murray, N., Spehar, B., Myerson, J., Sommers, M., & Hale, S. (2011). Crossmodal enhancement of speech detection in young and older adults: Does signal content matter? Ear and Hearing, 32, 650–655. doi:10.1097/aud.0b013e31821a4578
63. van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, USA, 102, 1181–1186. doi:10.1073/pnas.0408949102
64. Weatherly, S. L. (2002). Back to basics: Testing visual acuity with the Jaeger eye chart. Material Evaluation, 60, 928–929.
65. Wild, C. J., Yusuf, A., Wilson, D. E., Peelle, J. E., Davis, M. H., & Johnsrude, I. S. (2012). Effortful listening: The processing of degraded speech depends critically on attention. The Journal of Neuroscience, 32, 14010–14021. doi:10.1523/JNEUROSCI. 1528-12.2012
66. Wingfield, A., Amichetti, N. M., & Lash, A. (2015). Cognitive aging and hearing acuity: Modeling spoken language comprehension. Frontiers in Psychology, 6(684). doi:10.3389/fpsyg.2015.00684
67. Winneke, A. H., & Phillips, N. A. (2011). Does audiovisual speech offer a fountain of youth for old ears? An event-related brain potential study of age differences in audiovisual speech perception. Psychology and Aging, 26, 427–438. doi:10.1037/a0021683
68. Wiswede, D., Rüsseler, J., & Münte, T. F. (2007). Serial position effects in free memory recall–An ERP-study. Biological Psychology, 75, 185–193. doi:10.1016/j.biopsycho.2007.02.002
69. Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, S., Schevon, C. A., McKhann, G. M.,Schroeder, C. E. (2013). Mechanism underlying selective neuronal tracking of attended speech at a “cocktail party.” Neuron, 77, 980–991. doi:10.1016/j.neuron. 2012.12.037