Visual information can hinder working memory processing of speech

Journal of Speech, Language, and Hearing Research

Volumn 56, Issue 4, 2013, Pages 1120-1132

  Mishra, Sushmit ^a   Lunner, Thomas ^a,b   Stenfelt, Stefan ^a   Rönnberg, Jerker ^a   Rudner, Mary ^a  

^a LINKÖPING UNIVERSITY   (Sweden)

^b OTICON A S   (Denmark)

Author keywords

Auditory processing; Cognitive spare capacity; Executive functions; Free recall; Speech understanding; Working memory

Indexed keywords

ADULT; ASSOCIATION; CLINICAL TRIAL; COGNITION; COMPREHENSION; EXECUTIVE FUNCTION; FEMALE; HUMAN; MALE; MIDDLE AGED; NEUROPSYCHOLOGICAL TEST; PERCEPTION; PHOTOSTIMULATION; PROCEDURES; SHORT TERM MEMORY; SPEECH; SPEECH PERCEPTION; STANDARDS; VALIDATION STUDY; VISION; YOUNG ADULT; ARTICLE; AUDITORY PROCESSING; COGNITIVE SPARE CAPACITY; EXECUTIVE FUNCTIONS; METHODOLOGY; RECALL; SPEECH UNDERSTANDING; STANDARD; WORKING MEMORY;

ADULT; COGNITION; COMPREHENSION; CUES; EXECUTIVE FUNCTION; FEMALE; HUMANS; MALE; MEMORY, SHORT-TERM; MIDDLE AGED; NEUROPSYCHOLOGICAL TESTS; PERCEPTUAL MASKING; PHOTIC STIMULATION; SPEECH; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

AUDITORY PROCESSING; COGNITIVE SPARE CAPACITY; EXECUTIVE FUNCTIONS; FREE RECALL; SPEECH UNDERSTANDING; WORKING MEMORY;

EID: 84879735444     PISSN: 10924388     EISSN: None     Source Type: Journal    
DOI: 10.1044/1092-4388(2012/12-0033)     Document Type: Article

References (62)

1. Akeroyd, M. A. (2008). Are individual differences in speech perception related to individual differences in cognitive ability? A survey of twenty experimental studies with normal and hearing impaired adults. International Journal of Audiology, 47, 125-143. doi:10.1080/14992020802301142
2. Allen, R. J., Baddeley, A. D., & Hitch, G. J. (2006). Is the binding of visual features in working memory resource-demanding? Journal of Experimental Psychology: General, 135, 298-313. doi:10.1037/ 0096-3445.135.2.298
3. Andersson, U., Lyxell, B., Rönnberg, J., & Spens, K. E. (2001). Cognitive correlates of visual speech understanding in hearing impaired individuals. Journal of Deaf Studies and Deaf Education, 6, 103-116. doi:10.1093/deafed/6.2.103
4. Baddeley, A. (1996). Exploring the central executive. The Quarterly Journal of Experimental Psychology, 49, 5-28.
5. Baddeley, A. (1998). The central executive: A concept and some misconception. Journal of International Neuropsychological Society, 4, 523-526.
6. Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4, 417-423. doi:10.1016/j.bbr.2011.03.031
7. Baddeley, A. (2012). Working memory: Theories, models, and controversies. The Annual Review of Psychology, 63, 1-29. doi:10.1146/annurev-psych-120710-100422
8. Baddeley, A., & Hitch, G. (1974). Working memory. In G. A. Bower (Ed.), The psychology of learning and motivation (pp. 47-89). London, England: Academic Press.
9. Bernstein, J. G., & Grant, K. W. (2009). Auditory and auditory-visual intelligibility of speech in fluctuating maskers for normal-hearing and hearing-impaired listeners. The Journal of the Acoustical Society of America, 125, 3358-3372. doi:10.1121/1.3110132
10. Besser, J., Zekveld, A. A., Kramer, S. E., Rönnberg, J., & Festen, J. M. (2012). New measures of masked text recognition in relation to speech-in-noise perception and their associations with age and cognitive abilities. Journal of Speech,Language,and Hearing Research, 55, 194-209. doi:10.1044/1092-4388(2011/11-0008)
11. Braver, T. S., Cohen, J. D., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D. C. (1997). A parametric study of prefrontal cortex involvement in human working memory. NeuroImage, 5, 49-62. doi:10.1006/nimg.1996.0247
12. Conway, A. R. A., Cowan, N. & Bunting, M. F. (2001). The cocktail party phenomenon revisited: The importance of working memory capacity. Psychonomic Bulletin & Review, 8, 331-335. doi:10.1234/12345678
13. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450-466.
14. Erber, N. P. (1969). Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech,Language,and Hearing Research, 12, 423-425.
15. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. doi:10.3758/BRM.41.4.1149
16. Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London, England: Sage.
17. Field, A. (2009). Discovering statistics using SPSS (2nd ed.). London, England: Sage.
18. Forster, K. I., & Forster, J. C. (2003). DMDX: A Windows display program with millisecond accuracy. Behavior Research Methods, Instruments,&Computers, 35, 116-124.
19. Fraser, S., Gagné, J.-P., Alepins, M., & Dubois, P. (2010). Evaluating the effort expended to understand speech in noise using a dual-task paradigm: The effects of providing visual speech-cues. Journal of Speech, Language, and Hearing Research, 53, 18-33. doi:10.1044/1092-4388(2009/08-0140)
20. Glanzer, M., & Cunitz, A. R. (1966). Two storage mechanisms in free recall. Journal of Verbal Learning and Verbal Behavior, 5, 351-360.
21. Grant, K. W., & Seitz, P. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. The Journal of the Acoustical Society of America, 108, 1197-1208. doi:10.1121/ 1.1288668
22. Grant, K. W., Walden, B. E., & Seitz, P. F. (1998). Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration. The Journal of the Acoustical Society of America, 103, 2677-2690. doi:10.1121/1.422788
23. Hällgren, M., Larsby, B., & Arlinger, S. (2006). A Swedish version of the Hearing in Noise Test (HINT) for measurement of speech recognition. International Journal of Audiology, 45, 227-237. doi:10.1080/14992020500429583
24. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice Hall.
25. Kubo-Kawai, N., & Kawai, N. (2010). Elimination of the enhanced Simon effect for older adults in a three-choice situation: Ageing and the Simon effect in a go/no-go Simon task. The Quarterly Journal of Experimental Psychology, 63, 452-464. doi:10.1080/ 17470210902990829
26. Lavie, N. (2005). Distracted and confused? Selective attention under load. Trends in Cognitive Sciences, 9, 75-82. doi:10.1016/j.tics. 2004.12.004
27. Mattys, S. L., Davis, M. H., Bradlow, A. R., & Scott, S. K. (2012). Speech recognition in adverse conditions: A review. Language and Cognitive Processes, 27, 953-978. doi:10.1080/01690965. 2012.705006
28. Mishra, S., Rudner, M., Lunner, T., Stenfelt, S., & Rönnberg, J. (2010). Speech understanding and cognitive spare capacity. In J. M. Buchholz, T. Dau, J. Christensen-Dalsgaard, & T. Poulsen (Eds.), Binaural processing and spatial hearing (pp. 305-313). Elsinore, Denmark: ISAAR.
29. Miyake, A., Friedman, N. P., Emerson, M. J., Witziki, A. H., Howerter, A., & Wager, T. (2000). The unity and diversity of executive functions and their contribution to complex frontal lobe tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100. doi:10.1006/cogp.1999.0734
30. Morris, N., & Jones, D. M. (1990). Memory updating in working memory: The role of the central executive. British Journal of Psychology, 81, 111-121.
31. Murdock, B. B. (1974). Human memory: Theory and data. Potomac, MD: Erlbaum.
32. Ng, E.H.N., Rudner, M., Lunner, T., Pedersen, M.S.,& Rönnberg, J. (2013). Effects of noise and working memory capacity on memory processing of speech for hearing aid users. International Journal of Audiology, 52, 433-441.
33. Nilsson, M., Soli, S. D., & Sullivan, J. A. (1994). Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. The Journal of the Acoustical Society of America, 95, 1085-1099.
34. Nyberg, L., Brocki, K., Tillman, C., & Bohlin, G. (2008). The proposed interaction between working memory and inhibition. European Journal of Cognitive Psychology, 21, 84-111. doi:10.1080/ 09541440701862133
35. Pichora-Fuller, M. K. (2007). Audition and cognition: What audiologists need to know about listening. In C. Palmer & R. Seewald (Eds.), Hearing care for adults (pp. 71-85). Stäfa, Switzerland: Phonak.
36. Pichora-Fuller, M. K. (2008). Use of supportive context by younger and older adults listeners: Balancing bottom-up and top-down information processing. International Journal of Audiology, 47, S144-S154. doi:10.1080/14992020802307404
37. Prabhakaran, V., Narayanan, K., Zhao, Z., & Gabrielli, J.D.E. (2000). Integration of diverse information in working memory in the frontal lobe. Nature Neuroscience, 3, 85-90. doi:10.1038/71156
38. Pratte, M. S., Rouder, J. N., Morey, R. D., & Feng, C. (2010). Exploring the differences in distributional properties between Stroop and Simon effects using delta plots. Attention, Perception, and Psychophysics, 72, 2013-2025. doi:10.3758/APP.72.7.2013
39. Repovs, G., & Baddeley, A. (2006). The multi-component model of working memory: Explorations in experimental cognitive psychology. Neuroscience, 139, 5-21. doi:10.1016/j.bbr.2011. 03.031
40. Rönnberg, J. (1990). Cognitive and communicative function: The effects of chronological age and "handicap age." European Journal of Cognitive Psychology, 2, 253-273.
41. Rönnberg, J. (2003). Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: A framework and a model. International Journal of Audiology, 42, S68-S76.
42. Rönnberg, J., Rudner, M., & Foo, C. (2010). The cognitive neuroscience of signed language: Applications to a working memory system for sign and speech. In L. Bäckman & L. Nyberg (Eds.), Memory, aging and the brain: A Festschrift in honour of Lars-Göran Nilsson (pp. 265-286). London, England: Psychology Press.
43. Rönnberg, J., Rudner, M., Foo, C, & Lunner, T. (2008). Cognition counts: A working memory system for ease of language understanding (ELU). International Journal of Audiology, 47, S99-S105. doi: 10.1080/14992020802301167
44. Rönnberg, J., Rudner, M., Lunner, T., & Zekveld, A. A. (2010). When cognition kicks in: Working memory and speech understanding in noise. Noise & Health, 12, 263-269. doi:10.4103/ 1463-1741.70505
45. Rönnberg, N., Stenfelt, S., Rudner, M., & Lunner, T. (2012). Testing listening effort for speech comprehension. In J. M. Buchholz, T. Dau, J. Christensen-Dalsgaard, & T. Poulsen (Eds.), Speech perception andauditory disorders (pp. 73-80). Elsinore, Denmark: ISAAR.
46. Rudner, M., Rönnberg, J., & Lunner, T. (2011). Working memory supports listening in noise for persons with hearing impairment. Journal of the American Academy of Audiology, 22, 156-167. doi:10.3766/jaaa.22.3.4
47. Rudner, M., Lunner, T., Behrens, T., Thorén, E. S., & Rönnberg, J. (2012). Working memory capacity may influence perceived effort during aided speech recognition in noise. Journal of the American Academy of Audiology, 23, 577-589. doi:10.3766/jaaa.23.7.7
48. Rudner, M., Ng, E. H., Rönnberg, N., Mishra, S., Rönnberg, J., Lunner, T., & Stenfelt, S. (2011). Cognitive spare capacity as measure of listening effort. Journal of Hearing Science, 1, 1-3.
49. Rudner, M., & Rönnberg, J. (2004). Perceptual saliency in the visual channel enhances explicit language processing. Iranian Audiology, 3, 16-26.
50. Sarampalis, A., Kalluri, S., Edwards, B., & Hafter, E. (2009). Objective measures of listening effort: Effects of background noise and noise reduction. Journal of Speech, Language, and Hearing Research, 52, 1230-1240. doi:10.1044/1092-4388(2009/ 08-0111)
51. Simon, J. R. (1969). Reactions towards the source of stimulation. Journal of Experimental Psychology, 81, 174-176.
52. Sörqvist, P., Ljungberg, J. K., & Ljung, R. (2010). A sub-process view of working memory capacity: Evidence from effects of speech on prose memory. Memory, 18, 310-326. doi:10.1080/ 09658211003601530
53. St Clair-Thompson, H. L., & Gathercole, S. E. (2006). Executive functions and achievements in school: Shifting, updating, inhibition, and working memory. The Quarterly Journal of Experimental Psychology, 59, 745-759. doi:10.1080/ 17470210500162854
54. Studebaker, G. A. (1985). A "rationalized" arcsine transform. Journal of Speech, Language, and Hearing Research, 28, 455-462.
55. Tye-Murray, N., Sommers, M., Spehar, B., Myerson, J., & Hale, S. (2010). Aging, audiovisual integration, and principle of inverse effectiveness. Ear and Hearing, 31, 636-644. doi:10.1097/ AUD.0b013e3181ddf7ff
56. Weatherly, S. L. (2002). Testing visual acuity with the Jaeger eye chart. Material Evaluation, 60, 928-929.
57. Wiswede, D., Rüsseler, J., & Münte, T. F. (2007). Serial position effects in free memory recall: An ERP study. Biological Psychology, 75, 185-193. doi:10.1016/j.biopsycho.2007.02.002
58. Yumba, W. (2010). Executive function and working memory inyoung adults (Unpublished master's thesis). Department of Behavioural Sciences and Learning, Linköping University, Sweden.
59. Zekveld, A. A., George, E. L. J., Kramer, S. E., Goverts, S. T., & Houtgast, T. (2007). The development of the Text Reception Threshold test: A visual analogue of the Speech Reception Threshold test. Journal of Speech, Language, and Hearing Research, 50, 576-584. doi:10.1044/1092-4388(2007/040)
60. Zekveld, A. A., Heslenfeld, D. J., Festen, J. M., & Schoonhoven, R. (2006). Top-down and bottom-up process in speech comprehension. Neuroimage, 32, 1826-1836. doi:10.1016/j.neuroimage.2006.04.199
61. Zekveld, A. A., Rudner, M., Johnsrude, I. S., Festen, J. M., Van Beek, H. M., & Rönnberg, J. (2011). The influence of semantically related and unrelated text cues on the intelligibility of sentences in noise. Ear and Hearing, 32, 16-25. doi:10.1097/AUD. 0b013e318228036a
62. Zekveld, A. A., Rudner, M., Johnsrude, I. S., Heslenfeld, D. J., & Rönnberg, J. (2012). Behavioral and fMRI evidence that cognitive ability modulates the effect of semantic context on speech intelligibility. Brain and Language, 122, 103-113. doi:10.1016/ j.bandl.2012.05.006