Linking brain responses to naturalistic music through analysis of ongoing EEG and stimulus features

IEEE Transactions on Multimedia

Volumn 15, Issue 5, 2013, Pages 1060-1069

  Cong, Fengyu ^a   Alluri, Vinoo ^a   Nandi, Asoke K ^a,b   Toiviainen, Petri ^a   Fa, Rui ^b   Abu Jamous, Basel ^b   Gong, Liyun ^c   Craenen, Bart G W ^a,b   Poikonen, Hanna ^d   Huotilainen, Minna ^d,e   Ristaniemi, Tapani ^a  

^a UNIVERSITY OF JYVÄSKYLÄ   (Finland)

^b BRUNEL UNIVERSITY   (United Kingdom)

^c UNIVERSITY OF LIVERPOOL   (United Kingdom)

^d UNIVERSITY OF HELSINKI   (Finland)

^e FINNISH INSTITUTE OF OCCUPATIONAL HEALTH   (Finland)

Author keywords

Acoustical features; Clustering; EEG; Independent component analysis; Natural continuous music; Ongoing; Oscillation

Indexed keywords

ACOUSTICAL FEATURES; CLUSTERING; NATURAL CONTINUOUS MUSIC; ONGOING; OSCILLATION;

ELECTROENCEPHALOGRAPHY; INDEPENDENT COMPONENT ANALYSIS; NEUROIMAGING;

CURRICULA;

EID: 84880822422     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2013.2253452     Document Type: Article

References (54)

1. V. Alluri, P. Toiviainen, I. P. Jaaskelainen, E. Glerean, M. Sams, and E. Brattico, "Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm," Neuroimage, vol. 59, pp. 3677-3689, 2012.
2. U. Hasson, Y. Nir, I. Levy, G. Fuhrmann, and R. Malach, "Intersubject synchronization of cortical activity during natural vision," Science, vol. 303, pp. 1634-1640, 2004. (Pubitemid 38338315)
3. J. D. Haynes and G. Rees, "Decoding mental states from brain activity in humans," Nat. Rev. Neurosci., vol. 7, pp. 523-534, 2006. (Pubitemid 43989773)
4. J. P. Kauppi, I. P. Jaaskelainen, M. Sams, and J. Tohka, "Inter-subject correlation of brain hemodynamic responses during watching a movie: localization in space and frequency," Front. Neuroinform, vol. 4, 2010.
5. K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant, "Identifying natural images from human brain activity," Nature, vol. 452, pp. 352-355, 2008. (Pubitemid 351430789)
6. H. J. Spiers and E. A. Maguire, "Decoding human brain activity during real-world experiences," Trends Cogn. Sci., vol. 11, pp. 356-365, 2007. (Pubitemid 47077911)
7. A. Hyvarinen, P. Ramkumar, L. Parkkonen, and R. Hari, "Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis," Neuroimage, vol. 49, pp. 257-271, 2010.
8. M. Koskinen, J. Viinikanoja, M. Kurimo, A. Klami, S. Kaski, and R. Hari, "Identifying fragments of natural speech from the listener'sMEG signals," Hum. Brain Mapp., 2012.
9. H. Pan, J. Epstein, D. A. Silbersweig, and E. Stern, "New and emerging imaging techniques for mapping brain circuitry," Brain Res. Rev., vol. 67, pp. 226-251, 2011.
10. H. Berger, "Ueber das elektrenkephalogramm des menschen," Archives Fur Psychiatrie Nervenkrankheiten, vol. 87, pp. 527-570, 1929.
11. E. Niedermeyer and F. d. S. Lopes, Electroencephalography : Basic Principles, Clinical Applications, and Related Fields. Baltimore, MD, USA: Williams & Wilkins, 2004.
12. H. Davis, P. A. Davis, A. L. Loomis, E. N. Harvey, and G. Hobart, "Electrical reactions of the human brain to auditory stimulation during sleep," J. Neurophysiol., vol. 2, pp. 500-514, 1939.
13. S. J. Luck, An Introduction to the Event-Related Potential Technique. Cambridge, MA, USA: The MIT Press, 2005.
14. Z. Chen, J. Cao, Y. Cao, Y. Zhang, F. Gu, G. Zhu, Z. Hong, B. Wang, and A. Cichocki, "An empirical EEG analysis in brain death diagnosis for adults," Cogn. Neurodyn., vol. 2, pp. 257-271, 2008.
15. D. S. Tan and A. Nijholt, Brain-Computer Interfaces: Applying Our Minds to Human-Computer Interaction. London, U.K.: Springer, 2010.
16. S. Debener, F. Minow, R. Emkes, K. Gandras, and M. de Vos, "How about taking a low-cost, small, and wireless EEG for a walk?," Psychophysiology, vol. 49, pp. 1617-1621, 2012.
17. S. Makeig, K. Gramann, T. P. Jung, T. J. Sejnowski, and H. Poizner, "Linking brain, mind and behavior," Int. J. Psychophysiol., vol. 73, pp. 95-100, 2009.
18. S. Sanei and J. A. Chambers, EEG Signal Processing. New York, NY, USA: Wiley, 2007.
19. G. Y. Görsev and E. Basar, "Sensory evoked and event related oscillations in Alzheimer's disease: a short review," Cogn. Neurodyn., vol. 4, pp. 263-274, 2010.
20. E. Basar, M. Schurmann, T. Demiralp, C. Basar-Eroglu, and A. Ademoglu, "Event-related oscillations are 'real brain responses'-wavelet analysis and new strategies," Int. J. Psychophysiol., vol. 39, pp. 91-127, 2001. (Pubitemid 32042117)
21. A. Delorme and S. Makeig, "EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis," J. Neurosci. Meth., vol. 134, pp. 9-21, 2004. (Pubitemid 38224183)
22. G. Buzsáki, Rhythms of the Brain. Oxford, U.K.: Oxford Univ. Press, 2006.
23. D. Sammler, M. Grigutsch, T. Fritz, and S. Koelsch, "Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music," Psychophysiology, vol. 44, pp. 293-304, 2007. (Pubitemid 46354084)
24. R. S. Schaefer, R. J. Vlek, and P. Desain, "Music perception and imagery in EEG: alpha band effects of task and stimulus," Int. J. Psychophysiol., vol. 82, pp. 254-259, 2011.
25. A. J. Shahin, L. J. Trainor, L. E. Roberts, K. C. Backer, and L. M. Miller, "Development of auditory phase-locked activity for music sounds," J. Neurophysiol., vol. 103, pp. 218-229, 2010.
26. S. Makeig, T. P. Jung, A. J. Bell, D. Ghahremani, and T. J. Sejnowski, "Blind separation of auditory event-related brain responses into independent components," Proc. Nat. Acad. Sci. USA, vol. 94, pp. 10979-10984, 1997. (Pubitemid 27430846)
27. S. Makeig, M. Westerfield, T. P. Jung, J. Covington, J. Townsend, T. J. Sejnowski, and E. Courchesne, "Functionally independent components of the late positive event-related potential during visual spatial attention," J. Neurosci., vol. 19, pp. 2665-2680, 1999. (Pubitemid 29162415)
28. F. Cong, Y. Huang, I. Kalyakin, H. Li, T. Huttunen-Scott, H. Lyytinen, and T. Ristaniemi, "Frequency response based wavelet decomposition to extract children's mismatch negativity elicited by uninterrupted sound," J. Med. Biol. Eng., vol. 32, pp. 205-214, 2012.
29. F. Cong, P. H. Leppanen, P. Astikainen, J. Hämäläinen, J. K. Hietanen, and T. Ristaniemi, "Dimension reduction: Additional benefit of an optimal filter for independent component analysis to extract event-related potentials," J. Neurosci. Meth., vol. 201, pp. 269-280, 2011.
30. R. Vigario and E. Oja, "BSS and ICA in neuroinformatics: From current practices to open challenges," IEEE Rev. Biomed. Eng., vol. 1, pp. 50-61, 2008.
31. F. Cong, I. Kalyakin, H. Li, T. Huttunen-Scott, Y. X. Huang, H. Lyytinen, and T. Ristaniemi, "Answering six questions in extracting children's mismatch negativity through combining wavelet decomposition and independent component analysis," Cogn. Neurodyn., vol. 5, pp. 343-359, 2011.
32. N. P. Castellanos and V. A. Makarov, "Recovering EEG brain signals: artifact suppression with wavelet enhanced independent component analysis," J. Neurosci. Meth., vol. 158, pp. 300-312, 2006. (Pubitemid 44738074)
33. O. Lartillot and P. Toiviainen, "MIR in matlab (II): a toolbox for musical feature extraction from audio," in Proc. Int. Conf. Music Information Retrieval 2007, 2007, pp. 237-244.
34. F. Cong, I. Kalyakin, and T. Ristaniemi, "Can back-projection fully resolve polarity indeterminacy of ICA in study of ERP?," Biomed. Signal Process. Control, vol. 6, pp. 422-426, 2011.
35. F. Cong, I. Kalyakin, C. Zheng, and T. Ristaniemi, "Analysis on subtracting projection of extracted independent components from EEG recordings," Biomed. Tech., vol. 56, pp. 223-234, 2011.
36. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York, NY, USA: Wiley, 2001.
37. J. Onton, M. Westerfield, J. Townsend, and S. Makeig, "Imaging human EEG dynamics using independent component analysis," Neurosci. Biobehav. Rev., vol. 30, pp. 808-822, 2006. (Pubitemid 44292287)
38. M. de SVos, J. D. Thorne, G. Yovel, and S. Debener, "Let's face it, from trial to trial: Comparing procedures for N170 single-trial estimation," Neuroimage, vol. 63, pp. 1196-1202, 2012.
39. Y. Wang, Y. T. Wang, and T.P Jung, "Translation of EEG spatial filters from resting tomotor imagery using independent component analysis," PLoS One, vol. 7, p. E37665, 2012.
40. A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. Chichester, U.K.: Wiley, 2003.
41. F. Cong, I. Kalyakin, T. Huttunen-Scott, H. Li, H. Lyytinen, and T. Ristaniemi, "Single-trial based independent component analysis on mismatch negativity in children," Int. J. Neural Syst., vol. 20, pp. 279-292, 2010.
42. J. Himberg, A. Hyvärinen, and F. Esposito, "Validating the independent components of neuroimaging time series via clustering and visualization," Neuroimage, vol. 22, pp. 1214-1222, 2004. (Pubitemid 38829210)
43. A. J. Bell and T. J. Sejnowski, "An information maximisation approach to blind separation and blind deconvolution," Neural Computat., vol. 7, p. 1159, 1995.
44. A. Delorme, J. Palmer, J. Onton, R. Oostenveld, and S. Makeig, "Independent EEG sources are dipolar," PLoS One, vol. 7, p. E30135, 2012.
45. C. M. Bishop, Pattern Recognition and Machine Learning. Singapore: Springer, 2006.
46. L. Kaufman and P. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis. San Francisco, CA, USA: Wiley, 2005.
47. J. Rissanen, "Modeling by the shortest data description," Automatica, vol. 14, pp. 465-471, 1978.
48. D. M. Groppe, T. P. Urbach, and M. Kutas, "Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review," Psychophysiology, vol. 48, pp. 1711-1725, 2011.
49. D. A. Abrams, T. Nicol, S. Zecker, and N. Kraus, "Right-hemisphere auditory cortex is dominant for coding syllable patterns in speech," J. Neurosci., vol. 28, pp. 3958-3965, Apr. 2008.
50. S. J. Aiken and T. W. Picton, "Envelope and spectral frequency-following responses to vowel sounds," Hear. Res., vol. 245, pp. 35-47, 2008.
51. F. Cong, Z. He, J. Hämäläinen, P. H. T. Leppänen, H. Lyytinen, A. Cichocki, and T. Ristaniemi, "Validating rationale of group-level component analysis based on estimating number of sources in EEG through model order selection," J. Neurosci. Meth., vol. 212, pp. 165-172, 2013.
52. W. Gunther, R. Steinberg, P. Streck, J. P. Banquet, I. Bscheid, L. Raith, R. Riedel, U. Klages, and I. Stiltz, "Brain dysfunction in psychiatric patients during music perception measured by EEG mapping: relation to motor dysfunction and influence of neuroleptic drugs," Eur. Neuropsychopharmacol. , vol. 1, pp. 143-155, 1991.
53. M. H. Ruiz, S. Koelsch, and J. Bhattacharya, "Decrease in early right alpha band phase synchronization and late gamma band oscillations in processing syntax in music," Hum. Brain Mapp., vol. 30, pp. 1207-1225, 2009.
54. S. Elmer, M. Meyer, and L. Jancke, "The spatiotemporal characteristics of elementary audiovisual speech and music processing in musically untrained subjects," Int. J. Psychophysiol., 2011.