Context Sensitivity of EEG-Based Workload Classification under Different Affective Valence

IEEE Transactions on Affective Computing

Volumn 11, Issue 2, 2020, Pages 327-334

  Grissmann, Sebastian ^a   Spuler, Martin ^a   Faller, Josef ^b   Krumpe, Tanja ^a   Zander, Thorsten O ^c   Kelava, Augustin ^a   Scharinger, Christian ^d   Gerjets, Peter ^d  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b Columbia University ^*  (United States)

^c TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^d KNOWLEDGE MEDIA RESEARCH CENTER   (Germany)

Author keywords

affective valence (VAL); Brain computer interface (BCI); electroencephalography (EEG); emoback; working memory load (WML)

Indexed keywords

ADAPTIVE SYSTEMS; CLASSIFICATION (OF INFORMATION); DATA INTEGRATION; DATABASE SYSTEMS; ELECTRODES; ELECTROENCEPHALOGRAPHY; ELECTRONIC MAIL; ELECTROPHYSIOLOGY; INTERFACE STATES;

AFFECTIVE VALENCE (VAL); BRAIN COMPUTER INTERFACES (BCIS); CLASSIFICATION ACCURACY; CONTEXT INDEPENDENT; CONTEXT SENSITIVITY; EMOBACK; REAL-LIFE APPLICATIONS; WORKING MEMORY;

BRAIN COMPUTER INTERFACE;

EID: 85035803285     PISSN: None     EISSN: 19493045     Source Type: Journal    
DOI: 10.1109/TAFFC.2017.2775616     Document Type: Article

References (56)

1. J. Frey, C. Mühl, F. Lotte, and M. Hachet, "Review of the use of electroencephalography as an evaluation method for humancomputer interaction," in Proc. Int. Conf. Physiological Comput. Syst., 2014, pp. 214-23.
2. C. Walter, S. Schmidt, W. Rosenstiel, M. Bogdan, and P. Gerjets, "Alpha-and Theta-frequencies as indicators for optimal cognitive load during learning," in Proc. 6th Int. Cognitive Load Theory Conf., 2013. p. 3-4.
3. J. Reissland, and T. O. Zander, "Automated detection of bluffing in a game-Revealing a complex covert user state with a passive BCI, in Proc. Human Factors Conf. 2009, 2010, pp. 435-443.
4. M. Spüler, C. Walter, W. Rosenstiel, P. Gerjets, K. Moeller, and E. Klein, "EEG-based prediction of cognitive workload induced by arithmetic: A step towards online adaptation in numerical learning," Zdm Int. J. Math. Educ., vol. 48, pp. 267-78, 2016.
5. T. D Smedt and L. Menschaert, "VALENCE: Affective visualisation using EEG," Digit. Creat., 2012, pp. 37-41.
6. P. Gerjets, C. Walter, W. Rosenstiel, M. Bogdan, and T. O. Zander, "Cognitive state monitoring and the design of adaptive instruction in digital environments: Lessons learned from cognitive workload assessment using a passive brain-computer interface approach," Front. Neurosci., vol. 8, pp. 1-21, 2014.
7. S. H. Fairclough, "Fundamentals of physiological computing," Interact Comput. Elsevier B.V., vol. 21, no. 1/2, pp. 133-45, Jan. 2009.
8. T. O. Zander and S. Jatzev, "Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment," J. Neural. Eng., vol. 9, no. 1, Feb. 2012, Art. no. 16003.
9. A. Myrden and T. Chau, "Effects of user mental state on EEG-BCI performance," Front. Hum. Neurosci., vol. 9, pp. 1-11, 2015.
10. C. Berka, et al., "EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks," Aviat Space Environ Med., pp. 78, no. 5 Suppl, May 2007, pp. B231-B244.
11. J. Kohlmorgen, et al., "Improving human performance in a real operating environment through real-time mental workload detection," in Toward Brain-Computer Interfacing, G. Dornhege, J. del R. Millan, T. Hinterberger, D. J. McFarland, K.-R. Müller, Ed. Cambridge, MA, USA: MIT Press, 2007. pp. 409-422.
12. A. Baddeley, "Working memory: Looking back and looking forward," Nat. Rev. Neurosci., vol. 4, no. 10, pp. 829-39, 2003.
13. A. Baddeley, "Working memory: Theories, models, and controversies," Annu. Rev. Psychol., vol. 63, no. 1, pp. 1-29, 2012.
14. C. Scharinger, A. Soutschek, T. Schubert, and P. Gerjets, "When flanker meets the n-back: What EEG and pupil dilation data reveal about the interplay between the two central-executive working memory functions inhibition and updating," Psychophysiology, pp. 52, no. 10, pp. 1293-304, 2015.
15. T. Klingberg, The Overflowing Brain Information Overload and the Limits of Working Memory. Oxford, U.K.: Oxford University Press, 2009, Art. no. 202.
16. W. Klimesch, "EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis," Brain Res. Rev., vol. 29, no. 2/3, pp. 169-95, Apr. 1999.
17. M. E. Smith and A. Gevins, "Neurophysiologic monitoring of mental workload and fatigue during operation of a flight simulator," in Defense Secur., vol. 5797, pp. 116-126, 2005.
18. A. Gevins, et al., "Towards measurement of brain function in operational environments," Biol. Psychol., vol. 40, no. 1/2, pp. 169-186, May 1995.
19. A.-M. Brouwer, M. Hogervorst, J. van Erp, T. Heffelaar, P. H. Zimmerman, and R. Oostenveld, "Estimating workload using EEG spectral power and ERPs in the n-back task," J. Neural Eng., vol. 9, 2012, Art. no. 45008.
20. R. N. Roy, S. Bonnet, S. Charbonnier, and A. Campagne, "Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI," in Proc. 35th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 2013, pp. 6607-6610.
21. J. A. Russell, "A circumplex model of affect," J. Pers. Soc. Psychol., vol. 39, pp. 1161-1178, 1980.
22. C. Mühl, C. Jeunet, and F. Lotte, "EEG-based workload estimation across affective contexts," Front. Neurosci., vol. 8, Jan. 2014, Art. no. 114.
23. W. K. Kirchner, "Age differences in short-term retention of rapidly changing information," J. Exp. Psychol., vol. 55, pp. 352-358, 1958.
24. A. Gerrards-Hesse, K. Spies, and F. W. Hesse, "Experimental inductions of emotional states and their effectiveness: A review," Br. J. Psychol., vol. 85, pp. 55-78, 1994.
25. P. J. Lang, M. Bradley, and B. Cuthbert, "International affective picture system (IAPS): Technical manual and affective ratings," NIMH Cent. Study Emot. Atten., pp. 39-58, 1997.
26. M. Bradley and P. Lang, "Measuring emotion: The self-assessment manikin and the semantic differential," J. Behav. Ther. Exp. Psychiatry., vol. 25, no. I, pp. 49-59, 1994.
27. S. Alguacil, P. Tudela, and M. Ruz, "Cognitive and affective control in a flanker word task: Common and dissociable brain mechanisms," Neuropsychologia, vol. 51, no. 9, pp. 1663-1672, 2013.
28. A. MacNamara, J. Ferri, and G. Hajcak, "Working memory load reduces the late positive potential and this effect is attenuated with increasing anxiety," Cogn. Affect. Behav. Neurosci., vol. 11, no. 3, pp. 321-331, 2011.
29. H. T. Schupp, B. N. Cuthbert, M. M. Bradley, J. T. Cacioppo, T. Ito, and P. J. Lang, "Affective picture processing: The late positive potential is modulated by motivational relevance," Psychophysiology, vol. 37, no. 2, pp. 257-261, Mar. 2000.
30. J. Kopf, T. Dresler, P. Reicherts, M. J. Herrmann, and A. Reif, "The effect of emotional content on brain activation and the late positive potential in a word n-back task," PLoS One, vol. 8, no. 9, pp. 1-11, 2013.
31. S. Grissmann, J. Faller, C. Scharinger, M. Spüler, and P. Gerjets, "Electroencephalography based analysis of working memory load and affective valence in an N-back task with emotional stimuli," Front. Hum. Neurosci., in press.
32. P. J. Lang, M. K. Greenwald, M. M. Bradley, and O. Hamm, "Looking at pictures: Affective, facial, visceral, and behavioral reactions," Psychophysiology, vol. 30, no. 3, pp. 261-273, May 1993.
33. G. L. Ahern and G. E. Schwartz, "Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis," Neuropsychologia., vol. 23, no. 6, pp. 745-755, 1985.
34. F. Raimondo, J. E. Kamienkowski, M. Sigman, and D. Fernandez Slezak, "CUDAICA: GPU optimization of Infomax-ICA EEG analysis," Comput. Intell. Neurosci., vol. 2012, Jan. 2012, Art. no. 206972.
35. A. J. Bell and T. J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Comput., vol. 7, no. 6, pp. 1129-1159, Nov. 1995.
36. A. Mognon, J. Jovicich, L. Bruzzone, and M. Buiatti, "ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features," Psychophysiology, vol. 48, no. 2, pp. 229-240, 2011.
37. A. Delorme, et al., "EEGLAB, SIFT, NFT, BCILAB, and ERICA: New tools for advanced EEG processing," Comput. Intell. Neurosci., vol. 2011, 2011, Art. no. 130714.
38. J. K. Olofsson, S. Nordin, H. Sequeira, and J. Polich, "Affective picture processing: An integrative review of ERP findings," Biol. Psychol., vol. 77, no. 3, pp. 247-265, Mar. 2008.
39. A. M. Leventhal, R. L. Martin, R. W. Seals, E. Tapia, and L. P. Rehm, "Investigating the dynamics of affect: Psychological mechanisms of affective habituation to pleasurable stimuli," Motiv. Emot., vol. 31, no. 2, pp. 145-157, 2007.
40. H. Ayaz, M. Izzetoglu, S. Bunce, T. Heiman-Patterson, and B. Onaral, "Detecting cognitive activity related hemodynamic signal for brain computer interface using functional near infrared spectroscopy," in Proc. 3rd Int. IEEE/EMBS Conf. Neural Eng., 2007, pp. 342-345.
41. M. B. Priestly, Spectral Analysis and Time Series, vol. 1. London, U.K.: Academic, 1982, pp. 422-423.
42. H. Sheikh, D. J. McFarland, W. A. Sarnacki, and J. R. Wolpaw, "Electroencephalographic (EEG)-based communication: EEG control versus system performance in humans," Neurosci. Lett., vol. 345, no. 2, pp. 89-92, 2003.
43. V. Vapnik, The Nature of Statistical Learning Theory. Berlin, Germany: Springer, 1999.
44. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., 2011, vol. 2, no. 3, pp. 1-27, 2003.
45. F. Lotte and M. Congedo, "A review of classification algorithms for EEG-based brain-computer interfaces," J. Neural Eng., vol. 4, no. 2, pp. R1-R13, Jun. 2007.
46. E. Combrisson and K. Jerbi, "Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy," J. Neurosci. Methods, vol. 250, pp. 126-136, Jul. 30, 2015.
47. C. Kirschbaum, K. M. Pirke, and D. H. Hellhammer, "The "Trier Social Stress Test"-a tool for investigating psychobiological stress responses in a laboratory setting," Neuropsychobiology, vol. 28, pp. 76-81, 1993.
48. A. Kübler, N. Neumann, J. Kaiser, B. Kotchoubey, T. Hinterberger, andN. Birbaumer, "Brain-computer communication: Self-regulation of slow cortical potentials for verbal communication," Arch. Phys. Med. Rehabil., vol. 82, no. 11, pp. 1533-1539,Nov. 2001.
49. N. G.Mathieu, S. Bonnet, S. Harquel, E. Gentaz, and A. Campagne, "Single-trial ERP classification of emotional processing," in Proc. IEEE-EMBS Conf. Neural Eng. NER., 2013. pp. 101-104.
50. R. N. Roy, S. Bonnet, S. Charbonnier, and A. Campagne, "Efficient workload classification based on ignored auditory probes: A proof of concept," Front. Hum. Neurosci., 2016, Art. no. 10.
51. C. Mühl, A. Brouwer, N. Wouwe, E. L. Broek, F. Nijboer, and D. K. J. Heylen, "Modality-specific affective responses and their implications for affective BCI," in Proc. 5th Int. Brain-Comput. Interface Conf., 2011, pp. 4-7.
52. R. Oostenveld, P. Fries, E. Maris, and J.-M. Schoffelen, "FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data," Comput. Intell. Neurosci., vol. 2011, Jan. 2011, Art. no. 156869.
53. T.-P. Jung, S. Makeig, M. J. McKeown, A. J. Bell, T.-W. Lee, and T. J. Sejnowski, "Imaging brain dynamics using independent component analysis," IEEE Inst. Electr. Electron. Eng., vol. 89, no. 7, pp. 1107-1122, Jul. 1, 2001.
54. S. Makeig, A. J. Bell., T.-P. Jung, and T. J. Sejnowski, "Independent component analysis of electroencephalographic data," Adv. Neural Inf. Process. Syst., vol. 8, pp. 145-151, 1996.
55. S. Makeig, G. Leslie, and T. Mullen, "First demonstration of a musical emotion BCI," in Proc. Int. Conf. Affect Comput., 2011, pp. 487-496.
56. C. Kothe and S. Makeig, "Estimation of task workload from EEG data: new and current tools and perspectives," in Proc. 33rd Annu. Int. Conf. IEEE Eng. Med. Biology Soc., 2011, pp. 6547-6551.