Real time emotion aware applications: A case study employing emotion evocative pictures and neuro-physiological sensing enhanced by Graphic Processor Units

Computer Methods and Programs in Biomedicine

Volumn 107, Issue 1, 2012, Pages 16-27

  Konstantinidis, Evdokimos I ^a   Frantzidis, Christos A ^a   Pappas, Costas ^a   Bamidis, Panagiotis D ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Affective computing; Electroencephalography; Emotional processing; GPU; IAPS; Neuro physiological recordings; Parallel programming

Indexed keywords

AFFECTIVE COMPUTING; AFFECTIVE STATE; C LANGUAGE; CORRELATION DIMENSIONS; DESIGN TECHNIQUE; EXECUTION TIME; GPU; GRAPHIC PROCESSOR UNITS; IAPS; MEMORY MANAGEMENT; MULTI-CHANNEL; NEURO-PHYSIOLOGICAL RECORDINGS; PROGRAMMING ENVIRONMENT; REAL TIME; REAL TIME SENSING; ROBUST SOLUTIONS; SEQUENTIAL EXECUTION; SIGNAL PROCESSING ALGORITHMS; SKIN CONDUCTANCE; TIME MINIMIZATION;

C (PROGRAMMING LANGUAGE); ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; PARALLEL PROGRAMMING; SIGNAL PROCESSING;

COMPUTER GRAPHICS EQUIPMENT;

ADULT; ARTICLE; BRAIN RADIOGRAPHY; CONTROLLED STUDY; DATA ANALYSIS SOFTWARE; ELECTRODERMAL RESPONSE; FEASIBILITY STUDY; FEMALE; GRAPHIC PROCESSOR UNIT; HUMAN; HUMAN EXPERIMENT; MALE; MEMORY CONSOLIDATION; NEUROPHYSIOLOGY; NONLINEAR SYSTEM; NORMAL HUMAN; REAL TIME EMOTION AWARE COMPUTING; SIGNAL PROCESSING; VISUAL STIMULATION;

ALGORITHMS; COMPUTER GRAPHICS; COMPUTER SYSTEMS; ELECTROENCEPHALOGRAPHY; EMOTIONS; GALVANIC SKIN RESPONSE; HUMANS; NERVOUS SYSTEM PHYSIOLOGICAL PROCESSES; NONLINEAR DYNAMICS; SIGNAL PROCESSING, COMPUTER-ASSISTED; SOFTWARE; SOFTWARE DESIGN; USER-COMPUTER INTERFACE;

EID: 84861199627     PISSN: 01692607     EISSN: 18727565     Source Type: Journal    
DOI: 10.1016/j.cmpb.2012.03.008     Document Type: Article

References (41)

1. Berridge K.C., Kringelbach M.L. Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology 2008, 199(3):457-480.
2. Bechara A., Damasio H., Damasio A.R. Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 2000, 10(March (3)):295-307.
3. Sergerie K., Chochol C., Armony J.L. The role of the amygdala in emotional processing: a quantitative meta-analysis of functional neuroimaging studies. Neuroscience and Biobehavioral Reviews 2008, 32(4):811-830.
4. Vuilleumier P., Armony J.L., Driver J., Dolan R.J. Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nature Neuroscience 2003, May (6):624-631.
5. Pantic M., Rothkrantz L.J.M. Toward an affect-sensitive multimodal human-computer interaction. Proceedings of the IEEE 2003, 91(September (9)):1370-1390.
6. Hudlicka E. To feel or not to feel: the role of affect in human-computer interaction. International Journal of Human-Computer Studies 2003, 59(July (1-2)):1-32.
7. Picard R.W., Vyzas E., Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence 2001, 23(October (10)):1175-1191.
8. Picard R.W. Affective computing: challenges. International Journal of Human-Computer Studies 2003, 59(July (1-2)):55-64.
9. Klein J., Moon Y., Picard R.W. This computer responds to user frustration: theory, design, and results. Interacting with Computers 2002, 14(February (2)):119-140.
10. Nasoz F., Lisetti C.L., Alvarez K., Finkelstein N. Emotion recognition from physiological signals for user modeling of affect. 3rd Workshop on Affective and Attitude User Modeling 2003.
11. Kort B., Reilly R. An affective module for an intelligent tutoring system. Lecture Notes in Computer Science, Intelligent Tutoring Systems, vol. 2363/2002 2007, 955-962. Springer, Berlin/Heidelberg.
12. Hudlicka E. Affective computing for game design. Proc. 4th North American Conference on Intelligent Games and Simulation (GAMEON-NA) 2008, 5-12. McGill University, Montreal, Canada.
13. Frantzidis C.A., Bamidis P.D. Description and future trends of ICT solutions offered towards independent living: the case of LLM project. Proc. of the 2nd International Conference on PErvsive Technologies Related to Assistive Environments 2009.
14. Bamidis P.D., Frantzidis C.A., Konstantinidis E.I., Luneski A., Lithari C., Klados M.A., Bratsas C., Papadelis C.L., Pappas C. An integrated approach to emotion recognition for advanced emotional intelligence. Lecture Notes in Computer Science, Affective Dialogue Systems, vol. 3068/2004 2009, 36-48. Springer, Berlin/Heidelberg.
15. Reeves B., Nass C. The Media Equation 1996, Cambridge University Press; Center for the Study of Language and Information.
16. Picard R.W. Affective Computing 1997, MIT Press, Cambridge, MA.
17. Wehrle T., Kaiser S. Emotion and facial expression. Lecture Notes in Computer Science, Affective Interactions, vol. 1814/2000 2006, 49-63. Springer, Berlin/Heidelberg.
18. Kessous L., Castellano G., Caridakis G. Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis. Journal on Multimodal User Interfaces 2010, 3(March):33-48.
19. Castellano G., Aylett R., Paiva A., McOwan P.W. Affect recognition for interactive companions. 1st Workshop on Affective Interaction in Natural Environments (AFFINE), ACM International Conference on Multimodal Interfaces (ICMI'08) 2008.
20. Young J.E., Xin M., Sharlin E. Robot expressionism through cartooning. Proc. of the ACM/IEEE International Conference on Human-robot Interaction 2007.
21. Bamidis P.D., Papadelis C., Kourtidou-Papadeli C., Pappas C., Vivas A.B. Affective computing in the era of contemporary neurophysiology and health informatics. Interacting with Computers 2004, 16(August (4)):715-721.
22. Frantzidis C.A., Lithari C.D., Vivas A.B., Papadelis C.L., Pappas C., Bamidis P.D. Towards emotion aware computing: a study of arousal modulation with multichannel event related potentials, delta oscillatory activity and skin conductivity responses. Presented at the 8th IEEE Int. Conf. on BioInformatics and BioEngineering (BIBE 08) 2008.
23. Bradley M.M., Codispoti M., Cuthbert B.N., Lang P.J. Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion 2001, 1(September (3)):276-298.
24. Lang P.J., Bradley M.M., Cuthbert B.N. Motivated attention: affect, activation and action. Attention and Orienting: Sensory and Motivational Processes 1997, Erlbaum, Hillsdale, NJ. P.J. Lang, R.F. Simons, M.T. Balaban (Eds.).
25. Lang P.J., Bradley M.M., Cuthbert B.N. Emotion, motivation, and anxiety: brain mechanisms and psychophysiology. Biological Psychiatry 1998, 44(December (12)):1248-1263.
26. Lang P.J., Bradley M.M., Cuthbert B.N. International Affective Picture System (IAPS): Technical Manual and Affective Ratings 1997, NIMH Center for the Study of Emotion and Attention.
27. Schupp H.T., Junghofer M., Weike A.I., Hamm A.O. Attention and emotion: an ERP analysis of facilitated emotional stimulus processing. NeuroReport 2003, 14(June (8)):1107-1110.
28. Cuthbert B.N., Schupp H.T., Bradley M.M., Birbaumer N., Lang P.J. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biological Psychology 2000, 52(March (2)):95-111.
29. Amrhein C., Muhlberger A., Pauli P., Wiedermann G. Modulation of event-related brain potentials during affective picture processing: a complement to startle reflex and skin conductance response?. International Journal of Psychophysiology 2004, 54(November (3)):231-240.
30. Lithari C., Frantzidis C.A., Papadelis C., Vivas A.B., Klados M.A., Kourtidou-Papadeli C., Pappas C., Ioannides A.A., Bamidis P.D. Are females more responsive to emotional stimuli? A neurophysiological study across arousal and valence dimensions. Brain Topography 2010, 23(March (1)):27-40.
31. Frantzidis C.A., Bratsas C., Klados M., Konstantinidis E., Lithari C.D., Vivas A.B., Papadelis C., Kaldoudi E., Pappas C., Bamidis P.D. On the classification of emotional biosignals evoked while viewing affective pictures: an integrated data mining based approach for healthcare applications. IEEE Trans. on Information Technology in Biomedicine 2010, 14(March (2)):309-318.
32. C.A. Frantzidis, C. Bratsas, E. Konstantinidis, C. Papadelis, C. Pappas, P.D. Bamidis, Towards emotion aware computing: an integrated approach using multi-channel neuro-physiological recordings & affective visual stimuli, IEEE Transactions on Information Technology in Biomedicine (2010), . doi:10.1109/TITB.2010.2041553.
33. NVIDIA CUDA Programming Guide, Available . http://http%3a//www.nvidia.com/object/cuda_home.html.
34. Barros K., Babich R., Brower R., Clark M.A., Rebbi C. Blasting through lattice calculations using CUDA. PoS LATTICE2008:045 2008.
35. Polychronaki G.E., Ktonas P., Gatzonis S., Asvestas P.A., Spanou E., Siatouni A., Tsekou H., Sakas D., Nikita K.S. Comparison of fractal dimension estimation algorithms for epileptic seizure onset detection. Presented at the 8th IEEE Int. Conf. on BioInformatics and BioEngineering (BIBE 08) 2008.
36. B. Zink, A general relativistic evolution code on CUDA architectures, CCT Technical Report Series, Center for Computation and Technology, Louisiana State University, 2008.
37. Konstantinidis E.I., Frantzidis C.A., Tzimkas L., Pappas C., Bamidis P.D. Accelerating biomedical signal processing algorithms with parallel programming on graphic processor units. Final Program and Abstract Book - 9th International Conference on Information Technology and Applications in Biomedicine 2009, art. no. 5394314.
38. Playne P.D., Hawick K.A. Auto-Generation of Parallel Finite-Differencing Code for MPI, TBB and CUDA, appeared in the Proceedings of the 16th International Workshop on High-Level Parallel Programming Models with Supportive Environments 2011 2011.
39. Kasim H., March V., Zhang R., See S. Survey on parallel programming model, network and parallel computing. Lecture Notes in Computer Science, vol. 5245 2008, 266-275. 208.
40. Franco J., Bernabe G., Fernandez J., Ujaldon M. Parallel 3D fast wavelet transform on manycore GPUs and multicore CPUs. Procedia Computer Science 2010, 1(May (1)):1101-1110.
41. Noaje G., Krajecki M., Jaollet C. MultiGPU computing using MPI or OpenMP, appeared on the IEEE International Conference on Intelligent Computer Communication and Processing (ICCP) 2010, 347-354.