Using unlabeled data to improve classification of emotional states in human computer interaction

Journal on Multimodal User Interfaces

Volumn 8, Issue 1, 2014, Pages 5-16

  Schels, Martin ^a   Kächele, Markus ^a   Glodek, Michael ^a   Hrabal, David ^a   Walter, Steffen ^a   Schwenker, Friedhelm ^a  

^a UNIVERSITY OF ULM   (Germany)

Author keywords

Affective computing; Clustering; Partially supervised learning

Indexed keywords

CLASSIFICATION (OF INFORMATION);

AFFECTIVE COMPUTING; CLUSTERING; DENSITY ESTIMATION; EMOTION CLASSIFICATION; PARTIALLY SUPERVISED LEARNING; PHYSIOLOGICAL MEASUREMENT; STATISTICAL CLASSIFIER; SUPERVISED CLASSIFIERS;

HUMAN COMPUTER INTERACTION;

EID: 84902366569     PISSN: 17837677     EISSN: 17838738     Source Type: Journal    
DOI: 10.1007/s12193-013-0133-0     Document Type: Article

References (57)

1. Baluja S (1999) Probabilistic modeling for face orientation discrimination: learning from labeled and unlabeled data. In: Proceedings of the 1998 conference on advances in neural information processing systems II (NIPS), pp 854-860.
2. Baumert JH, Frey AW, Adt M (1995) Analysis of heart rate variability. Background, method, and possible use in anesthesia. Der Anaesth 44(10): 677-686.
3. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings conference on computational learning theory, pp 92-100.
4. Boiten FA, Frijda NH, Wientjes CJ (1994) Emotions and respiratory patterns: review and critical analysis. Int J Psychophysiol 17(2): 103-128.
5. Breiman L (1996) Bagging predictors. Mach Learn 24: 123-140.
6. Bruns T, Praun N (2002) Biofeedback: Ein Handbuch für die therapeutische Praxis. Vandenhoeck & Ruprecht, Germany.
7. Burkhardt F, Paeschke A, Rolfes M, Sendlmeier W, Weiss B (2005) A database of German emotional speech. In: Proceeding of interspeech, pp 1517-1520.
8. Christensen H, Fuglsang-Frederiksen A (1986) Power spectrum and turns analysis of EMG at different voluntary efforts in normal subjects. Electroencephalogr Clin Neurophysiol 64(6): 528-535.
9. Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4: 129-145.
10. Cooper DB, Freeman JH (1970) On the asymptotic improvement in the out-come of supervised learning provided by additional nonsupervised learning. IEEE Trans Comput 19(11): 1055-1063.
11. Cover TM (1965) Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans Electron 14(3): 326-334.
12. Deng J, Schuller B (2012) Confidence measures in speech emotion recognition based on semi-supervised learning. In: proceeding of the INTERSPEECH 2012, pp 2226-2229.
13. Eckmann JP, Kamphorst SO, Ruelle D (1987) Recurrence plots of dynamical systems. Europhys Lett 4(9): 973.
14. Ekman P, Friesen WV (1977) Facial action coding system. Consulting Psychologists Press, USA.
15. Esparza J, Scherer S, Schwenker F (2012) Studying self- and active-training methods for multi-feature set emotion recognition. In: Proceeding of the partially supervised learning (PSL 11), LNCS. Springer, New York, pp 19-31.
16. Fahrenberg J, Foerster F (1982) Covariation and consistency of activation parameters. Biol Psychol 15(3-4): 151-169.
17. Haynes JDD, Rees G (2006) Decoding mental states from brain activity in humans. Nature reviews. Neuroscience 7(7): 523-534.
18. Joachims T (2006) Transductive support vector machines. In: Semi-supervised learning. The MIT Press, London, pp 105-117.
19. Kanade T, Cohn J, Tian Y (2000) Comprehensive database for facial expression analysis. In: IEEE international conference on automatic face and gesture recognition, pp 46-53.
20. Karmakar C, Khandoker A, Voss A, Palaniswami M (2011) Sensitivity of temporal heart rate variability in poincar plot to changes in parasympathetic nervous system activity. BioMed Eng Online 10: 1-14.
21. Kelley JF (1983) An empirical methodology for writing user-friendly natural language computer applications. In: SIGCHI conference on human factors in computing systems, pp 193-196.
22. Kestler HA, Schwenker F, Hoher M, Palm G (1995) Adaptive class-specific partitioning as a means of initializing RBF-networks. IEEE Int Conf Syst Man Cybern 1: 46-49.
23. Kierkels JJM, Soleymani M, Pun T (2009) Queries and tags in affect-based multimedia retrieval. In: IEEE international conference on multimedia and expo, ICME '09, pp 1436-1439.
24. Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal 12(12): 2067-2083.
25. Krikler DM (1990) The QRS complex. Ann N Y Acad Sci 601(1): 24-30.
26. Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2): 181-207.
27. Kuppens P, van Mechelen I, Nezlek JB, Dossche D, Timmermans T (2007) Individual differences in core affect variability and their relationship to personality and psychological adjustment. Emotion 7(2): 262-274.
28. Lang PJ, Greenwald MK, Bradley MM, Hamm AO (1993) Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30(3): 261-273.
29. Lisetti CL, Nasoz F (2004) Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP J Appl Signal Process 2004(4): 1672-1687.
30. Malik M (1996) Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation 93(5): 1043-1065.
31. Mehrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol 14(4): 261-292.
32. Mewett D, Reynolds K, Nazeran H (1999) Recurrence plot features of ECG signals. In: Conference on engineering in medicine and biology and 1999 annual fall meeting of the Biomedical Engineering Society, vol 2, p 193.
33. Picard RW (2000) Affective computing. MIT Press, Cambridge.
34. Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10): 1175-1191.
35. Pincus SM (1991) Approximate entropy as a measure of system complexity. Natl Acad Sci USA 88(6): 2297-2301.
36. Qiu G (2002) Indexing chromatic and achromatic patterns for content-based colour image retrieval. Pattern Recogn 35(8): 1675-1686.
37. Richman JS, Moorman JR (2000) Physiological time-series analysis using approximate entropy and sample entropy. Am J Physiol Heart C 278(6): H2039-H2049.
38. Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection models. In: Workshop on application of computer vision, pp 29-36.
39. Rösner D, Frommer J, Friesen R, Haase M, Lange J, Otto M (2012) Last minute: a multimodal corpus of speech-based user-companion interactions. In: Conference on language resources and evaluation (LREC '12), pp 2559-2566.
40. Rudnicki M, Strumiłło P (2007) A real-time adaptive wavelet transform-based QRS complex detector. In: International conference on adaptive and natural computing algorithms, pp 281-289.
41. Russell J (1980) A circumplex model of affect. J Pers Soc Psychol 39: 1161-1178.
42. Schels M, Kächele M, Schwenker F (2012) Classification of emotional states in a Woz scenario exploiting labeled and unlabeled bio-physiological data. In: Proceedings of partially supervised learning 2011, pp 138-147.
43. Scherer KR (2005) What are emotions? and how can they be measured? Soc Sci Inf 44(4): 693-727.
44. Schuller B, Valstar M, Eyben F, McKeown G, Cowie R, Pantic M (2011) AVEC 2011-the first international audio visual emotion challenge. In: Proceeding of the affective computing and intelligent, interaction, pp 415-424.
45. Schwenker F, Kestler HA, Palm G (2001) Three learning phases for radial-basis-function networks. Neural Netw 14: 439-458.
46. Simson M (1981) Use of signals in the terminal QRS complex to identify patients with ventricular tachycardia after myocardial infarction. Circulation 64(2): 235-242.
47. Soleymani M, Chanel G, Kierkels J, Pun T (2008) Affective characterization of movie scenes based on multimedia content analysis and user's physiological emotional responses. In: IEEE international symposium on multimedia, pp 228-235.
48. Stemmler G (1989) The autonomic differentiation of emotions revisited: convergent and discriminant validation. Psychophysiology 26(6): 617-632.
49. Stemmler G, Heldmann M, Pauls CA, Scherer T (2001) Constraints for emotion specificity in fear and anger: the context counts. Psychophysiology 38(2): 275-291.
50. Strauss PM, Hoffmann H, Minker W, Neumann H, Palm G, Scherer S, Traue H, Weidenbacher U (2008) The pit corpus of German multi-party dialogues. In: International language resources and evaluation (LREC '08), pp 2442-2445.
51. van Boxtel A (2010) Facial EMG as a tool for inferring affective states. In: Proceedings of measuring behavior, pp 104-108.
52. Vapnik A (1998) Statistical learning theory. Wiley, New York.
53. Vogt T, André E (2009) Exploring the benefits of discretization of acoustic features for speech emotion recognition. In: Proceeding of the interspeech, pp 328-331.
54. Walter S, Kim J, Hrabal D, Crawcour S, Kessler H, Traue H (2013) Transsituational individual-specific biopsychological classification of emotions. IEEE Trans Syst Man Cybern 43(4): 988-995.
55. Wright RA, Dill JC (1993) Blood pressure responses and incentive appraisals as a function of perceived ability and objective task demand. Psychophysiology 30(2): 152-160.
56. Zhang Z, Deng J, Schuller B (2013) Co-training succeeds in computational paralinguistics. In: IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp 8505-8509.
57. Zhang Z, Schuller B (2012) Active learning by sparse instance tracking and classifier confidence in acoustic emotion recognition. In: Proceeding of the INTERSPEECH 2012, pp 362-365.