Issues in data labelling

Cognitive Technologies

Volumn , Issue 9783642151835, 2011, Pages 213-241

  Cowie, Roddy ^a   Cox, Cate ^a   Martin, Jean Claude ^b   Batliner, Anton ^c   Heylen, Dirk ^d   Karpouzis, Kostas ^e  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^b UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^c UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^d UNIVERSITY OF TWENTE   (Netherlands)

^e NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

Author keywords

[No Author keywords available]

Indexed keywords

DATA LABELLING; DESCRIPTORS; STATES AND EVENTS; TIME COURSE;

EID: 85028803032     PISSN: 16112482     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-642-15184-2_13     Document Type: Chapter

References (54)

1. Auberge V, Audibert N, Rülard A (2006) Auto-annotation: an alternative method to label expressive corpora. In: Proceedings LREC 2006, Genoa, pp 45-46
2. Bachorowski JA (1999) Vocal expression and perception of emotion. Curr Dir Psychol Sci 8(2): 53-57
3. Bänziger T, Tran V, Scherer KR (2005) The Geneva Emotion Wheel: a tool for the verbal report of emotional reactions. In: Poster presented at ISRE 2005, Bari
4. Baron-Cohen S (2007) Mind reading: the interactive guide to emotions - version 1.3. Jessica Kingsley, London
5. Batliner A, Steidl S, Schuller B, Seppi D, Laskowski K, Vogt T, Devillers L, Vidrascu L, Amir N, Kessous L, Aharonson V (2006) Combining efforts for improving automatic classification of emotional user states. In: Erjavec T, Gros J (eds) Language technologies, IS-LTC 2006. Infornacijska Druzba (Information Society), Ljubljana, Slovenia, pp 240-245
6. Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25(1):49-59
7. Campbell N (2004) Extra-semantic protocols: input requirements for the synthesis of dialogue speech. In: Andre E, Dybkjaer L, Minker W, Heisterkamp P (eds) Affective dialogue systems. Lecture notes in artificial intelligence. Springer, Berlin, pp 221-228
8. Cohen JA (1960). A coefficient of agreement for nominal scales. Educ Psychol Meas, 20(1):37-46
9. Cowie R, Cornelius R (2003) Describing the emotional states that are expressed in speech. Speech Commun 40:5-32
10. Cowie R, Douglas-Cowie E, Apolloni B, Taylor J, Romano A, Fellenz W (1999) What a neural net needs to know about emotion words. In: Mastorakis N (ed) Computational intelligence and applications. World Scientific Engineering Society, Dallas, TX, pp 109-114
11. Cowie R, Douglas-Cowie E, Savvidou S, McMahon E, Sawey M, Schroeder M (2000) ‘FEELTRACE’: an instrument for recording perceived emotion in real time. In: Proceedings of the ISCA ITRW on speech and emotion: developing a conceptual framework, Newcastle, 5-7 Sept 2000, Textflow, Belfast, pp 19-24.
12. Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor J (2001) Emotion recognition in human-computer interaction. IEEE Signal Process Mag 18(1):32-80
13. Craggs R, Wood M (2003) Annotating emotion in dialogue. In: Proceedings of the 4th SIGdial workshop on discourse and dialogue, Sapporot
14. Craggs R, Wood M (2004) A 2 dimensional annotation scheme for emotion in dialogue. In: AAAI spring symposium on exploring attitude and affect in text: theories and applications. Stanford University, AAAI Press, pp 44-49
15. Devillers L, Abiilian S, Martin J-C (2005) Representing real life emotions in audiovisual data with non basic emotional patterns and context features. In: Proceedings of the 1st international conference on affective computing and intelligent interaction (ACII’2005), Beijing, 22-24 Oct. Spinger, Berlin, pp 519-526
16. Devillers L, Cowie R, Martin J-C, Douglas-Cowie E, Abrilian S, McRorie M (2006) Real life emotions in French and English TV video clips: an integrated annotation protocol combining continuous and discrete approaches. In: Proceedings LREC 2006, Genoa
17. Devillers L, Martin J-C (2008) Coding emotional events in audiovisual corpora. In 6th international conference on language resources and evaluation (LREC 2008). Marrakech (Morocco)
18. Douglas-Cowie E, Campbell N, Cowie R, Roach P (2003) Emotional speech: towards a new generation of databases. Speech Commun 40(1-2):33-60
19. Douglas-Cowie E, Devillers L, Martin J-C, Cowie R, Savvidou R, Abrilian S, Cox C (2005) Multimodal databases of everyday emotion: facing up to complexity. In: Proceedings of the Interspeech 2005, Lisbon, pp 813-816
20. Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169-200
21. Ekman P (1999) Basic emotions. In: Dalgleish T, Power MJ (eds) Handbook of cognition and emotion. Wiley, New York, pp 301-320
22. Ekman P, Friesen W (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall, Englewood Cliffs, NJ
23. Ekman P, Friesen WC, Hager JC (2002). Facial action coding system. The manual on CD ROM. Research Nexus Division of Network Information Research Corporation, Salt Lake City, UT
24. El Kaliouby R, Robinson P (2004) Mind reading machines: automated inference of cognitive mental states from video. In: IEEE international conference on systems, man and cybernetics, vol 1, The Hague, pp 682-688
25. Fragopanagos N, Taylor J (2005) Emotion recognition in human-computer interaction. Neural Netw 18:389-405
26. Hall JA, Matsumoto D (2004) Gender differences in judgments of multiple emotions from facial expressions. Emotion 4(2):201-206
27. Hanson NR (1958) Patterns of discovery: an inquiry into the conceptual foundations of science. Cambridge University Press, Cambridge, MA
28. Hartmann B, Mancini M, Pelachaud C (2005) Implementing expressive gesture synthesis for embodied conversational agents. In: Gesture Workshop (GW’2005), Vannes, France, pp 1095-1096
29. Ioannou S, Raouzaiou A, Tzouvaras V, Mailis T, Karpouzis K, Kollias S (2005) Emotion recognition through facial expression analysis based on a neurofuzzy network Neural Netw, 18, 423-435
30. James W (1884) What is emotion? Mind 9:188-205
31. Kipp M (2001) Anvil - a generic annotation tool for multimodal dialogue. In: 7th European conference on speech communication and technology (Eurospeech’2001), Aalborg, Denmark, 3-7 Sept, pp 1367-1370
32. Kipp M (2004). Gesture generation by imitation. From human behavior to computer character animation, Florida, Boca Raton, Dissertation.com. 15811-22551
33. Kita S, van Gijn I, van der Hulst H (1998). Movement phases in signs and co-speech gestures, and their transcription by human coders. Gesture and sign language in human computer interaction: Proceedings/international gesture workshop, Bielefeld, 17-19 Sept. Springer, Berlin, Heidelberg
34. Ladd S, Silverman K, Bergmann G, Scherer K (1985) Evidence for independent function of intonation contour type, voice quality, and F0 in signalling speaker affect. J Acoust Soc Am 78(2):435-444
35. Martin J-C, Abrilian S, Devillers L (2005). Annotating multimodal behaviors occurring during non basic emotions. In: 1st international conference on affective computing and intelligent interaction (ACII’2005), Beijing, 22-24 Oct. Spinger, Berlin, pp 550-557
36. Martin JC, Devillers L, Zara A, Maffiolo V, LeChenadec G (2006) The EmoTABOU corpus. Humaine Summer School, Genova, pp 22-28
37. Matsumoto D (2001) Culture and emotion. In: Matsumoto D (ed) Handbook of culture and psychology. Oxford University Press, New York, NY, pp 171-194
38. Mayer JD, Salovey P, Caruso D (2000) Models of emotional intelligence. In: Sternberg R (ed) Handbook of intelligence. Cambridge University Press, Cambridge, MA, pp 396-421
39. McNeill D (1992) Hand and mind - what gestures reveal about thoughts. University of Chicago Press, Chicago, IL
40. McNeill D (2005) Gesture and thought. University of Chicago Press, Chicago, IL
41. Mozziconacci SJL (1998). Speech variability and emotion: production and perception. Ph.D. thesis, Eindhoven
42. Ortony A, Clore G, Collins A (1988) The cognitive structure of emotions. Cambridge University Press, Cambridge, MA
43. Ortony A, Turner TJ (1990) What’s basic about basic emotions? Psychol Rev 97:315-331
44. Rosenberg A, Binkowski E (2005). Augmenting the kappa statistic to determine interannotator reliability for multiple labelled data points. In: Proceeding of the HLT-NAACL, Boston
45. Russell J, Barrett-Feldman L (1999) Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J Pers Soc Psychol 5:37-63
46. Sander D, Grandjean D, Scherer KR (2005) A systems approach to appraisal mechanisms in emotion. Neural Netw 18:317-352
47. Savvidou S (submitted) Validation of the Feeltrace tool for recording impressions of expressed emotion. Thesis submitted for Ph.D., Queen’s University, Belfast.
48. Scherer KR (ed) (1988a) Facets of emotion: recent research. Erlbaum, Hillsdale, NJ. Appendix F. Labels describing affective states in five major languages, revised by the members of the Geneva Emotion Research Group retrieved Dec 2007 from http://www.unige.ch/cisa/gerg/research.html
49. Scherer KR (ed) (1988b) Appendix B. Antecedent and reaction codes used in the “Emotion in Social Interaction.” In: Scherer KR (ed) Facets of emotion: recent research. Erlbaum, Hillsdale, NJ, pp 241-243. http://www.unige.ch/cisa/gerg/research.html
50. Schreuder M, van Eerten L, Gilbers D (2006) Music as a method of identifying emotional speech. In: LREC Research Workshop on Corpora on Emotion and Affect, Genoa
51. Steidl S, Levit M, Batliner A, Nöth E, Niemann H (2005) “Of All Things the Measure is Man” -automatic classification of emotions and inter-labeller consistency. In: Proceedings of the ICASSP 2005, Philadelphia, pp 317-320
52. Tabachnik BG, Fidell LS (2001) Using multivariate statistics, 4th edn. Allyn & Bacon, Boston, MA
53. Wallbott HG (1998) Bodily expression of emotion. Eur J Soc Psychol 28:879-896
54. Whissell C (1989) The dictionary of affect in language. In: Plutchik R, Kellerman H (eds) Emotion: theory, research and experience: vol 4, The measurement of emotions. Academic, New York, NY, pp 113-131