Affective speech interface in serious games for supporting therapy of mental disorders

Expert Systems with Applications

Volumn 39, Issue 12, 2012, Pages 11072-11079

  Kostoulas, Theodoros ^a   Mporas, Iosif ^a   Kocsis, Otilia ^a   Ganchev, Todor ^a   Katsaounos, Nikos ^a   Santamaria, Juan J ^b   Jimenez Murcia, Susana ^b   Fernandez Aranda, Fernando ^b   Fakotakis, Nikos ^b  

^a UNIVERSITY OF PATRAS   (Greece)

^b HOSPITAL UNIVERSITARI DE BELLVITGE   (Spain)

Author keywords

Affect recognition; Emotion recognition; Mental disorders; Serious games; Speech interface; Speech recognition

Indexed keywords

AFFECT RECOGNITION; EMOTION RECOGNITION; MENTAL DISORDERS; SERIOUS GAMES; SPEECH INTERFACE;

SPEECH;

SPEECH RECOGNITION;

EID: 84861182398     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2012.03.067     Document Type: Article

References (49)

1. E.M. Álvarez-Moya, S. Jiménez-Murcia, R. Granero, J. Vallejo, I. Krug, and C.M. Bulik Comparison of personality risk factors in bulimia nervosa and pathological gambling Comprehensive Psychiatry 48 2007 452 457
2. S. Barab, M. Thomas, T. Dodge, R. Carteaux, and H. Tuzun Making learning fun: Quest Atlantis a game without guns Educational Technology Research and Development 53 2005 86 107
3. A. Batliner, S. Steidl, C. Hacker, and E. Nöth Private emotions versus social interaction: A data-driven approach towards analysing emotion in speech User Modeling and User-Adapted Interaction 18 2008 175 206
4. I.L. Beale, P.M. Kato, V.M. Marin-Bowling, N. Guthrie, and S.W. Cole Improvement in cancer-related knowledge following use of a psychoeducational video game for adolescents and young adults with cancer Journal of Adolescent Health 41 2007 263 270
5. B. Bergeron Learning & retention in adaptive serious games Studies in Health Technology and Informatics 132 2008 26 30
6. Bohus, D.; Rudnicky, A. I. (2003). RavenClaw: Dialog management using hierarchical task decomposition and an expectation agenda. In Eurospeech 2003 (pp. 597-600).
7. C. Botella, H. Villa, P.A. García, S. Quero, R. Baños, and M. Alcaniz The use of VR in the treatment of panic disorders and agoraphobia Studies in Health Technology and Informatics 99 2004 73 90
8. Brendel, M.; Zaccarelli, R.; Schuller, B.; Devillers, L. (2010). Towards measuring similarity between emotional corpora. In LREC 2010 (pp. 58-64).
9. V. Brezinka Treasure Hunt-a serious game to support psychotherapeutic treatment of children Studies in Health Technology and Informatics 136 2008 71 76
10. Clarkson, P.; Rosenfeld, R. (1997). Statistical language modeling using the CMU-Cambridge toolkit. In Eurospeech 2007 (pp. 2707-2710).
11. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, and W. Fellenz Emotion recognition in human-computer interaction Signal Processing Magazine, IEEE 18 2001 32 80
12. D2.4, D. (2010). Refined requirements and specifications. In PlayMancer project FP7 215839.
13. D3.3, D. (2011). Final Playmancer 3D dialogue-enabled game engine prototype. In PlayMancer project FP7 215839.
14. A.P. Dempster, N.M. Laird, and D.B. Rubin Maximum likelihood from incomplete data via the EM algorithm Journal of the Royal Statistical Society, Series B (Methodological) 39 1977 1 38
15. Eyben, F.; Wollmer, M.; Schuller, B. (2009). OpenEAR - introducing the Munich open-source emotion and affect recognition toolkit. In 4th International humaine association conference on affective computing and intelligent interaction (pp. 1-6): IEEE.
16. F. Fernández-Aranda, S. Jiménez-Murcia, E.M. Álvarez-Moya, R. Granero, J. Vallejo, and C.M. Bulik Impulse control disorders in eating disorders: clinical and therapeutic implications Comprehensive Psychiatry 47 2006 482 488
17. Fernandez-Aranda, F.; Jimenez-Murcia, S.; Santamaria, J. J.; Gunnard, K.; Soto, A.; Kalapanidas, E.; et al. (2012). Video games as a complementary tool in mental disorders: Playmancer a european multicenter study. Journal of Mental Health. doi: 10.3109/09638237.2012.664302.
18. F. Fernández-Aranda, A. Núñez, C. Martínez, I. Krug, M. Cappozzo, and I. Carrard Internet-based cognitive-behavioral therapy for bulimia nervosa: a controlled study CyberPsychology & Behavior 12 2009 37 41
19. M. Griffiths Can Videogames Be Good for Your Health? Health Psychology 9 2004 334 339
20. S. Jiménez-Murcia, E.M. Álvarez-Moya, R. Granero, M.N. Aymami, M. Gómez-Peña, and N. Jaurrieta Cognitive-behavioral group treatment for pathological gambling: analysis of effectiveness and predictors of therapy outcome Psychotherapy Research 17 2007 544 552
21. S. Jiménez-Murcia, F. Fernández-Aranda, E. Kalapanidas, D. Konstantas, T. Ganchev, and O. Kocsis Playmancer project: A serious videogame as an additional therapy tool for eating and impulse control disorders Annual Review of Cyber Therapy and Telemedicine 7 2009 163 166
22. E. Kalapanidas, F. Fernandez-Aranda, S. Jimenez-Murcia, O. Kocsis, T. Ganchev, and H. Kaufmann PlayMancer: Games for health with accessibility in mind Communications & Strategies DigiWorld Economic Journal 73 2009 105 119
23. O. Kocsis, T. Ganchev, I. Mporas, G. Papadopoulos, and N. Fakotakis Multi-modal system architecture for serious gaming Artificial Intelligence Applications and Innovations III 296 2009 441 447
24. Kostoulas, T.; Ganchev, T.; Fakotakis, N. (2008). Study on speaker-independent emotion recognition from speech on real-world data. In Verbal and nonverbal features of human-human and human-machine interaction (vol. 5042, pp. 235-242).
25. Kostoulas, T.; Ganchev, T.; Fakotakis, N. (2010). Affect recognition in real life scenarios. In Toward autonomous, adaptive, and context-aware multimodal interfaces. Theoretical and practical issues (pp. 429-435).
26. Kostoulas, T.; Ganchev, T.; Lazaridis, A.; Fakotakis, N. (2011). Enhancing emotion recognition from speech through feature selection. In TSD 2010 (pp. 338-344): Springer.
27. Kostoulas, T.; Ganchev, T.; Mporas, I.; Fakotakis, N. (2008). A real-world emotional speech corpus for modern Greek. In: LREC 2008 (pp. 2676-2680).
28. Kostoulas, T.; Kocsis, O.; Ganchev, T.; Santamaría, J. J.; Jiménez-Murcia, S.; Moussa, M. B.; Magnenat-Thalmann, N.; Fakotakis, N. (2010). The PlayMancer database: a multimodal affect database in support of research and development activities in serious game environment. In LREC 2010 (pp. 3011-3015).
29. Kostoulas, T.; Mporas, I.; Ganchev, T.; Fakotakis, N. (2008). The effect of emotional speech on a smart-home application. In IEA/AIE 2008 (pp. 305-310).
30. Moreno, A. (1997). SpeechDat Spanish database for fixed telephone network. Corpus Design Technical Report, SpeechDAT Project LE2-4001.
31. I. Mporas, T. Ganchev, O. Kocsis, and N. Fakotakis Context-adaptive pre-processing scheme for robust speech recognition in fast-varying noise environment Signal Processing 91 2011 2101 2111
32. Mporas, I.; Ganchev, T.; Kocsis, O.; Fakotakis, N. (2011b). Dynamic selection of a speech enhancement method for robust speech recognition in moving motorcycle environment. In 36th International conference on acoustics, speech and signal processing (pp. 5176-5179): IEEE.
33. Mporas, I.; Ganchev, T.; Siafarikas, M.; Kostoulas, T. (2007). Comparative evaluation of speech parameterizations for speech recognition. In 19th International conference on tools with artificial intelligence (vol. 2, pp. 510-513): IEEE.
34. I. Mporas, O. Kocsis, T. Ganchev, and N. Fakotakis Robust speech interaction in motorcycle environment Expert Systems with Applications 37 2010 1827 1835
35. M.P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann Service-oriented computing: State of the art and research challenges Computer 40 2007 38 45
36. D.A. Reynolds, T.F. Quatieri, and R.B. Dunn Speaker verification using adapted Gaussian mixture models Digital Signal Processing 10 2000 19 41
37. D.A. Reynolds, and R.C. Rose Robust text-independent speaker identification using Gaussian mixture speaker models Speech and Audio Processing, IEEE Transactions on 3 1995 72 83
38. M. Ribasés, F. Fernández-Aranda, M. Gratacòs, J.M. Mercader, C. Casasnovas, and A. Núñez Contribution of the serotoninergic system to anxious and depressive traits that may be partially responsible for the phenotypical variability of bulimia nervosa Journal of Psychiatric Research 42 2008 50 57
39. C. Russoniello, K. O'Brien, and J. Parks EEG, HRV and psychological correlates while playing bejeweled II: A randomized controlled study Studies in Health Technology and Informatics 144 2009 189 192
40. Santamaria, J. J.; Soto, A.; Fernandez-Aranda, F.; Krug, I.; Forcano, L.; Kalapanidas, E.; Gunnard, K.; Lam, T.; Raguin, T.; Davarakis, C.; Menchon, J. M.; Jimenez-Murcia, S. (2011). Serious Games as additional Psychological Support: A review of the literature. Cyberpsychology and Behaviour, Therapy, 469-476.
41. B. Schuller, A. Batliner, S. Steidl, and D. Seppi Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge Speech Communication 53 2011 1062 1087
42. Schuller, B.; Steidl, S.; Batliner, A. (2009). The Interspeech 2009 Emotion Challenge. In Interspeech 2009 (pp. 312-315).
43. Schuller, B.; Steidl, S.; Batliner, A.; Burkhardt, F.; Devillers, L.; Müller, C.; Narayanan, S. S. (2010). The Interspeech 2010 Paralinguistic Challenge. In Interspeech 2010 (pp. 2794-2797).
44. Sphinx3 (2011). The CMU sphinx group open source speech recognition engines. Available from cmusphinx.sourceforge.net/ (accessed 09.04.2012).
45. Unity (2010). Unity game development tool. Available from unity3d.com/ (accessed 09.04.2012).
46. K. van Bastelaar, F. Pouwer, P. Cuijpers, J. Twisk, and F. Snoek Web-based cognitive behavioural therapy (W-CBT) for diabetes patients with co-morbid depression: Design of a randomised controlled trial BMC Psychiatry 8 2008 9
47. A. Vicentic, and D.C. Jones The CART (cocaine-and amphetamine-regulated transcript) system in appetite and drug addiction Journal of Pharmacology and Experimental Therapeutics 320 2007 499 506
48. J.C. Wells SAMPA computer readable phonetic alphabet Handbook of Standards and Resources for Spoken Language Systems 4 1997
49. C. Whissell The dictionary of affect in language Emotion: Theory, Research, and Experience 4 1989 113 131