Gesture synthesis adapted to speech emphasis

Speech Communication

Volumn 57, Issue , 2014, Pages 331-350

  Fernández Baena, Adso ^a   Montaño, Raúl ^a   Antonijoan, Marc ^a   Roversi, Arturo ^a   Miralles, David ^a   Alías, Francesc ^a  

^a UNIVERSITAT RAMON LLULL   (Spain)

Author keywords

Body language; Character animation; Human computer interaction; Motion capture; Motion graphs; Speech analysis; Speech emphasis

Indexed keywords

BODY LANGUAGE; CHARACTER ANIMATION; MOTION CAPTURE; MOTION GRAPH; SPEECH EMPHASIS;

ANIMATION; HUMAN COMPUTER INTERACTION; SPEECH ANALYSIS;

SPEECH;

EID: 84888308215     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2013.06.005     Document Type: Article

References (72)

1. Antonijoan, M., 2012. Gesture-prosody correlations analysis. Master's Thesis, La Salle - Ramon Llull University.
2. O. Arikan, and D.A. Forsyth Interactive motion generation from examples ACM Trans. Graph. 21 2002 483 490
3. B.J.H. van Basten, and A. Egges Evaluating distance metrics for animation blending Proceedings of the Fourth International Conference on Foundations of Digital Games 2009 ACM New York, NY, USA 199 206
4. M. Beckman, M. Díaz-Campos, J.T. McGory, and T.A. Morgan Intonation across spanish in the tones and break indices framework Probus 14 2002 9 36
5. P. Boersma Praat, a system for doing phonetics by computer Glot Int. 5 2001 341 345
6. D.L.M. Bolinger Intonation and Its Parts: Melody in Spoken English 1986 Stanford University Press Stanford, CA
7. Bulut, M., Lee, S., Narayanan, S., 2007. A statistical approach for modeling prosody features using pos tags for emotional speech synthesis. In: ICASSP 2007, pp. 1237-1240.
8. J. Cassell Beat: The Behavior Expression Animation Toolkit 2001 ACM Press pp. 477-486
9. J. Cassell, C. Pelachaud, N. Badler, M. Steedman, B. Achorn, T. Becket, B. Douville, S. Prevost, and M. Stone Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques 1994 ACM New York, NY, USA 413 420
10. C.C. Chiu, and S. Marsella How to train your avatar: a data driven approach to gesture generation Proceedings of the 10th International Conference on Intelligent Virtual Agents 2011 Springer-Verlag Berlin, Heidelberg 127 140
11. W.S. Condon An analysis of behavioral organization Sign Lang. Stud. 1976 285 318
12. Escudero-Mancebo, D., Aguilar, L., 2010. Procedure for assessing the reliability of prosodic judgements using Sp-TOBI labelling system. In: Proceedings of Internationa Conference on Speech Prosody.
13. Fernández-Baena, A., Miralles, D., 2012. Fast response and quick progressive transitions using body part motion graphs. Eurographics Association, Cagliari, Sardinia, Italy, pp. 77-80.
14. Goldman, J.P., 2011. Easyalign: an automatic phonetic alignment tool under Praat. In: INTERSPEECH-2011, pp. 3233-3236.
15. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten The weka data mining software: an update SIGKDD Explor. 2009 11
16. ITU-T, 1996. P.800 - methods for subjective determination of transmission quality.
17. O. Kalinli, and S.S. Narayanan Prominence detection using auditory attention cues and task-dependent high level information IEEE Trans. Audio Speech Lang. Process. 17 2009 1009 1024
18. A. Kendon Gesture and speech: two aspects of the process utterances Nonverbal Commun. Lang. 1980 207 227
19. Kipp, M., 2001. Anvil a generic annotation tool for multimodal dialogue. In: Proceedings of the Seventh European Conference on Speech Communication and Technology, Aalborg, pp. 1367-1370 (EN).
20. Kipp, M., 2004. Gesture generation by imitation - from human behavior to computer character animation. Ph.D. Thesis. Boca Raton, FL.
21. M. Kipp, and J.C. Martin Gesture and emotion: can basic gestural form features discriminate emotions? Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII-09) 2009 IEEE Press
22. M. Kipp, M. Neff, K.H. Kipp, and I. Albrecht Towards natural gesture synthesis: evaluating g (Pubitemid 350284306)
23. S. Kita, I. van Gijn, and H. van der Hulst Movement phases in signs and co-speech gestures, and their transcription by human coders I. Wachsmuth, M. Frhlich, Gesture and Sign Language in Human-Computer Interaction Lecture Notes in Computer Science vol. 1371 1998 Springer Berlin/Heidelberg 23 35
24. S. Kopp, K. Bergmann, and I. Wachsmuth Multimodal communication from multimodal thinking - towards an integrated model of speech and gesture production Int. J. Semant. Comput. 2 2008 115 136
25. L. Kovar, M. Gleicher, and F. Pighin Motion graphs ACM Trans. Graph. 21 2002 473 482
26. J. Lee, J. Chai, P.S.A. Reitsma, J.K. Hodgins, and N.S. Pollard Interactive control of avatars animated with human motion data ACM Trans. Graph. 21 2002 491 500
27. T. Leonard, and F. Cummins The temporal relation between beat gestures and speech Lang. Cognit. Process. 26 2011 1457 1471
28. S. Levine, P. Krähenbühl, S. Thrun, and V. Koltun Gesture controllers ACM Trans. Graph. 29 2010 124:1 124:11
29. S. Levine, C. Theobalt, and V. Koltun Real-time prosody-driven synthesis of body language ACM Trans. Graph. 28 2009 172:1 172:10
30. Llull, L.S.U.R., 2012. MediaLab. motion capture, RV + RA, animation, videogames and CAD. .
31. Loehr, D., 2004. Gesture and intonation. Ph.D. Thesis. Georgetown University.
32. P. Luo, M. Kipp, and M. Neff Augmenting gesture animation with motion capture data to provide full-body engagement Intelligent Virtual Agents 2009 Springer Berlin, Heidelberg 405 417
33. B.W. Matthews Comparison of the predicted and observed secondary structure of T4 phage lysozyme Biochim. Biophys. Acta 405 1975 442 451
34. D. McNeill So you think gestures are nonverbal? Psychol. Rev. 92 1985 350 371
35. D. McNeill Hand and Mind: What Gestures Reveal about Thought 1992 University of Chicago Press Chicago
36. D. McNeill Gesture and Thought Phoenix Poets Series 2008 University of Chicago Press
37. M. Neff, M. Kipp, I. Albrecht, and H.P. Seidel Gesture modeling and animation based on a probabilistic re-creation of speaker style ACM Trans. Graph. 27 2008 5:1 5:24
38. Nobe, S., 1996. Representational gestures, cognitive rhythms, and acoustic aspects of speech: a network/threshold model of gesture production. University of Chicago, Department of Psychology.
39. Onuma, K., Faloutsos, C., Hodgins, J.K., 2008. FMDistance: a fast and effective distance function for motion capture data. In: Short Papers Proceedings of EUROGRAPHICS.
40. M. Ortega-Llebaria, and P. Prieto Acoustic correlates of stress in central catalan and castilian Spanish Lang. Speech 54 2011 73 97
41. H. Ortiz-Lira La aplicación de tobi a un corpus del español de chile Onomázein 4 1999 429 442
42. M. Oshita, and N. Masaoka Generating avoidance motion using motion graph Proceedings of the Fourth International Conference on Motion in Games 2011 Springer-Verlag Berlin, Heidelberg 120 131
43. T. Pejsa, and I. Pandzic State of the art in example-based motion synthesis for virtual characters in interactive applications Comput. Graph. Forum 29 2010 202 226
44. C. Pelachaud Multimodal expressive embodied conversational agents Proceedings of the 13th Annual ACM International Conference on Multimedia 2005 ACM New York, NY, USA 683 689
45. Pierrehumbert, J., 1980. The phonology and phonetics of english intonation. Ph.D. Thesis, MIT, Cambridge, MA.
46. Planet, S., Iriondo, I., Martnez, E., Montero, J.A., 2008. True: an online testing platform for multimedia evaluation. In: Proceedings of the Second International Workshop on EMOTION: Corpora for Research on Emotion and Affect at the Sixth Conference on Language Resources & Evaluation.
47. J.C. Platt Fast Training of Support Vector Machines using Sequential Minimal Optimization 1999 MIT Press Cambridge, MA, USA pp. 185-208
48. Qin, Y., Zhang, X., Ying, H., 2010. A HMM-based fuzzy affective model for emotional speech synthesis. In: International Conference on Signal Processing Systems.
49. Quintilian, Butler, H.E., 1920. Institutio oratoria/Quintilian; with an English translation by H.E. Butler. Heinemann, London.
50. Ren, Z., Meng, J., Yuan, J., 2011. Depth camera based hand gesture recognition and its applications in human-computer-interaction. In: 2011 Eighth International Conference on Information, Communications and Signal Processing (ICICS), pp. 1 -5.
51. Renwick, M., Yasinnik, Y., Shattuck-Hufnagel, S., 2004. The timing of speech-accompanying gestures with respect to prosody. In: From Sound to Sense: 50+ Years of Discoveries in Speech Communication, pp. 97-102.
52. C. van Rijsbergen Foundation of evaluation J. Docum. 30 1974 365 373
53. Roekhaut, S., Goldman, J., Simon, A., 2010. A model for varying speaking style in TTS systems. In: Fifth International Conference on Speech Prosody.
54. J. Russell A circumplex model of affect J. Personal. Social Psychol. 39 1980 1161 1178
55. Shalev-Shwartz, S., Singer, Y., Srebro, N., 2007. Pegasos: primal estimated sub-gradient solver for SVM. In: 24th International Conference on Machine Learning, pp. 807-814.
56. Shattuck-Hufnagel, S., Yasinnik, Y., Veilleux, N., Renwick, M., 2007. A method for studying the time alignment of gestures and prosody in american english: 'hits' and pitch accents in academic-lecture-style speech. In: Esposito, Anna, Bratanić, Maja, Keller, Eric, Marinaro, Maria (Eds.), Fundamentals of Verbal and Nonverbal Communication and the Biometric Issue, NATO Publishing Sub-Series E: Human and Societal Dynamics, vol. 18, Washington, DC.
57. Silipo, R., Greenberg, S., 1999. Automatic transcription of prosodic stress for spontaneous English discourse. In: Olds, J.J., Hasegawa, Y., Ohala, M., Bailey, A.C. (Eds.), Proceedings of the XIVth International Congress of Phonetic Sciences (ICPhS99), The Regents of the University of California, pp. 2351-2354.
58. Abete, G., Cutugno, F., Ludusan, B., Origlia, A., 2010. Pitch behavior detection for automatic prominence recognition. In Proc. of Speech Prosody 2010 Satellite Workshop.
59. Silverman, K., Beckman, M.E., Pitrelli, J.F., Ostendorf, M., Wightman, C.W., Price, P., Pierrehumbert, J.B., Hirschberg, J., 1992. Tobi: a standard for labeling english prosody. In: ICSLP, ISCA.
60. I. van der Sluis, and E. Krahmer Generating multimodal references Discourse Process. Multidisc. J. 44 2007 145 174
61. M. Stone, D. DeCarlo, I. Oh, C. Rodriguez, A. Stere, A. Lees, and C. Bregler Speaking with hands: creating animated conversational characters from recordings of human performance ACM Trans. Graph. 23 2004 506 513
62. Streefkerk, B.M., Pols, L.C.W., ten Bosch, L., 1999. Acoustical features as predictors for prominence in read aloud dutch sentences used in ann's. In: EUROSPEECH, ISCA.
63. A.K. Syrdal, J. Hirschberg, J. McGory, and M. Beckman Automatic Tobi prediction and alignment to speed manual labeling of prosody Speech Commun. 33 2001 135 151 (Pubitemid 32029456)
64. Tamburini, F., Wagner, P., 2007. On automatic prominence detection for german. In: INTERSPEECH, ISCA. pp. 1809-1812.
65. J. Terken Fundamental frequency and perceived prominence of accented syllables J. Acoust. Soc. Am. 89 1991 1768 1776
66. Valbonesi, L., 2002. Multimodal signal analysis of prosody and hand motion: temporal correlation in speech and gestures. University of Illinois at Chicago.
67. H. Van Welbergen, B.J.H. Van Basten, A. Egges, Z.M. Ruttkay, and M.H. Overmars Real time animation of virtual humans: a trade-off between naturalness and control Comput. Graph. Forum 29 2010 2530 2554
68. E. Vilaplana, and P. Prieto La notación prosodica en español. una revisión del sp-tobi Estudios de Fonética Experimental XVIII 2009 263 283
69. Vilhjálmsson, H., 2003. Avatar augmented online conversation. Ph.D. Thesis, Massachusetts Institute of Technology.
70. H.G. Wallbott Bodily expression of emotion Eur. J. Soc. Psychol. 28 1998 879 896
71. J. Wang, and B. Bodenheimer Synthesis and evaluation of linear motion transitions ACM Trans. Graph. 27 2008 1:1 1:15
72. J. Xu, K. Takagi, and S. Sakazawa Motion synthesis for synchronizing with streaming music by segment-based search on metadata motion graphs ICME 2011 IEEE 1 6