Data-driven multimodal synthesis

Speech Communication

Volumn 47, Issue 1-2, 2005, Pages 182-193

  Carlson, Rolf ^a   Granström, Björn ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Data driven synthesis; Multimodal synthesis; Speech synthesis

Indexed keywords

DATA ACQUISITION; DATA REDUCTION; KNOWLEDGE BASED SYSTEMS; MATHEMATICAL MODELS;

ACOUSTIC MODELING; DATA-DRIVEN SYNTHESIS; FORMANT-SYNTHESIS SYSTEM; MULTIMODAL SYNTHESIS;

SPEECH SYNTHESIS;

EID: 24144469759     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2005.02.015     Document Type: Article

References (57)

1. Acero, A., 1999. Formant analysis and synthesis using hidden Markov models. In: Proc. Eurospeech'99, pp. 1047-1050.
2. J. Allen, M.S. Hunnicut, and D. Klatt From text to speech: The MITalk system 1987 Cambridge University Press Cambridge MA
3. Bailly, G., Badin, P., 2002. Seeing tongue movements from outside. In: Proc. ICSLP2002, pp. 1913-1916.
4. Beskow, J., 1995. Rule-based visual speech synthesis. In: Proc. 4th European Conf. on Speech Communication and Technology (Eurospeech'95), Madrid, Spain, pp. 299-302.
5. Beskow, J., 1997. Animation of talking agents. In: Proc. Internat. Conf. on Auditory-Visual Speech Processing (AVSP'97), Rhodos, Greece, pp. 149-152.
6. Beskow, J., 2003. Talking heads-models and applications for multimodal speech synthesis. Doctoral thesis, Department of Speech, Music and Hearing, KTH, Stockholm, Sweden.
7. J. Beskow Trainable articulatory control models for visual speech synthesis J. Speech Technol. 7 4 2004 335 349
8. Beskow, J., Engwall, O., Granström, B., 2003. Resynthesis of facial and intraoral articulation from simultaneous measurements. In: Proc. ICPhS 2003, Barcelona, Spain.
9. Beskow, J., Karlsson, I., Kewley, J., Salvi, G., 2004. SYNFACE-a talking head telephone for the hearing-impaired. In: Miesenberger, K., Klaus, J., Zagler, W., Burger, D., (Eds.), Computers Helping People with Special Needs, pp. 1178-1186.
10. Branderud, P., 1985. Movetrack-a movement tracking system. In: Proc. French-Swedish Symposium on Speech, Grenoble, France, pp. 113-122.
11. Bregler, C., Covell, M., Laney, M., 1997. Video rewrite: Driving visual speech with audio. In: Proc. ACM SIGGRAPH'97, pp. 353-360.
12. Brooke, N.M., Scott, D.S., 1998. Two- and three-dimensional audio-visual speech synthesis. In: Proc. Internat. Conf. on Auditory-Visual Speech Processing (AVSP'98), Terrigal, Australia, pp. 213-218.
13. Carlson, R., Granström, B., 1976. A text-to-speech system based entirely on rules. In: Proc. ICASSP-76.
14. Carlson, R., Granström, B., Hunnicutt, S., 1982. A multi-language text-to-speech module. In: Proc. 7th Internat. Conf. on Acoustics, Speech, and Signal Processing (ICASSP'82), Paris, France, Vol. 3, pp. 1604-1607.
15. R. Carlson, B. Granström, and I. Karlsson Experiments with voice modelling in speech synthesis Speech Comm. 10 1991 481 489
16. Carlson, R., Granström, B., Nord, L., 1992. Experiments with emotive speech-acted utterances and synthesized replicas. In: Internat. Conf. on Spoken Language Processing, Banff, Canada, pp 671-674.
17. Carlson, R., Sigvardson, T., Sjölander, A., 2002. Data-driven formant synthesis. In: Fonetik 2002.
18. F. Charpentier, and E. Moulines Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones Speech Comm. 9 5/6 1990 435 467
19. Charpentier, F., Stella, M., 1986. Diphone synthesis using an overlap-add technique for speech waveforms concatenation. In: Proc. ICASSP 86, Vol. 3, pp. 2015-2018.
20. M.M. Cohen, and D.W. Massaro Modelling coarticulation in synthetic visual speech N. Magnenat Thalmann D. Thalmann Models and techniques in computer animation 1993 Springer Verlag Tokyo 139 156
21. N.R. Dixon, and H.D. Maxey Terminal analog synthesis of continuous speech using the diphone method of segment assembly IEEE Trans. AudioElectroacoust. AU-16 1968 40 50
22. Engwall, O., 2002a. Tongue talking-studies in intraoral speech synthesis. PhD thesis, KTH, Sweden.
23. Engwall, O., 2002b. Evaluation of a system for concatenative articulatory visual speech synthesis. In: Proc. ICSLP 2002.
24. Engwall, O., 2004. From real-time MRI to 3D tongue movements. In: Proc. ICSLP 2004.
25. Ezzat, T., Geiger, G., Poggio, T., 2002. Trainable videorealistic speech animation. In: Proc. ACM SIGGRAPH 2002, San Antonio, TX, pp. 388-398.
26. Hällgren, Å., Lyberg, B., 1998. Visual speech synthesis with concatenative speech. In: Proc. Internat. Conf. on Auditory-Visual Speech Processing (AVSP'98), Terrigal, Australia, pp. 181-183.
27. Hertz, S., 2002. Integration of rule-based formant synthesis and waveform concatenation: A hybrid approach to text-to-speech synthesis. In: Proc. IEEE 2002 Workshop on Speech Synthesis, Santa Monica, USA, 11-13 September 2002.
28. Högberg, J., 1997. Data driven formant synthesis. In: Proc. Eurospeech 97.
29. Jiang, J., Alwan, A., Bernstein, L., Keating, P., Auer, E., 2000. On the correlation between facial movements, tongue movements and speech acoustics. In: Proc. ICSLP2000, Vol. 1, 42-45.
30. Klatt, D., 1982. The Klattalk text-to-speech conversion system. In: Proc. ICASSP 82, pp. 1589-1592.
31. D. Klatt Review of text-to-speech conversion for English J. Acoust. Soc. Amer. 82 3 1987 737 793
32. Le Goff, B., 1997. Automatic modeling of coarticulation in text-to-visual speech synthesis. In: Proc. 5th European Conf. on Speech Communication and Technology (EUROSPEECH'97), Rhodos, Greece, pp. 1667-1670.
33. Lee, M., van Santen, J., Möbius, B., Olive, J., 1999. Formant tracking using segmental phonemic information. In: Proc. Eurospeech'99, Vol. 6, pp. 2789-2792.
34. A. Löfqvist Speech as audible gestures W.J. Hardcastle A. Marchal Speech production and speech modelling 1990 Kluwer Academic Publishers Dordrecht 289 322
35. A. MacLeod, and Q. Summerfield A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use Br. J. Audiol. 24 1990 29 43
36. Mannell, R.H., 1998. Formant diphone parameter extraction utilising a labeled single speaker database. In: Proc. ICSLP 98.
37. D.W. Massaro, M.M. Cohen, M. Tabain, J. Beskow, and R. Clark Animated speech: Research progress and applications E. Vatikiotis-Bateson G. Bailly P. Perrier Audiovisual Speech Processing 2005 MIT Press
38. Mori, H., Ohtsuka, T., Kasuya, H., 2002. A data-driven approach to source-formant type text-to-speech system. In: ICSLP-2002, pp. 2365-2368.
39. R. Ogden, S. Hawkins, J. House, M. Huckvale, J. Local, P. Carter, J. Dankovicova, and S. Heid Prosynth: An integrated prosodic approach to device-independent, natural-sounding speech synthesis Comput. Speech Language 14 2000 177 210
40. Öhlin, D., 2004. Formant extraction for data-driven formant synthesis. Master thesis, TMH, KTH, Stockholm (in Swedish).
41. Öhlin, D., Carlson, R., 2004. Data-driven formant synthesis. In: Proc. Fonetik, pp. 160-163.
42. Öhman, T., 1998. An audio-visual speech database and automatic measurements of visual speech. In: KTH TMH QPSR, Vols. 1-2, pp. 61-76.
43. Olive, J.P., 1977. Rule synthesis of speech from diadic units. In: Proc. ICASSP-77, pp. 568-570.
44. F.I. Parke Parameterized models for facial animation IEEE Comput. Graphics 2 9 1982 61 68
45. C. Pelachaud Visual text-to-speech I. Pandzic R. Forchheimer MPEG-4 facial animation-the standard, implementation and applications 2002 John Wiley & Sons 125 140
46. C. Pelachaud, N.I. Badler, and M. Steedman Generating facial expressions for speech Cognitive Sci. 20 1 1996 1 46
47. G. Peterson, W. Wang, and E. Sivertsen Segmentation techniques in speech synthesis J. Acoust. Soc. Amer. 32 1958 639 703
48. Reveret, L., Bailly, G., Badin, P., 2000. Mother: A new generation of talking heads providing a flexible articulatory control for video-realistic speech animation. In: Proc. 6th Internat. Conf. on Spoken Language Processing (ICSLP'2000). Bejing, China, pp. 755-758.
49. R.V. Shannon, F-G. Zeng, V. Kamath, J. Wygonski, and M. Ekelid Speech recognition with primarily temporal cues Science 270 1995 303 304
50. Siciliano, C., Williams, G., Beskow, J., Faulkner A., 2003. Evaluation of a multilingual synthetic talking face as a communication aid for the hearing impaired. To appear in Proc. 15th Internat. Congress of Phonetic Sciences, Barcelona, Spain.
51. Sigvardson, T., 2002. Data-driven methods for parameter synthesis-description of a system and experiments with CART-analysis. Master thesis, TMH, KTH, Stockholm, Sweden (in Swedish).
52. Sjölander, A., 2001. Data-driven formant synthesis. Master thesis, TMH, KTH, Stockholm (in Swedish).
53. Sjölander, K., 2003. An HMM-based system for automatic segmentation and alignment of speech. In: Proc. Fonetik 2003, Umeå Universitet, Umeå, Sweden, pp. 93-96.
54. K.N. Stevens, and C.A. Bickley Constraints among parameters simplify control of Klatt formant synthesizer J. Phonetics 19 1991 161 174
55. D. Talkin Looking at speech Speech Technol. 4 1989 74 77
56. Vinet, R., 2004. Enhancing rule-based synthesizer using concatenative synthesis. Master thesis, TMH, KTH, Stockholm, Sweden.
57. H. Yehia, P. Rubin, and E. Vatikiotis-Bateson Quantitative association of vocal-tract and facial behaviour Speech Comm. 26 1998 23 43