Audiovisual speech inversion by switching dynamical modeling governed by a Hidden Markov process

European Signal Processing Conference

Volumn , Issue , 2008, Pages

  Katsamanis, A ^a   Ananthakrishnan, G ^b   Papandreou, G ^a   Maragos, P ^a   Engwall, O ^b  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

ACTIVE APPEARANCE MODELS; AUDIO-VISUAL SPEECH; CLASSIFICATION ANALYSIS; CORRELATION COEFFICIENT; DYNAMICAL MODELING; EVALUATION SCHEME; HIDDEN MARKOV PROCESS; INVERSION PROBLEMS; MEL-FREQUENCY CEPSTRAL COEFFICIENTS; PREDICTION ERRORS; RADIAL BASIS FUNCTIONS; ROOT MEAN SQUARED ERRORS; STATE SEQUENCES; SWITCHING LINEAR DYNAMICAL SYSTEMS; UNIFIED FRAMEWORK; VISUAL ANALYSIS;

HIDDEN MARKOV MODELS; LINEAR CONTROL SYSTEMS; RADIAL BASIS FUNCTION NETWORKS; SIGNAL PROCESSING;

IMAGE SEGMENTATION;

EID: 84863731362     PISSN: 22195491     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. M. Sondhi, "Articulatory modeling: a possible role in concatenative text-to-speech synthesis," in IEEE Workshop on Speech Synthesis, Santa Monica, USA, 2002.
2. S. King, J. Frankel, K. Livescu, E. McDermott, K. Richmond, and M. Wester, "Speech production knowledge in automatic speech recognition," Journal of the Acoustical Society of America, vol. 121, no. 2, pp. 723-742, February 2007.
3. J. Schroeter and M. M. Sondhi, Speech coding based on physiological models of speech production. NewYork: Marcel Dekker Inc, 1992, ch. 8.
4. O. Engwall, O. Bälter, A.-M. Öster, and H. Sidenbladh- Kjellström, "Designing the user interface of the computer-based speech training system ARTUR based on early user tests," Journal of Behaviour and Information Technology, vol. 25, no. 4, pp. 353-365, 2006.
5. S. Ouni and Y. Laprie, "Modeling the articulatory space using a hypercube codebook for acoustic-to-articulatory inversion," Journal of Acoustical Society of America, vol. 118, no. 1, pp. 444-460, 2005.
6. K. Richmond, S. King, and P. Taylor, "Modelling the uncertainty in recovering articulation from acoustics," Computer Speech and Language, vol. 17, pp. 153-172, 2003.
7. T. Toda, A. W. Black, and K. Tokuda, "Statistical mapping between articulatory movements and acoustic spectrum using a gaussian mixture model," Speech Communication, vol. 50, pp. 215-227, 2008.
8. S. Dusan and L. Deng, "Acoustic-to-articulatory inversion using dynamical and phonological constraints," in Proceedings of Seminar on Speech Production, 2000, pp. 237-240.
9. S. Hiroya and M. Honda, "Estimation of articulatory movements from speech acoustics using an HMM-based speech production models," IEEE Transactions on Speech and Audio Processing, vol. 12, no. 2, pp. 175-185, March 2004.
10. H. Yehia, P. Rubin, and E. Vatikiotis-Bateson, "Quantitative association of vocal-tract and facial behavior," Speech Communication, vol. 26, pp. 23-43, 1998.
11. J. Jiang, A. Alwan, P. A. Keating, E. T. Auer Jr., and L. E. Bernstein, "On the relationship between face movements, tongue movements, and speech acoustics," EURASIP Journal on Applied Signal Processing, vol. 11, pp. 1174-1188, 2002.
12. O. Engwall, "Introducing visual cues in acoustic-to-articulatory inversion," in Interspeech, 2005, pp. 3205-3208.
13. H. Kjellström, O. Engwall, and O. Bälter, "Reconstructing tongue movements from audio and video," in Interspeech, 2006, pp. 2238-2241.
14. A. Katsamanis, G. Papandreou, and P.Maragos, "Audiovisual-to- articulatory speech inversion using active appearance models for the face and hidden markov models for the dynamics," in Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, 2008.
15. J. Beskow, O. Engwall, and B. Granström, "Resynthesis of facial and intraoral motion from simultaneous measurements," in Proc. of the 15th ICPhS, 2003, pp. 431-434.
16. O. Engwall, "Evaluation of speech inversion using an articulatory classifier," in Proc. of the Seventh International Seminar on Speech Production, 2006, pp. 431-434.
17. B. D. O. Anderson and J. B.Moore, Optimal Filtering. Prentice Hall, 1979.
18. Z. Ghahramani and G. E. Hinton, "Variational learning for switching state-space models," Neural Computation, vol. 12, no. 4, pp. 831-864, 2000.
19. A. Katsamanis, G. Papandreou, and P.Maragos, "Audiovisualto- articulatory speech inversion using HMMs," in Proceedings of International Workshop on Multimedia Signal Processing (MMSP), 2007.
20. S. Youn, Eun, "Feature selection in support vector machines," Master's thesis, The Graduate School of the University of Florida, 2002.
21. J. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. World Scientific, Singapore, 2002.
22. A. Wrench and W. Hardcastle, "A multichannel articulatory speech database and its application for automatic speech recognition," in In Proc. 5th Seminar on Speech Production, Kloster Seeon, Bavaria, 2000, pp. 305-308. [Online]. Available: http://www.cstr.ed.ac.uk/artic
23. S. Young, J. Odell, D. Ollason, V. Valtchev, and P.Woodland., The HTK Book. Cambridge University, 1997.