A simplified audiovisual fusion model with application to large-vocabulary recognition of French Canadian speech

Canadian Journal of Electrical and Computer Engineering

Volumn 33, Issue 2, 2008, Pages 109-119

  Gagnon, L ^a   Foucher, S ^a   Laliberte F ^a   Boulianne, G ^a  

^a Cent de Recherche Informatique de Montreal   (Canada)

Author keywords

Audiovisual speech recognition; Dempster Shafer theory; Mouth tracking; Multimodal fusion

Indexed keywords

AUDIO ACOUSTICS; CLASSIFICATION (OF INFORMATION); COMPUTER NETWORKS; CONTROL THEORY; DECISION THEORY; DISCRIMINANT ANALYSIS; ERROR ANALYSIS; FINANCIAL DATA PROCESSING; FUSION REACTIONS; IMAGE CLASSIFICATION; INFORMATION MANAGEMENT; SPEECH; SPEECH ANALYSIS; VISUAL COMMUNICATION;

AUDIOVISUAL SPEECH RECOGNITION; DEMPSTER-SHAFER THEORY; MOUTH TRACKING; MULTIMODAL FUSION; VISUAL INFORMATIONS;

SPEECH RECOGNITION;

EID: 51649128057     PISSN: 08408688     EISSN: None     Source Type: Journal    
DOI: 10.1109/CJECE.2008.4621835     Document Type: Article

References (25)

1. G. Potamianos, C Neti, J. Luettin, and I. Matthews, "Audio-visual automatic speech recognition: An overview," in Issues in Visual and Audio-Visual Speech Processing, ed. G. Balily, E. Vatikiotis-Bateson, and P. Perrier, Cambridge, Mass.: MIT Press, 2004.
2. G. Potamianos, C. Neti, G. Gravier, and A. Garg, "Recent advances in the automatic recognition of audio-visual speech," Proc. IEEE, vol. 91, no. 9, Sept. 2003, pp.1306-1326.
3. G. Potamianos, A. Verma, C. Neti, G. Iyengar, and S. Basu, "A cascade image transform, for speaker independent automatic speechreading," in IEEE Int. Conf. Multimedia and Expo, vol. 2, New York, 2000, pp. 1097-1100.
4. A. V. Nefian, L. Liang, X. Pi, X. Liu, C. Mao, and K. Murphy, "A coupled HMM for audio-visual speech recognition," in Proc. Int. Conf. Acoust., Speech, Signal Processing (ICASSP 2002), vol. 2, Orlando, Fla., May 2002, pp. 2013-2016.
5. C. Neti, G. Potamianos, J. Luettin, I. Matthews, H. Glotin, D. Vergyri, J. Sison, A. Mashari, and J. Zhou, "Audio-visual speech recognition," Center for Language and Speech Processing, The Johns Hopkins University, Baltimore, Md., Final Workshop 2000 Report, 2000.
6. M.E. Hennecke, D.G. Stork, and K.V. Prasad, "Visionary speech: Looking ahead to practical speechreading systems," in Speechreading by Humans and Machines, ed. D.G. Stork and M.E. Hennecke, Berlin, Germany: Springer, 1996, pp. 331-349.
7. G. Potamianos and C. Neti, "Improved ROI and within frame discriminant features for lipreading," in Proc. Int. Conf Image Processing (ICIP 2001), vol. 3, Thessaloniki, Greece, 2001, pp. 250-253.
8. P. Viola and M. Jones, "Robust real-time object detection," Cambridge Research Laboratory, Cambridge, U.K., Tech. Report No. CRL2001/01, 2001.
9. R. Lienhart and J. Maydt, "An. extended set of Haar-like features for rapid object detection," in Proc. IEEE Int. Conf. Image Processing (ICIP 2002), vol. 1, 2002, pp. 900-903.
10. G. Boulianne, J. Brousseau, P. Ouellet, and P. Dumouchel, "French large vocabulary recognition with cross-word phonology transducers," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (ICASSP 2000), vol. 3, Istanbul, Turkey, June 5-9, 2000, pp. 1675-1678.
11. G. Shafer, A Mathematical Theory of Evidence, Princeton, N.J.: Princeton University Press, 1976.
12. F. Delmotte and P. Smets, "Target, identification based on the transferable belief model interpretation of Dempster-Shafer model," IEEE Trans. Syst., Man, Cybern. A, vol. 34, July 2004, pp. 457-471.
13. S. Fabre, X. Briottet, and A. Appriou, "Impact of contextual information integration on pixel fusion," IEEE Trans. Geosci. Remote Sensing, vol. 40, no. 9, 2002, pp. 1997-2010.
14. A. Bendjebbour, Y. Delignon, L. Fouque, V. Samson, and W. Pieczynski, "Dempster-Shafer fusion in Markov fields context," IEEE Trans. Geosci. Remote Sensing, vol. 39, no. 8, 2001, pp. 1789-1798.
15. A. Varga et al., "The Noise-92 study on the effect of additive noise on automatic speech recognition," DRA Speech. Research. Unit, Malvern, Worcestershire, U.K., 1999.
16. National Institute of Standards and Technology (NIST), "Tools," Gaithersburg, Md.: NIST, Dec. 20, 2007, http://www.nist.gov/speech/tools.
17. J. Shawe-Taylor and N. Cristianmi, Kernel Methods for Pattern Analysis, New York: Cambridge University Press, 2004.
18. J. Fürnkranz, "Round robin classification," J. Machine Learning Research, vol. 2, Mar. 2002, pp. 721-747.
19. SourceForge, Inc., "Open Computer Vision Library," SourceForge, Inc., 2007, http://sourceforge.net/projects/opencvl.ibrary.
20. M.D. Cordea, E.M. Petriu, N.D. Georganas, D.C. Petriu, and TE. Whalen, "Real-time 2(l/2)-D head pose recovery for model-based video coding," IEEE Trans. Instrum. Meas., vol. 50, no. 4, 2001, pp. 1007-1013.
21. C. Benoît, T. Lallouache, T. Mohamadi, and C Abry, "A set of French, visemes for visual speech synthesis," in Talking Machines: Theories, Models, and Designs, ed. G. Bailly and C. Benoît, Amsterdam: Elsevier, 1992, pp. 485-504.
22. B. Jutras, J.-P. Gagné, M. Picard, and J. Roy, "Identification visuelle et catégorisation de consonnes en français québécois" (Visual, identification and categorization of consonants in Quebec French), Revues d'orthophonie et d'audiologie, vol. 22, no. 2, 1998, pp. 81-87.
23. R. Kneser and H. Ney, "Improved backing-off for m-gram language modeling," in Proc. IEEE Int. Conf Acoust, Speech, Signal Processing (ICASSP 1995), vol. 1, 1995, pp. 181-184.
24. VirtualDub v1.7.8, www.virtualdub.org.
25. P. Walley, "Belief function representation of statistical evidence," The Annals of Statistics, vol. 15, no. 4, 1987, pp. 1439-1465.