Models for Audiovisual Fusion in a Noisy-Vowel Recognition Task

Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology

Volumn 20, Issue 1-2, 1998, Pages 25-44

  Teissier, Pascal ^a,b   Guerin Dugue, Anne ^a   Schwartz, Jean Luc ^a,c,d,e,f,g  

^a GRENOBLE INSTITUTE OF TECHNOLOGY   (France)

^b Institut de la Communication Parlee   (France)

^c ECOLE NORMALE SUPÉRIEURE   (France)

^d Universite d'Orsay   (France)

^e ICP in Grenoble   (France)

^f CNRS   (France)

^g Bureau of the French Audition Group   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; MATHEMATICAL MODELS; SIGNAL TO NOISE RATIO; COMPUTER ARCHITECTURE; DATA ACQUISITION; NONLINEAR SYSTEMS;

AUDIOVISUAL FUSION; NONLINEAR DIMENSION REDUCTION;

SPEECH RECOGNITION;

AUDIOVISUAL (AV) FUSION; DIRECT IDENTIFICATION (DI); NOISY-VOWEL RECOGNITION TASK; NONLINEAR DIMENSION REDUCTION ALGORITHMS;

EID: 0032179738     PISSN: 13875485     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (24)

1. C. Benoit, T. Mohamadi, and S.D. Kandel, "Effects of phonetic context on audio-visual intelligibility of French," Journal of Speech and Hearing Research, Vol. 37, pp. 1185-1203, 1994.
2. N.P. Erber, "Auditory-visual perception of speech," Journal of Speech and Hearing Disorders, Vol. 40, pp. 481-492, 1975.
3. W.H. Sumby and I. Pollack, "Visual contribution to speech intelligibility in noise," J. Acoust. Soc. Am., Vol. 26, pp. 212-215, 1954.
4. E.D. Petajan, "Automatic lipreading to enhance speech recognition," Doctoral thesis, University of Illinois, 1984.
5. D.G. Stork and M.E. Hennecke (Eds.), Speechreading by Man and Machine: Models, Systems and Applications, NATO ASI Series, Springer, 1996.
6. J. Robert-Ribes, M. Piquemal, J.L. Schwartz, and P. Escudier, "Exploiting sensor fusion architectures and stimuli complementarity in AV speech recognition," in Speechreading by Man and Machine: Models, Systems and Applications, D.G. Stork and M.E. Hennecke (Eds.), NATO ASI Series, Springer, pp. 193-210, 1996.
7. P. Teissier, J. Robert-Ribes, J.L. Schwartz, and A. Guerin-Dugue, "Comparing models for audiovisual fusion in a noisyvowel recognition task," IEEE Trans. Speech and Audio Processing, submitted.
8. B.V. Dasarathy, Decision Fusion, IEEE Computer Society Press, Los Alamitos, CA, 1994.
9. J.L. Schwartz, J. Robert-Ribes, and P. Escudier, "Ten years after Summerfield... a taxinomy of models for audiovisual fusion in speech perception," in Hearing by Eye, II. Perspectives and Directions in Research on Audiovisual Aspects of Language Processing, R. Campbell, B. Dodd, and D. Burnham (Eds.), Erlbaum/Psychology Press, 1998.
10. A. Liberman and I. Mattingly, "The motor theory of speech perception revised," Cognition, Vol. 21, pp. 1-36, 1985.
11. I. Bloch, "Information combination operators for data fusion: A comparative review with classification," IEEE Trans, on SMC, A, Vol. 26, pp. 52-67, 1996.
12. M.T. Lallouache, "Un poste 'visage-parole'. Acquisition et traitement de contours labiaux," Proceedings of the XVIII Journes d'Etudes sur la Parole, Montral, pp. 282-286, 1990.
13. K. Fukunaga, Introduction to Statistical Pattern recognition, 2nd edition, Academic Press Inc., San Diego, 1990.
14. A. Guerin-Dugue, C. Aviles-Cruz, and P.M. Palagi, "Interpreting data through neural and statistical tools," ESANN'96, Bruges, pp. 229-236, 1996.
15. Calliope, La Parole et son Traitement Automatique, J.P. Tubach (Ed.), Masson, Paris, 1989.
16. J.W. Sammon, "A nonlinear mapping for data structure analysis," IEEE Trans. on Computers, Vol. 18, pp. 401-409, 1969.
17. T. Kohonen, "The self-organizing maps," Proc. of the IEEE, Vol. 78, pp. 1481-1497, 1990.
18. P. Demartines and J. Herault, "Curvilinear component analysis: A self-organizing neural network for nonlinear mapping of data sets," IEEE Trans, on Neural Networks, Vol. 8, pp. 148-154, 1997.
19. A. Guerin-Dugue, P. Teissier, J.L. Schwartz, and J. Herault, "Nonlinear representation for audio-visual fusion in a noisyvowel recognition task," NEURAP'97, Marseille, pp. 31-40, 1997.
20. J.R. Movellan and P. Mineiro, "Modularity and catastrophic fusion: A bayesian approach with application to audiovisual speech recognition," Technical report CogSci. USCD-97.01, Department of Cognitive Science, USCD, San Diego, CA, 92093-0515, 1996.
21. A. Adjoudani and C. Benoit, "On the integration of auditory and visual parameters in an HMM-based ASR," in Speechreading by Man and Machine: Models, Systems and Applications, D.G. Stork and M.E. Hennecke (Eds.), NATO ASI Series, Springer, pp. 461-472, 1996.
22. C. Bregler, H. Hild, S. Manke, and A. Waibel, "Improving connected letter recognition by lipreading," International Joint Conf. of Speech and Signal Processing, Minneapolis, pp. 557-560, 1993.
23. U. Meier, W. Hurst, and P. Duchnowski, "Adaptive bimodal sensor fusion for automatic speechreading," ICASSP, 1996.
24. A. Guerin-Dugue, P. Teissier, J.L. Schwartz, and J. Herault, "Constrained neural network for estimating sensor reliability in sensors fusion," IWANN'97, Lanzarotte, pp. 872-881, 1997.