Detecting local audio-visual synchrony in monologues utilizing vocal pitch and facial landmark trajectories

British Machine Vision Conference, BMVC 2009 - Proceedings

Volumn , Issue , 2009, Pages

  Cadavid, Steven ^a   Abdel Mottaleb, Mohamed ^a   Messinger, Daniel S ^b   Mahoor, Mohammad H ^c   Bahrick, Lorraine E ^d  

^a University of Miami College of Engineering   (United States)

^b Department of Biology   (United States)

^c University of Denver   (United States)

^d FLORIDA INTERNATIONAL UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FACE RECOGNITION; VIDEO CAMERAS;

ACTIVE SHAPE MODEL; ASYNCHRONOUS SEQUENCE; FUNDAMENTAL FREQUENCIES; HORIZONTAL AND VERTICAL DISPLACEMENT; HUMAN VISUAL SYSTEM; MUTUAL INFORMATIONS; STATISTICAL MODELING; VISUAL COMPONENTS;

COMPUTER VISION;

EID: 84898841607     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.5244/C.23.10     Document Type: Conference Paper

References (20)

1. J. P. Barker and F. Berthommier. Evidence of correlation between acoustic and visual features of speech. In Proceedings of the 14th International Congress of Phonetic Sciences, pages 199-202, June 1999.
2. Z. Barzelay and Y.Y. Schechner. Harmonyin motion. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8, June 2007. doi:10.1109/CVPR.2007. 383344.
3. H. Bredin, A. Miguel, I.H. Witten, and G. Chollet. Detecting replay attacks in audiovisual identity verification. In IEEE International Conference on Acoustics, Speech and Signal Processing, Volume 1, pages 621-624, May 2006.
4. C.C. Chibelushi, F. Deravi, and J.S.D. Mason. A review of speech-based bimodal recognition. IEEE Transactions on Multimedia, 4(1):23-37, March 2002.
5. T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. Active shape models-their training and application. Computer Vision and Image Understanding, 61(1):38-59, 1995.
6. R. Cutler and L. Davis. Look who's talking: speaker detection using video and audio correlation. In IEEE International Conference on Multimedia and Expo, Volume 3, pages 1589-1592, 2000.
7. N. Eveno and L. Besacier. Co-inertia analysis for "liveness™ test in audio-visual biometrics. In Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, pages 257-261, September 2005.
8. J.W. Fisher III and T. Darrell. Speaker association with signal-level audiovisual fusion. IEEE Transactions on Multimedia, 6(3):406-413, June 2004.
9. R. Goecke and B. Millar. Statistical analysis of the relationship between audio and video speech parameters for australian english. In Proceedings of the ISCA Tutorial and Research Workshop on Audio Visual Speech Processing, pages 133-138, Saint - Jorioz, France, September 2003.
10. D. J. Hermes. Measurement of pitch by subharmonic summation. Journal of Acoustic Society of America, 83:257-264, 1988.
11. J. Hershey and J. Movellan. Using audio-visual synchrony to locate sounds. Advances in Neural Information Processing Systems 12, pages 813-819, 1999.
12. G. Iyengar, H.J. Nock, and C. Neti. Audio-visual synchrony for detection of monologues in video archives. In International Conference on Multimedia and Expo, Volume 1, pages 329-332, July 2003.
13. E. Kidron, Y.Y. Schechner, and M. Elad. Pixels that sound. In IEEE Conference on Computer Vision and Pattern Recognition, Volume 1, pages 88-95, June 2005. doi:10.1109/CVPR.2005.274.
14. J. B. Macqueen. Some methods of classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, pages 281-297, 1967.
15. M. H. Mahoor and M. Abdel-Mottaleb. Facial features extraction in color images using enhanced active shape model. In Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pages 144-148, Washington, DC, USA, 2006. IEEE Computer Society.
16. H. J. Nock, G. Iyengar, and C. Neti. Assessing face and speech consistency for monologue detection in video. In Proceedings of the tenth ACM international conference on Multimedia, pages 303-306, Juanles-Pins, France, 2002. ACM.
17. C. G. Prince and G. J. Hollich. Synching models with infants: a perceptual-level model of infant audio-visual synchrony detection. Cognitive Systems Research, 6(3):205-228, 2005.
18. D. A. Reynolds, T. F. Quatieri, and R. B. Dunn. Speaker verification using adapted gaussian mixture models. Digital Signal Processing, 10(1-3):19-41, January 2000.
19. N. Sugamura and F. Itakura. Speech analysis and synthesis methods developed at ecl in ntt - from lpc to lsp. Speech Commununication, 5(2):199-215, 1986.
20. X. Sun. A pitch determination algorithm based on subharmonic-to-harmonic ratio. In the 6th International Conference of Spoken Language Processing, pages 676-679, 2000.