Learning dynamic audio-visual mapping with input-output hidden markov models

IEEE Transactions on Multimedia

Volumn 8, Issue 3, 2006, Pages 542-549

  Li, Yan ^b   Shum, Heung Yeung ^a,c  

^a IEEE

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c MICROSOFT RESEARCH ASIA   (China)

Author keywords

Animation; Audio visual mapping; HMM; IOHMM; Learning

Indexed keywords

ALGORITHMS; ANIMATION; COMPUTER ARCHITECTURE; MARKOV PROCESSES; MAXIMUM LIKELIHOOD ESTIMATION; NEURAL NETWORKS; PROBABILITY;

AUDIO-VISUAL MAPPING; HMM; IOHMM; LEARNING; MARKOV MODEL;

MULTIMEDIA SYSTEMS;

EID: 33646752807     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2006.870732     Document Type: Article

References (21)

1. T. Chen and R. Rao, "Audio-visual integration in multimodal communication," Proc. IEEE, vol. 86, no. 5, pp. 837-852, May 1998.
2. C. Bregler, M. Covell, and M. Slaney, "Video rewrite: driving visual speech with audio," in Proc. SIGGRAPH97, 1997, pp. 353-360.
3. K. H. Choi and J. N. Hwang, "Baum-welch hidden Markov model inversion for reliable audio-to-visual conversion," in Proc. 3rd IEEE Workshop on Multimedia Signal Processing, 1999, pp. 175-180.
4. M. Brand, "Voice puppetry," in Proc. SIGGRAPH99, 1999, pp. 21-28.
5. B. Guenter, C. Grimm, D. Wood, H. Malvar, and F. Pighin, "Making faces," in Proc. SIGGRAPH'98, 1998, pp. 55-66.
6. S. Levinson, L. Rabiner, and M. Sondhi, "An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition," Bell Syst. Tech. J., vol. 64, no. 4, pp. 1035-1074, 1983.
7. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 76, no. 2, pp. 257-286, Feb. 1989.
8. Y. Bengio and P. Frasconi, "Input/output HMMs for sequence processing," IEEE Trans. Neural Netw., vol. 3, pp. 1231-1249, 1996.
9. Y. Bengio, "Markovian models for sequential data," Neural Comput. Surv., vol. 2, pp. 129-162, 1999.
10. J. Pearl, Probabilistic Reasoning in Intelligent System: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
11. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum-likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc. B, vol. 39, pp. 1-38, 1977.
12. [Online]. Available: http://dynamo.ecn.purdue.edu/~boumari/software/ cluster
13. T. Beier and S. Neely, "Feature-based image metamorphosis," in Proc. ACM SIGGRAPH'92, 1992, pp. 35-42.
14. G. Wolberg, Digital Image Warping. Los Alamitos, CA: IEEE Comput. Soc. Press, 1994.
15. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
16. V. Pavlović, J. M. Rehg, T. J. Cham, and K. P. Murphy, "A dynamic Bayesian network approach to figure tracking using learned dynamic models," in IEEE Int. Conf. Computer Vision, 1999.
17. V. Pavlović, J. M. Rehg, and J. MacCormick, "Impact of dynamic model learning on classification of human motion," in IEEE Int. Conf. Computer Vision and. Pattern Recognition, 2000.
18. B. North, A. Blake, M. Isard, and J. Rittscher, "Learning and classification of complex dynamics," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 9, pp. 1016-1034, Sep. 2000.
19. S. Soatto, G. Doretto, and Y. N. Wu, "Dynamic textures," in Proc. IEEE Int. Conf. Computer Vision, 2001, pp. 439-446.
20. A. W. Fitzgibbon, "Stochastic rigidity: Image registration for nowhere-static scenes," in Proc. IEEE Int. Conf. Computer Vision, 2001, pp. 662-669.
21. Y. Li, T. Wang, and H. Shum, "Motion texture: a two-level statistical model for character motion synthesis," ACM Trans. Graph. (Siggraph), vol. 22, no. 3, pp. 465-472, 2002.