Photo-real talking head with deep bidirectional LSTM

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2015-August, Issue , 2015, Pages 4884-4888

  Fan, Bo ^a,b   Wang, Lijuan ^b   Soong, Frank K ^b   Xie, Lei ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

AAM; BLSTM; RNN; talking head

Indexed keywords

AUDIO SIGNAL PROCESSING; REGRESSION ANALYSIS; SPEECH COMMUNICATION; STEREO IMAGE PROCESSING;

ACTIVE APPEARANCE MODELS; BLSTM; LONG-RANGE DEPENDENCIES; OBJECTIVE MEASUREMENT; RECURRENT NEURAL NETWORK (RNN); SUM OF SQUARE ERRORS; TALKING HEADS; TEMPORAL SEQUENCES;

LONG SHORT-TERM MEMORY;

EID: 84946029513     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2015.7178899     Document Type: Conference Paper

References (22)

1. C. Bregler, M. Covell, and M. Slaney, Video rewrite: driving visual speech with audio, in Proc. ACM SIGGGRAPH 97, 1997, pp. 353-360
2. K. Liu and J. Ostermann, Realistic facial animation system for interactive services, in Proc. Interspeech, 2008, pp. 2330-2333
3. L.J. Wang, W. Han, and F. Soong, High quality lip-sync animation for 3D photo-realistic talking head, in Proc. ICASSP. IEEE, 2012, pp. 4529-4532
4. S. Sako, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, HMM-based text-to-audio-visual speech synthesis., in Proc. Interspeech, 2000, pp. 25-28
5. S. Fu, R. Gutierrez-Osuna, A. Esposito, P. Kakumanu, and O. Garcia, Audio/visual mapping with cross-modal hidden Markov models, IEEE Transactions on Multimedia, vol. 7, no. 2, pp. 243-252, 2005
6. L. Xie and Z.Q. Liu, Realistic mouth-synching for speechdriven talking face using articulatory modelling, IEEE Transactions on Multimedia, vol. 9, no. 3, pp. 500-510, 2007
7. L. Xie and Z.Q. Liu, A coupled HMM approach to videorealistic speech animation, Pattern Recognition, vol. 40, no. 8, pp. 2325-2340, 2007
8. L.J. Wang and F. Soong, HMM trajectory-guided sample selection for photo-realistic talking head, Multimedia Tools and Applications, pp. 1-21, 2014
9. L.J.Wang, Y.J.Wu, X.D. Zhuang, and F. Soong, Synthesizing visual speech trajectory with minimum generation error, in Proc. ICASSP. IEEE, 2011, pp. 4580-4583
10. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingsbury, Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012
11. H. Zen, A. Senior, and M. Schuster, Statistical parametric speech synthesis using deep neural networks, in Proc. ICASSP. IEEE, 2013, pp. 7962-7966
12. S. Mike and K. Paliwal, Bidirectional recurrent neural networks, IEEE Transactions on Signal Processing, vol. 45, no. 6, pp. 2673-2681, 1997
13. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997
14. L. Xie, N.C. Sun, and B. Fan, A statistical parametric approach to video-realistic text-driven talking avatar, Multimedia Tools and Applications, vol. 73, no. 1, pp. 377-396, 2014
15. T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appearance models, IEEE Transactions on pattern analysis and machine intelligence, vol. 23, no. 6, pp. 681-685, 2001
16. Q.Wang,W.W. Zhang, X.O. Tang, and H.Y. Shum, Real-time Bayesian 3-d pose tracking, IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 12, pp. 1533-1541, 2006
17. S. Roweis, EM algorithms for PCA and SPCA, Advances in neural information processing systems, pp. 626-632, 1998
18. A. Graves, Supervised sequence labelling with recurrent neural networks, vol. 385, Springer, 2012
19. R. J. Williams and D. Zipser, Gradient-based learning algorithms for recurrent networks and their computational complexity, Back-propagation: Theory, architectures and applications, pp. 433-486, 1995
20. P. J. Werbos, Consistency of HDP applied to a simple reinforcement learning problem, Neural networks, vol. 3, no. 2, pp. 179-189, 1990
21. J. Kominek and A. W. Black, The CMU Arctic speech databases, in Fifth ISCAWorkshop on Speech Synthesis, 2004
22. http://research.microsoft.com/en-us/ projects/blstmtalkinghead