Simulating speech with a physics-based facial muscle model

Computer Animation, Conference Proceedings

Volumn 02-04-September-2006, Issue , 2006, Pages 261-270

  Sifakis, Eftychios ^a   Selle, Andrew ^a   Robinson Mosher, Avram ^a   Fedkiw, Ronald ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMATION; INTERACTIVE COMPUTER GRAPHICS;

DATA-DRIVEN APPROACH; FACIAL EXPRESSIONS; IMAGE-BASED TECHNIQUES; MODEL AND SIMULATION; MUSCLE ACTIVATION; PHYSICALLY BASED; THREE-DIMENSIONAL SURFACE; TISSUE DEFORMATIONS;

MUSCLE;

EID: 84981340969     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (65)

1. [Ann06] ANNOSOFT, LLC: Basic phoneset, 2006. Available at http://www.annosoft.com/phoneset.htm.
2. [BB02] BYUN M., BADLER N. I.: FacEMOTE: Qualitative parametric modifiers for facial animations. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2002), ACM Press, pp. 65-71.
3. [BBPV03] BLANZ V., BASSO C., POGGIO T., VETTER T.: Reanimating faces in images and video. In Proc. of Eurographics (2003), vol. 22.
4. [BCS97] BREGLER C., COVELL M., SLANEY M.: Video Rewrite: driving visual speech with audio. In Proc. of ACM SIGGRAPH (1997), pp. 353-360.
5. [BOP98a] BASU S., OLIVER N., PENTLAND A.: 3D lip shapes from video: a combined physical-statistical model. Speech Communication 26 (1998), 131-148.
6. [BOP98b] BASU S., OLIVER N., PENTLAND A.: 3D modeling and tracking of human lip motions. IEEE Comput. Society, pp. 337-343.
7. [Bra99] BRAND M.: Voice puppetry. In Proc. of ACM SIGGRAPH (1999), pp. 21-28.
8. [BSVS04] BLANZ V., SCHERBAUM K., VETTER T., SEIDEL H. P.: Exchanging faces in images. In Proc. of Eurographics (2004), vol. 23.
9. [BTC99] BLACK A., TAYLOR P., CALEY R.: The festival speech synthesis system.
10. [BV99] BLANZ V., VETTER T.: A morphable model for the synthesis of 3D faces. In Proc. of ACM SIGGRAPH (1999), ACM Press, pp. 187-194.
11. [CB05] CHUANG E., BREGLER C.: Mood swings: expressive speech animation. ACM Trans. Graph. 24, 2 (2005), 331-347.
12. [CE05] CHANG Y., EZZAT T.: Transferable videorealistic speech animation. Eurographics/ACM SIGGRAPH Symp. on Comput. Anim. (2005).
13. [CFKP04] CAO Y., FALOUTSOS P., KOHLER E., PIGHIN F.: Real-time speech motion synthesis from recorded motions. In Proc. of 2003 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2004), pp. 347-355.
14. [CFP03] CAO Y., FALOUTSOS P., PIGHIN F.: Unsupervised learning for speech motion editing. In Proc. of the ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2003), pp. 225-231.
15. [CK01] CHOE B., KO H.-S.: Analysis and synthesis of facial expressions with hand-generated muscle actuation basis. In Proc. of Comput. Anim. (2001), pp. 12-19.
16. [CLK01] CHOE B., LEE H., KO H.-S.: Performance-driven muscle-based facial animation. J. Vis. and Comput. Anim. 12 (2001), 67-79.
17. [CM93] COHEN M., MASSARO D.: Modeling coarticulation in synthetic visual speech. Models and Techniques in Comput. Anim. (1993).
18. [CPB-94] CASSELL J., PELACHAUD C., BADLER N., STEEDMAN M., ACHORN B., BECKET T., DOUBILLE B., PREVOST S., STONE M.: Animated conversation: Rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. In Proc. of ACM SIGGRAPH (1994), ACM Press, pp. 413-420.
19. [CVB01] CASSELL J., VILHJÁLMSSON H. H., BICKMORE T.: BEAT: the Behavior Expression Animation Toolkit. In Proc. of ACM SIGGRAPH (2001), pp. 477-486.
20. [CXH03] CHAI J., XIAO J., HODGINS J.: Vision-based control of 3D facial animation. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2003), pp. 193-206.
21. [DLN05] DENG Z., LEWIS J., NEUMANN U.: Synthesizing speech animation by learning compact speech co-articulation models. Comput. Graph. Int. (2005), 19-25.
22. [DMS98] DECARLO D., METAXAS D., STONE M.: An anthropometric face model using variational techniques. In Proc. of ACM SIGGRAPH (1998), ACM Press, pp. 67-74.
23. [EBDP96] ESSA I., BASU S., DARRELL T., PENTLAND A.: Modeling, tracking and interactive animation of faces and heads using input from video. In Proc. of Comput. Anim. (1996), IEEE Comput. Society, pp. 68-79.
24. [EF78] EKMAN P., FRIESEN W. V.: Facial Action Coding System. Consulting Psychologist Press, Palo Alto, 1978.
25. [EGP02] EZZAT T., GEIGER G., POGGIO T.: Trainable videorealistic speech animation. In ACM Trans. Graph. (2002), vol. 21, ACM Press, pp. 388-398.
26. [EP97] ESSA I., PENTLAND A.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. on Pattern Analysis and Machine Intelligence 19, 7 (1997), 757-763.
27. [EP00] EZZAT T., POGGIO T.: Visual speech synthesis by morphing visemes. In Int. J. Comp. Vision (2000), vol. 38, pp. 45-37.
28. [GGW-98] GUENTER B., GRIMM C., WOOD D., MALVAR H., PIGHIN F.: Making faces. In Proc. ACM SIGGRAPH (1998), ACM Press, pp. 55-66.
29. [JTDP03] JOSHI P., TIEN W. C., DESBRUN M., PIGHIN F.: Learning controls for blend shape based realistic facial animation. In Proc. ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2003), pp. 365-373.
30. [KGB98] KOCH R., GROSS M., BOSSHARD A.: Emotion editing using finite elements. Proc. of Eurographics 1998 17, 3 (1998).
31. [KGC-96] KOCH R. M., GROSS M. H., CARLS F. R., VON BUREN D. F., FANKHAUSER G., PARISH Y. I. H.: Simulating facial surgery using finite element models. Comput. Graph. 30, Annual Conf. Series (1996), 421-428.
32. [KHS01] KAHLER K., HABER J., SEIDEL H.-P.: Geometrybased muscle modeling for facial animation. In Proc. of Graph. Interface (2001), pp. 37-46.
33. [KHS03] KAHLER K., HABER J., SEIDEL H.-P.: Reanimating the dead: Reconstruction of expressive faces from skull data. In ACM Trans. Graph. (2003), vol. 22, pp. 554-561.
34. [KHYS02] KAHLER K., HABER J., YAMAUCHI H., SEIDEL H.-P.: Head shop: Generating animated head models with anatomical structure. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2002), pp. 55-63.
35. [KMMTT92] KALRA P., MANGILI A., MAGNETATTHALMANN N., THALMANN D.: Simulation of facial muscle actions based on rational free form deformations. In Proc. of Eurographics (1992), pp. 59-69.
36. [KMT03] KSHIRSAGAR S., MAGNENAT-THALMANN N.: Visyllable based speech animation. In Proc. of Eurographics (2003), vol. 22.
37. [KP05] KING S., PARENT R.: Creating speech-synchronized animation. IEEE Transactions on Visualization and Computer Graphics 11, 3 (2005), 341-352.
38. [LM99] LUCERO J., MUNHALL K.: A model of facial biomechanics for speech production. Journal of the Accoustical Society of America 106, 5 (1999), 2834-2842.
39. [LMVB-97] LUCERO J., MUNHALL K., VATIKIOTIS-BATESON E., GRACCO V., TERZOPOULOS D.: Muscle-based modeling of facial dynamics during speech production. Journal of the Acoustical Society of America 101, 5 (May 1997), 3175-3176.
40. [LTW95] LEE Y., TERZOPOULOS D., WATERS K.: Realistic modeling for facial animation. Comput. Graph. (SIGGRAPH Proc.) (1995), 55-62.
41. [MIT98] MORISHIMA S., ISHIKAWA T., TERZOPOULOS D.: Facial muscle parameter decision from 2D frontal image. In Proc. of the Int. Conf. on Pattern Recognition (1998), vol. 1, pp. 160-162.
42. [MTPT88] MAGNENAT-THALMANN N., PRIMEAU E., THALMANN D.: Abstract muscle action procedures for human face animation. The Vis. Comput. 3, 5 (1988), 290-297.
43. [NJ04] NA K., JUNG M.: Hierarchical retargetting of fine facial motions. In Proc. of Eurographics (2004), vol. 23.
44. [NN01] NOH J., NEUMANN U.: Expression cloning. In Proc. of ACM SIGGRAPH (2001), Fiume E., (Ed.), ACM Press, pp. 277-288.
45. [Par72] PARKE F. I.: Computer generated animation of faces. In Proc. of ACM Conf. (1972), ACM Press, pp. 451-457.
46. [PB81] PLATT S. M., BADLER N. I.: Animating facial expressions. Comput. Graph. (SIGGRAPH Proc.) 1981 245-252.
47. [PHL-98] PIGHIN F., HECKER J., LISCHINSKI D., SZELISKI R., SALESIN D. H.: Synthesizing realistic facial expressions from photographs. In Proc. of ACM SIGGRAPH (1998), ACM Press, pp. 75-84.
48. [PKC-03] PYUN H., KIM Y., CHAE W., KANG H. W., SHIN S. Y.: An example-based approach for facial expression cloning. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2003), pp. 167-176.
49. [PLB-05] PIGHIN F., LEWIS J., BORSHUKOV G., BENNETT D., DEBEVEC P., HERY C., SULLIVAN S., WILLIAMS L., ZHANG L.: Digital face cloning. In SIGGRAPH Course Notes (2005), ACM.
50. [PSS99] PIGHIN F., SZELISKI R., SALESIN D.: Resynthesizing facial animation through 3D model-based tracking. In Proc. of Int. Conf. on Comput. Vision (1999), pp. 143-150.
51. [RE01] REVERET L., ESSA I.: Visual coding and tracking of speech related facial motion. In Proc. of IEEE CVPR Int. Wrkshp. on Cues in Communication (2001).
52. [RGTC98] ROTH S. H., GROSS M., TURELLO M. H., CARLS S.: A Bernstein-Bézier based approach to soft tissue simulation. In Proc. of Eurographics (1998), vol. 17, pp. 285-294.
53. [SNF05] SIFAKIS E., NEVEROV I., FEDKIW R.: Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. (SIGGRAPH Proc.) (2005).
54. [SP04] SUMNER R., POPOVÍC J.: Deformation transfer for triangle meshes. In ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) (2004), vol. 23, pp. 399-405.
55. [TSIF05] TERAN J., SIFAKIS E., IRVING G., FEDKIW R.: Robust quasistatic finite elements and flesh simulation. Proc. of the 2005 ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2005).
56. [TW90] TERZOPOULOS D., WATERS K.: Physically-based facial modeling, analysis, and animation. J. Vis. and Comput. Anim. 1 (1990), 73-80.
57. [TW93] TERZOPOULOS D., WATERS K.: Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. on Pattern Analysis and Machine Intelligence 15, 6 (1993).
58. [VBMH-96] VATIKIOTIS-BATESON E., MUNHALL K., HIRAYAMA M., LEE Y., TERZOPOULOS D.: Dynamics of facial motion in speech: Kinematic and electromyographic studies of orofacial structures. In Speechreading by Humans and Machines, vol. 150 of NATO ASI Series on Computer and System Sciences. Springer-Verlag, March 1996, ch. 16, pp. 231-232.
59. [VBPP05] VLASIC D., BRAND M., PFISTER H., POPOVÍC J.: Face transfer with multilinear models. In ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) (2005), vol. 24, pp. 426-433.
60. [Wat87] WATERS K.: A muscle model for animating threedimensional facial expressions. Comput. Graph. (SIGGRAPH Proc.) (1987), 17-24.
61. [WF95] WATERS K., FRISBIE J.: A coordinated muscle model for speech animation. In Proc. of Graph. Interface (May 1995), pp. 163-170.
62. [WHL-04] WANG Y., HUANG X., LEE C. S., ZHANG S., LI Z., SAMARAS D., METAXAS D., ELGAMMAL A., HUANG P.: High resolution acquisition, learning and transfer of dynamic 3-D facial expressions. In Proc. of Eurographics (2004), pp. 677-686.
63. [Wil90] WILLIAMS L.: Performance-driven facial animation. In Comput. Graph. (Proc. of Int. Conf. on Comput. Graph. and Int. Techniques) (1990), ACM Press, pp. 235-242.
64. [ZLGS03] ZHANG Q., LIU Z., GUO B., SHUM H.: Geometrydriven photorealistic facial expression synthesis. In Proc. of ACM SIGGRAPH/Eurographics Symp. on Comput. Anim. (2003), ACM Press, pp. 16-22.
65. [ZSCS04] ZHANG L., SNAVELY N., CURLESS B., SEITZ S.: Spacetime faces: High resolution capture for modeling and animation. In ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) (2004), vol. 23, ACM Press, pp. 548-558.