Modeling and animating realistic faces from images

International Journal of Computer Vision

Volumn 50, Issue 2, 2002, Pages 143-169

  Pighin, Frideric ^a   Szeliski, Richard ^b   Salesin, David H ^b  

^a University of Southern California ^*  (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Analysis by synthesis; Facial animation; Image based modeling; Model based tracking; Performance driven animation

Indexed keywords

ALGORITHMS; ANIMATION; COMPUTATIONAL GEOMETRY; COMPUTER SIMULATION; FACE RECOGNITION; FINITE DIFFERENCE METHOD; IMAGE ANALYSIS; INTERACTIVE COMPUTER GRAPHICS; MOTION ESTIMATION; OPTIMIZATION; PHOTOGRAMMETRY; THREE DIMENSIONAL COMPUTER GRAPHICS;

FACIAL ANIMATION; IMAGE BASED MODELING; IMAGE SYNTHESIS; MODEL BASED TRACKING; PERFORMANCE DRIVEN SIMULATION; TEXTURE MAPPING;

COMPUTER VISION;

EID: 0036844217     PISSN: 09205691     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1020393915769     Document Type: Article

References (67)

1. Akimoto, T., Suenaga, Y., and Wallace, R. 1993. Automatic creation of 3D facial models. IEEE Computer Graphics and Applications, 13(4):16-22.
2. Ali. 1995. Alias V7.0.
3. Anjyo, K., Usami, Y., and Kurihara, T. 1992. A simple method for extracting the natural beauty of hair. In SIGGRAPH 92 Conference Proceedings, ACM SIGGRAPH, pp. 111-120.
4. Ballard, D. and Brown, C. 1982. Computer Vision. Prentice-Hall: Englewood Cliffs, NJ.
5. Black, M. and Yacoob, Y. 1995. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image otions. In Proceedings, International Conference on Computer Vision, pp. 374-381.
6. Blake, A. and Isard, M. 1998. Active Contours: The Application of Techniques from Graphics, Vision, Control Theory and Statistics to Visual Tracking of Shapes in Motion. Addision Wesley: Reading, MA.
7. Blanz, T., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In SIGGRAPH 99 Conference Proceedings, ACM SIGGRAPH.
8. Bregler, C., Covell, M., and Slaney, M. 1997. Video rewrite: Driving visual speech with audio. In SIGGRAPH 97 Conference Proceedings, ACM SIGGRAPH, pp. 353-360.
9. Bright Start Technologies Inc. 1993. In Beginning Reading Software, Sierra On-Line, Inc.
10. Cascia, M.L., Isidoro, J., and Sclaroff, S. 1998. Head tracking via robust registration in texture map images. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
11. Chen, D., State, A., and Banks, D. 1995. Interactive shape metamorphosis. In 1995 Symposium on Interactive 3D Graphics, ACM SIGGRAPH, pp. 43-44.
12. Choi, C., Aizawa, K., Harshima, H., and Takebe, T. 1994. Analysis and synthesis of facial image sequences in model-based image coding. IEEE Transactions on Circuits and Systems for Video Technology, 4(3):257-274.
13. Choi, C.S., Kiyoharu, Harashima, H., and Takebe, T. 1994. Analysis and synthesis of facial image sequences in model-based image coding. IEEE Transactions on Circuits and Systems for Video Technology, 4:257-275.
14. Covell, M. 1996. Eigen-points: Control-point location using principal component analysis. In Proceedings, Second International Conference on Automatic Face and Gesture Recognition, pp. 122-127.
15. Cyb. 1990. 4020/RGB 3D Scanner with Color Digitizer.
16. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In SIGGRAPH 2000 Conference Proceedings, ACM SIGGRAPH. pp. 35-42.
17. Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIGGRAPH 96 Conference Proceedings. ACM SIGGRAPH, pp. 11-20.
18. Decarlo, D. and Metaxas, D. 1998. Deformable model-based shape and motion analysis from images using motion residual error. In Proceedings, First International Conference on Computer Vision, pp. 113-119.
19. Devernay, F. and Faugeras, O. 1994. Computing differential properties of 3d shapes from stereoscopic images without 3d models. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition, pp. 208-213.
20. Edwards, G., Taylor, C., and Cootes, T. 1998. Interpreting face images using active appearance models. In Proceedings, Third Workshop on Face and Gesture Recognition, pp. 300-305.
21. Essa, I. and Pentland, A. 1997. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):757-763.
22. Faugeras, O. 1993. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press: Cambridge, MA.
23. Fua, P. and Miccio, C. 1998. From regular images to animated heads: A least square approach. In Proceedings, European Conference on Computer Vision, pp. 188-202.
24. Golub, G. and van Loan, C. 1996. Matrix Computation, 3rd edn. The John Hopkins University Press: Baltimore/London.
25. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The Lumigraph. in SIGGRAPH 96 Conference Proceedings, ACM SIGGRAPH, pp. 43-54.
26. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. in SIGGRAPH 98 Conference Proceedings. ACM SIGGRAPH, pp. 55-66.
27. Hager, G., and Belhumeur, P. 1996. Real-time tracking of image regions with changes in geometry and illumination. in Proceedings, Computer Vision and Pattern Recognition, pp. 403-410.
28. Hallinan, P. 1994. A low-dimensional representation of human faces for arbitrary lighting conditions. in Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. pp. 58-66.
29. Hanrahan, P. and Kruger, W. 1993, Reflection from layered surfaces due to surface scattering. in SIGGRAPH 93 Conference Proceedings, pp. 165-174.
30. Ip, H. and Yin, L. 1996. Constructing a 3D individualized head model from two orthogonal views. The Visual Computer, 12:254-266.
31. Jones, M. and Poggio, T. 1998. Hierarchical morphable models. in Proceedings, International Conference on Computer Vision, pp. 820-826.
32. Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. in Proceedings, First International Conference on Computer Vision, pp. 259-268.
33. Koch, R., Gross, M., Carls, F.R., von Büren, D., Fankhauser, G., and Parish, Y. 1996. Simulating facial surgery using finite element methods. in SIGGRAPH 96 Conference Proceedings, ACM SIGGRAPH, pp. 421-428.
34. Kurihara, T. and Arai, K. 1991 A transformation method for modeling and animation of the human face from photographs. in Computer Animation, Vol. 91, N.M. Thalmann and D. Thalmann (Eds.), Springer-Verlag, Tokyo, pp. 45-58.
35. Lanitis, A., Taylor, C., and Cootes, T. 1995. A unified approach for coding and interpreting face images. in Fifth International Conference on Computer Vision (ICCV 95), Cambridge, Massachusetts, pp. 368-373.
36. Lawson, C. and Hansen, R. 1974. Solving Least Squares Problems. Prentice-Hall: Englewood Cliffs.
37. Leclerc, Y. and Bobick, A. 1991. The direct computation of height from shading. in Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
38. Lee, S., Chwa, K., Shin, S., and Wolberg, G. 1995. Image metamorphosis using snakes and free-form deformations. in SIGGRAPH 95 Conference Proceedings, ACM SIGGRAPH, pp. 439-448.
39. Lee, S., Wolberg, G., Chwa, K., and Shin, S. 1996. Image metamorphosis with scattered feature constraints. in IEEE Transactions on Visualization and Computer Graphics. 2(4):337-354.
40. Lee, Y., Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. in SIGGRAPH 95 Conference Proceedings, ACM SIGGRAPH, pp. 55-62.
41. Lengagne, R., Fua, P., and Monga, O. 1998. 3d face modeling using stereo and differential constraints. In Proceedings of the Third International Conference on Automatic Face and Gesture Recognition.
42. Li, H., Roivainen, P., and Forschheimer, R. 1993. 3-d motion estimation in model-based facial image coding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6):545-555.
43. Matsino, K., Lee, C., Kimura, S., and Tsuji, S. 1995. Automatic recognition of human facial expressions. In Proceedings of the IEEE. pp. 352-359.
44. Moffitt, F., and Mikhail, E. 1980, Photogrammetry, 3rd Edn. Harper & Row, New York.
45. Nielson, G. 1993. Scattered data modeling. IEEE Computer Graphics and Applications, 13(1):60-70.
46. Ostby, E. Pixar animation studios: 1997. Personal communication.
47. Parke, F. 1972. Computer generated animation of faces. In Proceedings ACM annual conference.
48. Parke, F. 1974. A Parametric Model for Human Faces. PhD Thesis, University of Utah, Salt Lake City, Utah, UTEC-CSc-75-047.
49. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. 1998. Synthesizing realistic facial expression from photographs. in SIGGRAPH 98 Conference Proceedings, ACM SIGGRAPH, pp. 75-84.
50. Pighin, F., Szeliski, R., and Salesin, D. 1999. Resynthesizing facial animation through 3d model-based tracking. in Proceedings, International Conference on Computer Vision.
51. Press, W., Flannery, B., Teukolsky, S., and Vettering, W. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd. edn. Cambridge University Press: Cambridge, England.
52. Proesman, M., Gool, L.V., and Osterlinck, A. 1996. Active acuisition of 3d shape for moving objects. International Conference on Image Processing.
53. Pulli, K., Cohen, M., Duchamp, T., Hoppe, H., Shapiro, L., and Stuetzle, W. 1997. View-based rendering: Visualizing real objects from scanned range and color data. In Proc. 8th Eurographics Workshop on Rendering.
54. Rosenblum, R., Carlson, W., and III, E.T. 1991. Physically-based facial modeling, analysis, and animation. Journal of Visualization and Computer Animation, 2(4):141-148.
55. Schodl, A., Ario, A., and Essa, I. 1998. Head tracking using a textured polygonal model. In Workshop on Perceptual User Interfaces, pp. 43-48.
56. Slama, C. (Ed.). 1980 In Manual of Photogrammetry, 4th edn., American Society of Photogrammetry, Falls Church, Virginia.
57. Szeliski, R. and Kang, S. 1994. Recovering 3D shape and motion from image streams using nonlinear least squares. Journal of Visual Communication and Image Representation, 5(1):10-28.
58. Szeliski, R. and Shum, H. 1997. Creating full view panoramic image mosaics and texture-mapped models. In SIGGRAPH 97 Conference Proceedings, ACM SIGGRAPH, pp. 251-258.
59. Terzopoulos, D. and Waters, K. 1993. Analysis and synthesis of facial image sequences using physical an anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6):569-579.
60. Thomas, F., Johnston, O., and Johnston, C. 1995. The Illusion of Life. Hyperion.
61. Thórisson, K. 1997. Gandalf: An embodied humanoid capable of real-time multimodal dialogue with people. In First ACM International Conference on Autonomous Agents.
62. Turk, M. and Pentland, A. 1987. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71-87.
63. Vanier, M., Marsh, J., and Warren, J.O. 1983. Three-dimentional computer graphics for craniofacial surgical planning and evaluation. In SIGGRAPH 83 Conference Proceedings, Vol. 17, ACM SIGGRAPH, pp. 263-273.
64. Vetter, T. and Blanz, V. 1998. Estimating coloured 3d face models from single images: An example based approach. In Proceedings, European Conference on Computer Vision, pp. 499-513.
65. Watanabe, Y. and Suenaga, Y. 1992. A trigonal prism-based method for hair image generation. IEEE Computer Graphics and Applications, 12(1):47-53.
66. Williams, L. 1990. Performance-driven facial animation. In SIGGRAPH 90 Conference Proceedings, Vol. 24. pp. 235-242.
67. Yu, Y. and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. In SIGGRAPH 98 Conference Proceedings, ACM SIGGRAPH, pp. 207-217.