M-Face: An appearance-based photorealistic model for multiple facial attributes rendering

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 16, Issue 7, 2006, Pages 830-842

  Fu, Yun ^b   Zheng, Nanning ^a,c  

^a IEEE   (China)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^c XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Aging and rejuvenating; Classes and objects; Expression; Face modeling; Facial attributes; Illumination variance; Merging Face (M Face); Photorealistic rendering; Ratio image; View rotation

Indexed keywords

AGING AND REJUVENATING; CLASSES AND OBJECTS; EXPRESSION; FACE MODELING; FACIAL ATTRIBUTES; ILLUMINATION VARIANCE; MERGING FACE (M-FACE); PHOTOREALISTIC RENDERING; RATIO IMAGE; VIEW ROTATION;

COMPUTATIONAL COMPLEXITY; COMPUTER SIMULATION; CONFORMAL MAPPING; HUMAN ENGINEERING; INFORMATION ANALYSIS; MATHEMATICAL MODELS; THREE DIMENSIONAL; VIDEO SIGNAL PROCESSING;

FACE RECOGNITION;

EID: 33746894562     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2006.877398     Document Type: Article

References (60)

1. H. Y. Shum, S. B. Kang, and S. C. Chan, "Survey of image-based representations and compression techniques," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 11, pp. 1020-1037, Nov. 2003.
2. D. Burschka, G. D. Hager, Z. Dodds, M. Jägersand, D. Cobzas, and K. Yerex, Recent Methods for Image-Based Modeling and Rendering. Piscataway, NJ: IEEE Virtual Reality, 2003, tutorial 1.
3. F. Pighin, R. Szeliski, and D. Salesin, "Modeling and animating realistic faces from images," Int. J. Comput. Vision, vol. 50, no. 2, pp. 143-169, 2002.
4. J. Y. Noh, "Facial animation by expression cloning," Ph.D. dissertation, Dept. Comput. Sci., Univ. of Southern California, Los Angeles, 2002.
5. Y. C. Lee, D. Terzopoulos, and K. Waters, "Realistic modeling for facial animation," in Proc. SIGGRAPH'95, 1995, pp. 55-62.
6. R. M. Koch, M. H. Gross, F. R. Carls, D. F. Von Buren, F. Fankhauser, and Y. I. H. Parish, "Simulating facial surgery using finite element models," in Proc. SIGGRAPH'96, 1996, pp. 421-428.
7. F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. Salesin, "Synthesizing realistic facial expressions from photographs," in Proc. SIGGRAPH'98, 1998, pp. 75-84.
8. V. Blanz and T. Vetter, "A morphable model for the synthesis of 3D faces," in Proc. SIGGRAPH'99, 1999, pp. 187-194.
9. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681-685, Jun. 2001.
10. L. Liang, H. Chen, Y. Q. Xu, and H. Y. Shum, "Example-based caricature generation with exaggeration," in Proc. 10th Pacific Comput. Graphics Applic. Conf., 2002, pp. 386-393.
11. P. Y. Hong, Z. Wen, and T. S. Huang, "Real-time speech-driven face animation with expressions using neural networks," IEEE Trans. Neural Netw., vol. 13, no. 4, pp. 916-927, Jul. 2002.
12. Q. Zhang, "Face image representation and facial attribute transformation based on dense correspondence," Master's thesis, Inst. Artif. Intelligence & Robotics, Xi'an Jiaotong Univ., 2003.
13. Y. Fu and N. N. Zheng, "Image-based photorealistic fusion model of multi facial object class," in Proc. Chinagraph'2004, Xi'an, China, 2004, pp. 210-217.
14. F. Zhuo, "Face modeling based on linear feature correspondence and expression synthesis research," Master's thesis, Inst. Artif. Intelligence & Robotics, Xi'an Jiaotong Univ., 2004.
15. A. J. Calder, A. W. Young, D. Rowland, and D. I. Perrett, "Computer-enhanced emotion in facial expressions," Proc. R. Soc. Lond. B. Biol. Sci., vol. 264, no. 1383, pp. 919-925, 1997.
16. Z. C. Liu, Y. Shan, and Z. Y. Zhang, "Expressive expression mapping with ratio images," in Proc. SIGGRAPH'01, 2001, pp. 271-276.
17. Y. Shan, Z. C. Liu, and Z. Y. Zhang, "Image-based surface detail transfer," in Proc. IEEE CVPR'01, 2001, pp. 794-799.
18. A. Shashua and T. Riklin-Raviv, "The quotient image: class-based re-rendering and recognition with varying illuminations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 129-139, Feb. 2001.
19. T. Vetter and T. Poggio, "Linear object classes and image synthesis from a single example image," IEEE Trans. Pattern Anal Mach. Intell., vol. 19, no. 7, pp. 733-742, Jul. 1997.
20. R. Basri and D. W. Jacobs, "Lambertian reflectance and linear subspaces," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 2, pp. 218-233, Feb. 2003.
21. Y. Fu, "Photorealistic face rendering based on a fusion model of linear and Lambertian object class," Master;s thesis, Inst. Artif. Intelligence & Robotics, Xi'an Jiaotong Univ., 2004.
22. T. Zhang, "Face age transformation based on shape and texture and face gender classification," Master's thesis, Inst. Artif. Intelligence & Robotics, Xi'an Jiaotong Univ., 2004.
23. Y. Fu, N. N. Zheng, J. Y. Liu, and T. Zhang, "FaceTransfer: A system model of facial image rendering," in Proc. IEEE SMC'04, 2004, pp. 4528-4534.
24. Y. Fu, "Merging ratio images based realistic object class re-rendering," in Proc. IEEE ICIP'04, Singapore, 2004, pp. 3523-3526.
25. F. I. Parke, "Computer generated animation of faces," in Proc. ACM Nat. Conf., 1972, pp. 451-457.
26. H. Pyun, Y. Kim, W. Chae, H. Kang, and S. Shin, "An example-based approach for cloning facial expressions," in Proc. 2003 ACM SIGGRAPH/Eurographics, 2003, pp. 167-176.
27. L. Boissieux, G. Kiss, N. M. Thalmann, and P. Kalra, "Simulation of skin aging and wrinkles with cosmetics insight," in Proc. Eurographics, Workshop Animation and Simulation, 2000.
28. T. Kuratate and T. Nishita, "A simple method for modeling wrinkles on human skin," in Proc. PG'02, 2002, pp. 166-175.
29. S. M. Seitz and C. R. Dyer, "View morphing," in Proc. SIGGRAPH'96, New Orleans, LA, 1996, pp. 21-30.
30. _, "Complete structure from four point correspondences," in Proc. IEEE ICCV'95, 1995, pp. 330-337.
31. A. J. O'Toole, T. Vetter, H. Volz, and E. M. Salter, "Three- demensional caricatures of human heads: Distinctiveness and the perception of facial age," Perception, vol. 26, pp. 719-732, 1997.
32. _, As we get older, do we get more distinct? MPI. Memo TR49, Max-Planck Institute, 1997.
33. K. A. Deffenbacher, T. Vetter, J. Johanson, and O'Toole, "Facial aging, attractiveness, and distinctiveness," Perception, vol. 27, pp. 1233-1243, 1998.
34. A. J. O'Toole, T. Price, T. Vetter, J. C. Bartlett, and V. Blanz, "Three-dimensional shape and two-dimensional surface textures of human faces: the role of "averages" in attractiveness and age," Image Vision Computing J., vol. 18, pp. 9-19, 1999.
35. C. Braun, M. Gruendl, C. Marberger, C. Scherber, T. Vetter, H. Volz, and E. M. Salter, Beautycheck-Causes and Consequences of Human Facial Attractiveness (Summary) The German Students Award, 2001.
36. T. Valentine, "A unified account of the effects of distinctiveness, inversion, and race in face recognition," Quart. J. Exper. Psychol., vol. 43, no. 2, pp. 161-204, 1991.
37. A. Stoschek, "Image-based re-rendering of faces for continuous pose and illumination directions," in Proc. IEEE CVPR'00, 2000, pp. 582-587.
38. Z. Wen, Z. C. Liu, and T. S. Huang, "Face relighting with radiance environment maps," in Proc. IEEE CVPR'03, 2003, pp. 158-165.
39. P. H. Tu, I. C. Lin, J. S. Yeh, R. H. Liang, and O. Y. Ming, "Expression detail mapping for realistic facial animation," in Proc. CAD/CG '2003, Taipa, Macau, 2003, pp. 20-25.
40. D. L. Jiang, W. Gao, Z. Q. Wang, and Y. Q. Chen, "Realistic 3D facial animations with subtle texture changes," in Proc. ICICS-PCM'2003, Singapore, 2003.
41. V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 9, pp. 1063-1074, Sep. 2003.
42. T. Vetter, "Synthesis of novel views from a single face image," Int. J. Comput. Vision, vol. 28, no. 2, pp. 103-116, 1998.
43. T. Vetter, M. J. Jones, and T. Poggio, "A bootstrapping algorithm for learning linear models of object classes," in Proc. IEEE CVPR '97, 1997, pp. 40-47.
44. M. J. Jones and T. Poggio, "Multidimensional morphable models," in Proc. IEEE ICCV'98, Bombay, India, 1998, pp. 683-688.
45. _, "Model-based matching of line drawings by linear combinations of examples," in Proc. IEEE ICCV'95, Boston, MA, 1995, pp. 532-536.
46. D. M. Burt and D.I. Perrett, "Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information," Proc. R. Soc. Lond. B. Biol. Sci., vol. 259, no. 1355, pp. 137-143, 1995.
47. F. D. Torre and M. J. Black, "Robust principal component analysis for computer vision," in Proc. IEEE ICCV'01, 2001, pp. 362-369.
48. D. Beymer and T. Poggio, "Image representations for visual learning," Science, vol. 272, pp. 1905-1909, 1996.
49. D. Beymer, Vectorizing face images by interleaving shape and texture computations A.I. Memo 1537, MIT, 1995.
50. T. Beier and S. Neely, "Feature-based image metamorphosis," Comput. Graphics, vol. 26, no. 2, pp. 35-42, 1992.
51. Y. Chang, C. Hu, and M. Turk, "Manifold of facial expression," in Proc. IEEE Workshop AMFG'03, 2003, pp. 28-35.
52. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, pp. 2319-2323, 2000.
53. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, pp. 2323-2326, 2000.
54. S. E. Brennan, "The caricature generator," Leonardo, vol. 18, pp. 170-178, 1985.
55. P. J. Benson and D. I. Perrett, "Synthesising continuous-tone caricatures," Image Vision Comput., vol. 9, pp. 123-129, 1991.
56. H. Chen, "Statistic learning for facial image analysis and rendering," Ph.D. dissertation, Inst. Artif. Intelligence & Robotics, AI&R, Xi'an Jiaotong Univ., 2002.
57. R. L. Hsu and A. K. Jain, "Generating discriminating cartoon faces using interacting snakes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 11, pp. 1388-1398, Nov. 2003.
58. R. W. G. Hunt, Measuring Color, ser. Ellis Norwood series in applied science and industrial technology. New York: Halsted, 1989.
59. J. Q. Wang, K. H. Wong, P. A. Pheng, H. M. Meng, and T. T. Wong, "A real-time Cantonese text-to-audiovisual speech synthesizer," in Proc. IEEE ICASSP'04, 2004, vol. 1, pp. 653-656.
60. J. Hershey, H. Attias, N. Jojic, and T. Kristjansson, "Audio-visual graphical models for speech processing," in Proc. IEEE ICASSP'04, 2004, vol. 5, pp. 649-652.