3D facial expression recognition: A perspective on promises and challenges

2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011

Volumn , Issue , 2011, Pages 603-610

  Fang, T ^a   Zhao, X ^a   Ocegueda, O ^a   Shah, S K ^a   Kakadiaris, I A ^a  

^a ODNI ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

2D TEXTURES; 3-D FACIAL EXPRESSION RECOGNITION; 3D FACES; EXPERIMENTAL PROTOCOLS; FACIAL ACTION UNIT RECOGNITION; RESEARCH TRENDS;

GESTURE RECOGNITION; THREE DIMENSIONAL;

FACE RECOGNITION;

EID: 79958727625     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FG.2011.5771466     Document Type: Conference Paper

References (73)

1. K. Etemad and R. Chellappa, "Discriminant analysis for recognition of human face images," Journal of the Optical Society of America A, vol. 14, no. 8, pp. 1724-1733, August 1997. (Pubitemid 127569239)
2. B. Fasel and J. Luettin, "Automatic facial expression analysis: A survey," Pattern Recognition, vol. 36, no. 1, pp. 259-275, 1999.
3. M. Pantic and L. Rothkrantz, "Automatic analysis of facial expressions: The state of the art," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, December 2000. (Pubitemid 32088469)
4. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, 2007.
5. I. Kakadiaris, G. Passalis, G. Toderici, M. Murtuza, Y. Lu, N. Karampatziakis, and T. Theoharis, "Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 640-649, Apr. 2007. (Pubitemid 46464404)
6. P. J. Phillips, W. T. Scruggs, A. J. O'Toole, P. J. Flynn, K. W. Bowyer, C. L. Schott, and M. Sharpe, "FRVT 2006 and ICE 2006 largescale experimental results," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 831-846, 2010.
7. L. Yin, X. Wei, Y. Sun, J. Wang, and M. Rosato, "A 3D facial expression database for facial behavior research," in Proc. 7th International Conference on Automatic Face and Gesture Recognition, Southampton, UK, Apr. 10-12 2006, pp. 211-216. (Pubitemid 44711352)
8. J. F. Cohn, "Foundations of human computing: facial expression and emotion," in Proc. 8th International Conference on Multimodal Interfaces, Banff, Alberta, Canada, 2006, pp. 233-238. (Pubitemid 47113454)
9. P. Ekman and W. Friesen, "Constants across cultures in the face and emotion," Journal of Personality and Social Psychology, vol. 17, no. 2, pp. 124-129, 1971.
10. -, Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press, 1978.
11. www.face-and-emotion.com/dataface/facs/emfacs.jsp.
12. www.face-and-emotion.com/dataface/facsaid/description.jsp.
13. L. Yin, X. Wei, P. Longo, and A. Bhuvanesh, "Analyzing facial expressions using intensity-variant 3D data for human computer interaction," in Proc. 18th International Conference on Pattern Recognition, Atlanta, GA, Oct. 8-11 2006, pp. 1248-1251. (Pubitemid 46532125)
14. Y. Sun and L. Yin, "Facial expression recognition based on 3D dynamic range model sequences," in Proc. 10th European Conference on Computer Vision, Marseille, France, 2008, pp. 58-71.
15. H. Soyel and H. Demirel, "Facial expression recognition using 3D facial feature distances," in Proc. International Conference on Image Analysis and Recognition, vol. 4633, Montreal, Canada, Aug. 22-24 2007, pp. 831-838.
16. -, "Optimal feature selection for 3D facial expression recognition with geometrically localized facial features," in Proc. Soft Computing, Computing with Words and Perceptions in System Analysis, Decision and Control, Famagusta, Cyprus, September 2009.
17. I. Mpiperis, S. Malassiotis, and M. Strintzis, "Bilinear models for 3-D face and facial expression recognition," IEEE Transactions on Information Forensics and Security, vol. 3, no. 3, pp. 498-511, 2008.
18. B. Gong, Y. Wang, J. Liu, and X. Tang, "Automatic facial expression recognition on a single 3D face by exploring shape deformation," in Proc. 17th ACM International Conference on Multimedia, New York, NY, 2009, pp. 569-572.
19. Y. Venkatesh, A. Kassim, and O. Murthy, "A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA," Pattern Recognition Letters, vol. 30, no. 12, pp. 1128-1137, 2009.
20. X. Zhao, D. Huang, E. Dellandŕea, and L. Chen, "Automatic 3D facial expression recognition based on a bayesian belief net and a statistical facial feature model," in Proc. 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010, pp. 3724-3727.
21. Y. Sun, M. Reale, and L. Yin, "Recognizing partial facial action units based on 3D dynamic range data for facial expression recognition," in Proc. 8th IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands, 2008.
22. A. Savran, B. Sankur, and M. Bilge, "Facial action unit detection: 3D versus 2D modality," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, June 2010, pp. 71-78.
23. X. Zhao, E. Dellandŕea, L. Chen, and D. Samaras, "AU recognition on 3D faces based on an extended statistical facial feature model," in Proc. 4th IEEE International Conference on Biometrics: Theory, Applications and Systems, Washington, DC, 2010.
24. L. Yin, X. Chen, Y. Sun, T. Worm, and M. Reale, "A high-resolution 3D dynamic facial expression database," in Proc. 8th IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands, Sep. 17-19 2008.
25. A. Savran, N. Alyuz, H. Dibeklioglu, O. Celiktutan, B. Gokberk, B. Sankur, and L. Akarun, "Bosphorus database for 3D face analysis," in Proc. First COST 2101 Workshop on Biometrics and Identity Management, Roskilde University, Denmark, May 7-9 2008, pp. 47-56.
26. K. Schmidt and J. Cohn, "Dynamics of facial expression: normative characteristics and individual differences," in Proc. IEEE International Conference on Multimedia and Expo, Tokyo, Japan, August 2001, pp. 728-731.
27. Y. Chang, M. Vieira, M. Turk, and L. Velho, "Automatic 3D facial expression analysis in videos," in Proc. Analysis and Modeling of Faces and Gestures, Beijing, China, 2005, pp. 293-307.
28. M. Rosato, X. Chen, and L. Yin, "Automatic registration of vertex correspondences for 3D facial expression analysis," in Proc. 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems, Arlington, VA, Sep. 29-Oct. 1 2008.
29. P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the face recognition grand challenge," in Proc. International Conference on Computer Vision and Pattern Recognition, vol. 1, San Diego, CA, 2005, pp. 947-954. (Pubitemid 43897294)
30. A. B. Moreno and A. Śanchez, "GavabDB: a 3D face database," in Proc. 2nd COST275 Workshop on Biometrics on the Internet, Vigo, Spain, March 2004, pp. 77-85.
31. www.3dmd.com.
32. www.di3d.com.
33. www.inspeck.com.
34. J. Wang, L. Yin, X. Wei, and Y. Sun, "3D facial expression recognition based on primitive surface feature distribution," in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, New York, NY, Jun. 17-22 2006, pp. 1399-1406. (Pubitemid 44931484)
35. S. Ramanathan, A. Kassim, Y. Venkatesh, and W. S. Wah, "Human facial expression recognition using a 3D morphable model," in Proc. IEEE International Conference on Image Processing, Atlanta, GA, 2006, pp. 661-664.
36. P. Wang, C. Kohler, F. Barrett, R. Gur, and R. Verma, "Quantifying facial expression abnormality in schizophrenia by combining 2D and 3D features," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota, Jun. 17-22 2007.
37. I. Mpiperis, S. Malassiotis, V. Petridis, and M. Strintzis, "3D facial expression recognition using swarm intelligence," in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, Mar. 31-Apr. 4 2008, pp. 2133-2136.
38. H. Soyel and H. Demirel, "3D facial expression recognition with geometrically localized facial features," in Proc. 23rd International Symposium on Computer and Information Sciences, Istanbul, Turkey, 2008.
39. H. Tang and T. S. Huang, "3D facial expression recognition based on automatically selected features," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Anchorage, AK, June 2008.
40. H. Tang and T. Huang, "3D facial expression recognition based on properties of line segments connecting facial feature points," in Proc. 8th IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands, 2008, pp. 1-6.
41. H. Soyel and H. Demirel, "Optimal feature selection for 3D facial expression recognition using coarse-to-fine classification," Turkish Journal of Electrical Engineering and Computer Sciences, vol. 18, no. 6, pp. 1031-1040, 2010.
42. U. Tekg̈uç, H. Soyel, and H. Demirel, "Feature selection for personindependent 3D facial expression recognition using NSGA-II," in Proc. 24rd International Symposium on Computer and Information Sciences, Guzelyurt, Turkey, 2009, pp. 35-38.
43. A. Savran and B. Sankur, "Automatic detection of facial actions from 3D data," in Proc. 12th IEEE International Conference on Computer Vision Workshops, Kyoto, Japan, 2009, pp. 1993-2000.
44. S. Berretti, A. D. Bimbo, P. P. B. B. Amor, and M. Daoudi, "A set of selected SIFT features for 3D facial expression recognition," in Proc. 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010, pp. 4125-4128.
45. A. Maalej, B. Amor, M. Daoudi, A. Srivastava, and S. Berretti, "Local 3D shape analysis for facial expression recognition," in Proc. 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010, pp. 4129-4132.
46. Y. Venkatesh, A. Kassim, and O. Murthy, "Resampling approach to facial expression recognition using 3D meshes," in Proc. 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010, pp. 3772-3775.
47. F. Tsalakanidou and S. Malassiotis, "Real-time 2D+3D facial action and expression recognition," Pattern Recognition, vol. 43, no. 5, pp. 1763-1775, 2010.
48. J. Goldfeather and V. Interrante, "A novel cubic-order algorithm for approximating principal direction vectors," ACM Transactions on Graphics, vol. 23, no. 1, pp. 45-63, 2004. (Pubitemid 44743514)
49. I. Pandzic and R. Forchheimer, MPEG-4 facial animation: the standard, implementation and applications. Chichester: John Wiley & Sons Ltd, 2002.
50. Y. Wang, J. Liu, and X. Tang, "Robust 3D face recognition by local shape difference boosting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1858-1870, 2010.
51. C. Padgett and G. Cottrell, "Representing face images for emotion classification," in Proc. Advances in Neural Information Processing Systems, vol. 9, 1997, pp. 894-900.
52. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information: criteria of max-dependency, max-relevance, and minredundancy," IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, 2005. (Pubitemid 41245053)
53. S. H. Joshi, E. Klassen, A. Srivastava, and I. H. Jermyn, "A novel representation for efficient computation of geodesics between ndimensional curves," in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota, 2007.
54. P. Viola and M. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004.
55. T. Cootes, G. Edwards, and C. J. Taylor, "Active appearance models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, 2001. (Pubitemid 32585279)
56. E. Trucc and R. Fisher, "Experiments in curvature-based segmentation of range data," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 2, pp. 177-182, 1995.
57. C. Holmes and N. Adams, "A probabilistic nearest neighbor method for statistical pattern recognition," Journal of the Royal Sta-tistical Society: Series B (Statistical Methodology), vol. 64, pp. 295-306, 2002.
58. M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Automatic recognition of facial actions in spontaneous expressions," Journal of Multimedia, vol. 1, no. 6, pp. 22-35, 2006.
59. P. Campadelli and R. Lanzarotti, "Fiducial point localization in color images of face foregrounds," Image and Vision Computing, vol. 22, no. 11, pp. 863-872, 2004.
60. C. Barber, D. Dobkin, and H. Huhdanpaa, "The quickhull algorithm for convex hulls," ACM Transactions on Mathematical Software, vol. 22, no. 4, pp. 469-483, 1996. (Pubitemid 126417394)
61. J. Kennedy and R. Eberhart, "Particle swarm optimization," in Proc. IEEE International Conference on Neural Networks, Perth, Australia, 1995, pp. 1942-1948.
62. B. Ĺevy, S. Petitjean, N. Ray, and J. Maillot, "Least squares conformal maps for automatic texture atlas generation," ACM Transactions on Graphics, vol. 21, no. 3, pp. 362-371, 2002.
63. C. Shelton, "Morphable surface models," International Journal of Computer Vision, vol. 38, pp. 75-91, 2000.
64. J. Bourgain, "On lipschitz embedding of finite metric spaces in hilbert space," Israel Journal of Mathematics, vol. 52, pp. 46-52, 1985.
65. L. Yin and A. Basu, "Generating realistic facial expressions with wrinkles for model-based coding," Computer Vision and Image Understanding, vol. 84, no. 2, pp. 201-240, November 2001. (Pubitemid 34222295)
66. L. Kharevych, B. Springborn, and P. Schr̈oder, "Discrete conformal mappings via circle patterns," ACM Transactions on Graphics, vol. 25, no. 2, pp. 412-438, 2006. (Pubitemid 44023265)
67. L. Rabiner, "A tutorial on hidden markov models and selected applications in speech recognition," Proceedings of IEEE, vol. 77, no. 2, pp. 257-286, 1989.
68. A. Lanitis, C. Taylor, and T. Cootes, "Automatic interpretation and coding of face images using flexible models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743-756, 1997. (Pubitemid 127762266)
69. Anitha C, M. Venkatesha, and B. S. Adiga, "A survey on facial expression databases," International Journal of Engineering Science and Technology, vol. 2, no. 10, pp. 5158-5174, 2010.
70. K. Bowyer, K. Chang, and P. Flynn, "A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition," Computer Vision and Image Understanding, vol. 101, no. 1, pp. 1-15, Jan. 2006. (Pubitemid 41642085)
71. P. Perakis, G. Passalis, T. Theoharis, G. Toderici, and I. Kakadiaris, "Partial matching of interpose 3D facial data for face recognition," in Proc. 3rd IEEE International Conference on Biometrics: Theory, Applications and Systems, Arlington, VA, Sep. 28-30 2009, pp. 439-446.
72. P. Nair and A. Cavallaro, "3-D face detection, landmark localization, and registration using a point distribution model," IEEE Transactions on Multimedia, vol. 11, no. 4, pp. 611-623, Jun. 2009.
73. Y. Wang, X. Huang, C.-S. Lee, S. Zhang, Z. Li, D. Samaras, D. Metaxas, A. Elgammal, and P. Huang, "High resolution acquisition, learning and transfer of dynamic 3-D facial expressions," Computer Graphics Forum, vol. 23, no. 3, pp. 677-686, 2004.