Face landmark fitting via optimized part mixtures and cascaded deformable model

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 11, 2016, Pages 2212-2226

  Yu, Xiang ^a   Huang, Junzhou ^b   Zhang, Shaoting ^c   Metaxas, Dimitris N ^a  

^a RUTGERS UNIVERSITY   (United States)

^b University of Texas at Arlington   (United States)

^c University of North Carolina at Charlotte   (United States)

Author keywords

Deformable shape model; Face landmark localization; Face tracking; Part based model

Indexed keywords

DEFORMATION; IMAGE SEGMENTATION; MIXTURES; OPTIMIZATION;

CONSTRAINED LOCAL MODELS; DEFORMABLE SHAPES; ENVIRONMENTAL DATABASE; FACE LANDMARKS; FACE TRACKING; LANDMARK LOCALIZATION; PART-BASED MODELS; STATE-OF-THE-ART METHODS;

FACE RECOGNITION;

EID: 84990933998     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2015.2509999     Document Type: Article

References (59)

1. W. Zhao, R. Chellappa, P. Phillips, and A. Rosenfeld, "Face recognition: A literature survey," ACM Comput. Surveys, vol. 35, pp. 399-458, Dec. 2003.
2. V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 9, pp. 1063-1074, Sep. 2003.
3. M. Pantic and I. Patras, "Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 2, pp. 433-449, Apr. 2006.
4. I. Mpiperis, S. Malassiotis, and M. Strintzis, "Bilinear elastically deformable models with application to 3d face and facial expression recognition," in Proc. IEEE 8th Int. Conf. Autom. Face Gesture Recog., 2008, pp. 1-8.
5. V. Bettadapura, "Face expression recognition and analysis: The state of the art," in Arxiv, 2012.
6. I. Kemelmacher-Shlizerman and R. Basri, "3D face reconstruction from a single image using a single reference face shape," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 394-405, Feb. 2011.
7. F. Yang, L. Bourdev, E. Shechtman, J. Wang, and D. Metaxas, "Facial expression editing in video using a temporally-smooth factorization," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 861-868.
8. J. Saragih, S. Lucey, and J. Cohn, "Deformable model fitting by regularized landmark mean-shift," Int. J. Comput. Vis., vol. 91, pp. 200-215, 2011.
9. X. Xiong and F. De la Torre, "Supervised descent method and its applications to face alignment," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 532-539.
10. M. Jones and P. Viola, "Fast multi-view face detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2003.
11. C. Zhang and Z. Zhang, "A survey of recent advances in face detection," Microsoft Res., Redmond, WA, USA, Tech. Rep., 2010.
12. H. Li, Z. Lin, J. Brandt, X. Shen, and G. Hua, "Efficient boosted exemplar-based face detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 1843-1850.
13. T. Cootes, G. Edwards, and C. Taylor, "Active appearance models," in Proc. 5th Eur. Conf. Comput. Vis., 1998, pp. 484-498 .
14. F. Yang, J. Huang, and D. Metaxas, "Sparse shape registration for occluded facial feature localization," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog. Workshops, 2011, pp. 272-277.
15. M. Roh, T. Oguri, and T. Kanade, "Face alignment robust to occlusion," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., 2011, pp. 239-244.
16. T. Cootes and C. Taylor, "A mixture model for representing shape variation," in Proc. Brit. Mach. Vis. Conf., 1997.
17. X. Yu, Z. Lin, J. Brandt, and D. Metaxas, "Consensus of regression for occlusion-robust facial feature localization," in Proc. 13th Eur. Conf. Comput. Vis., 2014, pp. 105-118.
18. X. Burgos-Artizzu, P. Perona, and P. Dollar, "Robust face landmark estimation under occlusion," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1513-1520.
19. G. Ghiasi and C. C. Fowlkes, "Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 1899- 1906.
20. Y. Zhou, W. Zhang, X. Tang, and H. Shum, "A Bayesian mixture model for multi-view face alignment," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 2005, pp. 741-746.
21. C. Vogler, Z. Li, A. Kanaujia, S. Goldenstein, and D. Metaxas, "The best of both worlds: Combining 3d deformable models with active shape models," in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1-7.
22. E. M. Chutorian and M. Trivedi, "Head pose estimation in computer vision: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 607-626, Apr. 2009.
23. T. Cootes, C. Taylor, D. Cooper, and J. Graham, "Active shape models-their training and application," Comput. Vis. Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
24. I. Matthews and S. Baker, "Active appearance models revisited," Int. J. Comput. Vis., vol. 60, no. 2, pp. 135-164, 2004.
25. D. Cristinacce and T. Cootes, "Automatic feature localization with constrained local models," Pattern Recog., vol. 41, pp. 3054-3067, 2008.
26. Y. Wang, S. Lucey, and J. Cohn, "Enforcing convexity for improved alignment with constrained local models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
27. D. Cristinacce and T. Cootes, "Boosted regression active shape models," in Proc. Brit. Mach. Vis. Conf., 2007, pp. 880-889.
28. X. Cao, Y. Wei, F. Wen, and J. Sun, "Face alignment by explicit shape regression," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 2887-2894.
29. X. Zhu and D. Ramanan, "Face detection, pose estimation and landmark localization in the wild," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 2879-2886.
30. X. Yu, J. Huang, S. Zhang, W. Yan, and D. Metaxas, "Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1944-1951.
31. S. Rivera and A. Martinez, "Learning deformable shape manifolds," Pattern Recog., vol. 45, no. 4, pp. 1792-1801, Apr. 2012.
32. Z. Kalal, J. Matas, and K. Mikolajczyk, "Weighted sampling for large-scale boosting," in Proc. Brit. Mach. Vis. Conf., 2008, pp. 42.1- 42.10.
33. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627- 1645, Sep. 2009.
34. M. Everingham, J. Sivic, and A. Zisserman, "'Hello! my name is. buffy'-automatic naming of characters in tv video," in Proc. 17th Brit. Mach. Vis. Conf., 2006.
35. L. Karlinsky and S. Ullman, "Using linking features in learning non-parametric part models," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 326-339.
36. M. Uricar, V. Franc, and V. Hlavac, "Detector of facial landmarks learned by the structured output SVM," in Proc. VISAPP, 2012.
37. P. Felzenszwalb and D. Huttenlocher, "Pictorial structures for object recognition," Int. J. Comput. Vis., vol. 61, pp. 55-79, 2003.
38. Y. Huang, Q. Liu, and D. Metaxas, "A component based deformable model for generalized face alignment," in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1-8.
39. X. Liu, "Generic face alignment using boosted appearance model," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
40. T. Cootes, M. Ionita, C. Lindner, and P. Sauer, "Robust and accurate shape model fitting using random forest regression voting," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 278-291.
41. G. Tzimiropoulos and M. Pantic, "Optimization problems for fast AAM fitting in-the-wild," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 593-600.
42. X. Cheng, S. Sridharan, J. Saragih, and S. Lucey, "Rank minimization across appearance and shape for AAM ensemble fitting," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 577-584.
43. A. Asthana, S. Cheng, S. Zafeiriou, and M. Pantic, "Robust discriminative response map fitting with constrained local models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 3444-3451.
44. P. Belhumeur, D. Jacobs, D. Kriegman, and N. Kumar, "Localizing parts of faces using a consensus of exemplars," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 545-552.
45. B. Smith and L. Zhang, "Joint face alignment with non-parametric shapemodels," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 43-56.
46. F. Zhou, J. Brandt, and Z. Lin, "Exemplar-based graph matching for robust facial landmark localization," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1025-1032.
47. H. Gao, H. Ekenel, and R. Stiefelhagen, "Face alignment using a ranking model based on regression trees," in Proc. Brit. Mach. Vis. Conf., 2012.
48. M. Valstar, B. Martinez, X. Binefa, and M. Pantic, "Facial point detection using boosted regression and graph models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 2729-2736.
49. B. Martinez, M. Valstar, X. Binefa, and M. Pantic, "Local evidence aggregation for regression-based facial point detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 5, pp. 1149-1163, May 2013.
50. M. Dantone, J. Gall, G. Fanelli, and L. Gool, "Real-time facial feature detection using conditional regression forests," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 2578-2585.
51. P. Dollar, P. Welinder, and P. Perona, "Cascaded pose regression," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 1078-1085.
52. L. Liang, R. Xiao, F. Wen, and J. Sun, "Face alignment via component- based discriminative search," in Proc. 10th Eur. Conf. Comput. Vis., 2008, pp. 72-85.
53. G. Huang, M. Ramesh, T. Berg, and E. Miller, "Labeled faces in the wild: A database for studying face recognition in unconstrained environments," Univ. Massachusetts, Amherst, MA, USA, Tech. Rep. 07-49, 2007.
54. A. Martinez and R. Benavente, "The AR face database," CVC Tech. Rep. 24, 1998.
55. J. Liu, S. Ji, and J. Ye. (2009). SLEP: Sparse Learning with Efficient Projections. Arizona State Univ., Tempe, AZ, USA [Online]. Available: http://www.public.asu.edu/jye02/Software/SLEP
56. R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi- PIE," Image Vis. Comput., vol. 28, pp. 807-813, 2010.
57. L. Yin, X. Wei, Y. Sun, J. Wang, and M. Rosato, "A 3D facial expression database for facial behavior research," in Proc. 7th Int. Conf. Autom. Face Gesture Recog., 2006, pp. 211-216.
58. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "300 faces in-the-wild challenge: The first facial landmark localization challenge," in Proc. IEEE Int. Conf. Comput. Vis. Workshop, 2013, pp. 397-403.
59. [Online]. Available: http://www-prima.inrialpes.fr/FGnet/data/01-TalkingFace/talking-face.html