Texture and shape information fusion for facial expression and facial action unit recognition

Pattern Recognition

Volumn 41, Issue 3, 2008, Pages 833-851

  Kotsia, Irene ^a   Zafeiriou, Stefanos ^a   Pitas, Ioannis ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Discriminant Non Negative Matrix Factorization; Facial Action Unit recognition; Facial expression recognition; Fusion; Multidimensional embedding; Radial Basis Functions; Support Vector Machines

Indexed keywords

DATABASE SYSTEMS; FEATURE EXTRACTION; IMAGE FUSION; RADIAL BASIS FUNCTION NETWORKS; SUPPORT VECTOR MACHINES;

DISCRIMINANT NON-NEGATIVE MATRIX FACTORIZATION; FACIAL ACTION UNIT RECOGNITION; FACIAL EXPRESSION RECOGNITION; MULTIDIMENSIONAL EMBEDDING;

FACE RECOGNITION;

EID: 35448954479     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2007.06.026     Document Type: Article

References (68)

1. Pentland A., and Choudhury T. Face recognition for smart environments. IEEE Comput. 33 2 (2000) 50-55
2. Pantic M., and Rothkrantz L. Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91 9 (2003) 1370-1390
3. Ekman P., and Friesen W.V. Emotion in the Human Face (1975), Prentice-Hall, New Jersey
4. T. Kanade, J. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in: Proceedings of IEEE International Conference on Face and Gesture Recognition, 2000, pp. 46-53.
5. Pantic M., and Rothkrantz L.J.M. Expert system for automatic analysis of facial expressions. Image Vision Comput. 18 11 (2000) 881-905
6. Pantic M., and Rothkrantz L.J.M. Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22 12 (2000) 1424-1445
7. Fasel B., and Luettin J. Automatic facial expression analysis: a survey. Pattern Recognition 36 1 (2003) 259-275
8. Gu H., Zhang Y., and Ji Q. Task oriented facial behavior recognition with selective sensing. CVIU 100 3 (2005) 385-415
9. Bartlett M.S., Littlewort G., Fasel I., and Movellan JR. Real time face detection and facial expression recognition: development and applications to human computer interaction. Proceedings of Conference on Computer Vision and Pattern Recognition Workshop vol. 5 (2003), Madison, Wisconsin 53-58
10. G. Littlewort, M. Bartlett, I. Fasel, J. Susskind, J. Movellan, Dynamics of facial expression extracted automatically from video, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Face Processing in Video, 2004.
11. C. Shan, S. Gong, P. McOwan, Appearance manifold of facial expression, in: IEEE International Workshop on Human-Computer Interaction, 2005.
12. Y. Kosaka, K. Kotani, Facial expression analysis by kernel eigenspace method based on class features (KEMC) using non-linear basis for separation of expression-classes, in: Proceedings of IEEE International Conference on Image Processing (ICIP 2004), Singapore, 2004.
13. Dubuisson S., Davoine F., and Masson M. A solution for facial expression representation and recognition. Signal Process. Image Commun. 17 9 (2002) 657-673
14. Thomaz C., and Feitosa D.G.A. Using mixture covariance matrices to improve face and facial expression recognitions. Pattern Recognition Lett. 24 13 (2003) 2159-2165
15. Ma L., and Khorasani K. Facial expression recognition using constructive feedforward neural networks. IEEE Trans. Syst. Man Cybern. B: Cybern. 34 3 (2004) 1588-1595
16. Gizatdinova Y., and Surakka V. Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans. Pattern Anal. Mach. Intell. 28 1 (2006) 135-139
17. Matsugu M., Mori K., Mitari Y., and Kaneda Y. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Networks 16 5-6 (2003) 555-559
18. Liang D., Yang J., Zheng Z., and Chang Y. A facial expression recognition system based on supervised local linear embedding. Pattern Recognition Lett. 26 (2005) 2374-2389
19. T. Yabui, Y. Kenmochi, K. Kotani, Facial expression analysis from 3d range images; comparison with the analysis from 2d images and their integration, in: Proceedings of IEEE International Conference on Image Processing, 2003, pp. 879-882.
20. Chen X.-W., and Huang T. Facial expression recognition: a clustering-based approach. Pattern Recognition Lett. 24 9-10 (2003) 1295-1302
21. Y.L. Tian, T. Kanade, J. Cohn, Evaluation of Gabor wavelet-based facial action unit recognition in image sequences of increasing complexity, in: Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, 2002, pp. 229-234.
22. Roweis S., and Saul L. Nonlinear dimensionality reduction by locally linear embedding. Science 290 (2000) 2323-2326
23. Gao Y., Leung M., Hui S., and Tananda M. Facial expression recognition from line-based caricatures. IEEE Trans. Systems Man Cybern. A: Syst. Hum. 33 3 (2003) 407-412
24. Zhang Y., and Ji Q. Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27 5 (2005) 699-714
25. P. Michel, R. Kaliouby, Real time facial expression recognition in video using support vector machines, in: Proceedings of Fifth International Conference on Multimodal Interfaces, Vancouver, British Columbia, Canada, 2003, pp. 258-264.
26. F. Dornaika, F. Davoine, View- and texture-independent facial expression recognition in videos using dynamic programming, in: Proceedings of IEEE International Conference on Image Processing, Genova, Italy, 2005.
27. F. Dornaika, F. Davoine, Online appearance-based face and facial feature tracking, in: Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, 2004.
28. Abboud B., Davoine F., and Dang M. Facial expression recognition and synthesis based on an appearance model. Signal Process. Image Commun. 19 8 (2004) 723-740
29. M. Malciu, F. Preteux, Tracking facial features in video sequences using a deformable model-based approach, in: Proceedings of the SPIE, vol. 4121, 2000, pp. 51-62.
30. Zhu Y., Silva L.C.D., and Ko C.C. Using moment invariants and HMM in facial expression recognition. Pattern Recognition Lett. 23 1-3 (2002) 83-91
31. Chang Y., Hu C., Feris R., and Turk M. Manifold based analysis of facial expression. Image Vision Comput. 24 (2006) 605-614
32. Guo G., and Dyer C.R. Learning from examples in the small sample case: face expression recognition. IEEE Trans. Syst. Man Cybern. B: Cybern. 35 3 (2005) 477-488
33. S.B. Gokturk, C. Tomasi, B. Girod, J.-Y. Bouguet, Model-based face tracking for view-independent facial expression recognition, in: Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, Cambridge, UK, 2002, pp. 287-293.
34. Kotsia I., and Pitas I. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16 1 (2007) 172-187
35. D. Lee, H. Seung, Algorithms for non-negative matrix factorization, in: NIPS, 2000, pp. 556-562.
36. Zafeiriou S., Tefas A., Buciu I., and Pitas I. Exploiting discriminant information in non-negative matrix factorization with application to frontal face verification. IEEE Trans. Neural Networks 17 3 (2006) 683-695
37. Donato G., Bartlett M., Hager J., Ekman P., and Sejnowski T. Classifying facial actions. IEEE Trans. Pattern Anal. Mach. Intell. 21 10 (1999) 974-989
38. I. Kotsia, S. Zafeiriou, N. Nikolaidis, I. Pitas, Multiclass support vector machines and metric multidimensional scaling for facial expression recognition, 2007.
39. Buciu I., and Pitas I. DNMF modeling of neural receptive fields involved in human facial expression perception. J. Visual Commun. Image Representation 17 5 (2006) 958-969
40. Li S., Hou X., and Zhang H. Learning spatially localized, part-based representation. CVPR (2001), Kauai, HI, USA 207-212
41. Cox M.F., and Cox M.A.A. Multidimensional Scaling (2001), Chapman & Hall, London
42. Borg I., and Groenen P. Modern Multidimensional Scaling: Theory and Applications (1997), Springer, New York
43. Torgerson W.S. Theory and Methods of Scaling (1958), Wiley, New York
44. S. Krinidis, I. Pitas, Statistical analysis of facial expressions for facial expression synthesis, IEEE Trans. Pattern Anal. Mach. Intell., submitted for publication.
45. M. Collins, R.E. Schapire, Y. Singer, Logistic regression, adaboost and bregman distances, Comput. Learn. Theory (2000) 158-169.
46. J.Y. Bouguet, Pyramidal implementation of the Lucas-Kanade feature tracker, Technical Report, Intel Corporation, Microprocessor Research Labs, 1999.
47. Wiskott L., Fellous J.M., Kuiger N., and von der Malsburg C. Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19 7 (1997) 775-779
48. Pekalska E., Paclik P., and Duin R. A generalized kernel approach to dissimilarity-based classification. J. Mach. Learn. Res. 2 (2001) 175-211
49. Drucker H., Donghui W., and Vapnik V. Support vector machines for spam categorization. IEEE Trans. Neural Networks 10 5 (1999) 1048-1054
50. Ganapathiraju A., Hamaker J., and Picone J. Applications of support vector machines to speech recognition. IEEE Trans. Signal Process. 52 8 (2004) 2348-2355
51. Pontil M., and Verri A. Support vector machines for 3D object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 20 6 (1998) 637-646
52. Tefas A., Kotropoulos C., and Pitas I. Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication. IEEE Trans. Pattern Anal. Mach. Intell. 23 7 (2001) 735-746
53. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
54. Hsu C.W., and Lin C.J. A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Networks 13 2 (2002) 415-425
55. J. Weston, C. Watkins, Multi-class Support Vector Machines, Technical Report CSD-TR-98-04, 2004.
56. Weston J., and Watkins C. Multi-class support vector machines. Proceedings of ESANN99 (1999), Brussels, Belgium
57. C.J.C. Burges, A tutorial on support vector machines for pattern recognition, Data Min. Knowl. Discovery 2(2).
58. Snelick R., Uludag U., Mink A., Indovina M., and Jain A. Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Trans. Pattern Anal. Mach. Intell. 27 3 (2005) 450-455
59. Bors A.G., and Pitas I. Median radial basis function neural network. IEEE Trans. Neural Networks 7 (1996) 1351-1364
60. Chatzis V., Bors A., and Pitas I. Multimodal decision-level fusion for person authentication. IEEE Trans. Syst. Man Cybern. A 29 6 (1999) 674-680
61. C.C. Hung, Y. Kim, T. Coleman, A comparative study of radial basis function neural networks and wavelet neural networks in classification of remotely sensed data, in: Fourth Biannual World Automation Congress (WAC'2002), Orlando, FL, USA, 2002.
62. Schalkof R.J. Pattern Recognition: Statistical, Structural and Neural Approaches (1992), Wiley, New York
63. Scholkopf B., Sung K.-K., Burges C.J.C., Girosi F., Niyogi P., Poggio T., and Vapnik V. Comparing support vector machines with gaussian kernels to radial basis function classifiers. IEEE Trans. Signal Process. 45 11 (1997) 2758-2765
64. Pitas I., and Venetsanopoulos A.N. Nonlinear Digital Filters: Principles and Applications (1990), Kluwer Academic, Dordrecht
65. Cohen I., Sebe N., Garg S., Chen L.S., and Huanga T.S. Facial expression recognition from video sequences: temporal and static modelling. Comput. Vision Image Understanding 91 (2003) 160-187
66. Aleksic S., and Katsaggelos K. Automatic facial expression recognition using facial animation parameters and multi-stream HMMs. IEEE Trans. Inf. Forensics Secur. 1 1 (2006) 3-11
67. Yeasin M., Bullot B., and Sharma R. Recognition of facial expressions and measurement of levels of interest from video. IEEE Transactions on Multimedia 8 3 (2006) 500-508
68. Zhao G., and Pietikainen M. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29 6 (2007) 915-928