Facial expression recognition in image sequences using geometric deformation features and support vector machines

IEEE Transactions on Image Processing

Volumn 16, Issue 1, 2007, Pages 172-187

  Kotsia, Irene ^a   Pitas, Ioannis ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Candide grid; Facial Action Coding S (FACS); Facial Action Unit (FAU); Facial expression recognition; Machine vision; Pattern recognition; Support Vector Machines (SVMs)

Indexed keywords

COMPUTER VISION; GRID COMPUTING; PATTERN RECOGNITION; SUPPORT VECTOR MACHINES;

CANDIDE GRID; FACIAL ACTION UNIT (FAU); FACIAL EXPRESSION RECOGNITION;

FACE RECOGNITION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; EVALUATION; FACE; FACIAL EXPRESSION; HISTOLOGY; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; METHODOLOGY; VIDEORECORDING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; FACE; FACIAL EXPRESSION; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; SUBTRACTION TECHNIQUE; VIDEO RECORDING;

EID: 33847747430     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2006.884954     Document Type: Article

References (54)

1. P. Ekman and W. V. Friesen, Emotion in the Human Face. Englewood Cliffs, NJ: Prentice-Hall, 1975.
2. T. Kanade, J. Cohn, and Y. Tian, "Comprehensive database for facial expression analysis," in Proc. IEEE Int. Conf. Face and Gesture Recognition, Mar. 2000, pp. 46-53.
3. M. Pantic and L. J. M. Rothkrantz, "Expert system for automatic analysis of facial expressions," Image Vis. Comput., vol. 18, no. 11, pp. 881-905, Aug. 2000.
4. M. Pantic and L. J. M. Rothkrantz, "Automatic analysis of facial expressions: The state of the art," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1424-1445, Dec. 2000.
5. B. Fasel and J. Luettin, "Automatic facial expression analysis: A survey," Pattern Recognit., vol. 36, no. 1, pp. 259-275, 2003.
6. Y. Zhang and Q. Ji, "Active and dynamic information fusion for facial expression understanding from image sequences," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 699-714, May 2005.
7. I. Cohen, N. Sebe, S. Garg, L. S. Chen, and T. S. Huanga, "Facial expression recognition from video sequences: Temporal and static modelling," Comput. Vis. Image Understand., vol. 91, pp. 160-187, 2003.
8. M. S. Bartlett, G. Littlewort, I. Fasel, and J. R. Movellan, "Real time face detection and facial expression recognition: Development and applications to human computer interaction," in Proc. Conf. Computer Vision and Pattern Recognition Workshop, Madison, WI, Jun. 16-22, 2003, vol. 5, pp. 53-58.
9. M. J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, "Coding facial expressions with G abor wavelets," in Proc. 3rd IEEE Int. Conf. Automatic Face and Gesture Recognition, 1998, pp. 200-205.
10. M. J. Lyons, J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 12, pp. 1357-1362, Dec. 1999.
11. L. Wiskott, J. Fellous, N. Krüger, and C. v. d. Malsburg, "Face recognition by elastic bunch graph matching," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 775-779, Jul. 1997.
12. G. Guo and C. R. Dyer, "Learning from examples in the small sample case: Face expression recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 3, pp. 477-488, Jun. 2005.
13. Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu, "Comparison between geometry-based and G abor-wavelets-based facial expression recognition using multi-layer perceptron," in Proc. 3rd IEEE Int. Conf. Automatic Face and Gesture Recognition, Nara, Japan, Apr. 14-16, 1998, pp. 454-459.
14. B. Fasel, "Multiscale Facial Expression Recognition Using Convolutional Neural Networks," Tech. Rep., IDIAP, 2002,.
15. M. Matsugu, K. Mori, Y. Mitari, and Y. Kaneda, "Subject independent facial expression recognition with robust face detection using a convolutional neural network," Neural Netw., vol. 16, no. 5-6, pp. 555-559, Jun.-Jul. 2003.
16. M. Rosenblum, Y. Yacoob, and L. S. Davis, "Human expression recognition from motion using a radial basis function network architecture," IEEE Trans. Neural Netw., vol. 7, no. 5, pp. 1121-1138, Sep. 1996.
17. L. Ma and K. Khorasani, "Facial expression recognition using constructive feedforward neural networks," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 3, pp. 1588-1595, Jun. 2004.
18. S. Dubuisson, F. Davoine, and M. Masson, "A solution for facial expression representation and recognition," Signal Process.: Image Commun., vol. 17, no. 9, pp. 657-673, Oct. 2002.
19. X.-W. Chen and T. Huang, "Facial expression recognition: A clustering-based approach," Pattern Recognit. Lett., vol. 24, no. 9-10, pp. 1295-1302, Jun. 2003.
20. Y. Gao, M. Leung, S. Hui, and M. Tananda, "Facial expression recognition from line-based caricatures," IEEE Trans. Syst., Man, Cybern. A: Syst. Humans, vol. 33, no. 3, pp. 407-412, May 2003.
21. B. Abboud, F. Davoine, and M. Dang, "Facial expression recognition and synthesis based on an appearance model," Signal Process.: Image Commun., vol. 19, no. 8, pp. 723-740, 2004.
22. I. A. Essa and A. P. Pentland, "Facial expression recognition using a dynamic model and motion energy," presented at the Int. Conf. Computer Vision, Cambrdige, MA, Jun. 20-23, 1995.
23. I. A. Essa and A. P. Pentland, "Coding, analysis, interpretation, and recognition of facial expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 757-763, Jul. 1997.
24. M. S. Bartlett, G. Littlewort, B. Braathen, T. J. Sejnowski, and J. R. Movellan, "An approach to automatic analysis of spontaneous facial expressions," presented at the 5th IEEE Int. Conf. Automatic Face and Gesture Recognition, Washington, DC, 2002.
25. G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski, "Classifying facial actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 974-989, Oct. 1999.
26. Y. L. Tian, T. Kanade, and J. F. Cohn, "Recognizing action units for facial expression analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp. 97-115, Feb. 2001.
27. J. J. Lien, T. Kanade, J. Cohn, and C. C. Li, "Automated facial expression recognition based on facs action units," in Proc. 3rd IEEE Int. Conf. Automatic Face and Gesture Recognition, Apr. 1998, pp. 390-395.
28. J. J. Lien, T. Kanade, J. F. Cohn, and C. Li, "Detection, tracking, and classification of action units in facial expression," J. Robot. Auton. Syst., Jul. 1999.
29. Y. L. Tian, T. Kanade, and J. Cohn, "Evaluation of G abor wavelet-based facial action unit recognition in image sequences of increasing complexity," in Proc. 5th IEEE Int. Conf. Automatic Face and Gesture Recognition, 2002, pp. 229-234.
30. A. Tefas, C. Kotropoulos, and I. Pitas, "Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 7, pp. 735-746, Jul. 2001.
31. H. Drucker, W. Donghui, and V. Vapnik, "Support vector machines for spam categorization," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1048-1054, Sep. 1999.
32. A. Ganapathiraju, J. Hamaker, and J. Picone, "Applications of support vector machines to speech recognition," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2348-2355, Aug, 2004.
33. M. Pontil and A. Verri, "Support vector machines for 3D object recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 6, pp. 637-646, Jun. 1998.
34. M. Rydfalk, "CANDIDE: A Parameterized Face," Tech. Rep., Linkoping Univ., 1978.
35. F. Dornaika and F. Davoine, "Simultaneous facial action tracking and expression recognition using a particle filter," presented at the IEEE Int. Conf. Computer Vision, Beijing, China, Oct. 17-20, 2005.
36. S. B. Gokturk, C. Tomasi, B. Girod, and J.-Y. Bouguet, "Model-based face tracking for view-independent facial expression recognition," in Proc. 5th IEEE Int. Conf. Automatic Face and Gesture Recognition, Cambridge, U.K., May 2002, pp. 287-293.
37. M. Malciu and F. Preteux, "Tracking facial features in video sequences using a deformable model-based approach," Proc. SPIE, vol. 4121, pp. 51-62, 2000.
38. P. Michel and R. Kaliouby, "Real time facial expression recognition in video using support vector machines," in Proc. 5th Int. Conf. Multi-modal interfaces, Vancouver, BC, Canada, 2003, pp. 258-264.
39. V. Vapnik, Statistical learning theory. New York: Wiley, 1998.
40. J. Y. Bouguet, "Pyramidal Implementation of the Lucas-Kanade Feature Tracker," Tech. Rep., Intel Corporation, Microprocessor Research Labs, 1999.
41. C. W. Hsu and C. J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002.
42. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, no. 2, 1998.
43. B. Scholkopf, S. Mika, C. Burges, P. Knirsch, K.-R. Muller, G. Ratsch, and A. Smola, "Input space vs. feature space in kernel-based methods," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1000-1017, Sep. 1999.
44. K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, and B. Scholkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001.
45. R. Duda and P. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
46. S. Gunn, "Support vector machines for classification and regression," MP-TR-98-05, Image, Speech, Intell. Syst. Group, Univ. Southampton, 1998.
47. MATLAB User's Guide. Natick, MA: The MathWorks, Inc., 1994-2001.
48. L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard, and V. Vapnik, "Comparison of classifier methods: A case study in handwritting digit recognition," in Proc. Int. Conf. Pattern Recognition, 1994, pp. 77-87.
49. J. Weston and C. Watkins, "Multi-Class Support Vector Machines," Tech. Rep. CSD-TR-98-04, 2004.
50. J. Weston and C. Watkins, "Multi-class support vector machines," presented at the ESANN, Brussels, Belgium, 1999.
51. S. Krinidis and I. Pitas, "2-D physics-based deformable shape models: Explicit governing equations," in Proc. 1st Int. Workshop on Interactive Rich Media Content Production: Architectures, Technologies, Applications, Tools, Lausanne, Switzerland, Oct. 16-17, 2003, pp. 43-55.
52. P. Ekman and W. V. Friesen, "FACS - Facial Action Coding System," 1978 [Online]. Available: http://www-2.cs.cmu.edu/afs/cs/project/face/www/facs.htm
53. F. Dornaika and F. Davoine, "View- and texture-independent facial expression recognition in videos using dynamic programming," presented at the IEEE Int. Conf. Image Processing Genova, Italy, Sep. 11-14, 2005.
54. L. Gillick and S. Cox, "Some statistical issues in the comparison of speech recognition algorithms," in Proc. ICASSP, 1989, pp. 532-535.