Recognition of genuine smiles

IEEE Transactions on Multimedia

Volumn 17, Issue 3, 2015, Pages 279-294

  Dibeklioʇlu, Hamdi ^a,b   Salah, Albert Ali ^c   Gevers, Theo ^a,d  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^c BOGAZICI UNIVERSITY   (Turkey)

^d UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

Author keywords

Affective computing; expression dynamics; expression spontaneity; face analysis; genuine smile; human computer interaction; social signals

Indexed keywords

DYNAMICS;

AFFECTIVE COMPUTING; EXPRESSION DYNAMICS; EXPRESSION SPONTANEITY; FACE ANALYSIS; GENUINE SMILE; SOCIAL SIGNALS;

HUMAN COMPUTER INTERACTION;

EID: 84923270197     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2015.2394777     Document Type: Article

References (55)

1. P. Ekman and W. V. Friesen, The Facial Action Coding System: A Technique for the Measurement of Facial Movement. San Francisco, CA, USA: Consulting Psychol. Press Inc., 1978.
2. M. Pantic and L. Rothkrantz, "Automatic analysis of facial expressions: The state of the art," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1424-1445, Dec. 2000.
3. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., pp. 39-58, Jan. 2009.
4. Z. Ambadar, J. F. Cohn, and L. I. Reed, "All smiles are not created equal: Morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous," J. Nonverbal Behav., vol. 33, no. 1, pp. 17-34, 2009.
5. P. Ekman, W. V. Friesen, and M. O'Sullivan, "Smiles when lying," J. Personality Social Psychol., vol. 54, no. 3, pp. 414-420, 1988.
6. P. Ekman, Telling Lies: Cues to Deceit in the Marketplace, Politics, and Marriage. New York, NY, USA: Norton, 1992.
7. P. Ekman, J. C. Hager, and W. V. Friesen, "The symmetry of emotional and deliberate facial actions," Psychophysiol., vol. 18, pp. 101-106, 1981.
8. P. Ekman and W. V. Friesen, "Felt, false, and miserable smiles," J. Nonverbal Behav., vol. 6, pp. 238-252, 1982.
9. J. F. Cohn and K. L. Schmidt, "The timing of facial motion in posed and spontaneous smiles," Int. J. Wavelets, Multiresolution Inf. Process., vol. 2, no. 2, pp. 121-132, 2004.
10. H. Dibeklioʇlu, A. A. Salah, and T. Gevers, "Are you really smiling at me? Spontaneous versus posed enjoyment smiles," in Proc. ECCV, 2012, pp. 526-539.
11. B. Duchenne, The Mechanism of Human Facial Expression. Cambridge, U.K.: Cambridge Univ. Press, 1990.
12. P. Ekman, R. J. Davidson, and W. V. Friesen, "The Duchenne smile: Emotional expression and brain physiology II," J. Personality Social Psychol., vol. 58, no. 2, pp. 342-353, 1990.
13. P. Ekman, G. Roper, and J. C. Hager, "Deliberate facial movement," Child Develop., vol. 51, pp. 886-891, 1980.
14. P. Ekman, "Darwin, deception, and facial expression," Ann. New York Academy Sci., vol. 1000, no. 1, pp. 205-221, 2003.
15. K. L. Schmidt, S. Bhattacharya, and R. Denlinger, "Comparison of deliberate and spontaneous facial movement in smiles and eyebrow raises," J. Nonverbal Behav., vol. 33, pp. 35-45, 2009.
16. P. Gosselin, M. Perron, and M. Beaupré, "The voluntary control of facial action units in adults," Emotion, vol. 10, no. 2, p. 266, 2010.
17. S. D. Gunnery, J. A. Hall, and M. A. Ruben, "The deliberate duchenne smile: Individual differences in expressive control," J. Nonverbal Behav., vol. 37, no. 1, pp. 29-41, 2013.
18. E. G. Krumhuber and A. S. R. Manstead, "Can Duchenne smiles be feigned? New evidence on felt and false smiles," Emotion, vol. 9, no. 6, pp. 807-820, 2009.
19. V. Manera, M. D. Giudice, E. Grandi, and L. Colle, "Individual differences in the recognition of enjoyment smiles: No role for perceptual-attentional factors and autistic-like traits," Frontiers Psychol., vol. 2, 2011.
20. G. Littlewort-Ford, M. S. Bartlett, and J. R. Movellan, "Are your eyes smiling? Detecting genuine smiles with support vector machines and Gabor wavelets," in Proc. Joint Symp. Neural Comput., 2001.
21. M. Del Giudice and L. Colle, "Differences between children and adults in the recognition of enjoyment smiles," Develop. Psychol., vol. 43, no. 3, p. 796, 2007.
22. D. M. Shore and E. A. Heerey, "The value of genuine and polite smiles," Emotion, vol. 11, no. 1, p. 169, 2011.
23. J. C. Borod, E. Koff, and B. White, "Facial asymmetry in posed and spontaneous expressions of emotion," Brain Cognition, vol. 2, no. 2, pp. 165-175, 1983.
24. M. F. Valstar and M. Pantic, "How to distinguish posed from spontaneous smiles using geometric features," in Proc. ACM ICMI, 2007, pp. 38-45.
25. H. Dibeklioʇlu, R. Valenti, A. A. Salah, and T. Gevers, "Eyes do not lie: Spontaneous versus posed smiles," in Proc. ACM Multimedia, 2010, pp. 703-706.
26. L. Zhang, D. Tjondronegoro, and V. Chandran, "Geometry vs. appearance for discriminating between posed and spontaneous emotions," in Proc. ICONIP, 2011, pp. 431-440.
27. M. He, S. Wang, Z. Liu, and X. Chen, "Analyses of the differences between posed and spontaneous facial expressions," in Proc. ACII, 2013, pp. 79-84.
28. J. F. Cohn, L. I. Reed, T. Moriyama, J. Xiao, K. L. Schmidt, and Z. Ambadar, "Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., May 2004, pp. 129-135.
29. L. K. Schmidt, Z. Ambadar, J. F. Cohn, and L. I. Reed, "Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling," J. Nonverbal Behav., vol. 30, no. 1, pp. 37-52, 2006.
30. M. F. Valstar, M. Pantic, Z. Ambadar, and J. F. Cohn, "Spontaneous vs. posed facial behavior: Automatic analysis of brow actions," in Proc. ACM ICMI, 2006, pp. 162-170.
31. T. Pfister, X. Li, G. Zhao, and M. Pietikainen, "Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework," in Proc. ICCV Workshops, 2011, pp. 868-875.
32. H. Tao and T. Huang, "Explanation-based facial motion tracking using a piecewise Bézier volume deformation model," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 1999, vol. 1, pp. 611-617.
33. H. Dibeklioʇlu, A. A. Salah, and T. Gevers, "A statistical method for 2-D facial landmarking," IEEE Trans. Image Process., vol. 21, no. 2, pp. 844-858, Feb. 2012.
34. T. W. Sederberg and S. R. Parry, "Free-form deformation of solid geometric models," ACM SIGGRAPH Comput. Graphics, vol. 20, no. 4, pp. 151-160, 1986.
35. K. L. Schmidt, J. F. Cohn, and Y. Tian, "Signal characteristics of spontaneous facial expressions: Automatic movement in solitary and social smiles," Biological Psychol., vol. 65, no. 1, pp. 49-66, 2003.
36. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
37. S. Petridis, B. Martinez, and M. Pantic, "The MAHNOB laughter database," Image Vis. Comput., vol. 31, no. 2, pp. 186-202, 2013.
38. M. F. Valstar and M. Pantic, "Induced disgust, happiness and surprise: An addition to the MMI facial expression database," in Proc. LREC, Workshop Emotion, 2010, pp. 65-70.
39. S. Wang, Z. Liu, S. Lv, Y. Lv, G. Wu, P. Peng, F. Chen, and X. Wang, "A natural visible and infrared facial expression database for expression recognition and emotion inference," IEEE Trans. Multimedia, vol. 12, no. 7, pp. 682-691, Nov. 2010.
40. M. Pantic and I. Patras, "Detecting facial actions and their temporal segments in nearly frontal-view face image sequences," in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 2005, vol. 4, pp. 3358-3363.
41. M. F. Valstar and M. Pantic, "Fully automatic recognition of the temporal phases of facial actions," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 1, pp. 28-43, Feb. 2012.
42. S. Koelstra and M. Pantic, "Non-rigid registration using free-form deformations for recognition of facial actions and their temporal dynamics," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., Sep. 2008, pp. 1-8.
43. S. Chen, Y. Tian, Q. Liu, and D. N. Metaxas, "Segment and recognize expression phase by fusion of motion area and neutral divergence features," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., Mar. 2011, pp. 330-335.
44. L. I. Kuncheva, "Fusion of continuous-valued outputs," in Combining Pattern Classifiers: Methods and Algorithms. New York, NY, USA: Wiley, 2004, pp. 151-188.
45. L. G. Farkas, Anthropometry of the Head and Face. New York, NY, USA: Raven Press, 1994.
46. H. Dibeklioʇlu, T. Gevers, A. A. Salah, and R. Valenti, "A smile can reveal your age: Enabling facial dynamics in age estimation," in Proc. ACM Multimedia, 2012, pp. 209-218.
47. N. Ramanathan, R. Chellappa, and S. Biswas, "Computational methods for modeling facial aging: A survey," J. Vis. Lang. Comput., vol. 20, no. 3, pp. 131-144, 2009.
48. Y. Fu, G. Guo, and T. S. Huang, "Age synthesis and estimation via faces: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 1955-1976, Nov. 2010.
49. G. Guo, R. Guo, and X. Li, "Facial expression recognition influenced by human aging," IEEE Trans. Affective Comput., vol. 4, no. 3, pp. 291-298, Jul.-Sep. 2013.
50. G. Guo and X. Wang, "A study on human age estimation under facial expression changes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun, 2012, pp. 2547-2553.
51. C. Zhang and G. Guo, "Age estimation with expression changes using multiple aging subspaces," in Proc. BTAS, 2013, pp. 1-6.
52. Y. Saatci and C. Town, "Cascaded classification of gender and facial expression using active appearance models," in Proc. IEEE Int. Conf. Autom. Face Gesture Recog., Apr. 2006, pp. 393-398.
53. K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, "XM2VTS: The extended M2VTS database," AVBPA, vol. 964, pp. 965-966, 1999.
54. A. Savran, N. Alyüz, H. Dibeklioʇlu, O. Çeliktutan, B. Gökberk, B. Sankur, and L. Akarun, "Bosphorus database for 3D face analysis," in Biometrics and Identity Management, ser. Lecture Notes Comput. Sci. Berlin, Germany: Springer, 2008, vol. 5372, pp. 47-56.
55. A. Martinez and R. Benavente, "The AR Face Database," CVC Tech. Rep. 24, 1998.