Learning a sparse codebook of facial and body microexpressions for emotion recognition

ICMI 2013 - Proceedings of the 2013 ACM International Conference on Multimodal Interaction

Volumn , Issue , 2013, Pages 237-244

  Song, Yale ^a   Morency, Louis Philippe ^b   Davis, Randall ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

audio visual emotion recognition; data fusion; dictionary learning; microexpressions; sparse coding; spatio temporal interest points

Indexed keywords

DICTIONARY LEARNING; EMOTION RECOGNITION; MICRO-EXPRESSIONS; SPARSE CODING; SPATIO-TEMPORAL INTEREST POINTS;

INTERACTIVE COMPUTER SYSTEMS; TIME AND MOTION STUDY;

DATA FUSION;

EID: 84892611788     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2522848.2522851     Document Type: Conference Paper

References (51)

1. Y.-L. Boureau, J. Ponce, and Y. LeCun. A theoretical analysis of feature pooling in visual recognition. In ICML, 2010.
2. S. Chen, Y. Tian, Q. Liu, and D. N. Metaxas. Recognizing expressions from face and body gesture by temporal normalized motion and appearance features. Image Vision Comput., 31(2), 2013.
3. A. Coates and A. Y. Ng. The importance of encoding versus training with sparse coding and vector quantization. In ICML, 2011.
4. T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. PAMI, 23(6), 2001.
5. D. Cristinacce and T. F. Cootes. Feature detection and tracking with constrained local models. In BMVC, 2006.
6. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.
7. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-temporal features. In VS-PETS, October 2005.
8. P. Ekman. Telling lies: Clues to deceit in the marketplace, politics, and marriage. WW Norton & Company, 2009.
9. P. Ekman and W. V. Friesen. Facial action coding system. 1977.
10. P. Ekman and W. V. Friesen. Unmasking the face: A guide to recognizing emotions from facial clues. I S H K, 2003.
11. E. A. Haggard and K. S. Isaacs. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy. In Methods of research in psychotherapy. Springer, 1966.
12. D. R. Hardoon, S. Szedmák, and J. Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. NECO, 16(12), 2004.
13. J. M. Iverson and S. Goldin-Meadow. Why people gesture when they speak. Nature, 396(6708), 1998.
14. Y. Jia, O. Vinyals, and T. Darrell. On compact codes for spatially pooled features. In ICML, 2013.
15. B. Jiang, M. F. Valstar, and M. Pantic. Action unit detection using sparse appearance descriptors in space-time video volumes. In FG, 2011.
16. J. Joshi, R. Goecke, M. Breakspear, and G. Parker. Can body expressions contribute to automatic depression analysis? In FG, 2013.
17. A. Kendon. Gesture - Visible Action as Utterance. Cambridge University Press, 2004.
18. I. Laptev. On space-time interest points. IJCV, 64(2-3), 2005.
19. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In CVPR, 2008.
20. S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR, 2006.
21. X. Li, T. Pfister, X. Huang, G. Zhao, and M. Pietikainen. A spontaneous micro-expression database: Inducement, collection and baseline. In FG, 2013.
22. M. Liu, S. Li, S. Shan, and X. Chen. AU-aware Deep Networks for Facial Expression Recognition. In FG, 2013.
23. D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2), 2004.
24. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. JMLR, 11, 2010.
25. G. McKeown, M. F. Valstar, R. Cowie, M. Pantic, and M. Schröder. The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent. T. Affective Computing, 3(1), 2012.
26. D. McNeill. Gesture and thought. University of Chicago Press, 2008.
27. J. Nicolle, V. Rapp, K. Bailly, L. Prevost, and M. Chetouani. Robust continuous prediction of human emotions using multiscale dynamic cues. In ICMI, 2012.
28. T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. PAMI, 24(7), 2002.
29. B. A. Olshausen and D. J. Fieldt. Sparse coding with an overcomplete basis set: a strategy employed by V1. Vision Research, 37, 1997.
30. D. Ozkan, S. Scherer, and L.-P. Morency. Step-wise emotion recognition using concatenated-hmm. In ICMI, 2012.
31. M. Pantic and L. J. M. Rothkrantz. Automatic analysis of facial expressions: The state of the art. PAMI, 22(12), 2000.
32. T. Pfister, X. Li, G. Zhao, and M. Pietikäinen. Recognising spontaneous facial micro-expressions. In ICCV, 2011.
33. R. Poppe. A survey on vision-based human action recognition. IVC, 28(6), 2010.
34. T. Qin, T.-Y. Liu, X.-D. Zhang, D.-S. Wang, and H. Li. Global ranking using continuous conditional random fields. In NIPS, 2008.
35. E. Rahtu, J. Heikkilä, V. Ojansivu, and T. Ahonen. Local phase quantization for blur-insensitive image analysis. Image Vision Comput., 30(8), 2012.
36. A. Savran, H. Cao, M. Shah, A. Nenkova, and R. Verma. Combining video, audio and lexical indicators of affect in spontaneous conversation via particle filtering. In ICMI, 2012.
37. B. Schuller, M. Valstar, F. Eyben, R. Cowie, and M. Pantic. Avec 2012: the continuous audio/visual emotion challenge. In ICMI, 2012.
38. C. Shan, S. Gong, and P. W. McOwan. Beyond facial expressions: Learning human emotion from body gestures. In BMVC, 2007.
39. M. Shreve, S. Godavarthy, D. B. Goldgof, and S. Sarkar. Macro- and micro-expression spotting in long videos using spatio-temporal strain. In FG, 2011.
40. A. J. Smola and B. Scholkopf. A tutorial on support vector regression. Technical report, Statistics and computing, 2003.
41. C. Soladié, H. Salam, C. Pelachaud, N. Stoiber, and R. Séguier. A multimodal fuzzy inference system using a continuous facial expression representation for emotion detection. In ICMI, 2012.
42. Y. Song, L.-P. Morency, and R. Davis. Multimodal human behavior analysis: learning correlation and interaction across modalities. In ICMI, 2012.
43. A. Vinciarelli, M. Pantic, D. Heylen, C. Pelachaud, I. Poggi, F. D'Errico, and M. Schröder. Bridging the gap between social animal and unsocial machine: A survey of social signal processing. T. Affective Computing, 3(1), 2012.
44. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Action recognition by dense trajectories. In CVPR, 2011.
45. H. Wang, M. M. Ullah, A. Kläser, I. Laptev, and C. Schmid. Evaluation of local spatio-temporal features for action recognition. In BMVC, 2009.
46. J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. S. Huang, and S. Yan. Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 98(6), 2010.
47. Q. Wu, X. Shen, and X. Fu. The machine knows what you are hiding: An automatic micro-expression recognition system. In ACII, 2011.
48. W.-J. Yan, Q. Wu, Y.-J. Liu, S.-J. Wang, and X. Fu. Casme database: A dataset of spontaneous micro-expressions collected from neutralized faces. In FG, 2013.
49. J. Yang, K. Yu, Y. Gong, and T. S. Huang. Linear spatial pyramid matching using sparse coding for image classification. In CVPR, 2009.
50. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. PAMI, 31(1), 2009.
51. Y. Zhu, X. Zhao, Y. Fu, and Y. Liu. Sparse coding on local spatial-temporal volumes for human action recognition. In ACCV, 2010.