Spontaneous facial micro-expression analysis using Spatiotemporal Completed Local Quantized Patterns

Neurocomputing

Volumn 175, Issue PartA, 2015, Pages 564-578

  Huang, Xiaohua ^a,b   Zhao, Guoying ^a   Hong, Xiaopeng ^a   Zheng, Wenming ^b   Pietikäinen, Matti ^a  

^a UNIVERSITY OF OULU   (Finland)

^b SOUTHEAST UNIVERSITY   (China)

Author keywords

Discriminative; LBP TOP; Micro expression; Vector quantization

Indexed keywords

STRUCTURAL OPTIMIZATION;

COMPONENT ANALYSIS; CRITICAL PROBLEMS; DISCRIMINATIVE; FACIAL EXPRESSIONS; INTERESTING INFORMATION; LOCAL BINARY PATTERNS; MICRO-EXPRESSIONS; SPATIO TEMPORAL FEATURES;

VECTOR QUANTIZATION;

ALGORITHM; ANALYSIS OF VARIANCE; ARTICLE; CALCULATION; CONTROLLED STUDY; DISCRIMINATION LEARNING; FACIAL EXPRESSION; FACIAL MICROEXPRESSION; FEMALE; FISHER EXACT TEST; HISTOGRAM; HUMAN; INFORMATION PROCESSING; MACHINE LEARNING; MALE; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; SPATIOTEMPORAL COMPLETED LOCAL QUANTIZATION PATTERN;

EID: 84947342915     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2015.10.096     Document Type: Article

References (51)

1. Belongie, S., Malik, J., Puzicha, J., Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24:4 (2002), 509–522.
2. Chan, C., Goswami, B., Kittler, J., Christmas, W., Local ordinal contrast pattern histograms for spatiotemporal, lip-based speaker authentication. IEEE Trans. Inf. Forensics Secur. 7:2 (2012), 602–612.
3. Chang, C., Lin, C., A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2:27 (2011), 1–27.
4. Chen, J., Shan, C., He, C., Zhao, G., Pietikäinen, M., Chen, X., Gao., W., WLD: a robust local image descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 32:9 (2010), 1705–1720.
5. Cootes, T., Taylor, C., Cooper, D., Graham, J., Active shape models-their training and application. Comput. Vis. Image Underst. 61:1 (1995), 38–59.
6. A. Davison, M. Yap, N. Costen, K. Tan, C. Lansley, D. Leightley, Micro-facial movements: an investigation on spatio-temporal descriptors, in: Proceeding of ECCV workshop on Spontaneous Behavior Analysis, 2014, pp. 111–123.
7. P. Dollar, V. Rabaud, G. Cottrell, S. Belongie, Behavior recognition via sparse spatio-temporal features, in: Proceedings of IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65–72.
8. P. Ekman, Lie catching and micro expressions, in: Ed. Clancy Martin (Ed.), The Philosophy of Deception, 2009, pp. 118–133.
9. X. Feng, Y. Lai, X. Mao, J. Peng, X. Jiang, A. Hadid, Extracting local binary patterns from image key points: application to automatic facial expression recognition, in: Proceedings of Scandinavian Conference on Image Analysis, 2013, pp. 339–348.
10. M. Frank, M. Herbasz, K. Sinuk, A. Keller, C. Nolan, I see how you feel: training laypeople and professionals to recognize fleeting emotions, in: The Annual Meeting of International Communication Association, 2009.
11. Gizatdinova, Y., Surakka, V., Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans. Pattern Anal. Mach. Intell. 28:1 (2006), 135–139.
12. Gonzalez, R., Woods, R., Digital Image Processing, 1992, Addison Wesley, Boston, MA, USA, 414–428.
13. Guo, Z., Zhang, L., Zhang, D., A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19:6 (2010), 1657–1663.
14. Hand, D., Till, R., A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach. Learn. 45:2 (2001), 171–186.
15. X. He, P. Niyogi, Locality preserving projections, in: Proceedings of Neural Information Processing Systems, 2003, pp. 153–160.
16. Huang, X., Zhao, G., Zheng, M., Pietikäinen, M., Spatiotemporal local monogenic binary patterns for facial expression recognition. IEEE Signal Process. Lett. 19:5 (2012), 243–246.
17. X. Huang, G. Zhao, X. Hong, M. Pietikäinen, Texture description with completed local quantized patterns, in: Proceedings of Scandinavian Conference on Image Analysis, 2013, pp. 1–10.
18. S. Hussain, B. Triggs, Visual recognition using local quantized patterns, in: Proceedings of European Conference on Computer Vision, 2012, pp. 716–729.
19. S. Hussain, T. Napoleon, F. Jurie, Face recognition using local quantized patterns, in: Proceedings of British Conference on Computer Vision, 2012, pp. 1–11.
20. S. Jain, C. Hu, J. Aggarwal, Facial expression recognition with temporal modeling of shapes, in: Proceedings of IEEE International Conference on Computer Vision, 2011, pp. 1642–1649.
21. F. Kang, R. Jin, R. Sukthankar, Correlated label propagation with application to multi-label learning, in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, 2006, pp. 1719–1726.
22. X. Li, T. Pfister, X. Huang, G. Zhao, M. Pietikäinen, A spontaneous micro-expression database: Inducement, collection and baseline, in: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, 2013, pp. 1–6.
23. Littlewort, G., Bartlett, M., Fasel, I., Susskind, J., Movellan, J., Dynamics of facial expression extracted automatically from video. Image Vis. Comput. 24:6 (2006), 615–625.
24. Liu, Y., Ma, C., Fu, Q., Fu, X., Qin, S., Xie, L., A sketch-based approach for interactive organization of video clips. ACM Trans. Multimed. Comput. Commun. Appl. 11:1 (2014), 2:1–2:21.
25. Nanni, L., Brahnam, S., Lumini, A., Local ternary patterns from three orthogonal planes for human action clssification. Expert Syst. Appl. 38:5 (2011), 5125–5128.
26. Ojala, T., Pietikäinen, M., Mäenpää, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24:7 (2002), 971–987.
27. T. Pfister, X. Li, G. Zhao, M. Pietikäinen, Recognising spontaneous facial micro-expressions, in: Proceedings of IEEE International Conference on Computer Vision, 2011, pp. 1449–1456.
28. T. Pfister, X. Li, G. Zhao, M. Pietikäinen, Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework, in: Proceedings of IEEE International Conference on Computer Vision, 2011, pp. 868–875.
29. S. Polikovsky, Y. Kameda, Y. Ohta, Facial micro-expressions recognition using high speed camera and 3-D gradient descriptor, in: Proceedings of International Conference on Crime Detection and Prevention, 2009, pp. 1–6.
30. S. Polikovsky, Y. Kameda, Y. Ohta, Detection and measurement of facial micro-expression characteristics for psychological analysis, IEICE Technical Report, vol. 110, no. 98, 2009, pp. 57–64.
31. Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T., Computer Vision using Local Binary Patterns, 2011, Springer, London.
32. J. Ruiz-Hernandez, M. Pietikäinen, Encoding local binary patterns using re-parameterization of the second order Gaussian jet, in: Proceedings of International Conference on Automatic Face and Gesture Recognition, 2013, pp. 1–6.
33. Shan, C., Gong, S., McOwan, P., Facial expression recognition based on local binary pattern: a comprehensive study. Image Vis. Comput. 27:6 (2009), 803–816.
34. Shin, G., Chun, J., Spatio-temporal facial expression recognition using optical flow and HMM. Softw. Eng. Artif. Intell. Netw. Parallel/Distrib. Comput., 2008, 27–38.
35. Tzimiropoulos, G., Zafeiriou, S., Pantic, M., Subspace learning from image gradient orientations. IEEE Trans. Pattern Anal. Mach. Intell. 34:12 (2012), 2454–2466.
36. N.G. Vu, A. Caplier, Mining patterns of orientations and magnitudes for face recognition, in: Proceedings of International Joint Conference on Biometrics, 2011, pp. 1–8.
37. S. Wang, W. Yan, X. Li, G. Zhao, X. Fu, Micro-expression recognition using dynamic textures on tensor independent color space, in; Proceedings of International Conference on Pattern Recognition, 2014, pp. 4678–4683.
38. S. Wang, W. Yan, G. Zhao, X. Fu, Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features, in: Proceedings of ECCV Workshop on Spontaneous Behavior Analysis, 2014, pp. 325–338.
39. Y. Wang, J. See, R. Phan, Y. Oh, LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition, in: Proceedings of Asian Conference on Computer Vision, 2014, pp. 525–537.
40. Xie, S., Shan, S., Chen, X., Chen, J., Fusing local patterns of gabor magnitude and phase for face recognition. IEEE Trans. Image Process. 19:5 (2010), 1349–1361.
41. W. Yan, Q. Wu, Y. Liu, S. Wang, X. Fu, CASME database: a dataset of spontaneous micro-expressions collected from neutralized faces, in: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, 2013, pp. 1–7.
42. Yan, W., Li, X., Wang, S., Zhao, G., Liu, Y., Chen, Y., Fu, X., CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLOS One 9:1 (2014), 1–8.
43. W. Yan, S. Wang, Y. Chen, G. Zhao, X. Fu, Quantifying micro-expressions with constraint local model and local binary pattern, in: Proceedings of ECCV Workshop on Spontaneous Behavior Analysis, 2014, pp. 296–305.
44. Yang, Y., An evaluation of statistical approaches to text categorization. Inf. Retr. 1:12 (1999), 69–90.
45. J. Yang, Y. Jiang, A. Hauptmann, C. Ngo, Evaluating bag-of-the-visual-words representations in scene classification, in: Proceedings of the International Workshop on Multimedia Information Retrieval, 2007, pp. 192–206.
46. J. Ylioinas, A. Hadid, Y. Guo, M. Pietikäinen, Efficient image appearance description using dense sampling based local binary patterns, in: Proceedings of Asian Conference on Computer Vision, 2012, pp. 375–388.
47. Z. Zhang, M. Lyons, M. Schuster, S. Akamatsu, Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron, in: Proceedings of International Conference on Automatic Face and Gesture Recognition, 1998, pp. 454–459.
48. Zhang, B., Shan, C., Chen, X., Gao, W., Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Trans. Image Process. 16:1 (2007), 57–68.
49. Zhou, Z., Zhao, G., Guo, Y., Pietikäinen, M., An image-based visual speech animation system. IEEE Trans. Circuits Syst. Video Technol. 22:10 (2012), 1420–1432.
50. Zhao, G., Pietikäinen, M., Dynamic texture recognition using local binary pattern with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29:6 (2007), 915–928.
51. Zhao, G., Ahonen, T., Matas, J., Pietikäinen, M., Rotation invariant image and video description with local binary pattern features. IEEE Trans. Image Process. 21:4 (2012), 1465–1467.