Spatiotemporal analysis of RGB-D-T facial images for multimodal pain level recognition

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Volumn 2015-October, Issue , 2015, Pages 88-95

  Irani, Ramin ^a   Nasrollahi, Kamal ^a   Simon, Marc O ^b,c   Corneanu, Ciprian A ^b,c   Escalera, Sergio ^b,c   Bahnsen, Chris ^a   Lundtoft, Dennis H ^a   Moeslund, Thomas B ^a   Pedersen, Tanja L ^d   Klitgaard, Maria Louise ^d   Petrini, Laura ^d  

^a Visual Analysis of People (VAP) Laboratory   (Denmark)

^b Campus UAB ^*  (Spain)

^c UNIVERSITY OF BARCELONA   (Spain)

^d AALBORG UNIVERSITY   (Denmark)

Author keywords

Calibration; Face; Face recognition; Feature extraction; Histograms; Pain; Spatiotemporal phenomena

Indexed keywords

CALIBRATION; COMPUTER VISION; FEATURE EXTRACTION; HEALTH;

COMPUTER VISION TECHNIQUES; FACE; HISTOGRAMS; MULTI-MODAL APPROACH; PAIN; SPATIO-TEMPORAL FILTER; SPATIOTEMPORAL ANALYSIS; SPATIOTEMPORAL PHENOMENA;

FACE RECOGNITION;

EID: 84951947131     PISSN: 21607508     EISSN: 21607516     Source Type: Conference Proceeding    
DOI: 10.1109/CVPRW.2015.7301341     Document Type: Conference Paper

References (44)

1. Z. Ambadar, W. Schooler, and J. Cohn. The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16 (5): 403-410, 2005.
2. N. Ambady and R. Rosenthal. Thin slices of expressive behavior as predictors of interpersonal consequences: A metaanalysis. Psychological Bull., 111 (2): 256-274, 1992.
3. A. B. Ashraf, S. Lucey, J. F. Cohn, T. Chen, Z. Ambadar, K. M. Prkachin, and P. E. Solomon. The painful face-pain expression recognition using active appearance models. Image and vision computing, 27 (12): 1788-1796, 2009.
4. P. Bateson. Assessment of pain in animals. Animal Behaviour, 52: 827-839, 1991.
5. P. N. Belhumeur, D.W. Jacobs, D. Kriegman, and N. Kumar. Localizing parts of faces using a consensus of exemplars. In CVPR, pages 545-552. IEEE, 2011.
6. S. Brahnam, C.-F. Chuang, F. Y. Shih, and M. R. Slack. Machine recognition and representation of neonatal facial displays of acute pain. Artificial Intelligence in Medicine, 36 (3): 211-222, 2006.
7. K. Cannons and R.Wildes. The applicability of spatiotemporal oriented energy features to region tracking. IEEE Trans. Pattern Anal. Mach. Intell., 36 (4): 784-796, 2014.
8. S. Coffman, Y. Alvarez, M. Pyngolil, R. Petit, C. Hall, and M. Smyth. Nursing assessment and management of pain in critically ill children. Heart Lung, 26 (221), 1997.
9. J. Cohn and K. Schmidt. The timing of facial motion in posed and spontaneous smiles. Journal of Wavelets, Multiresolution & Information Processing, 2 (2): 121-132, 2004.
10. A. R. Damasio. The feeling of what happens: Body, emotion and the making of consciousness. 1999.
11. P. Ekman and E. Rosenberg. What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system, 2nd edn. Oxford University Press, London, 2005.
12. T. Hadjistavropoulos. Social influences and the communication of pain. Pain: psychological perspectives, page 87, 2004.
13. Z. Hammal and J. F. Cohn. Automatic detection of pain intensity. In ACM-ICMI, pages 47-52. ACM, 2012.
14. Z. Hammal and J. F. Cohn. Towards multimodal pain assessment for research and clinical use. In Roadmapping the Future of Multimodal Interaction Research, Business Opportunities and Challenges, pages 13-17. ACM, 2014.
15. Z. Hammal and M. Kunz. Pain monitoring: A dynamic and context-sensitive system. Pattern Recognition, 45 (4): 1265-1280, 2012.
16. R. Irani, K. Nasrollahi, and T. B. Moeslund. Pain recognition using spatiotemporal oriented energy of facial muscles. In Computer Vision and Pattern Recognition Workshop, 2015 IEEE Conference on, pages 679-692. IEEE, 2015.
17. W. Janig. The sympathetic nervous system in pain. Eur. J. Anaesthesiol, (10): 53-60.
18. S. Kaltwang, O. Rudovic, and M. Pantic. Continuous pain intensity estimation from facial expressions. Advances in Visual Computing, pages 368-377, 2012.
19. L. I. Kuncheva. Combining Pattern Classifier: Methods and Algorithms. John Wiley & Sons, 2004.
20. V. Le, J. Brandt, Z. Lin, L. Bourdev, and T. S. Huang. Interactive facial feature localization. In Computer Vision-ECCV 2012, pages 679-692. Springer, 2012.
21. G. C. Littlewort, M. S. Bartlett, and K. Lee. Faces of pain: automated measurement of spontaneous facial expressions of genuine and posed pain. In Proceedings of the 9th international conference on Multimodal interfaces, pages 15-21. ACM, 2007.
22. G. C. Littlewort, M. S. Bartlett, and K. Lee. Automatic coding of facial expressions displayed during posed and genuine pain. ICV, 27 (12): 1797-1803, 2009.
23. P. Lucey, J. Cohn, S. Lucey, I. Matthews, S. Sridharan, and K. M. Prkachin. Automatically detecting pain using facial actions. In Affective Computing and Intelligent Interaction, pages 1-8. IEEE, 2009.
24. P. McGrath. Pain in children: nature, assessment and treatment. Guildford Press, New York.
25. N. B. Patel. Physiology of pain. Guide to pain management in low-resource settings, page 13, 2010.
26. K. M. Prkachin. The consistency of facial expressions of pain: a comparison across modalities. Pain, 51 (3): 297-306, 1992.
27. K. M. Prkachin. Assessing pain by facial expression: facial expression as nexus. Pain Research & Management: The Journal of the Canadian Pain Society, 14 (1): 53, 2009.
28. K. M. Prkachin and K. D. Craig. Expressing pain: The communication and interpretation of facial pain signals. Journal of Nonverbal Behavior, 19 (4): 191-205, 1995.
29. K. M. Prkachin and P. E. Solomon. The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain, 139 (2): 267-274, 2008.
30. K. M. Prkachin, P. E. Solomon, and J. Ross. Underestimation of pain by health-care providers: towards a model of the process of inferring pain in others. CJNR (Canadian Journal of Nursing Research), 39 (2): 88-106, 2007.
31. O. Rudovic, V. Pavlovic, and M. Pantic. Automatic pain intensity estimation with heteroscedastic conditional ordinal random fields. Advances in Visual Computing, pages 234-243, 2013.
32. J. A. Russell, J. Bachorowski, and J. Fernandez-Dols. Facial and Vocal Expressions of Emotion. 54: 329-349, 2003.
33. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on, pages 397-403. IEEE, 2013.
34. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. A semi-automatic methodology for facial landmark annotation. In CVPR Workshops (CVPRW), 2013, pages 896-903. IEEE, 2013.
35. K. R. Scherer. Appraisal theory. Handbook of cognition and emotion, pages 637-663, 1999.
36. M. Valstar, M. Pantic, Z. Ambadar, and J. Cohn. Spontaneous versus posed facial behavior: automatic analysis of Brow actions. Proc eight intl conf multimodal interfaces (ICMI06), pages 162-170, 2006.
37. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001, volume 1, pages I-511. IEEE, 2001.
38. S. Wang, Z. Liu, S. Lv, Y. Lv, G. Wu, P. Peng, F. Chen, and X. Wang. A natural visible and infrared facial expression database for expression recognition and emotion inference. T. Multimedia, 12 (7): 682-691, 2010.
39. P. Werner, A. Al-Hamadi, and R. Niese. Pain recognition and intensity rating based on comparative learning. In ICIP, pages 2313-2316. IEEE, 2012.
40. P. Werner, A. Al-Hamadi, R. Niese, S. Walter, S. Gruss, and H. C. Traue. Towards pain monitoring: Facial expression, head pose, a new database, an automatic system and remaining challenges. In BMVC, pages 119-1, 2013.
41. P. Werner, A. Al-Hamadi, R. Niese, S. Walter, S. Gruss, and H. C. Traue. Automatic pain recognition from video and biomedical signals. In ICPR, pages 4582-4587. IEEE, 2014.
42. A. C. d. C. Williams. Facial expression of pain, empathy, evolution, and social learning. Behavioral and brain sciences, 25 (04): 475-480, 2002.
43. X. Xiong and F. De la Torre. Supervised descent method and its applications to face alignment. In CVPR, pages 532-539. IEEE, 2013.
44. X. Zhu and D. Ramanan. Face detection, pose estimation, and landmark localization in the wild. In CVPR.