Classifying affective states using thermal infrared imaging of the human face

IEEE Transactions on Biomedical Engineering

Volumn 57, Issue 4, 2010, Pages 979-987

  Nhan, Brian R ^a   Chau, Tom ^a,b  

^a University of Toronto   (Canada)

^b HOLLAND BLOORVIEW KIDS REHABILITATION HOSPITAL   (Canada)

Author keywords

Access technology; Emotion; Genetic algorithm; Infrared; Pattern classification; Thermography

Indexed keywords

IMAGING SYSTEMS; INFRARED DEVICES; REMOTE SENSING; THERMOGRAPHY (TEMPERATURE MEASUREMENT);

ACCESS TECHNOLOGY; AFFECTIVE STATE; BINARY CLASSIFICATION; CLASSIFICATION ACCURACY; HUMAN FACES; PASSIVE COMMUNICATION; PATTERN CLASSIFICATION; THERMAL INFRARED DATA; THERMAL INFRARED IMAGING; TIME FREQUENCY FEATURES; VISUAL STIMULUS;

THERMOGRAPHY (IMAGING);

ACCURACY; AFFECT; ARTICLE; CLASSIFICATION; FACE; FEMALE; FREQUENCY ANALYSIS; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; RESPONSE TIME; SELF REPORT; THERMOGRAPHY; TIME; VISUAL STIMULATION; ADULT; ALGORITHM; ANALYSIS OF VARIANCE; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; EMOTION; IMAGE PROCESSING; PHYSIOLOGY; PROCEDURES;

ADULT; ALGORITHMS; ANALYSIS OF VARIANCE; EMOTIONS; FACE; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MALE; MODELS, GENETIC; PATTERN RECOGNITION, AUTOMATED; THERMOGRAPHY;

EID: 77950213405     PISSN: 00189294     EISSN: None     Source Type: Journal    
DOI: 10.1109/TBME.2009.2035926     Document Type: Article

References (46)

1. K. Tai, S. Blain, and T. Chau, "A review of emerging access technologies for individuals with severe motor impairments," Assist. Technol., vol.20, pp. 204-219, 2008.
2. S. Blain, A. Mihailidis, and T. Chau, "Assessing the potential of electrodermal activity as an alternate access pathway," Med. Eng. Phys., vol.30, pp. 498-505, 2008.
3. N. Memarian, A. Venetsanopoulos, and T. Chau, "Mutual information as a measure of contextual effects on single switch use," Open Rehabil. J., vol.2, pp. 1-10, 2009.
4. N. Birbaumer and L. G. Cohen, "Brain-computer interfaces: Communication and restoration of movement in paralysis," J. Physiol., vol.579, no.3, pp. 621-636, 2007.
5. R. W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: analysis of affective physiological state," IEEE Trans. Pattern Anal. Mach. Intell., vol.23, no.10, pp. 1175-1191, Oct. 2001.
6. I. Pavlidis, J. Dowdall, N. Sun, C. Puri, J. Fei, andM. Garbey, "Interacting with human physiology," Comput. Vis. Image Understanding, vol.108, no.1-2, pp. 150-170, 2007.
7. D. Shastri, A.Merla, P. Tsiamyrtzis, and I. Pavlidis, "Imaging facial signs of neurophysiological responses," IEEE Trans. Biomed. Eng., vol.56, no.2, pp. 477-484, Feb. 2009.
8. J. A. Levine, I. Pavlidis, and M. Cooper, "The face of fear," The Lancet, vol.357, no.9270, pp. 1757-1757, 2001.
9. A. Nozawa and M. Tacano, "Correlation analysis on alpha attenuation and nasal skin temperature," J. Stat. Mech.: Theory Exp., vol.P01007, pp. 1-10, 2009.
10. R. Nakanishi and K. Imai-Matsumura, "Facial skin temperature decreases in infants with joyful expression," Infant Behav. Dev., vol.31, pp. 137- 144, 2008.
11. N. Memarian, A. Venetsanopoulos, and T. Chau, "Infrared thermography as an access pathway for individuals with severe motor impairments," J. Neuroeng. Rehabil., vol.6, no.11, pp. 1-8, 2009.
12. P. J. Lang, M. M. Bradley, and B. N. Cuthbert, "International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report A-6," Univ. Florida, Gainesville, FL, Tech. Rep. 6, 2005.
13. J. A. Russell, "A circumplex model of affect," J. Pers. Soc. Psychol., vol.39, pp. 1161-1178, 1980.
14. J. Britton, K. Phan, S. Taylor, R. Welsh, K. Berridge, and I. Liberzon, "Neural correlates of social and nonsocial emotions: An fMRI study," Neuroimage, vol.31, no.1, pp. 397-409, 2006.
15. S. Anders, M. Lotze, M. Erb, W. Grodd, and N. Birbaumer, "Brain activity underlying emotional valence and arousal: a response-related fMRI study," Human Brain Mapp., vol.23, no.4, pp. 200-209, 2004.
16. F. Dolcos, K. S. LaBar, and R. Cabeza, "Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: An event-related fMRI study," Neuroimage, vol.23, no.1, pp. 64-74, 2004.
17. J. Steketee, "Spectral emissivity of the skin and pericardium," Phys.Med. Biol., vol.18, no.5, pp. 686-694, 1973.
18. B. R. Nhan and T. Chau, "Infrared thermal imaging as a physiological access pathway: A study of the baseline characteristics of facial skin temperatures," Physiol. Meas., vol.30, pp. N23-N35, 2009.
19. A. Haag, S. Gronzy, P. Schaich, and J. Williams, "Emotion recognition using bio-sensors: First steps towards an automatic system," Lecture Notes Comput. Sci., vol.3068, pp. 36-48, 2004.
20. E. F. Ring and K. Ammer, The Biomedical Engineering Handbook, 3rd ed. Boca Raton, FL: CRC Press, 2006, vol.2, pp. 1-9, ch. 36.
21. N. Charkoudian, "Skin blood flow in adult human thermoregulation: how it works, when it does not, and why," Mayo Clinic Proc., vol.78, pp. 603- 612, 2003.
22. N. Otsu, "A threshold selection method for gray-level histogram," IEEE Trans. Syst., Man, Cybern., vol.SMC-9, no.1, pp. 62-66, Jan. 1979.
23. J. Lee, S. Blain, and T. Chau, "A radial basis classifier for automatic detection of aspiration in children," J. Neural Eng. Rehabil., vol.3, pp. 1- 17, 2006.
24. T. Chau, D. Chau, M. Casas, G. Berall, and D. J. Kenny, "Investigating the stationarity of paediatric aspiration signals," IEEE Trans. Neural Syst. Rehabil. Eng., vol.13, no.1, pp. 99-105, Mar. 2005.
25. S. Uematsu, "Symmetry of skin temperature comparing one side of the body to the other," Thermology, vol.1, pp. 4-7, 1986.
26. J. Kim and E. Andŕe, "Emotion recognition based on physiological changes in music listening," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.12, pp. 2067-2083, Dec. 2008.
27. J. Yang and V. Honavar, "Feature subset selection using a genetic algorithm," IEEE Intell. Syst. Appl., vol.13, no.2, pp. 44-49, Mar./Apr. 1998.
28. J. Hong and S. Cho, "Efficient huge-scale feature selection with speciated genetic algorithm," Pattern Recog. Lett., vol.27, pp. 143-150, 2006.
29. I. Oh, J. Lee, and B. Moon, "Hybrid genetic algorithms for feature selection," IEEE Trans. Pattern Anal. Mach. Intell., vol.26, no.11, pp. 1424- 1437, Nov. 2004.
30. S. J. Raudys and A. K. Jain, "Small sample size effects in statistical pattern recognition: Recommendations for practitioners," IEEE Trans. Pattern Anal. Mach. Intell., vol.13, no.3, pp. 252-264, Mar. 1991.
31. G. Tzanis, C. Berberidis, A. Alexandridou, and I. Vlahavas, "Improving the accuracy of classifiers for the prediction of translation initiaion sites in genomic sequences," Lecture Notes Comput. Sci., vol.3746, pp. 426-436, 2005.
32. S. C. Manoharan, M. Veezhinathan, and S. Ramakrishman, "Comparison of two ANN methods for classification of spirometer data," Meas. Sci. Rev., vol.8, no.3, pp. 53-57, 2008.
33. F. Zeng, R. Yap, and L. Wong, "Using feature generation and feature selection for accurate prediction of translation initiation sites," Genome Inf., vol.13, pp. 192-200, 2002.
34. B. Efron, "Bootstrap methods: Another look at the jackknife," Ann. Stat., vol.7, no.1, pp. 1-26, 1979.
35. H. Xiong, Y. Zhang, and X. Chen, "Data-dependent kernel machines for microarray data classification," IEEE/ACM Trans. Comput. Biol. Bioinformatics, vol.4, no.4, pp. 583-595, Oct.-Dec. 2007.
36. W. Fu, R. Carroll, and S. Wang, "Estimating misclassification error with small samples via bootstrap cross-validation," Bioinformatics, vol.21, no.9, pp. 1979-1986, 2005.
37. A. K. Jain, R. Dubes, and C. Chen, "Bootstrap techniques for error estimation," IEEE Trans. Pattern Recog. Mach. Intell., vol.PAMI-9, no.5, pp. 628-633, Sep. 1987.
38. R. Vetrugno, R. Liguori, P. Cortelli, and P. Montagna, "Sympathetis skin response: Basic mechanisms and clinical applications," Clin. Auton. Res., vol.13, no.4, pp. 256-270, 2003.
39. L. Li and J. Chen, "Emotion recognition using physiological signals," Lecture Notes Comput. Sci., vol.4282, pp. 437-446, 2006.
40. C. L. Stanfield and W. J. Germann, Principles of Human Physiology, 3rd ed. San Francisco, CA: Pearson Benjamin Cummings, 2007.
41. K. Krauchi and A. Wirz-Justice, "Circadian rhythm of heat production, heart rate, and skin and core temperature under unmasking conditions in men," Amer. J. Appl. Physiol.: Regul., Integr. Comparitive Physiol., vol.267, no.3, pp. 819-829, 1994.
42. N. Zaproudina,V.Varmavuo, O. Airaksinen, and M.Närhi, "Reproducibility of infrared thermography measurements in healthy individuals," Physiol. Meas., vol.29, pp. 515-524, 2008.
43. E. F. J. Ring, "The historical development of thermal imaging in medicine," Rheumatology, vol.43, no.6, pp. 800-802, 2004.
44. I. T. Pavlidis, "Coalitional tracking," Comput. Vis. Image Understanding, vol.106, no.2-3, pp. 205-219, 2007.
45. B. F. Jones, "A reappraisal of the use of infrared thermal image analysis in medicine," IEEE Trans. Med. Imag., vol.17, no.6, pp. 1019-1027, Dec. 1998.
46. M. Naito,Y.Michioka,K.Ozawa,Y. Ito,M.Kiguchi, and T. Kanazawa, "A communication means for totally locked-in als patients based on changes in cerebral blood volume measured with near-infreared light," IEICE Trans. Inf. Syst., vol.90, pp. 1028-1037, 2007.