Objective measures, sensors and computational techniques for stress recognition and classification: A survey

Computer Methods and Programs in Biomedicine

Volumn 108, Issue 3, 2012, Pages 1287-1301

  Sharma, Nandita ^a   Gedeon, Tom ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Computational stress model; Pattern recognition; Stress classification; Stress computational techniques; Stress prediction; Stress sensors

Indexed keywords

COMPUTATIONAL TECHNIQUE; OBJECTIVE MEASURE; OPTIMAL SENSOR; STRESS LEVELS; STRESS MODELS; STRESS PREDICTION; STRESS SENSOR; WORK PRODUCTIVITY;

BAYESIAN NETWORKS; NEURAL NETWORKS; PATTERN RECOGNITION; RESEARCH; SENSOR DATA FUSION; STRESS MEASUREMENT; SURVEYS;

SENSORS;

ALPHA RHYTHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTONOMIC NERVOUS SYSTEM; BAYES THEOREM; BEHAVIOR; BETA RHYTHM; BLOOD PRESSURE MEASUREMENT; BLOOD VOLUME; BODY MOVEMENT; BREATHING; CIRCULATION; CLINICAL CLASSIFICATION; DECISION TREE; DENDRITE; ELECTRIC POTENTIAL; ELECTRODERMAL RESPONSE; ELECTROENCEPHALOGRAM; ELECTROENCEPHALOGRAPHY; ELECTROMYOGRAPHY; EMISSION TOMOGRAPHY; EYE MOVEMENT; EYELID REFLEX; FACIAL EXPRESSION; FACIES; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUZZY SYSTEM; GAZE; GENETIC ALGORITHM; GESTURE; HEAD MOVEMENT; HEART RATE VARIABILITY; HIDDEN MARKOV MODEL; MATHEMATICAL MODEL; MOVEMENT (PHYSIOLOGY); MUSCLE ACTION POTENTIAL; MUSCLE TONE; NERVE CELL STIMULATION; NEUROFEEDBACK; PATTERN RECOGNITION; POSITRON; POSITRON EMISSION TOMOGRAPHY; PROBABILITY; PUPIL; SENSOR; SKIN CONDUCTANCE; SKIN TEMPERATURE; STRESS; STRESS CLASSIFICATION; STRESS MANAGEMENT; SUPPORT VECTOR MACHINE; VOICE;

COMPUTATIONAL BIOLOGY; DATA COLLECTION; HEART RATE; HUMANS; SKIN; STRESS, PHYSIOLOGICAL;

EID: 84868578631     PISSN: 01692607     EISSN: 18727565     Source Type: Journal    
DOI: 10.1016/j.cmpb.2012.07.003     Document Type: Article

References (130)

1. Feuerstein M., et al. Health Psychology: A Psychobiological Perspective 1986, Springer.
2. Steckler T., et al. Handbook of Stress and the Brain 2005, Elsevier Science, Amsterdam.
3. Segerstrom S.C., Miller G.E. Psychological stress and the human immune system: a meta-analytic study of 30 years of inquiry. Psychological Bulletin 2004, 130:601.
4. Lifeline Australia Stress Costs Taxpayer $300K Every Day 2009, Available: http://www.lifeline.org.au/.
5. International Stress Management Association UK, 2010. Available: http://www.isma.org.uk/.
6. SupportLine UK Stress 2010, Available: http://www.supportline.org.uk/problems/stress.php.
7. Lifeline Australia Stress Down Day 2010, Available: http://www.lifeline.org.au/About-Lifeline/Lifeline-Locations/Central-Australia-NT/News-Articles/Stress-Down-Day/blog.aspx.
8. Kumar M., et al. Fuzzy evaluation of heart rate signals for mental stress assessment. IEEE Transactions on Fuzzy Systems 2007, 15:791-808.
9. Jin P. Efficacy of tai chi, brisk walking, meditation, and reading in reducing mental and emotional stress. Journal of Psychosomatic Research 1992, 36:361-370.
10. Lerner J.S., et al. Facial expressions of emotion reveal neuroendocrine and cardiovascular stress responses. Biological Psychiatry 2007, 61:253-260.
11. F.E. Ritter, et al., Measuring the effect of dental work as a stressor on cognition (Tech. Report No. 2005-1). Applied Cognitive Science Lab, School of Information Sciences and Technology, Penn State. acs.ist.psu.edu/acslab/reports/ritterCRK05.pdf, 2005.
12. Lundberg U., et al. Psychophysiological stress and EMG activity of the trapezius muscle. International Journal of Behavioral Medicine 1994, 1:354-370.
13. Dishman R.K., et al. Heart rate variability, trait anxiety, and perceived stress among physically fit men and women. International Journal of Psychophysiology 2000, 37:121-133.
14. Steptoe A., Marmot M. Impaired cardiovascular recovery following stress predicts 3-year increases in blood pressure. Journal of Hypertension 2005, 23:529.
15. Partala T., Surakka V. Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies 2003, 59:185-198.
16. Lundberg U. Stress, subjective and objective health. International Journal of Social Welfare 2006, 15:S41-S48.
17. Labbé E., et al. Coping with stress: the effectiveness of different types of music. Applied Psychophysiology and Biofeedback 2007, 32:163-168.
18. Katsis C.D., et al. Toward emotion recognition in car-racing drivers: a biosignal processing approach. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 2008, 38:502-512.
19. Rothkrantz L.J.M., et al. Voice stress analysis. Lecture Notes in Computer Science 2004, 3206:449-456.
20. Schindler K., et al. Recognizing emotions expressed by body pose: a biologically inspired neural model. Neural Networks 2008, 21:1238-1246.
21. Niculescu A., et al. Manipulating stress and cognitive load in conversational interactions with a multimodal system for crisis management support. Development of Multimodal Interfaces: Active Listening and Synchrony 2010, 134-147.
22. Vizer L.M., et al. Automated stress detection using keystroke and linguistic features: an exploratory study. International Journal of Human-Computer Studies 2009, 67:870-886.
23. Healey J.A., Picard R.W. Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems 2005, 6:156-166.
24. Chi E.H., et al. Guest editors' introduction: pervasive computing in sports technologies. IEEE Pervasive Computing 2005, 4:22-25.
25. Kompier M., Cooper C.L. Preventing Stress Improving Productivity: European Case Studies in the Workplace 1999, Routledge.
26. Sem-Jacobsen C.W. Electroencephalographic Study of Pilot Stresses in Flight 1961, Gaustad Hospital-EEG Research Lab, Oslo, Norway.
27. Hennessy D.A., Wiesenthal D.L. Traffic congestion, driver stress, and driver aggression. Aggressive Behavior 1999, 25:409-423.
28. Healey J., Picard R. Smartcar: Detecting Driver Stress 2000, p. 4218.
29. Zhai J., Barreto A. Stress detection in computer users based on digital signal processing of noninvasive physiological variables. Proceedings of the 28th IEEE EMBS Annual International Conference 2006, 1355-1358.
30. Hopkins C.S., et al. Evaluation of voice stress analysis technology. Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS '05) 2005, 20b.
31. Haddad D., et al. Investigation and Evaluation of Voice Stress Analysis Technology 2001, Air Force Research Lab Rome NY Information Directorate.
32. Roscoe A. Assessing pilot workload. Why measure heart rate, HRV and respiration?. Biological Psychology 1992, 34:259-287.
33. Sexton J.B., et al. Error, stress, and teamwork in medicine and aviation: cross sectional surveys. British Medical Journal 2000, 320:745.
34. Kemper K.J., Danhauer S.C. Music as therapy. Southern Medical Journal 2005, 98:282.
35. Ulstein I., et al. High score on the Relative Stress Scale, a marker of possible psychiatric disorder in family carers of patients with dementia. International Journal of Geriatric Psychiatry 2007, 22:195-202.
36. Miller J.J., et al. Three-year follow-up and clinical implications of a mindfulness meditation-based stress reduction intervention in the treatment of anxiety disorders. General Hospital Psychiatry 1995, 17:192-200.
37. Weidner G., et al. Hostility and cardiovascular reactivity to stress in women and men. Psychosomatic Medicine 1989, 51:36.
38. Lemyre L., Tessier R. Measuring psychological stress. Concept, model, and measurement instrument in primary care research. Canadian Family Physician 2003, 49:1159.
39. Wartella J.E., et al. Emotional distress, coping and adjustment in family members of neuroscience intensive care unit patients. Journal of Psychosomatic Research 2009, 66:503-509.
40. I. BIOPAC Systems. MP System Hardware Guide, 2010. Available: http://www.biopac.com/.
41. Jänig W., McLachlan E.M. Characteristics of function-specific pathways in the sympathetic nervous system. Trends in Neurosciences 1992, 15:475-481.
42. Lang P.J., et al. Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 1993, 30:261-273.
43. Seyle H. The Stress of Life 1956, McGraw-Hill, New York.
44. Ferreira P., et al. License to chill!: how to empower users to cope with stress. Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges 2008, 123-132.
45. Liao W., et al. A real-time human stress monitoring system using dynamic Bayesian network. Computer Vision and Pattern Recognition-Workshops, CVPR Workshops 2005.
46. Jacobs S.C., et al. Use of skin conductance changes during mental stress testing as an index of autonomic arousal in cardiovascular research. American Heart Journal 1994, 128:1170-1177.
47. D. Bersak, et al., Intelligent biofeedback using an immersive competitive environment, presented at the Designing Ubiquitous Computing Games Workshop at UbiComp 2001, Atlanta, GA, USA, 2001.
48. Shi Y., et al. Galvanic skin response (GSR) as an index of cognitive load. CHI '07 extended abstracts on Human Factors in Computing Systems 2007, 2651-2656.
49. Lin T., et al. Do physiological data relate to traditional usability indexes?. Proceedings of the 17th Australia Conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future 2005, 1-10.
50. Miller L.H., Shmavonian B.M. Replicability of two GSR indices as a function of stress and cognitive activity. Journal of Personality and Social Psychology 1965, 753-756.
51. McEwen B.S., Sapolsky R.M. Stress and cognitive function. Journal of Current Opinion in Neurobiology 1995, 5:205-216.
52. Zhai J., Barreto A. Stress recognition using non-invasive technology. Proceedings of the 19th International Florida Artificial Intelligence Research Society Conference FLAIRS 2006, 395-400.
53. Affectiva Q Sensor 2012, Available: http://www.affectiva.com/q-sensor/.
54. Lisetti C.L., Nasoz F. Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP Journal on Applied Signal Processing 2004, 2004:1672-1687.
55. Apex-Fitness BodyBugg 2012, Available: http://www.bodybugg.com/.
56. J.A. Healey, Wearable and automotive systems for affect recognition from physiology, Doctor of Philosophy, Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge, 2000.
57. Rajendra Acharya U., et al. Heart rate variability: a review. Medical and Biological Engineering and Computing 2006, 44:1031-1051.
58. Niskanen J.P., et al. Software for advanced HRV analysis. Computer Methods and Programs in Biomedicine 2004, 76:73-82.
59. Rowe D.W., et al. Heart rate variability: indicator of user state as an aid to human-computer interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1998, 480-487.
60. Western Cape Direct StressEraser 2010, Available: http://stresseraser.com/.
61. HeartMath Australasia emWave 2010, Available: http://www.emwave.com.au/.
62. G.D. Clifford, Signal processing methods for heart rate variability, Doctor of Philosophy, Engineering Science, University of Oxford, 2002.
63. Bansal D., et al. A review of measurement and analysis of heart rate variability. International Conference on Computer and Automation Engineering (ICCAE '09) 2009, 243-246.
64. Devereux R.B., et al. Left ventricular wall stresses and wall stress-mass-heart rate products in hypertensive patients with electrocardiographic left ventricular hypertrophy: the LIFE study. Journal of Hypertension 2000, 18:1129-1138.
65. Bonoris P.E., et al. Significance of changes in R wave amplitude during treadmill stress testing: angiographic correlation. The American Journal of Cardiology 1978, 41:846-851.
66. Robinson B.F., et al. Control of heart rate by the autonomic nervous system: studies in man on the interrelation between baroreceptor mechanisms and exercise. Circulation Research 1966, 19(August):400-411.
67. Salahuddin L., Kim D. Detection of acute stress by heart rate variability using a prototype mobile ECG sensor. ICHIT Hybrid Information Technology 2006, 6:453-459.
68. Bailón R., et al. Analysis of heart rate variability using time-varying frequency bands based on respiratory frequency,. 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2007) 2007, 6674-6677.
69. Force T. Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation 1996, 93:1043-1065.
70. Hjortskov N., et al. The effect of mental stress on heart rate variability and blood pressure during computer work. European Journal of Applied Physiology 2004, 92:84-89.
71. Z. Dharmawan, Analysis of computer games player stress level using EEG data, Master of Science Thesis Report, Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Netherlands, 2007.
72. Interactive Productline IP AB-Mindball, 2010. Available: http://www.mindball.se/index.html.
73. D. Novák, et al., EEG and VEP signal processing, Technical Report. Czech Technical University in Prague, Department of Cybernetics, 2004.
74. Horlings R., et al. Emotion recognition using brain activity. Proceedings of the 9th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing 2008, pp. II.1-1.
75. Morilak D.A., et al. Role of brain norepinephrine in the behavioral response to stress. Progress in Neuro-Psychopharmacology and Biological Psychiatry 2005, 29:1214-1224.
76. Hoffmann E. Brain training against stress: theory methods and results from an outcome study. Stress Report 2005, 4.
77. Lin T., John L. Quantifying mental relaxation with EEG for use in computer games. International Conference on Internet Computing 2006, 409-415.
78. Simmons A.N., et al. Functional activation and neural networks in women with posttraumatic stress disorder related to intimate partner violence. Biological Psychiatry 2008, 64:681-690.
79. Hjorth B. EEG analysis based on time domain properties. Electroencephalography and Clinical Neurophysiology 1970, 29:306-310.
80. Sulaiman N., et al. Initial investigation of human physical stress level using brainwaves. IEEE Student Conference on Research and Development (SCOReD) 2009, 230-233.
81. Hamid N.H.A., et al. Evaluation of human stress using EEG power spectrum. 6th International Colloquium on Signal Processing and Its Applications (CSPA) 2010, 1-4.
82. M. Honal, T. Schultz, Identifying user state using electroencephalographic data, presented at the Workshop on Multimodal Multipary Meeting, Trento, Italy, 2005.
83. Pickering T., et al. Environmental influences on blood pressure and the role of job strain. Journal of Hypertension. Supplement: Official Journal of the International Society of Hypertension 1996, 14:179-185.
84. Reisman S. Measurement of physiological stress. Bioengineering Conference 1997, 21-23.
85. ® in the detection of cardiac rhythms. Applied Psychophysiology and Biofeedback 2008, 33:83-89.
86. HeartMath The Science Behind the emWave® Desktop & emWave2 Products 2010, Available: http://www.heartmathstore.com/cgi-bin/category.cgi?category=sciencebehind.
87. Carter N., et al. A combined Bayesian Markovian approach for behaviour recognition. Pattern Recognition 2006, 1:761-764.
88. Liu A., Salvucci D. Modeling and prediction of human driver behavior. 9th International Conference on Human Computer Interaction 2001, 235-244.
89. Pentland A., Liu A. Modeling and prediction of human behavior. Neural Computation 1999, 11:229-242.
90. Google NEVEN Vision 2010, Available: http://www.nevenvision.com/.
91. Dinges D.F., et al. Optical computer recognition of facial expressions associated with stress induced by performance demands. Aviation, Space, and Environmental Medicine 2005, 76:B172-B182.
92. Jabon M., et al. Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Computing 2010, 10:84-95.
93. Susskind J., et al. Human and computer recognition of facial expressions of emotion. Neuropsychologia 2007, 45:152-162.
94. Dubuisson S., et al. A solution for facial expression representation and recognition. Signal Processing: Image Communication 2002, 17:657-673.
95. Barreto A., et al. Non-intrusive physiological monitoring for automated stress detection in human-computer interaction. Human-Computer Interaction 2007, 29-38.
96. Peavler W.S. Pupil size, information overload, and performance differences. Psychophysiology 1974, 11:559-566.
97. Barreto A., et al. Pupil diameter measurements: untapped potential to enhance computer interaction for eye tracker users?. Proceedings of the 10th international ACM SIGACCESS conference on Computers and Accessibility 2008, 269-270.
98. S.Machines FaceLAB 2010, Available: http://www.seeingmachines.com/product/facelab/.
99. Haak M., et al. Detecting stress using eye blinks and brain activity from EEG signals. Proceeding of the 1st Driver Car Interaction and Interface (DCII 2008) 2008.
100. Murray I.R., et al. Towards a definition and working model of stress and its effects on speech. Speech Communication 1996, 20:3-12.
101. Wittels P., et al. Voice monitoring to measure emotional load during short-term stress. European Journal of Applied Physiology 2002, 87:278-282.
102. Williams C.E., Stevens K.N. On determining the emotional state of pilots during flight: an exploratory study. Aerospace Medicine 1969, 40:1369-1372.
103. Nwe T.L., et al. Speech emotion recognition using hidden Markov models. Speech Communication 2003, 41:603-623.
104. Gopalan K. On the effect of stress on certain modulation parameters of speech. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '01) 2001, 101-104.
105. Scherer S., et al. Emotion recognition from speech: stress experiment. Proceedings of the 6th International Language Resources and Evaluation (LREC 2008) 2008.
106. Hansen J., Patil S. Speech under stress: analysis, modeling and recognition. Speaker Classification I 2007, 108-137.
107. Fernandez R., Picard R.W. Modeling drivers' speech under stress. Speech Communication 2003, 40:145-159.
108. Ververidis D., Kotropoulos C. Emotional speech recognition: resources, features, and methods. Speech Communication 2006, 48:1162-1181.
109. Ang J., et al. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. 7th International Conference on Spoken Language Processing 2002, 2037-2040.
110. Rahurkar M., et al. Frequency band analysis for stress detection using a Teager energy operator based feature. 7th International Conference on Spoken Language Processing 2002, 2021-2024.
111. X. Liu, Voice stress analysis: detection of deception, Master's Thesis at the University of Sheffield, Department of Computer Science, 2005.
112. Sakr G.E., et al. Multi level SVM for subject independent agitation detection. IEEE/ASME International Conference on Advanced Intelligent Mechatronics 2009, 538-543.
113. Hosseini S.A., Khalilzadeh M.A. Emotional stress recognition system using EEG and psychophysiological signals: using new labelling process of EEG signals in emotional stress state. International Conference of Biomedical Engineering and Computer Science (ICBECS) 2010, 1-6.
114. Mindplace ThoughtStream 2012, Available: http://www.mindplace.com/Mindplace-Thoughtstream-USB-Personal-Biofeedback/dp/B005NDGPLC.
115. Mindplace Procyon 2012, Available: http://www.mindplace.com/Mindplace-Procyon-System-Meditation-Machine/dp/B005NDHWWI.
116. Mindplace Proteus 2012, Available: http://www.mindplace.com/MindPlace-Proteus-Light-Meditation-Machine/dp/B005NDED02.
117. Fukuda O., et al. Evaluation of heart rate variability by using wavelet transform and a recurrent neural network. Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 2 2001, 1769-1772.
118. Mathworks. Matlab., 2010. Available: http://www.mathworks.com.au/products/matlab/.
119. I. BIOPAC Systems. AcqKnowledge Software, 2010. Available: http://www.biopac.com/acqknowledge-data-acquisition-analysis-software-win.
120. Han J., Kamber M. Data Mining: Concepts and Techniques 2006, Morgan Kaufmann, San Francisco. 2nd ed.
121. Fukuoka Y., Ishida A. Chronic stress evaluation using neural networks. IEEE Engineering in Medicine and Biology Magazine 2000, 19:34-38.
122. Bundele M.M., Banerjee R. Detection of fatigue of vehicular driver using skin conductance and oximetry pulse: a neural network approach. Proceedings of the 11th International Conference on Information Integration and Web-based Applications and Services 2009, 739-744.
123. Zhou G., et al. Nonlinear feature based classification of speech under stress. IEEE Transactions on Speech and Audio Processing 2002, 9:201-216.
124. Schuller B., et al. Hidden Markov model-based speech emotion recognition. IEEE International Conference on Acoustics Speech, and Signal Processing (ICASSP '03) 2003, pp. II-1-4.
125. Kumar M., et al. Fuzzy techniques for subjective workload-score modeling under uncertainties. IEEE Transactions on Systems, Man, and Cybernetics: Part B: Cybernetics 2008, 38:1449-1464.
126. Kumar M., et al. Fuzzy filtering for physiological signal analysis. IEEE Transactions on Fuzzy Systems 2010, 18:208-216.
127. Kurose H., et al. A method for selecting facial expression based on emotions and its application for text reading. IEEE International Conference on Systems, Man and Cybernetics (SMC '06) 2006, 4028-4033.
128. Yoo S.K., et al. Neural network based emotion estimation using heart rate variability and skin resistance. Advances in Natural Computation 2005, 818-824.
129. Wilson G.F., Russell C.A. Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Human Factors: The Journal of the Human Factors and Ergonomics Society 2003, 45:635.
130. Loizides A., et al. Measuring facial emotional expressions using genetic programming. Soft Computing and Industry Recent Applications 2001, 545-554.