Improving medical students' awareness of their non-verbal communication through automated non-verbal behavior feedback

Frontiers in ICT

Volumn 3, Issue JUN, 2016, Pages

  Liu, Chunfeng ^a   Calvo, Rafael A ^a   Lim, Renee ^a,b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Automated feedback; Clinical consultation; Communication skills; Medical education; Non verbal behavior; Non verbal behavior detection; Non verbal communication

Indexed keywords

EID: 84996539523     PISSN: None     EISSN: 2297198X     Source Type: Journal    
DOI: 10.3389/fict.2016.00011     Document Type: Article

References (48)

1. BioID. (2016). BioID Face Database. Available at: https://www.bioid.com
2. Buller, D., and Street, R. L. (1992). "Physician-patient relationships," In R. Feldman Editor. Applications of Nonverbal Behavioral Theories and Research. Hillsdale, NJ: Lawrence Erlbaum, 119-141
3. Burgoon, J. K., Guerrero, L. K., and Floyd, K. (2016). Nonverbal Communication. Abingdon: Routledge
4. Calvo, R. A. (2015). "Affect-aware reflective writing studios," in Chapter 33. Handbook of Affective Computing, eds R. A. Calvo, S. K. D'Mello, J. Gratch and A. Kappas (New York, NY: Oxford University Press), 447-457
5. Calvo, R. A., D'Mello, S., Gratch, J., and Kappas, A. (eds) (2015). The Oxford Handbook of Affective Computing. New York, NY: Oxford University Press
6. Calvo, R. A., and D'Mello, S. K. (eds) (2011). New Perspectives on Affect and Learning Technologies, Vol. 3. Berlin: Springer Science & Business Media
7. Chung, S.-C., Barma, S., Kuan, T.-W., and Lin, T.-W. (2014). Frowning expression detection based on SOBEL filter for negative emotion recognition. Paper Presented at the Orange Technologies (ICOT), 2014 IEEE International Conference on, Xi'an
8. Collins, L. G., Schrimmer, A., Diamond, J., and Burke, J. (2011). Evaluating verbal and non-verbal communication skills, in an ethnogeriatric OSCE. Patient Educ. Couns. 83, 158-162. doi: 10.1016/j.pec.2010.05.012
9. Dent, J., and Harden, R. M. (2013). "Clinical communication," in A Practical Guide for Medical Teachers, eds J. A. Dent and R. M. Harden (Sydney: Elsevier Health Sciences), 265-272
10. DiMatteo, M. R., Taranta, A., Friedman, H. S., and Prince, L. M. (1980). Predicting patient satisfaction from physicians' nonverbal communication skills. Med. Care 18, 376-387. doi:10.1097/00005650-198004000-00003
11. Duggan, P., and Parrott, L. (2001). Physicians' nonverbal rapport building and patients' talk about the subjective component of illness. Hum. Commun. Res. 27, 299-311. doi:10.1093/hcr/27.2.299
12. Fung, M., Jin, Y., Zhao, R., and Hoque, M. E. (2015). ROC speak: semi-automated personalized feedback on nonverbal behavior from recorded videos. Paper Presented at the Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka
13. GENKI-4K. (2016). The MPLab GENKI Database, GENKI-4K Subset. Available at: http://mplab.ucsd.edu
14. Gibbings-Isaac, D., Iqbal, M., Tahir, M. A., Kumarapeli, P., and de Lusignan, S. (2012). The pattern of silent time in the clinical consultation: an observational multichannel video study. Fam. Pract. 29, 616-621. doi:10.1093/fampra/cms001
15. Gorawara-Bhat, R., Cook, M. A., and Sachs, G. A. (2007). Nonverbal communication in doctor-elderly patient transactions (NDEPT): development of a tool. Patient Educ. Couns. 66, 223-234. doi:10.1016/j.pec.2006.12.005
16. Harrigan, J. A., Oxman, T. E., and Rosenthal, R. (1985). Rapport expressed through nonverbal behavior. J. Nonverbal Behav. 9, 95-110. doi:10.1007/BF00987141
17. Hartzler, A. L., Patel, R. A., Czerwinski, M., Pratt, W., Roseway, A., Chandrasekaran, N., et al. (2014). Real-time feedback on nonverbal clinical communication. Methods Inf. Med. 53, 389-405. doi:10.3414/ME13-02-0033
18. Hoque, M. E., Courgeon, M., Martin, J.-C., Mutlu, B., and Picard, R. W. (2013). Mach: my automated conversation coach. Paper Presented at the Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Zurich
19. Ishikawa, H., Hashimoto, H., Kinoshita, M., Fujimori, S., Shimizu, T., and Yano, E. (2006). Evaluating medical students' non-verbal communication during the objective structured clinical examination. Med. Educ. 40, 1180-1187. doi:10.1111/j.1365-2929.2006.02628.x
20. Jang, J. S. R. (2016). Utility Toolbox. Available at: http://mirlab.org/jang
21. Kawato, S., and Ohya, J. (2000). Real-time detection of nodding and head-shaking by directly detecting and tracking the "between-eyes". Paper Presented at the Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, Grenoble
22. Kelley, J. F. (1984). An iterative design methodology for user-friendly natural language office information applications. ACM Trans. Inf. Syst. 2, 26-41. doi:10.1145/357417.357420
23. Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall
24. Larsen, K. M., and Smith, C. K. (1981). Assessment of nonverbal communication in the patient-physician interview. J. Fam. Pract. 12, 481-488
25. Leppink, J., van Gog, T., Paas, F., and Sweller, J. (2015). Cognitive load theory: researching and planning teaching to maximise learning. In: J. Cleland and S. J. Durning Editors. Res. Med. Edu. pp. 207-218 Oxford, UK: Wiley Blackwell
26. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7, 57-78. doi:10.1080/10447319509526110
27. Liu, C., Scott, K., and Calvo, R. A. (2015). "Towards a reflective online clinic (ROC): a tool to support interviewing skills," in CHI'15 Workshop on Developing Skills for Wellbeing, Seoul
28. Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010). The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. Paper Presented at the Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, San Francisco
29. Mann, K. V. (2011). Theoretical perspectives in medical education: past experience and future possibilities. Med. Educ. 45, 60-68. doi:10.1111/j.1365-2923.2010.03757.x
30. Martin, L., and Friedman, H. S. (2005). "Nonverbal communication and health care," In R. E. Riggio and R. S. Feldman Editors. London: Lawrence Erlbaum Applications of Nonverbal Communication, 3-16
31. Mast, M. S., Hall, J. A., Klöckner, C., and Choi, E. (2008). Physician gender affects how physician nonverbal behavior is related to patient satisfaction. Med. Care 46, 1212-1218. doi:10.1097/MLR.0b013e31817e1877
32. McLeod, S. A. (2013). Kolb-Learning Styles. Available at: www.simplypsychology.org/learning-kolb.html
33. Milborrow, S., and Nicolls, F. (2008). "Locating facial features with an extended active shape model," In David, F., Philip, T., and Andrew, Z. Computer Vision-ECCV 2008 (Springer), 504-513
34. Otsu, N. (1975). A threshold selection method from gray-level histograms. Automatica 11, 23-27
35. Pentland, A., and Heibeck, T. (2010). Honest Signals: How They Shape Our World. Cambridge, MA: MIT press
36. Picard, R. W. (2000). Affective Computing, Cambridge: MIT Press
37. Ross, M. J., Shaffer, H. L., Cohen, A., Freudberg, R., and Manley, H. J. (1974). Average magnitude difference function pitch extractor. IEEE Trans. Acoust. Speech Signal Process. 22, 353-362. doi:10.1109/TASSP.1974.1162598
38. Roter, D., and Larson, S. (2002). The Roter interaction analysis system (RIAS): utility and flexibility for analysis of medical interactions. Patient Educ. Couns. 46, 243-251. doi:10.1016/S0738-3991(02)00012-5
39. Simmenroth-Nayda, A., Heinemann, S., Nolte, C., Fischer, T., and Himmel, W. (2014). Psychometric properties of the Calgary Cambridge guides to assess communication skills of undergraduate medical students. Int. J. Med. Educ. 5, 212. doi:10.5116/ijme.5454.c665
40. Smith, L. I. (2002). A Tutorial on Principal Components Analysis, Vol. 51. Ithaca, NY: Cornell University, 65
41. Stewart, M. A. (1995). Effective physician-patient communication and health outcomes: a review. CMAJ 152, 1423
42. Tanveer, M. I., Zhao, R., Chen, K., Tiet, Z., and Hoque, M. E. (2016). AutoManner: An Automated Interface for Making Public Speakers Aware of Their Mannerisms
43. Vinciarelli, A., Pantic, M., and Bourlard, H. (2009). Social signal processing: survey of an emerging domain. Image Vision Comput. 27, 1743-1759. doi:10.1016/j.imavis.2008.11.007
44. Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D'Errico, F., et al. (2012). Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affective Comput. 3, 69-87. doi:10.1109/T-AFFC.2011.27
45. Viola, P., and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Paper Presented at the Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, Kauai
46. Wei, Y. (2009). Research on Facial Expression Recognition and Synthesis. Master Thesis, Department of Computer Science and Technology, Nanjing
47. Weinberger, M., Greene, J. Y., and Mamlin, J. J. (1981). The impact of clinical encounter events on patient and physician satisfaction. Soc. Sci. Med. E Med. Psychol. 15, 239-244. doi:10.1016/0271-5384(81)90019-3
48. Young, J. Q., Van Merrienboer, J., Durning, S., and Ten Cate, O. (2014). Cognitive load theory: implications for medical education: AMEE guide no. 86. Med. Teach. 36, 371-384. doi:10.3109/0142159X.2014.889290