A technology prototype system for rating therapist empathy from audio recordings in addiction counseling

PeerJ Computer Science

Volumn 2016, Issue 4, 2016, Pages

  Xiao, Bo ^a   Huang, Chewei ^a   Imel, Zac E ^b   Atkins, David C ^c   Georgiou, Panayiotis ^a   Narayanan, Shrikanth S ^a  

^a University of Southern California   (United States)

^b UNIVERSITY OF UTAH   (United States)

^c UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

Author keywords

Addiction counseling; Automatic speech recognition; Empathy modeling; Language modeling; Prototype system; Speech processing

Indexed keywords

AUDIO RECORDINGS; AUTOMATION; COMPUTATIONAL LINGUISTICS; MAXIMUM ENTROPY METHODS; MAXIMUM LIKELIHOOD; MODELING LANGUAGES; SPEECH PROCESSING;

ADDICTION COUNSELING; AUTOMATIC SPEECH RECOGNITION; AUTOMATIC SPEECH RECOGNIZERS; LANGUAGE MODEL; MAXIMUM ENTROPY MODELS; PROTOTYPE SYSTEM; SPEECH TECHNOLOGY; VOICE ACTIVITY DETECTION;

SPEECH RECOGNITION;

EID: 84994306642     PISSN: None     EISSN: 23765992     Source Type: Journal    
DOI: 10.7717/peerj-cs.59     Document Type: Article

References (47)

1. Baer JS, Wells EA, Rosengren DB, Hartzler B, Beadnell B, Dunn C. 2009. Agency context and tailored training in technology transfer: a pilot evaluation of motiva- tional interviewing training for community counselors. Journal of Substance Abuse Treatment 37(2):191 DOI 10.1016/j.jsat.2009.01.003.
2. Barrett-Lennard GT. 1981. The empathy cycle: refinement of a nuclear concept. Journal of Counseling Psychology 28(2):91 DOI 10.1037/0022-0167.28.2.91.
3. Batson CD. 2009. These things called empathy: eight related but distinct phenomena. In: The Social Neuroscience of Empathy. Cambridge: MIT Press, 3-15.
4. Berger AL, Pietra VJD, Pietra SAD. 1996. A maximum entropy approach to natural language processing. Computational linguistics 22(1):39-71.
5. Black MP, Katsamanis A, Baucom BR, Lee C-C, Lammert AC, Christensen A, Georgiou PG, Narayanan SS. 2013. Toward automating a human behavioral coding system for married couples interactions using speech acoustic features. Speech Communication 55(1):1-21 DOI 10.1016/j.specom.2011.12.003.
6. Bone D, Lee C-C, Black MP, Williams ME, Lee S, Levitt P, Narayanan S. 2014. The psychologist as an interlocutor in autism spectrum disorder assessment: Insights from a study of spontaneous prosody. Journal of Speech, Language, and Hearing Research 57(4):1162-1177 DOI 10.1044/2014_JSLHR-S-13-0062.
7. Can D, Georgiou P, Atkins D, Narayanan SS. 2012. A case study: detecting counselor reflections in psychotherapy for addictions using linguistic features. In: Proceedings of interspeech, 2254-2257.
8. Eisenberg N, Eggum ND. 2009. Empathic responding: sympathy and personal distress. In: The Social Neuroscience of Empathy. Cambridge: MIT Press, 71-83.
9. Elliott R, Bohart AC, Watson JC, Greenberg LS. 2011. Empathy, Psychotherapy 48(1):43 DOI 10.1037/a0022187.
10. Georgiou P, Black M, Lammert A, Baucom B, Narayanan S. 2011. "That's aggravating, very aggravating": is it possible to classify behaviors in couple interactions using automatically derived lexical features? In: Affective Computing and Intelligent Interaction, 87-96.
11. Godfrey JJ, Holliman EC, McDaniel J. 1992. Switchboard: telephone speech corpus for research and development. In: Proc. ICASSP. Vol. 1. Piscataway: IEEE, 517-520.
12. Guha T, Yang Z, Ramakrishna A, Grossman R, Hedley D, Lee S, Narayanan S. 2015. On quantifying facial expression-related atypicality of children with autism spectrum disorder. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). Piscataway: IEEE, 803-807.
13. Huang CW, Xiao B, Georgiou P, Narayanan S. 2014. Unsupervised speaker diarization using riemannian manifold clustering. In: Proceedings of interspeech, 567-571.
14. Iacoboni M. 2009. Imitation, empathy, and mirror neurons. Annual Review of Psychology 60:653-670 DOI 10.1146/annurev.psych.60.110707.163604.
15. Imel ZE, Steyvers M, Atkins DC. 2015. Computational psychotherapy research: scaling up the evaluation of patient-provider interactions. Psychotherapy 52(1):19-30 DOI 10.1037/a0036841.
16. Krupski A, Joesch JM, Dunn C, Donovan D, Bumgardner K, Lord SP, Ries R, Roy- Byrne P. 2012. Testing the effects of brief intervention in primary care for problem drug use in a randomized controlled trial: rationale, design, and methods. Addiction Science & Clinical Practice 7(1):27 DOI 10.1186/1940-0640-7-27.
17. Kumano S, Otsuka K, Matsuda M, Yamato J. 2013. Analyzing perceived empathy/an- tipathy based on reaction time in behavioral coordination. In: Automatic face and gesture recognition. Piscataway: IEEE, 1-8.
18. Kumano S, Otsuka K, Mikami D, Yamato J. 2011. Analyzing empathetic interactions based on the probabilistic modeling of the co-occurrence patterns of facial expres- sions in group meetings. In: Automatic face and gesture recognition. Piscataway: IEEE, 43-50.
19. Lee C-C, Katsamanis A, Black MP, Baucom BR, Christensen A, Georgiou PG, Narayanan SS. 2014. Computing vocal entrainment: a signal-derived pca-based quantification scheme with application to affect analysis in married couple interac- tions. Computer Speech & Language 28(2):518-539 DOI 10.1016/j.csl.2012.06.006.
20. Lee CM, Kilmer JR, Neighbors C, Atkins DC, Zheng C, Walker DD, Larimer ME. 2013. Indicated prevention for college student marijuana use: a randomized controlled trial. Journal of Consulting and Clinical Psychology 81(4):702 DOI 10.1037/a0033285.
21. Lee CM, Neighbors C, Lewis MA, Kaysen D, Mittmann A, Geisner IM, Atkins DC, Zheng C, Garberson LA, Kilmer JR, Larimer ME. 2014. Randomized controlled trial of a spring break intervention to reduce high-risk drinking. Journal of Consulting and Clinical Psychology 82(2):189 DOI 10.1037/a0035743.
22. Liu DC, Nocedal J. 1989. On the limited memory BFGS method for large scale optimiza- tion. Mathematical Programming 45(1-3):503-528 DOI 10.1007/BF01589116.
23. Metallinou A, Grossman RB, Narayanan S. 2013. Quantifying atypicality in affective facial expressions of children with autism spectrum disorders. In: 2013 IEEE international conference on multimedia and expo (ICME). Piscataway: IEEE, 1-6.
24. Miller WR, Rollnick S. 2012. Motivational interviewing: helping people change. New York: Guilford Press.
25. Miller WR, Rose GS. 2009. Toward a theory of motivational interviewing. American Psychologist 64(6):527 DOI 10.1037/a0016830.
26. Moyers TB, Martin T, Manuel JK, Hendrickson SM, Miller WR. 2005. Assessing competence in the use of motivational interviewing. Journal of Substance Abuse Treatment 28(1):19-26 DOI 10.1016/j.jsat.2004.11.001.
27. Moyers T, Martin T, Manuel J, Miller W, Ernst D. 2007. Revised global scales: Motiva- tional Interviewing Treatment Integrity 3.0.
28. Moyers TB, Miller WR. 2013. Is low therapist empathy toxic? Psychology of Addictive Behaviors 27(3):878 DOI 10.1037/a0030274.
29. Narayanan s, Georgiou P. 2013. Behavioral signal processing: deriving human behavioral informatics from speech and language. Proceeding of the IEEE 101(5):1203-1233 DOI 10.1109/JPROC.2012.2236291.
30. Neighbors C, Lee CM, Atkins DC, Lewis MA, Kaysen D, Mittmann A, Fossos N, Geisner IM, Zheng C, Larimer ME. 2012. A randomized controlled trial of event- specific prevention strategies for reducing problematic drinking associated with 21st birthday celebrations. Journal of Consulting and Clinical Psychology 80(5):850 DOI 10.1037/a0029480.
31. Paul DB, Baker JM. 1992. The design for the Wall Street Journal-based CSR Corpus. In: DARPA speech and language workshop, Stroudsburg: ACL, 357-362.
32. Povey D, Ghoshal A, Boulianne G, Burget L, Glembek O, Goel N, Hannemann M, Motlicek P, Qian Y, Schwarz P, Silovsky J, Stemmer G, Vesely K. 2011. The kaldi speech recognition toolkit. In: Proceedings of the ASRU. Piscataway: IEEE.
33. Preston SD, De Waal F. 2002. Empathy: its ultimate and proximate bases. Behavioral and Brain Sciences 25(01):1-20.
34. Rosenfeld R. 1996. A maximum entropy approach to adaptive statistical language mod- elling. Computer Speech & Language 10(3):187-228 DOI 10.1006/csla.1996.0011.
35. Roy-Byrne P, Bumgardner K, Krupski A, Dunn C, Ries R, Donovan D, West II, Maynard C, Atkins DC, Graves MC, Joesch JM, Zarkin GA. 2014. Brief intervention for problem drug use in safety-net primary care settings: a randomized clinical trial. Jama 312(5):492-501 DOI 10.1001/jama.2014.7860.
36. Stolcke A. 2002. Srilm-an extensible language modeling toolkit. In: Proceedings of interspeech, 901-904.
37. Substance Abuse and Mental Health Services Administration. 2013. Results from the 2012 national survey on drug use and health: summary of national findings. In: NSDUH Series H-46, HHS Publication No. (SMA) 13-4795. Rockville: Substance Abuse and Mental Health Services Administration.
38. Tollison SJ, Lee CM, Neighbors C, Neil TA, Olson ND, Larimer ME. 2008. Questions and reflections: the use of motivational interviewing microskills in a peer-led brief alcohol intervention for college students. Behavior Therapy 39(2):183-194 DOI 10.1016/j.beth.2007.07.001.
39. Van Segbroeck M, Tsiartas A, Narayanan SS. 2013. A robust frontend for VAD: exploit- ing contextual, discriminative and spectral cues of human voice. In: Proceedings of interspeech, 704-708.
40. Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D'Errico F, Schröder M. 2012. Bridging the gap between social animal and unsocial machine: A survey of social signal processing. IEEE Transactions on Affective Computing 3(1):69-87 DOI 10.1109/T-AFFC.2011.27.
41. Wang W, Lu P, Yan Y. 2008. An improved hierarchical speaker clustering. ACTA ACUSTICA 33(1):9.
42. Xiao B, Bone D, Van Segbroeck M, Imel ZE, Atkins D, Georgiou P, Narayanan S. 2014. Modeling therapist empathy through prosody in drug addiction counseling. In: Proceedings of interspeech, 213-217.
43. Xiao B, Can D, Georgiou PG, Atkins DC, Narayanan SS. 2012. Analyzing the language of therapist empathy in Motivational Interview based psychotherapy. In: Signal & information processing association annual summit and conference (APSIPA ASC), 1-4.
44. Xiao B, Georgiou PG, Baucom BR, Narayanan SS. 2014. Power-spectral analysis of head motion signal for behavioral modeling in human interaction. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), Piscataway: IEEE, 4593-4597.
45. Xiao B, Georgiou PG, Imel ZE, Atkins DC, Narayanan SS. 2013. Modeling therapist empathy and vocal entrainment in drug addiction counseling. In: Proceedings of interspeech, 2861-2865.
46. Xiao B, Imel ZE, Georgiou P, Atkins DC, Narayanan SS. 2015. "Rate my therapist": automated detection of empathy in drug and alcohol counseling via speech and language processing. PLoS ONE 10(12):1-15 DOI 10.1371/journal.pone.0143055.
47. Zhang L. 2013. Maximum entropy modeling toolkit for Python and C++. Available at https:// github.com/ lzhang10/maxent.