Computing vocal entrainment: A signal-derived PCA-based quantification scheme with application to affect analysis in married couple interactions

Computer Speech and Language

Volumn 28, Issue 2, 2014, Pages 518-539

  Lee, Chi Chun ^a   Katsamanis, Athanasios ^a   Black, Matthew P ^a   Baucom, Brian R ^a   Christensen, Andrew ^b   Georgiou, Panayiotis G ^a   Narayanan, Shrikanth S ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Affect recognition; Behavioral signal processing (BSP); Couple therapy; Dyadic interaction; Entrainment; Factorial Hidden Markov Model; Interaction synchrony; Principal component analysis (PCA)

Indexed keywords

AFFECT RECOGNITION; BEHAVIORAL SIGNAL PROCESSING (BSP); COUPLE THERAPY; DYADIC INTERACTION; INTERACTION SYNCHRONY;

AIR ENTRAINMENT; COMMUNICATION; HIDDEN MARKOV MODELS; PRINCIPAL COMPONENT ANALYSIS; SIGNAL PROCESSING;

BEHAVIORAL RESEARCH;

EID: 84890562000     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2012.06.006     Document Type: Article

References (60)

1. P.A. Andersen, and J.F. Andersen The exchange of nonverbal intimacy: a critical review of dyadic models Journal of Nonverbal Behavior 8 4 1984 328 349
2. Baucom, B.; Atkins, D. Polarization in marriage. In: Fine, M.; Fincham, F. (Eds.), Family Theories: A Content-Based Approach. Routledge, New York, NY, in press.
3. F.J. Bernieri, J.S. Reznick, and R. Rosenthal Synchrony, pseudosynchrony, and dissynchrony: measuring the entrainment process in mother-infant interactions Journal of Personality and Social Psychology 54 2 1988 243 253
4. M.P. Black, P.G. Georgiou, A. Katsamanis, B.R. Baucom, and S.S. Narayanan "You made me do it": classification of blame in married couples interactions by fusing automatically derived speech and language information Proceedings of Interspeech 2011 89 92
5. Black, M.P.; Katsamanis, A.; Baucom, B.R.; Lee, C.C.; Lammert, A.C.; Christensen, A.; Georgiou, P.G.; Narayanan, S.S. Toward automating a human behavioral coding system for married couples' interactions using speech acoustic features. Speech Communication, in press 2011, http://dx.doi.org/10.1016/j. specom.2011.12.003.
6. M.P. Black, A. Katsamanis, C.C. Lee, A.C. Lammert, B.R. Baucom, A. Christensen, P.G. Georgiou, and S.S. Narayanan Automatic classification of married couples' behavior using audio features Proceedings of Interspeech 2010 2030 2033
7. P. Boersma Praat, a system for doing phonetics by computer Glot International 5 2001 341 345
8. S.M. Boker, and J.P. Laurenceau Dynamical systems modeling: an application to the regulation of intimacy and disclosure in marriage Models for Intense Longitudinal Data 2006 pp. 195-218
9. J.K. Burgoon, L.A. Stern, and L. Dillman Interpersonal Adaptation: Dyadic Interaction Patterns 1995 Cambridge University Press
10. T.L. Chartrand, and J.A. Bargh The chameleon effect: the perception-behavior link and social interaction Journal of Personality and Social Psychology 76 6 1999 893 910
11. A. Christensen, D. Atkins, S. Berns, J. Wheeler, D.H. Baucom, and L. Simpson Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples Journal of Consulting and Clinical Psychology 72 2004 176 191
12. J.V. Cordova, N.S. Jacobson, J.M. Gottman, R. Rushe, and G. Cox Negative reciprocity and communication in couples with a violent husband Journal of Abnormal Psychology 102 1994 559 564
13. J. Dauwels, F. Vialatte, and A. Cichocki Diagnosis of Alzheimer's disease from EEG signals: where are we standing? Current Alzheimer's Research 7 2010 487 505 (Invited paper)
14. K. Eldridge, and B. Baucom Couples and consequences of the demand-withdraw interaction pattern Positive Pathways for Couples and Families: Meeting the Challenges of Relationships 2010 Wiley Blackwell
15. K.A. Eldridge, M. Sevier, J. Jones, D.C. Atkins, and A. Christensen Demand-withdraw communication in severely distressed, moderately distressed, and nondistressed couples: rigidity and polarity during relationship and personal problem discussions Journal of Family Psychology 21 2007 218 226
16. F. Eyben, M. Wöllmer, and B. Schuller OpenSMILE - the Munich versatile and fast open-source audio feature extractor ACM Multimedia Firenze, Italy 2010 1459 1462
17. P. Georgiou, M. Black, A. Lammert, B. Baucom, and S. Narayanan "That's aggravating, very aggravating": is it possible to classify behaviors in couple interactions using automatically derived lexical features? Affective Computing and Intelligent Interaction 2011 87 96
18. Z. Ghahramani, and M.I. Jordan Factorial hidden markov models Machine Learning 29 1997 245 274
19. P.K. Ghosh, A. Tsiartas, and S.S. Narayanan Robust voice activity detection using long-term signal variability IEEE Transactions on Audio, Speech, and Language Processing 19 2010 600 613
20. J. Gibson, A. Katsamanis, M. Black, and S. Narayanan Automatic identification of salient acoustic instances in couples behavioral interactions using diverse density support vector machines Proceedings of Interspeech 2011 1561 1564
21. J. Gottman, C. Swanson, and K. Swanson A general system theory of marriage: nonlinear difference equation modeling of marital interaction Personality and Social Psychology Review 6 2002 326 340
22. J.M. Gottman The roles of conflict engagement, escalation, and avoidance in marital interaction: a longitudinal view of five types of couples Journal of Consulting and Clinical Psychology 61 1993 6 15
23. S. Gregory, K. Dagan, and S. Webster Evaluating the relation between vocal accommodation in conversational partners' fundamental frequencies to perceptions of communication quality Journal of Nonverbal Behavior 21 1997 23 43
24. S. Gregory, and B. Hoyt Conversation partner mutual adaptation as demonstrated by Fourier series analysis Journal of Psycholinguistic Research 11 1982 35 46
25. S. Gregory, and S. Webster A nonverbal signal in voices of interview partners effectively predicts communication accommodation and social status perceptions Journal of Personality and Social Psychology 70 1996 1231 1240
26. S. Gregory, S. Webster, and G. Huang Voice pitch and amplitude convergence as a metric of quality in dyadic interviews Language and Communication 13 1993 195 217
27. C. Heavey, D. Gill, and A. Christensen Couples Interaction Rating System 2 (CIRS2) 2002 University of California Los Angeles
28. N.S. Jacobson, J.M. Gottman, J. Waltz, R. Rushe, J. Babcock, and A. Holtzworth-Munroe Affect, verbal content, and psychophysiology in the arguments of couples with a violent husband Journal of Consulting and Clinical Psychology 62 1994 982 988
29. Johnannesmeyer, M.; 1999. Abnormal situation analysis using pattern recognition techniques and historical data. Master's Thesis. UCSB, Santa Barbara, CA.
30. S.L. Johnson, and T. Jacob Sequential interactions in marital communication of depressed men and women Journal of Consulting and Clinical Psychology 68 2000 4 12
31. J. Jones, and A. Christensen Couples Interaction Study: Social Support Interaction Rating System 1998 University of California Los Angeles
32. D. Jurafsky, R. Ranganath, and D. McFarland Extracting social meaning: identifying interactional style in spoken conversation Human Language Technologies Boulder, CO, USA 2009 638 646
33. A. Katsamanis, M.P. Black, P.G. Georgiou, L. Goldstein, and S.S. Narayanan SailAlign: robust long speech-text alignment Very-Large-Scale Phonetics Workshop 2011
34. A. Katsamanis, J. Gibson, M. Black, and S. Narayanan Multiple instance learning for classification of human behavior observations Affective Computing and Intelligent Interaction 2011 145 154
35. P. Kerig, and D.E. Baucom Couple Observational Coding Systems 2004 Lawrence Erlbaum Associates Mahwah, NJ, USA
36. M. Kimura, and I. Daibo Interactional synchrony in conversations about emotional episodes: a measurement by 'the between-participants pseudosynchrony experimental paradigm' Journal of Nonverbal Behavior 30 2006 115 126
37. W. Krzanowski Between-groups comparison of principal components Journal of the American Statistical Association 74 1979 703 707
38. O.W. Kwon, C. Kwokleung, J. Hao, and T.W. Lee Emotion recognition by speech signals Proc. EUROSPEECH 2003 125 128
39. C.C. Lee, M.P. Black, A. Katsamanis, A.C. Lammert, B.R. Baucom, A. Christensen, P.G. Georgiou, and S.S. Narayanan Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples Proceedings of Interspeech 2010 793 796
40. C.C. Lee, A. Katsamanis, M.P. Black, B.R. Baucom, P.G. Georgiou, and S.S. Narayanan Affective state recognition in married couples' interactions using PCA-based vocal entrainment measures with multiple instance learning Affective Computing and Intelligent Interaction 2011 31 41
41. C.C. Lee, A. Katsamanis, M.P. Black, B.R. Baucom, P.G. Georgiou, and S.S. Narayanan An analysis of PCA-based vocal entrainment measures in married couples' affective spoken interactions Proceedings of Interspeech 2011 3101 3104
42. R. Levitan, and J. Hirschberg Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions Proceedings of Interspeech 2011 3081 3084
43. A.R. McGarva, and R.M. Warner Attraction and social coordination: mutual entrainment of vocal activity rhymes Journal of Psycholinguistic Research 32 2003 335 354
44. C.M. Murphy, and T.J. O'Farrell Couple communication patterns of maritally aggressive and nonaggressive male alcoholics Journal of Studies on Alcohol 58 1997 83 90
45. K.P. Murphy The Bayes net toolbox for Matlab Computing Science and Statistics 2001
46. A. Nenkova, A. Gravano, and J. Hirschberg High frequency word entrainment in spoken dialogue Proceedings of ACL-08: HLT, Short Papers Columbus, OH 2008 169 172
47. K.G. Niederhoffer, and J.W. Pennebaker Linguistic style matching in social interaction Journal of Language and Social Psychology 21 2002 337 360
48. M.E. Otey, and S. Parthasarathy A dissimilarity measure for comparing subsets of data: application to multivariate time series Proceedings of ICDM Workshop on Temporal Data Mining Houston, TX 2005
49. J.S. Pardo On phonetic convergence during conversational interaction Journal of Acoustical Society of America 119 2006 2382 2393
50. L.A. Pasch, T.N. Bradbury, and J. Davila Gender, negative affectivity, and observed social support behavior in marital interaction Personal Relationships 4 1997 278 361
51. R. Ranganath, D. Jurafsky, and D. McFarland It's not you, it's me: detecting flirting and its misperception in speed-dates Conference on Empirical Methods in Natural Language Processing Suntec City, Singapore 2009 334 342
52. D.A. Reynolds Experimental evaluation of features for robust speaker identification IEEE Transactions on Speech and Audio Processing 2 1994 639 643
53. M.J. Richardson, K.L. Marsh, and R. Schmit Effects of visual and verbal interaction on unintentional interpersonal coordination Journal of Experimental Psychology: Human Perception and Performance 31 2005 62 79
54. J.M. Romano, J.A. Turner, L.S. Friedman, R.A. Bulcroft, M.P. Jensen, H. Hops, and S.F. Wright Sequential analysis of chronic pain behaviors and spouse responses Journal of Consulting and Clinical Psychology 60 1992 777 782
55. V. Rozgić, B. Xiao, A. Katsamanis, B. Baucom, P.G. Georgiou, and S.S. Narayanan Estimation of ordinal approach-avoidance labels in dyadic interactions: ordinal logistic regression approach ICASSP 2011 2368 2371
56. V. Rozgic, V. Xiao, A. Katsamanis, B.R. Baucom, P.G. Georgiou, and S.S. Narayanan A new multichannel multimodal dyadic interaction database Proceedings of Interspeech 2010 1982 1985
57. P.E. Shrout, and J.L. Fleiss Intraclass correlation: uses in assessing rater reliability Psychological Bulletin 86 1979 420 428
58. L.L. Verhofstadt, A. Buysse, W. Ickes, M. Davis, and I. Devoldre Support provision in marriage: the role of emotional similarity and empathic accuracy Emotion 8 2008 792 802
59. A. Vinciarelli, M. Pantic, and H. Bourlard Social signal processing: survey of an emerging domain Image and Vision Computing 27 2009 1743 1759
60. J.H. Watt, and C.A. VanLearn Dynamic Patterns in Communication Processes 1996 Sage Publication, Inc.