Interpersonal coordination of headmotion in distressed couples

IEEE Transactions on Affective Computing

Volumn 5, Issue 2, 2014, Pages 155-167

  Hammal, Zakia ^a   Cohn, Jeffrey F ^b   George, David Ted ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

^c NATIONAL INSTITUTE ON ALCOHOL ABUSE AND ALCOHOLISM   (United States)

Author keywords

actor partner analysis; head motion; Interpersonal coordination; windowed cross correlation

Indexed keywords

COMPUTER PROGRAMMING; COMPUTER SCIENCE;

ACTOR-PARTNER ANALYSIS; ANGULAR DISPLACEMENT; CONFLICT INTERACTION; CROSS CORRELATIONS; EMOTIONAL EXPRESSIONS; EXPRESSION DETECTIONS; HEAD MOTION; INTERPERSONAL COORDINATIONS;

ANGULAR VELOCITY;

EID: 84905681393     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAFFC.2014.2326408     Document Type: Article

References (68)

1. N. Ambady and R. Rosenthal, "Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness," J. Personality Social Psychol., vol. 64, no. 3, pp. 431-441, 1993.
2. K. T. Ashenfelter, S. M. Boker, J. R. Waddell, and N. Vitanov, "Spatiotemporal symmetry and multifractal structure of head movements during dyadic conversation," J. Exp. Psychol.: Human Percept. Perform., vol. 35, no. 4, pp. 1072-1091, 2009.
3. J. N. Bailenson and N. Yee, "A longitudinal study of task performance, head movements, subjective report, simulator sickness, and transformed social interaction in collaborative virtual environments," Presence, vol. 15, no. 6, pp. 699-716, 2006. (Pubitemid 44879106)
4. L. Brent and S. C. Marsella, "Emotionally expressive head and body movement during gaze shifts," in Proc. 7th Int. Conf. Intell. Virtual Agents, 2007, pp. 72-85. (Pubitemid 350271476)
5. S. M. Boker, J. L. Rotondo, M. Xu, and K. King, "Windowed crosscorrelation and peak picking for the analysis of variability in the association between behavioral time series," Psychol. Methods, vol. 7, no. 1, pp. 338-355, 2002.
6. S. M. Boker, J. F. Cohn, B. J. Theobald, I. Matthews, T. Brick, and J. Spies, "Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars," Philosoph. Trans. Roy. Soc. London B: Biol. Sci., vol. 364, no. 1535, pp. 3485-3495, 2009.
7. S. M. Boker, J. F. Cohn, B. J. Theobald, I. Matthews, M. Mangini, J. R. Spies, and T. R. Brick, "Something in the way we move: Motion, not perceived sex, influences nods in conversation," J. Exp. Psychol.: Human Percept. Perform., vol. 37, no. 3, pp. 874-891 2011.
8. C. Busso, Z. Deng, M. Grimm, U. Neumann, and S. Narayanan, "Rigid head motion in expressive speech animation: Analysis and synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 3, pp. 1075-1086, Mar. 2007.
9. M. O. B. Caughy, K.-Y. Huang, and J. Lima, "Patterns of conflict interaction in mothertoddler dyads: Differences between depressed and non-depressed mothers," J. Child Family Stud., vol. 18, pp. 10-20, 2009.
10. J. P. Caughlin and T. L. Huston, "A contextual analysis of the association between demand/withdraw and marital satisfaction," Pers. Relationships, vol. 9, no. 1, pp. 95-119, 2002. (Pubitemid 36836899)
11. J. P. Caughlin and A. M. Scott, "Toward a communication theory of the demand/withdraw pattern of interaction in interpersonal relationships," in New Directions in Interpersonal Communication Research, S. W. Smith and S. R. Wilson, Eds. Thousand Oaks, CA, USA: Sage, 2010, pp. 180-200.
12. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ, USA: Lawrence Erlbaum, 1988.
13. J. F. Cohn, L. I. Reed, T. Moriyama, J. Xiao, K. L. Schmidt, and Z. Ambadar, "Multimodal coordination of facial action, head rotation, and eye motion," in Proc. 6th IEEE Int. Conf. Autom. Face Gesture Recog., Seoul, Korea, 2004, pp. 129-138.
14. J. F. Cohn and M. A. Sayette, "Spontaneous facial expression in a small group can be automatically measured: An initial demonstration," Behav. Res. Methods, vol. 42, no. 4, pp. 1079-1086, 2010.
15. M. Cox, J. Nuevo-Chiquero, J. M. Saragih, and S. Lucey, "CSIRO Face Analysis SDK," Brisbane, Qld., Australia, 2013.
16. J. W. Davidson, "What type of information is conveyed in the body movements of solo musician performers?" J. Human Movement Stud., vol. 6, pp. 279-301, 2004.
17. R. De Silva and N. Bianchi-Berthouze, "Modeling human affective postures: An information theoretic characterization of posture features," J. Comput. Anim. Virtual World, vol. 15, nos. 3/4, pp. 269-276, 2004. (Pubitemid 41108005)
18. J. Dunn and P. Munn, "Becoming a family member: Family conflict and the development of social understanding in the second year," Child Develop., vol. 56, pp. 480-492, 1985.
19. S. Duncan, "Some signals and rules for taking speaking turns in conversations," J. Personality Social Psychol., vol. 23, no. 2, pp. 283-292, 1972.
20. R. el Kaliouby, and P. Robinson, "Real-time inference of complex mental states from facial expressions and head gestures," in Proc. Real-Time Vis. HCI, 2005, pp. 181-200.
21. B. Xiao, P. G. Georgiou, B. Baucom, and S. S. Narayanan, "Data driven modeling of head motion toward analysis of behaviors in couple interactions," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Vancouver, BC, Canada, 2013, pp. 3766-3770.
22. C. G. Fisher, "The visibility of terminal pitch contour," J. Speech Hearing Res., vol. 12, pp. 379-382, 1969.
23. T. D. George, M. J. Phillips, M. Lifshitz, T. A. Lionetti, D. E. Spero, and N. Ghassemzedah, "Fluoxetine treatment of alcoholic perpetrators of domestic violence: A 12-week, double-blind, randomized placebo-controlled intervention study," J. Clin. Psychiatry, vol. 72, no. 1, pp. 60-65, 2011.
24. H. Gunes and M. Pantic, "Dimensional emotion prediction from spontaneous head gestures for interaction with sensitive artificial listeners," in Proc. 10th Int. Conf. Intell. Virtual Agents, 2010, pp. 371-377.
25. Z. Hammal, J. F. Cohn, D. S. Messinger, W. Masson, and M. Mahoor, "Head movement dynamics during normal and perturbed parent-infant interaction," in Proc. 5th Humaine Assoc. Conf. Affective Comput. Intell. Interact., Geneva, Sep. 2013, pp. 276-282.
26. D. F. Hay, "Social conflict in early childhood," in Annals of Child Development, G. J. Whitehurst, Ed. Greenwich, CT, USA: JAI Press, 1984, pp. 1-44.
27. J. S. Jang and T. Kanade, "Robust 3D head tracking by online feature registration," in Proc. IEEE 8th Int. Conf. Autom. Face Gesture Recog., Amsterdam, The Netherlands, Sep. 2008, pp. 1-6.
28. D. A. Kashy and D. A. Kenny, "The analysis of data from dyads and groups," in Handbook of Research Methods in Social Psychology, I. H. Reis and C. M. Judd, Eds. New York, NY, USA: Cambridge Univ. Press, 2000, pp. 451-477.
29. D. Keltner, "Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement and shame," J. Personality Social Psychol., vol. 68, no. 3, pp. 441-454, 1995.
30. D. A. Kenny, L. Mannetti, A. Pierro, S. Livi, and D. A. Kashy, "The statistical analysis of data from small groups," J. Personality Social Psychol., vol. 83, pp. 126-137, 2002.
31. M. L. Knapp and J. A. Hall, Nonverbal behavior in human communication, 7th ed. Boston, MA, USA: Cengage, 2010.
32. N. M. Laird and J. H. Ware, "Random-effects models for longitudinal data," Biometrics, vol. 38, pp. 963-974, 1982.
33. G. Laurent and H. Davidowitz, "Encoding of olfactory information with oscillating neural assemblies," Science, vol. 265, pp. 1872-1875, 1994. (Pubitemid 24324244)
34. G. Laurent, M. Wehr, and H. Davidowitz, "Temporal representations of odors in an olfactory network," J. Neurosci., vol. 16, no. 12, pp. 3837-3847, 1996. (Pubitemid 26176048)
35. J. R. Landis and G. G. Koch, "The measurement of observer agreement for categorical data," Biometrics, vol. 33, pp. 159-174, 1977. (Pubitemid 8054849)
36. R. W. Levenson and J. M. Gottman, "Marital interaction: Physiological linkage and affective exchange," J. Personality Social Psychol., vol. 45, no. 3, pp. 587-597, 1983.
37. T. M. Madhyastha, E. L. Hamaker, and J. M. Gottman, "Investigating spousal influence using moment-to-moment affect data from marital conflict," J. Family Psychol., vol. 25, no. 2, pp. 292-300, 2011.
38. I. Matthews, J. Xiao, and S. Baker, "2D vs. 3D deformable face models: Representational power, construction, and real-time fitting," Int. J. Comput. Vis., vol. 75, no. 1, pp. 93-113, 2007. (Pubitemid 47120247)
39. D. S. Messinger, M. H. Mahoor, S. M. Chow, and J. F. Cohn, "Automated measurement of facial expression in infant-mother interaction: A pilot study," Infancy, vol 14, no. 3, pp. 285-305, 2009.
40. D. S. Messinger, W. I. Mattson, M. H. Mahoor, and J. F. Cohn, "The eyes have it: Making positive expressions more positive and negative expressions more negative," Emotion, vol. 12, no. 3, pp. 430-436, 2012.
41. A. Mignault and A. Chaudhuri, " The many faces of a neutral face: Head tilt and perception of dominance and emotion," J. Nonverbal Behav., vol. 27, no. 2, pp. 111-132, 2003.
42. L. P. Morency, C. Sidner, C. Lee, and T. Darrell, "Head gestures for perceptual interfaces: The role of context in improving recognition," Artif. Intell., vol. 171, nos. 8/9, pp. 568-585, Jun. 2007. (Pubitemid 46828980)
43. K. G. Munhall, J. A. Jones, D. E. Callan, T. Kuratate, and E. Vatikiotis-Bateson "Visual prosodyand speech intelligibility: Head movement improves auditory speech perception," Psychol. Sci., vol. 30, no. 2, pp. 133-137, 2004. (Pubitemid 38361306)
44. K. G. Nicholson, S. Baum, L. Cuddy, and K. G. Munhall, "A case of impaired auditory and visual speech prosody perception after right hemisphere damage," Neurocase, vol. 8, pp. 314-322, 2002.
45. H. Paterson, F. Pollick, and A. G. Sanford, "The role of velocity in affect discrimination," in Proc. 23rd Annu. Conf. Cognition Sci. Soc., 2001, pp. 1-6.
46. C. Pelachaud, "Modelling multimodal expression of emotion in a virtual agent," Philos. Trans. Roy. Soc. B, vol. 364, pp. 3539-3548, 2009.
47. J. Pinheiro, D. Bates, S. DebRoy, D. Sarkar, and R Core Team, "nlme: Linear and Nonlinear Mixed Effects Models," R Package Version 3.1-109, 2013.
48. U. J. Pfeiffer, B. Timmermans, G. Bente, K. Vogeley, and L. Schilbach, "A non-verbal turing test: Differentiating mind from machine in gaze-based social interaction," PLoS ONE, vol. 6, no. 11, p. e27591, 2011.
49. U. J. Pfeiffer, B. Timmermans, G. Bente, K. Vogeley, and L. Schilbach R Core Team. (2013). R: A language and environment for statistical computing [Online]. Available: http://www.R-project.org/
50. J. L. Rotondo and S. M. Boker, "Behavioral synchronization in human conversational interaction," in Mirror Neurons and the Evolution of Brain and Language, M. Stamenov, and V. Gallese, Eds. Amsterdam, The Netherlands: John Benjamins, 2002.
51. N. A. Roberts, J. L. Tsai, and J. A. Coan, "Emotion elicitation using dyadic interaction tasks," in Handbook of Emotion Elicitation and Assessment, J. A. Coan, and J. J. B. Allen, Eds. New York, NY, USA: Oxford Univ. Press, 2007, pp.106-123.
52. A. Risberg and J. Lubker, "Prosody and speechreading," Speech Transm. Lab. Q. Progress Report and Status Report, vol. 4, pp. 1-16, 1978.
53. M. A. Sayette, K. G. Creswell, J. D. Dimoff, C. E. Fairbairn, J. F. Cohn, B. W. Heckman, and R. L. Moreland, "Alcohol and group formation: A multimodal investigation of the effects of alcohol on emotion and social bonding," Psychol. Sci., vol. 23, no. 8, pp. 869-878, 2012.
54. M. Cox, J. Nuevo-Chiquero, J. M. Saragih, and S. Lucey, "CSIRO Face Analysis SDK," ed. Brisbane, Australia, 10th IEEE Int. Conf. Autom. Face Gesture Recog. Shangai, China, Apr. 2013.
55. M. Cox, J. Nuevo-Chiquero, J. M. Saragih, and S. Lucey The SEMAINE project. (2008-2010) [Online]. Available: http://www.semaine-project.eu/
56. M. Stel, R. B. van Baaren, J. Blascovich, E. Van Dijk, C. McCall, and M. M. Pollmann, " Effects of a priori liking on the elicitation of mimicry," Exp. Psychol., vol. 57, no. 6, pp. 412-418, 2010.
57. M. Schrder, E. Bevacqua, R. Cowie, F. Eyben, H. Gunes, D. Haylen, M. ter Maat, G. McKeown, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. De Sevin, M. Valstar, and M. Wollmer, "Building autonomous sensitive artificial listeners," IEEE Trans. Affective Comput., vol. 3, no. 2, pp. 165-183, Apr.-Jun. 2012.
58. M. A. Straus, "Measuring intrafamily conflict and violence: The conflict tactics (CT) scales," Phys. Violence Amer. Families, vol. 41, no. 1, pp. 75-88, 1979.
59. C. Sidner, C. Lee, L. p. Morency, and Clifton Forlines, "The effect of head-nod recognition in human-robot conversation," in Proc. 1st ACM SIGCHI/SIGART Conf. Human-Robot Interact., Salt Lake City, Utah, USA, Mar. 2006, pp. 290-296. (Pubitemid 44032419)
60. B. J. Theobald, I. Matthews, M. Mangini, J. R. Spies, T. Brick, J. F. Cohn, and S. M. Boker, "Mapping and manipulating facial expression," Lang. Speech, vol. 52, nos. 2/3, pp. 369-386, 2009.
61. F. De La Torre, and J. F. Cohn, "Visual analysis of humans: Facial expression analysis," in Visual Analysis of Humans: Looking at People, T. B. Moeslund, A. Hilton, A. U. Volker Krger, and L. Sigal, Eds. Berlin, Germany: Springer, 2011, pp. 377-410.
62. A. G. Van Kleef and S. Cote, "Expressing anger in conflict: When it helps and when it hurts," J. Appl. Psychol., vol. 92, no. 6, pp. 1557-1569, 2007.
63. A. J. Viera and J. M. Garrett, "Understanding interobserver agreement: The kappa statistic," Family Med., vol. 37, no. 5, pp. 360-363, 2005. (Pubitemid 40656185)
64. A. M. Von der Ptten, N. C. Krmer, J. Gratch, and S. Kang, "It doesn't matter what you are: Explaining social effects of agents and avatars," Comput. Human Behav., vol. 26, p. 10, 2010.
65. J. Xiao, S. Baker, I. Matthews, and T. Kanade, "Real-time combined 2D+3D active appearance models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2004, pp. 535-542.
66. B. Xiao, P. G. Georgiou, B. Baucom, and S. S. Narayanan, "Data driven modeling of head motion toward analysis of behaviors in couple interactions," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Vancouver, Canada, 2013, pp. 3766-3770.
67. H. Yehia, T. Kuratate, and E. Vatikiotis-Bateson, "Linking facial animation, head motion and speech acoustics," J. Phonetics, vol. 30, pp. 555-568, 1994.
68. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intel., vol. 31, no. 1, pp. 39-58, Jan. 2009.