Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars

Philosophical Transactions of the Royal Society B: Biological Sciences

Volumn 364, Issue 1535, 2009, Pages 3485-3495

  Boker, Steven M ^a   Cohn, Jeffrey F ^b,c   Theobald, Barry John ^d   Matthews, Iain ^c,e   Brick, Timothy R ^a   Spies, Jeffrey R ^a  

^a UNIVERSITY OF VIRGINIA   (United States)

^b 3137 SQ ^*  (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

^d UNIVERSITY OF EAST ANGLIA   (United Kingdom)

^e DISNEY RESEARCH   (United States)

Author keywords

Dynamics; Facial expression; Symmetry

Indexed keywords

ADAPTATION; BEHAVIORAL RESPONSE; COMMUNICATION BEHAVIOR; EXPERIMENTAL DESIGN; MOVEMENT; PSYCHOLOGY; REAL TIME; SYMMETRY;

EID: 73549097793     PISSN: 09628436     EISSN: 14712970     Source Type: Journal    
DOI: 10.1098/rstb.2009.0152     Document Type: Article

References (40)

1. Ambadar, Z., Cohn, J. F. & Reed, L. I. 2009 All smiles are not created equal: morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. J. Nonverbal Behav. 33, 17-34. (doi:10.1007/s10919008-0059- 5)
2. Ashenfelter, K. T., Boker, S. M., Waddell, J. R., Vitanov, N. & Abadjieva, E. 2009 Spatiotemporal symmetry and multifractal structure of head movements during dyadic conversation. J. Exp. Psychol. Hum. Percept. Perform. 35(4), 1072-1091.
3. Bernieri, F. J. 1988 Coordinated movement and rapport in teacher-student interactions. J. Nonverbal Behav. 12, 120-138. (doi:10.1007/BF00986930)
4. Boker, S. M. & Cohn, J. F. In press. Real time dissociation of facial appearance and dynamics during natural conversation. In Dynamic faces: insights from experiments and computation (eds M. Giese, C. Curio & H. Bültoff). Cambridge, MA: MIT Press.
5. Boker, S. M. & Rotondo, J. L. 2002 Symmetry building and symmetry breaking in synchronized movement. In Mirror neurons and the evolution of brain and language (eds M. Stamenov & V. Gallese), pp. 163-171. Amsterdam, The Netherlands: John Benjamins.
6. Boker, S. M., Xu, M., Rotondo, J. L. & King, K. 2002 Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychol. Meth. 7, 338-355. (doi:10.1037/ 1082-989X.7.3.338)
7. Boker, S. M., Deboeck, P. R., Edler, C. & Keel, P. K. In press. Generalized local linear approximation of derivatives from time series. In Statistical methods for modeling human dynamics: an interdisciplinary dialogue (eds S.-M. Chow & E. Ferrar). Boca Raton, FL: Taylor & Francis.
8. Cappella, J. N. & Panalp, S. 1981 Talk and silence sequences in informal conversations: III: interspeaker influence. Hum. Commun. Res. 7, 117-132. (doi:10.1111/j.l4682958.1981.tb00564.x)
9. Cohn, J. F. & Tronick, E. Z. 1983 Three month old infants' reaction to simulated maternal depression. Child Dev. 54, 185-193. (doi: 10.2307/1129876)
10. Cohn, J. F., Kreuze, T. S., Yang, Y., Gnuyen, M. H., Padilla, M. T. & Zhou, F. 2009 Detecting depression from facial actions and vocal prosody. Affective computing and intelligent interaction (ACII 2009). Amsterdam, The Netherlands: IEEE.
11. Cohn, J. F., Reed, L. I., Moriyama, T., Xiao, J., Schmidt, K. L. & Ambadar, Z. 2004 Multimodal coordination of facial action, head rotation, and eye motion. 6th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 645-650. Seoul, Korea: IEEE.
12. Cootes, T. F., Edwards, G. & Taylor, C. J. 2001 Active appearance models. IEEE Trans. Pattern Anal. Machine Intell. 23, 681-685. (doi: 10.1109/34.927467)
13. Cootes, T. F., Wheeler, G. V., Walker, K. N. & Taylor, C. J. 2002 View-based active appearance models. Image Vis. Comput. 20, 657-664. (doi:10.1016/S0262-8856(02) 00055-0)
14. Coyne, J. C. 1976 Depression and the response of others. J. Abnorm. Psychol. 85, 186-193. (doi:10.1037/0021843X.85.2.186)
15. Ekman, P., Friesen, W. & Hager, J. 2002 Facial action coding system. Salt Lake City, UT: Research Nexus.
16. Ekman, P., Matsumoto, D. & Friesen, W. V. 2005 Facial expression in affective disorders. In What the face reveals (eds P. Ekman & E. Rosenberg), pp. 331-341. New York, NY: Oxford University Press.
17. Gehricke, J.-G. & Shapiro, D. 2000 Reduced facial expression and social context in major depression: discrepancies between facial muscle activity and self-reported emotion. Psychiatry Res. 95, 157-167. (doi:10.1016/ S0165-1781(00)00168-2) (Pubitemid 30645889)
18. Gross, R., Matthews, I. & Baker, S. 2004 Constructing and fitting active appearance models with occlusion. 1st IEEE Workshop on Face Processing in Video. Washington, DC: IEEE.
19. Hsee, C. K., Hatfield, E., Carlson, J. G. & Chemtob, C. 1990 The effect of power on susceptibility to emotional contagion. Cogn. Emotion 4, 327-340. (doi:10.1080/ 02699939008408081)
20. Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C. & Rizzolatti, G. 1999 Cortical mechanisms of human imitation. Science, 286, 2526-2528.
21. Kashy, D. A. & Kenny, D. A. 2000 The analysis of data from dyads and groups. In Handbook of research methods in social psychobgy (eds H. Reis & C. M. Judd), pp. 451-477. New York, NY: Cambridge University Press.
22. Keltner, D. 1995 Signs of appeasement: evidence for the distinct displays of embarrassment, amusement, and shame. J. Pers. Soc. Psychol. 68, 441-454. (doi:10.1037/ 0022-3514.68.3.441)
23. Kenny, D. A. & Judd, C. M. 1986 Consequences of violating the independence assumption in analysis of variance. Psychol. Bull. 99, 422-431. (doi:10.1037/0033-2909.99.3. 422)
24. Kenny, D. A., Kashy, D. A. & Cook, W. L. 2006 Dyadic data analysis. New York, NY: Guilford.
25. LaFrance, M. 1982 Posture mirroring and rapport. In Interaction rhythms: periodicity in communicative behavwr (ed. M. Davis), pp. 279-298. New York, NY: Human Sciences Press.
26. Lucey, S., Wang, Y., Cox, M., Sridharan, S. & Cohn, J. F. In press. Efficient constrained local model fitting for non-rigid face alignment. Image Vis. Comp. J.
27. Matthews, I. & Baker, S. 2004 Active appearance models revisited. Int. J. Comput. Vis. 60, 135-164. (doi:10. 1023/B:VISI.0000029666.37597.d3)
28. Messinger, D. S., Chow, S. M. & Cohn, J. F. 2009 Automated measurement of smile dynamics in motherinfant interaction: a pilot study. Infancy 14, 285-305. (doi:10.1080/15250000902839963)
29. Neumann, R. & Strack, F. 2000 'Mood contagion': the automatic transfer of mood between persons. J. Pers. Soc. Psychol. 79, 158-163. (doi:10.1037/0022-3514.79.2. 211)
30. Redlich, N. A. 1993 Redundancy reduction as a strategy for unsupervised learning. Neural Comput. 5, 289-304. (doi:10.1162/neco.1993.5.2.289)
31. Reed, L. I., Sayette, M. A. & Cohn, J. F. 2007 Impact of depression on response to comedy: a dynamic facial coding analysis. J. Abnorm. Psychol. 116, 804-809. (doi: 10.1037/0021-843X.116.4.804)
32. Rizzolatti, G. & Craighero, L. 2004 The mirror-neuron system. Ann. Rev. Neurosci. 27, 169-192. (doi: 10.1146/ annurev.neuro.27.070203.144230)
33. Rizzolatti, G. & Fadiga, L. 2007 Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex. In Novartis Foundation Symposium 218-Sensory Guidance of Movement (eds P. Ekman & E. Rosenberg), pp. 81-108. New York, NY: Novartis Foundation.
34. Rottenberg, J. 2005 Mood and emotion in major depression. Curr. Dir. Psychol. Sci. 14,167-170. (doi:10.1111/j.09637214.2005.00354.x)
35. Saragih, J., Lucey, S. & Cohn, J. F. 2009 Probabilistic constrained adaptive local displacement experts. Workshop at the Int. Conf. on Computer Vision, Kyoto, Japan.
36. Shannon, C. E. & Weaver, W 1949 The mathematical theory of communication. Urbana, IL: The University of Illinois Press.
37. Sheikh, Y. A., Datta, A. & Kanadě, T. 2008 On the sustained tracking of human motion. Proc. 8th Int. Conf. on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands.
38. Sloan, D. M., Bradley, M. M., Dimoulas, E. & Lang, P. J. 2002 Looking at facial expressions: dysphoria and facial EMG. Biol. Psychol 60, 79-90. (doi:10.1016/S0301- 0511(02)00044-3)
39. Theobald, B., Matthews, I., Cohn, J. F. & Boker, S. 2007 Real-time expression cloning using appearance models. Proc. 9th Int. Conf. on Multimodal Interfaces, pp. 134- 139. New York, NY: Association for Computing Machinery.
40. Young, R. D. & Frye, M. 1966 Some are laughing; some are not-why? Psychol. Rep. 18, 747-752.