Automated measurement of facial expression in infant - Mother interaction: A pilot study

Infancy

Volumn 14, Issue 3, 2009, Pages 285-305

  Messinger, Daniel S ^a   Mahoor, Mohammad H ^b   Chow, Sy Miin ^c   Cohn, Jeffrey F ^d,e  

^a UNIVERSITY OF MIAMI   (United States)

^b UNIVERSITY OF DENVER   (United States)

^c UNIVERSITY OF NORTH CAROLINA   (United States)

^d UNIVERSITY OF PITTSBURGH   (United States)

^e CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUTOMATION; CHILD BEHAVIOR; COMPUTER ANALYSIS; EMOTION; FACIAL EXPRESSION; FEMALE; HUMAN; IMAGE ANALYSIS; INFANT; INTERPERSONAL COMMUNICATION; MALE; MOTHER CHILD RELATION;

EID: 67651017903     PISSN: 15250008     EISSN: None     Source Type: Journal    
DOI: 10.1080/15250000902839963     Document Type: Article

References (66)

1. Ariely, D., Au, W. T., Bender, R. H., Budescu, D. V., Dietz, C. Gu, H. et al. (2000) The effects of averaging subjective probability estimates between and within judges. Journal of Experimental Psychology: Applied, 6, pp. 30-147.
2. Ashraf, A. B. L., Chen, S., Prkachin, T., Solomon, K., Ambadar, P. Z. and Cohn, J. (2007) The painful face: Pain expression recognition using active appearance models.
3. Bakeman, R. and Gottman, J. (1986) Observing interaction: An introduction to sequential analysis., Cambridge University Press, New York
4. Baker, S., Matthews, I. and Schneider, J. (2004) Automatic construction of active appearance models as an image coding problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, pp. 1380-1384.
5. Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I. and Movellan, J. (2005) Recognizing facial expression: Machine learning and application to spontaneous behavior. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, pp. 568-573.
6. Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I. and Movellan, J. (2006) Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia, 1:6, pp. 22-35.
7. Beebe, B. (1973) Ontongeny of positive affect in the third and fourth months of life of one infant.
8. Beebe, B. and Gerstman, L. (1984) A method of defining "packages" of maternal stimulation and their functional significance for the infant with mother and stranger. International Journal of Behavioral Development, 7, pp. 423-440.
9. Beebe, B., Jaffe, J., Buck, K., Chen, H., Cohen, P. Blatt, S. et al. (2007) Six-week postpartum maternal self-criticism and dependency and 4-month mother-infant self- and interactive contingencies. Developmental Psychology, 43, pp. 1360-1376.
10. Belkin, M. and Niyogi, P. (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation Archive, 15, pp. 1373-1396.
11. Boker, S. M., Rotondo, J. L., Xu, M. and King, K. (2002) Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7, pp. 338-355.
12. Bolzani-Dinehart, L., Messinger, D. S., Acosta, S., Cassel, T., Ambadar, Z. and Cohn, J. (2005) Adult perceptions of positive and negative infant emotional expressions. Infancy, 8, pp. 279-303.
13. Bruckner, C. T. and Yoder, P. (2006) Interpreting kappa in observational research: Baserate matters. American Journal on Mental Retardation, 111, pp. 433-441.
14. Chang, C. C. and Lin, C. J. (2001) LIBSVM: A library for support vector machines.
15. Chong, S. C. F., Werker, J. F., Russell, J. A. and Carroll, J. M. (2003) Three facial expressions mothers direct to their infants. Infant and Child Development, 12, pp. 211-232.
16. Cohn, J. and Kanade, T. (2007) Automated facial image analysis for measurement of emotion expression. The handbook of emotion elicitation and assessment, pp. 222-238. Oxford University Press, New York
17. Cohn, J., Reed, L. I., Moriyama, T., Xiao, J., Schmidt, K. and Ambadar, Z. (2004) Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles. Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 129-135.
18. Cohn, J. F. and Tronick, E. Z. (1987) Mother-infant face-to-face interaction: The sequence of dyadic states at 3, 6, and 9 months. Developmental Psychology, 23, pp. 68-77.
19. Cohn, J. F. and Tronick, E. Z. (1988) Mother-infant face-to-face interaction: Influence is bidirectional and unrelated to periodic cycles in either partner's behavior. Developmental Psychology, 24, pp. 386-392.
20. Cohn, J. F., Xiao, J., Moriyama, T., Ambadar, Z. and Kanade, T. (2003) Automatic recognition of eye blinking in spontaneously occurring behavior. Behavior Research Methods, Instruments, and Computers, 35, pp. 420-428.
21. Cohn, J. F., Zlochower, A. J., Lien, J. and Kanade, T. (1999) Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology, 36, pp. 35-43.
22. Ekman, P., Davidson, R. J. and Friesen, W. (1990) The Duchenne smile: Emotional expression and brain physiology II. Journal of Personality and Social Psychology, 58, pp. 342-353.
23. Ekman, P. and Friesen, W. V. (1982) Felt, false, and miserable smiles. Journal of Nonverbal Behavior, 6, pp. 238-252.
24. Ekman, P., Friesen, W. and Hager, J. C. (2002) The Facial Action Coding System on CD ROM., Network Information Research Center, Salt Lake City, UT
25. Ekman, P., Friesen, W. V. and O'Sullivan, M. (1988) Smiles when lying. Journal of Personality and Social Psychology, 54, pp. 414-420.
26. Fasel, B. and Luettin, J. (2003) Automatic facial expression analysis: A survey. Pattern Recognition, 36, pp. 259-275.
27. Feldman, R. (2006) From biological rhythms to social rhythms: Physiological precursors of mother-infant synchrony. Developmental Psychology, 42, pp. 175-188.
28. Feldman, R. (2007) On the origins of background emotions: From affect synchrony to symbolic expression. Emotion, 7, pp. 601-611.
29. Feldman, R., Greenbaum, C. W. and Yirmiya, N. (1999) Mother-infant affect synchrony as an antecedent of the emergence of self-control. Developmental Psychology, 35, pp. 223-231.
30. Fogel, A., Hsu, H. -C., Shapiro, A. F., Nelson-Goens, G. C. and Secrist, C. (2006) Effects of normal and perturbed social play on the duration and amplitude of different types of infant smiles. Developmental Psychology, 42, pp. 459-473.
31. Fox, N. and Davidson, R. J. (1988) Patterns of brain electrical activity during facial signs of emotion in 10 month old infants. Developmental Psychology, 24, pp. 230-236.
32. Frank, M. G., Ekman, P. and Friesen, W. V. (1993) Behavioral markers and the recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64, pp. 83-93.
33. Gross, R., Matthews, I. and Baker, S. (2006) Active appearance models with occlusion. Image and Vision Computing, 24, pp. 593-604.
34. Harker, L. and Keltner, D. (2001) Expressions of positive emotion in women's college yearbook pictures and their relationship to personality and life outcomes across adulthood. Journal of Personality and Social Psychology, 80, pp. 112-124.
35. Harris, C. R. (1999) The mystery of ticklish laughter. American Scientist, 87:4, p. 344.
36. Ibanez, L., Messinger, D., Newell, L., Sheskin, M. and Lambert, B. (2008) Visual disengagement in the infant siblings of children with an autism spectrum disorder (ASD). Autism: International Journal of Research and Practice, 12, pp. 523-535.
37. Kaye, K. and Fogel, A. (1980) The temporal structure of face-to-face communication between mothers and infants. Developmental Psychology, 16, pp. 454-464.
38. Kochanska, G. (2002) Mutually responsive orientation between mothers and their young children: A context for the early development of conscience. Current Directions in Psychological Science, 11:6, pp. 191-195.
39. Kochanska, G., Forman, D. R. and Coy, K. C. (1999) Implications of the mother-child relationship in infancy socialization in the second year of life. Infant Behavior and Development, 22, pp. 249-265.
40. Kochanska, G. and Murray, K. T. (2000) Mother-child mutually responsive orientation and conscience development: From toddler to early school age. Child Development, 71, pp. 417-431.
41. Lester, B. M., Hoffman, J. and Brazelton, T. B. (1985) The rhythmic structure of mother-infant interaction in term and preterm infants. Child Development, 56, pp. 15-27.
42. Littlewort, G., Bartlett, M. S. and Movellan, J. R. (2001) Are your eyes smiling? Detecting genuine smiles with support vector machines and Gabor wavelets.
43. Meng, X. -L., Rosenthal, R. and Rubin, D. B. (1992) Comparing correlated correlation coefficients. Psychological Bulletin, 111, pp. 172-175.
44. Messinger, D. (2002) Positive and negative: Infant facial expressions and emotions. Current Directions in Psychological Science, 11:1, pp. 1-6.
45. Messinger, D. S., Cassel, T., Acosta, S., Ambadar, Z. and Cohn, J. (2008) Infant smiling dynamics and perceived positive emotion. Journal of Nonverbal Behavior, 32, pp. 133-155.
46. Messinger, D. and Fogel, A. (2007) The interactive development of social smiling. Advances in child development and behavior, 35, pp. 328-366. Elsevier, Oxford, UK
47. Messinger, D., Fogel, A. and Dickson, K. (2001) All smiles are positive, but some smiles are more positive than others. Developmental Psychology, 37, pp. 642-653.
48. Oster, H. (2006) Baby FACS: Facial action coding system for infants and young children (Unpublished monograph and coding manual), New York University, New York
49. Reed, L. I., Sayette, M. A. and Cohn, J. F. (2007) Impact of depression on response to comedy: A dynamic facial coding analysis. Journal of Abnormal Psychology, 116, pp. 804-809.
50. Rogers, C. R., Schmidt, K. L., VanSwearingen, J. M., Cohn, J. F., Wachtman, G. S. Manders, E. K. et al. (2007) Automated facial image analysis: Detecting improvement in abnormal facial movement after treatment with Botulinum toxin: A. Annals of Plastic Surgery, 58, pp. 39-47.
51. Ruch, W. (1993) Exhilaration and humor. Handbook of emotions, pp. 605-616. Guilford, New York
52. Schmidt, K. L., Ambadar, Z., Cohn, J. F. and Reed, L. (2006) Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling. Journal of Nonverbal Behavior, 30, pp. 37-52.
53. Schmidt, K. L., Liu, Y. and Cohn, J. F. (2006) The role of structural facial asymmetry in asymmetry of peak facial expressions. Laterality: Asymmetries of Body, Brain and Cognition, 11:6, pp. 540-561.
54. Schneider, K. and Uzner, L. (1992) Preschoolers' attention and emotion in an achievement and an effect game: A longitudinal study. Cognition and Emotion, 6, pp. 37-63.
55. Schore, A. N. (1994) Affect regulation and the origin of self: The neurobiology of emotional development., Lawrence Erlbaum Associates, Inc, Hillsdale, NJ
56. Stern, D. N. (1974) Mother and infant at play: The dyadic interaction involving facial, vocal, and gaze behaviors. The effect of the infant on its caregiver, pp. 187-213. Wiley, New York
57. Stern, D. N. (1985) The interpersonal world of the infant: A view from psychoanalysis and developmental psychology., Basic Books, New York
58. Tian, Y., Kanade, T. and Cohn, J. F. (2001) Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, pp. 97-116.
59. Tian, Y. L., Kanade, T. and Cohn, J. F. (2002) Evaluation of Gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity.
60. Tronick, E. Z. (1989) Emotions and emotional communication in infants. American Psychologist, 44, pp. 112-119.
61. Tronick, E. Z. and Cohn, J. F. (1989) Infant-mother face-to-face interaction: Age and gender differences in coordination and the occurrence of miscoordination. Child Development, 60, pp. 85-92.
62. Tronick, E. Z., Messinger, D., Weinberg, K. M., Lester, B. M., LaGasse, L. Seifer, R. et al. (2005) Cocaine exposure compromises infant and caregiver social emotional behavior and dyadic interactive features in the face-to-face still-face paradigm. Developmental Psychology, 41, pp. 711-722.
63. Van Egeren, L. A., Barratt, M. S. and Roach, M. A. (2001) Mother-infant responsiveness: Timing, mutual regulation, and interactional context. Developmental Psychology, 37, pp. 684-697.
64. Wang, Y., Lucey, S., Saragih, J. and Cohn, J. F. (2008) Non-rigid tracking with local appearance consistency constraint.
65. Weinberg, K. M. and Tronick, E. Z. (1998) Infant and caregiver engagement phases system., Harvard Medical School, Boston
66. Weinberg, M. K., Tronick, E. Z., Cohn, J. F. and Olson, K. L. (1999) Gender differences in emotional expressivity and self-regulation during early infancy. Developmental Psychology, 35, pp. 175-188.