Social signal processing: Understanding social interactions through nonverbal behavior analysis

2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009

Volumn , Issue , 2009, Pages 42-49

  Vinciarelli, A ^a,b   Salamin, H ^a,b   Pantic, M ^b  

^a IDIAP RESEARCH INSTITUTE   (Switzerland)

^b UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SCIENCE; COMPUTERS; PATTERN RECOGNITION; SOFTWARE ENGINEERING;

AUTOMATIC UNDERSTANDING; BODY POSTURES; FACIAL EXPRESSIONS; NONVERBAL BEHAVIOR; SOCIAL INTERACTIONS; SOCIAL SIGNAL PROCESSING; SOCIAL SIGNALS;

COMPUTER VISION;

EID: 70449576066     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2009.5204290     Document Type: Conference Paper

References (76)

1. J. Ajmera, I. McCowan, and H. Bourlard. Speech/music segmentation using entropy and dynamism features in a HMM classification framework. Speech Communication, 40(3):351-363, 2003. 4
2. K. Albrecht. Social Intelligence: The new science of success. John Wiley & Sons Ltd, 2005. 6
3. N. Ambady, F. Bernieri, and J. Richeson. Towards a histology of social behavior: judgmental accuracy from thin slices of behavior. In M. Zanna, editor, Advances in Experimental Social Psychology, pages 201-272. 2000. 2, 3
4. M. Argyle. The Psychology of Interpersonal Behaviour. Penguin, 1967. 1
5. J. Bailenson and N. Yee. Virtual interpersonal touch and digital chameleons. Journal of Nonverbal Behavior, 31(4):225-242, 2007. 6
6. J. Bailenson, N. Yee, K. Patel, and A. Beall. Detecting digital chameleons. Computers in Human Behavior, 24(1):66-87, 2008. 6
7. S. Banerjee and A. Rudnicky. Using simple speech based features to detect the state of a meeting and the roles of the meeting participants. In Proceedings of International Conference on Spoken Language Processing, pages 2189-2192, 2004. 5
8. M. Buchanan. The science of subtle signals. Strategy+Business, 48:68-77, 2007. 6
9. J. Burgoon, L. Stern, and L. Dillman. Interpersonal Adaptation: Dyadic Interaction Patterns. Cambridge University Press, 1995. 6
10. J. Cassell. Embodied conversational interface agents. Communications of the ACM, 43(4):70-78, 2000. 1
11. T. Chartrand and J. Bargh. The chameleon effect: the perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76(6):893-910, 1999. 6
12. J. Cortes and F. Gatti. Physique and self-description of temperament. Journal of Consulting Psychology, 29(5):432-439, 1965. 2
13. R. Coulston, S. Oviatt, and C. Darves. Amplitude convergence in children's conversational speech with animated personas. In International Conference on Spoken Language Processing, pages 2689-2692, 2002. 6
14. J. Curhan and A. Pentland. Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. Journal of Applied Psychology, 92(3): 802-811, 2007. 5
15. T. Darrell, G. Gordon, M. Harville, and J. Woodfill. Integrated person tracking using stereo, color, and pattern detection. International Journal of Computer Vision, 37(2):175-185, 2000. 2
16. C. Darwin. The Expression of the Emotions in Man and Animals. J. Murray, 1872. 2
17. A. Dielmann and S. Renais. Automatic meeting segmentation using dynamic bayesian networks, IEEE Transactions on Multimedia, 9(1):25, 2007. 3, 5
18. K. Dion, E. Berscheid, and E. Walster. What is beautiful is good. Journal of Personality and Social Psychology, 24(3):285-290, 1972. 2
19. W. Dong, B. Lepri, A. Cappelletti, A. Pentland, F. Pianesi, and M. Zancanaro. Using the influence model to recognize functional roles in meetings. In Proceedings of the International Conference on Multimodal Interfaces, pages 271-278, 2007. 5
20. N. Eagle and A. Pentland. Reality mining: sensing complex social signals. Journal of Personal and Ubiquitous Computing, 10(4):255-268, 2006. 3, 6
21. P. Ekman and E. Rosenberg. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, 2005. 2, 3
22. A. Elgammal, Human-Centered Multimedia: representations and challenges. In Proc. ACM Intl. Workshop on Human-Centered Multimedia, pages 11-18, 2006. 1
23. N. Garg, S. Favre, H. Salamin, D. Hakkani-Tür, and A. Vinciarelli. Role recognition for meeting participants: an approach based on lexical information and social network analysis. In Proceedings of the ACM International Conference on Multimedia, pages 693-696, 2008. 5
24. J. Gemmell, K. Toyama, C. Zitnick, T. Kang, and S. Seitz. Gaze awareness for video-conferencing: A software approach. IEEE Multimedia, 7(4):26-35, 2000. 1
25. E. Goffman. The presentation of self in everyday life. Anchor Books, 1959. 2
26. K. Greene. 10 emerging technologies 2008.MIT Technology Review, february 2008. 6
27. H. Gunes and M. Piccardi. Assessing facial beauty through proportion analysis by image processing and supervised learning. International Journal of Human-Computer Studies, 64(12):1184-1199, 2006. 2
28. H. Gunes, M. Piccardi, and M. Pantic. From the lab to the real world: Affect recognition using multiple cues and modalities. In J. Or, editor, Affective Computing: Focus on Emotion Expression, Synthesis, and Recognition, pages 185-218.2008.2
29. E. Hall. The silent language. Doubleday, 1959. 3
30. M. Hecht, J. De Vito, and L. Guerrero. Perspectives on non-verbal communication, codes, functions and contexts. In L. Guerrero, J. De Vito, and M. Hecht, editors, The nonverbal communication reader, pages 201-272. 2000. 2
31. D. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez. Modeling dominance in group conversations using non-verbal activity cues. IEEE Transactions on Audio, Speech and Language: Special Issue on Multimedia, to appear, 2009. 5
32. D. Keltner and P. Ekman. Facial expression of emotion. In M. Lewis and J. Haviland-Jones, editors, Handbook of Emotions, pages 236-249. 2000. 2
33. M. Knapp and J. Hall. Nonverbal Communication in Human Interaction. Harcourt Brace College Publishers, 1972. 1, 2
34. K. Laskowski, M. Ostendorf, and T. Schultz. Modeling vocal interaction for text-independent participant characterization in multi-party conversation. In Proceedings of the SIGdial Workshop on Discourse and Dialogue, pages 148-155, 2008. 5
35. J. Levine and R. Moreland. Small groups. In D. Gilbert and G. Lindzey, editors, The handbook of social psychology, volume 2, pages 415-469. Oxford University Press, 1998. 4, 5
36. I. McCowan, S. Bengio, D. Gatica-Perez, G. Lathoud, F. Monay, D. Moore, P. Wellner, and H. Bouriard. Modeling human interaction in meetings. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pages 748-751, 2003. 3
37. I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard, and D. Zhang. Automatic analysis of multi-modal group actions in meetings. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3):305-317, 2005. 3, 5
38. L. Morency, I. de Kok, and J. Gratch. Context-based recognition during human interactions: automatic feature selection and encoding dictionary. In Proceedings of the 10th international conference on Multimodal interfaces, pages 181-188, 2008. 5
39. L. Morency, I. de Kok, and J. Gratch. Predicting listener backchannels: A probabilistic multimodal approach. In Lecture Notes in Computer Science, volume 5208, pages 176-190. Springer, 2008. 5
40. N. Morgan, E. Fosler, and N. Mirghafori. Speech recognition using on-line estimation of speaking rate. In Proceedings of Eurospeech, pages 2079-2082, 1997. 2
41. S. Mota and R. Picard. Automated posture analysis for detecting learners interest level. In Proceedings of Conference on Computer Vision and Pattern Recognition, pages 49-56, 2003. 3
42. C. Nass and S. Brave. Wired for speech: How voice activates and advances the Human-Computer relationship. The MIT Press, 2005. 1, 6
43. C. Nass and K. Lee. Does computer-synthesized speech manifest personality? Experimental tests of recognition, similarity-attraction, and consistency-attraction. Journal of Experimental Psychology: Applied, 7(3): 171-181, 2001. 6
44. C. Nass and J. Steuer. Computers and social actors. Human Communication Research, 19(4):504-527, 1993. 6
45. S. Oviatt, C. Darves, and R. Coulston. Toward adaptive conversational interfaces: Modeling speech convergence with animated personas. ACMTransactions on Computer-Human Interaction, 11(3):300-328, 2004. 6
46. M. Pantic, A. Pentland, A. Nijholt, and T. Huang. Human computing and machine understanding of human behavior: A survey. In Lecture Notes in Artificial Intelligence, volume 4451, pages 47-71. Springer Verlag, 2007. 1
47. 2): How far are we from attaining it? International Journal of Autonomous and Adaptive Communications Systems, 1(2):168-187, 2008. 4
48. A. Pentland. Social dynamics: Signals and behavior. In International Conference on Developmental Learning, 2004. 5
49. A. Pentland. Socially aware computation and communication. IEEE Computer, 38(3):33-40, 2005. 5
50. A. Pentland. Automatic mapping and modeling of human networks. Physica A, 378:59-67, 2007. 6
51. A. Pentland. Social Signal Processing. IEEE Signal Processing Magazine, 24(4): 108-111, 2007. 1, 5
52. S. Petridis and M. Pantic. Audiovisual laughter detection based on temporal features. In Proceedings of the 10th international conference on Multimodal interfaces, pages 37-44. ACM New York, NY, USA, 2008. 2
53. G. Psathas. Conversation Analysis - The study of talk-ininteraction. Sage Publications, 1995. 2, 4
54. V. Richmond and J. McCroskey. Nonverbal Behaviors in interpersonal relations. Allyn and Bacon, 1995. 1, 2
55. R. Rienks, D. Zhang, and D. Gatica-Perez. Detection and application of influence rankings in small group meetings. In Proceedings of the International Conference on Multimodal Interfaces, pages 257-264, 2006. 5
56. K. Scheret Personality markers in speech. Cambridge University Press, 1979. 2
57. K. Scherer. Vocal communication of emotion: A review of research paradigms. Speech Communication, 40(1-2):227-256, 2003. 2
58. E. Shriberg. Phonetic consequences of speech disfluency. Proceedings of the International Congress of Phonetic Sciences, 1:619-622, 1999. 2
59. L. Smith-Lovin and C. Brody. Interruptions in group discussions: the effects of gender and group composition. American Sociological Review, 54(3):424-435, 1989. 2
60. Y. Tong, W. Liao, and Q. Ji. Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10): 1683-1699, 2007. 3
61. S. Tranter and D. Reynolds. An overview of automatic speaker diarization systems. IEEE Transactions on Audio, Speech, and Language Processing, 14(5): 1557-1565, 2006. 3, 4
62. K. Truong and D. Leeuwen. Automatic detection of laughter. In Proceedings of the European Conference on Speech Communication and Technology, pages 485-488, 2005. 2
63. M. Valstar, H. Gunes, and M. Pantic. How to distinguish posed from spontaneous smiles using geometric features. In Proceedings of the International Conference on Multimodal Interfaces, pages 38-45, 2007. 3
64. R. Vertegaal, R. Slagter, G. van der Veer, and A. Nijholt. Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 301-308, 2001. 2
65. A. Vinciarelli. Sociometry based multiparty audio recording segmentation. In Proceedings of IEEE International Conference on Multimedia and Expo, pages 1801-1804, 2006, 6 (Pubitemid 46680087)
66. A. Vinciarelli. Speakers role recognition in multiparty audio recordings using social network analysis and duration distribution modeling. IEEE Transactions on Multimedia, 9(9):1215-1226, 2007. 6
67. A. Vinciarelli and S. Favre. Broadcast news story segmentation using social network analysis and hidden markov models. In Proceedings of the ACM International Conference on Multimedia, pages 261-264, 2007. 4
68. A. Vinciarelli, M. Pantic, and H. Bourlard. Social Signal Processing: survey of an emerging domain. Image and Vision Computing, to appear, 2009. 1, 2, 3, 5
69. A. Vinciarelli, M. Pantic, H. Bourlard, and A. Pentland. Social Signal Processing: State-of-the-art and future perspectives of an emerging domain. In Proceedings of the ACM International Conference on Multimedia, pages 1061-1070, 2008. 1, 3, 5
70. A. Waibel, T. Schultz, M. Bett, M. Denecke, R. Malkin, I. Rogina, and R. Stiefelhagen. SMaRT: the Smart Meeting Room task at ISL. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 752-755, 2003. 3
71. C. Weng, W. Chu, and J. Wu. Rolenet: Movie analysis from the perspective of social networks. IEEE Transactions on Multimedia, 11(2):256-271, 2009. 6
72. Y. Wu and T. Huang. Vision-based gesture recognition: A review. In A. Braffort, R. Gherbi, S. Gibet, J. Richardson, and D, Teil, editors, Gesture based communication in Human-Computer Interaction, pages 103-114. 1999. 2
73. M. Yang, D. Kriegman, and N. Ahuja. Detecting faces in images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1):34-58, 2002. 3
74. G. Yule. Pragmatics. Oxford University Press, 1996. 2, 4
75. M. Zancanaro, B. Lepri, and F. Pianesi. Automatic detection of group functional roles in face to face interactions. In Proceedings of the International Conference on Multimodal Interfaces, pages 28-34, 2006. 5
76. Z. Zeng, M. Pantic, G. Roisman, and T. Huang. A survey of affect recognition methods: audio, visual and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1):39-58, 2009. 2, 3