Mining group nonverbal conversational patterns using probabilistic topic models

IEEE Transactions on Multimedia

Volumn 12, Issue 8, 2010, Pages 790-802

  Jayagopi, Dinesh Babu ^a,b   Gatica Perez, Daniel ^a,b  

^a IDIAP RESEARCH INSTITUTE   (Switzerland)

^b EPFL   (Switzerland)

Author keywords

Discovery; group behavior descriptor; meetings; nonverbal cues

Indexed keywords

AUTOMATIC DISCOVERY; DESCRIPTORS; DISCOVERY; GROUP BEHAVIOR; GROUP INTERACTION; HUMAN ANNOTATIONS; MEETINGS; MINING GROUPS; MULTIPLE OBSERVERS; NONVERBAL CUES; PROBABILISTIC TOPIC MODELS; SOCIAL COMPUTING; TOPIC MODEL;

BEHAVIORAL RESEARCH;

EID: 78649436072     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2010.2065218     Document Type: Article

References (50)

1. K. Bachour, F. Kaplan, and P. Dillenbourg, "Reflect: An interactive table for regulating face-to-face collaborative learning", in Proc. 3rd Eur. Conf. Technology Enhanced Learning: Times of Convergence: Technologies Across Learning Contexts, 2008, pp. 39-48.
2. R. F. Bales, Interaction Process Analysis: A Method for the Study of Small Groups. Cambridge, MA: Addison-Wesley, 1950.
3. S. Basu, T. Choudhury, B. Clarkson, and A. Pentland, "Towards measuring human interactions in conversational settings", in Proc. IEEE CVPR Workshop Cues in Communication, 2001.
4. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent dirichlet allocation", J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
5. Y. Boureau, F. Bach, Y. LeCun, and J. Ponce, "Learning mid-level features for recognition", in Proc. Int. Conf. Computer Vision and Pattern Recognition (CVPR), 2010.
6. J. Boyd-Graber, J. Chang, S. Gerrish, C. Wang, and D. Blei, "Reading tea leaves: How humans interpret topic models", Adv. Neural Inf. Process. Syst. (NIPS), vol. 31, 2009.
7. J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, W. Post, D. Reidsma, and P. Wellner, "The AMI meeting corpus:A pre-announcement", Lecture Notes in Computer Science, vol. 3869, pp. 28-28, 2006.
8. A. Dielmann and S. Renals, "Automatic meeting segmentation using dynamic Bayesian networks", IEEE Trans. Multimedia, vol. 9, no. 1, pp. 25-25, Jan. 2007.
9. J. M. DiMicco, K. J. Hollenbach, and W. Bender, "Using visualizations to review a group's interaction dynamics", in Proc. Conf. Human Factors in Computing Systems, New York, 2006, pp. 706-711.
10. J. Dines, J. Vepa, and T. Hain, "The segmentation of multi-channel meeting recordings for automatic speech recognition", in Proc. 9th Int. Conf. Spoken Language Processing, 2006.
11. W. Dong, B. Lepri, A. Cappelletti, A. S. Pentland, F. Pianesi, and M. Zancanaro, "Using the influence model to recognize functional roles in meetings", in Proc. Int. Conf. Multimodal Interfaces (ICMI), Nagoya, Japan, 2007, pp. 271-278.
12. W. Dong, A. Mani, A. Pentland, B. Lepri, and F. Pianesi, "Modeling group discussion dynamics", IEEE Trans. Auton. Mental Develop., submitted for publication.
13. P. Dourish and V. Bellotti, "Awareness and coordination in shared workspaces", in Proc. 1992 ACM Conf. Computer-Supported Cooperative Work, Toronto, ON, Canada, 1992, pp. 114-114.
14. N. P. Garg, S. Favre, H. Salamin, D. H. Tur, and A. Vinciarelli, "Role recognition for meeting participants: An approach based on lexical information and social network analysis", in Proc. ACM Int. Conf. Multimedia, Vancouver, BC, Canada, 2008.
15. D. Gatica-Perez, "Automatic nonverbal analysis of social interaction in small groups: A Review", Image Vis. Comput., Special Issue on Human Behavior, vol. 27, no. 12, pp. 1775-1787, Nov. 2009.
16. D. Gatica-Perez, I. McCowan, D. Zhang, and S. Bengio, "Detecting group interest-level in meetings", in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), 2005, vol. 1.
17. T. L. Griffiths and M. Steyvers, "Finding scientific topics", Proc. Nat. Acad. Sci., vol. 101, no. 1, pp. 5228-5228, 2004.
18. J. A. Hall, E. J. Coats, and L. S. LeBeau, "Nonverbal behavior and the vertical dimension of social relations: A meta-analysis", Psychol. Bull., vol. 131, pp. 898-924, 2005.
19. R. R. Hassin, J. S. Uleman, and J. A. Bargh, The New Unconscious. New York: Oxford Univ. Press, 2005.
20. G. Heinrich, Parameter Estimation for Text Analysis, 2005. [Online]. Available: http://www.arbylon. net/publications/text-est.pdf.
21. D. Jayagopi, S. Ba, J. M. Odobez, and D. Gatica-Perez, "Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues", in Proc. Int. Conf. Multimodal Interfaces (ICMI), New York, 2008, pp. 45-52.
22. D. Jayagopi and D. Gatica-Perez, "Discovering group nonverbal conversational patterns with topics", in Proc. Int. Conf. Multimodal Interfaces (ICMI-MLMI), Boston, MA, Nov. 2009.
23. D. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez, "Modeling dominance in group conversations using nonverbal activity cues", IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 3, pp. 501-513, Mar. 2009.
24. D. Jayagopi, B. Raducanu, and D. Gatica-Perez, "Characterizing conversational group dynamics using nonverbal behaviour", in Proc. Int. Conf. Multimedia and Expo (ICME), New York, 2009.
25. O. P. John and S. Srivastava, "The Big-Five trait taxonomy: History, measurement, and theoretical perspectives", in Handbook of Personality: Theory and Research. New York: Guilford, 1999, vol. 2, pp. 102-135.
26. T. Kim, A. Chang, L. Holland, and A. S. Pentland, "Meeting mediator: Enhancing group collaboration using sociometric feedback", in Proc. ACM 2008 Conf. Computer Supported Cooperative Work, New York, 2008, pp. 457-466.
27. M. L. Knapp and J. A. Hall, Nonverbal Communication in Human Interaction. New York: Holt, Rinehart and Winston, 1978.
28. J. M. Levine and R. L. Moreland, "Progress in small group research", Annu. Rev. Psychol., vol. 41, no. 1, pp. 585-634, 1990.
29. K. Lewin, R. Llippit, and R. K. White, "Patterns of aggressive behavior in experimentally created social climates", J. Social Psychol., vol. 10, pp. 271-301, 1939.
30. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge, U. K.: Cambridge Univ. Press, 2003.
31. V. L. Manusov and M. L. Patterson, The SAGE Handbook of Nonverbal Communication. Newbury Park, CA: Sage, 2006.
32. J. E. McGrath, Groups: Interaction and Performance.:, 1984.
33. D. McNeill, Language and Gesture. Cambridge, U. K.: Cambridge Univ. Press, 2000.
34. D. Moore, "The IDIAP smart meeting room", in Proc. IDIAP COM 02, vol. 7, no. 2002.
35. K. Otsuka, H. Sawada, and J. Yamato, "Automatic inference of cross-modal nonverbal interactions in multiparty conversations: who responds to whom, when, and how? from gaze, head gestures, and utterances", in Proc. Int. Conf. Multimodal Interfaces (ICMI), Nagoya, Japan, 2007, pp. 255-262.
36. K. Otsuka, J. Yamato, Y. Takemae, and H. Murase, "Quantifying interpersonal influence in face-to-face conversations based on visual attention patterns", in Proc. CHI'06 Extended Abstracts on Human Factors in Computing Systems, 2006, pp. 1180-1180.
37. A. S. Pentland, Honest Signals: How They Shape Our World. Cambridge, MA: The MIT Press, 2008.
38. F. Pianesi, N. Mana, A. Cappelletti, B. Lepri, and M. Zancanaro, "Multimodal recognition of personality traits in social interactions", in Proc. Int. Conf. Multimodal Interfaces (ICMI), Chania, Greece, 2008, pp. 53-60.
39. F. Pianesi, M. Zancanaro, E. Not, C. Leonardi, and V. Falcon, "Multimodal support to group dynamics", Pers. Ubiq. Comput., vol. 12, no. 3, pp. 181-195, 2008.
40. M. S. Poole, A. B. Hollingshead, J. E. McGrath, R. L. Moreland, and J. Rohrbaugh, "Interdisciplinary perspectives on small groups", Small Group Res., vol. 35, no. 1, pp. 3-3, 2004.
41. B. Raducanu, J. Vitria, and D. Gatica-Perez, "You are fired! nonverbal role analysis in competitive meetings", in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), 2009.
42. M. Remland, Uses and Consequences of Nonverbal Communication in the Context of Organizational Life. Newbury Park, CA: Sage, 2006.
43. R. J. Rienks and D. Heylen, "Automatic dominance detection in meetings using easily detectable features", in Proc. Machine Learning and Multimodal Interaction (MLMI) Workshop, Edinburgh, U. K., 2005.
44. J. B. Rotter, Generalized Expectancies for Internal Versus External Control of Reinforcement. Washington, DC: American Psychological Assoc, 1966.
45. D. Sanchez-Cortes, D. Jayagopi, and D. Gatica-Perez, "Predicting remote versus collocated group interactions using nonverbal cues", in Proc. Workshop Multimodal Sensor-Based Systems and Mobile Phones for Social Computing-Proc. Int. Conf. Multimodal Interfaces (ICMI-MLMI), Boston, MA, Nov. 2009.
46. M. Steyvers and T. Griffiths, "Probabilistic topic models", in Latent Semantic Analysis: A Road to Meaning, D. McNamara, S. Dennis, and W. Kintsch, Eds. Mahwah, NJ: Laurence Erlbaum, 2007, vol. 427.
47. J. Sturm, O. H. Herwijnen, A. Eyck, and J. Terken, "Influencing social dynamics in meetings through a peripheral display", in Proc. 9th Int. Conf. Multimodal Interfaces, 2007, pp. 263-270.
48. A. Vinciarelli, "Speakers role recognition in multiparty audio recordings using social network analysis and duration distribution modeling", IEEE Trans. Multimedia, vol. 12, no. 6, pp. 1215-1226, Oct. 2007.
49. B. Wrede and E. Shriberg, "Spotting hot spots in meetings: Human judgments and prosodic cues", in Proc. 8th Eur. Conf. Speech Communication and Technology, 2003.
50. D. Zhang, D. Gatica-Perez, S. Bengio, I. McCowan, and G. Lathoud, "Modeling individual and group actions in meetings with layered HMMs", IEEE Trans. Multimedia, vol. 8, no. 3, pp. 509-520, Jun. 2006.