Generative modeling and classification of dialogs by a low-level turn-taking feature

Pattern Recognition

Volumn 44, Issue 8, 2011, Pages 1785-1800

  Cristani, Marco ^c   Pesarin, Anna ^a   Drioli, Carlo ^a   Tavano, Alessandro ^b   Perina, Alessandro ^d   Murino, Vittorio ^c  

^a UNIVERSITY OF VERONA   (Italy)

^b UNIVERSITY OF LEIPZIG   (Germany)

^c ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^d MICROSOFT RESEARCH   (United States)

Author keywords

Classification; Dialog analysis; Feature extraction; Generative modeling

Indexed keywords

ACOUSTIC FEATURES; ARTIFICIAL SYSTEMS; AUTOMATIC SYSTEMS; CLASSIFICATION; CLASSIFICATION METHODS; CLASSIFICATION PERFORMANCE; COMPARATIVE TESTS; DATA SETS; DIALOG ANALYSIS; GAUSSIAN MIXTURE MODEL; GENERATIVE MODEL; GENERATIVE MODELING; HUMAN-HUMAN COMMUNICATION; INFLUENCE MODEL; MARKOVIAN; TURN-TAKING;

COMMUNICATION; FEATURE EXTRACTION;

CLASSIFICATION (OF INFORMATION);

EID: 79953059545     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.01.013     Document Type: Article

References (62)

1. A. Pentland Social signal processing Signal Process. Mag. IEEE 24 4 2007 108 111 (Pubitemid 47316171)
2. T. Choudhury, and S. Basu Modeling conversational dynamics as a mixed-memory Markov process L.K. Saul, Y. Weiss, L. Bottou, Advances in Neural Information Processing Systems vol. 17 2005 MIT Press Cambridge, MA 281 288
3. S. Basu, T. Choudhury, B. Clarkson, A. Pentland, Learning human interaction with the influence model, MIT MediaLab, Technical Report 539, 2001.
4. C. Asavathiratham, A tractable representation for the dynamics of networked Markov chain, Ph.D. Dissertation, Department of ECS, MIT, 2000.
5. I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard, and D. Zhang Automatic analysis of multimodal group actions in meetings IEEE Trans. Pattern Anal. Mach. Intell. 27 3 2005
6. T. Jebara, Y. Ivanov, A. Rahimi, and A. Pentland Tracking conversational context for machine mediation of human discourse AAAI Fall 2000 Symposium - Socially Intelligent Agents - The Human in the Loop 2000 AAAI Press
7. J. Edlund, and M. Heldner Exploring prosody in interaction control Phonetica 62 2005 215 226 (Pubitemid 43030014)
8. M. Pantic, A. Pentland, and A. Nijholt Special issue on human computing IEEE Trans. Syst. Man Cybern. Part B 39 1 2009
9. M. Kotti, D. Ververidis, G. Evangelopoulos, I. Panagakis, C. Kotropoulos, P. Maragos, and I. Pitas Audio-assisted movie dialogue detection IEEE Trans. Circuits Syst. Video Technol. 18 2008 1618 1627
10. P. Dai, H. Di, L. Dong, L. Tao, and G. Xu Group interaction analysis in dynamic context IEEE Trans Syst. Man Cybern. Part B 38 1 2008 275 282 (Pubitemid 351285031)
11. T. Choudhury, and A. Pentland Characterizing social interactions using the sociometer Proceedings of NAACOS 2004
12. A.S. Pentland Socially aware computation and communication Computer 38 3 2005 33 40
13. J. Curhan, and A. Pentland Thin slices of negotiation: predicting outcomes from conversational dynamics within the first five minutes J. Appl. Psychol. 92 2007 802 811
14. A. Vinciarelli Speakers role recognition in multiparty audio recordings using social network analysis and duration distribution modeling IEEE Trans. Multimedia 9 6 2007 1215 1226
15. D. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez Modeling dominance in group conversations using nonverbal activity cues IEEE Trans. Audio Speech Lang. Process. 17 3 2009 501 513
16. R. Duda, P. Hart, and D. Stork Pattern Classification 2001 John Wiley and Sons
17. D. McFarland Respiratory markers of conversational interaction J. Speech Lang. Hear. Res. 44 1 2001 128 143 (Pubitemid 33649980)
18. S. Hurley The shared circuits model (scm): how control, mirroring, and simulation can enable imitation, deliberation, and mindreading Behav. Brain Sci. 31 1 2008 1 22
19. J. Pineda Sensorimotor cortex as a critical component of an 'extended' mirror neuron system: does it solve the development, correspondence, and control problems in mirroring? Behav. Brain Functions 4 1 2008 47
20. E. Nilsen, and S. Graham The relations between children's communicative perspective-taking and executive functioning Cognitive Psychol. 58 2 2009 220 249
21. M. Cristani, A. Pesarin, C. Drioli, A. Perina, A. Tavano, V. Murino, Auditory dialog analysis and understanding by generative modelling of interactional dynamics, in: Second IEEE Workshop on CVPR for Human Communicative Behavior Analysis, Miami, Florida, 2009.
22. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van, and E.M. Meteer Dialogue act modeling for automatic tagging and recognition of conversational speech Comput. Linguist. 26 2000 339 373
23. D. Surendran, G. Levow, Dialog act tagging with support vector machines and hidden markov models, in: INTERSPEECH 2006 - ICSLP, 2006.
24. C. Cortes, P. Haffner, and M. Mohri A machine learning framework for spoken-dialog classification L. Rabiner, F. Juang, Handbook on Speech Processing and Speech Communication, Part E: Speech recognition 2008 Springer-Verlag Heidelberg, Germany 585 595
25. J. Jaffe, S. Feldstein, and L. Cassotta Markovian models of dialogic time patterns Nature 216 1967 93 94
26. J. Schwartz, and J. Jaffe Markovian prediction of sequential temporal patterns in spontaneous speech Lang. Speech 11 1 1968 27
27. P. Brady A model for generating on-off speech patterns in two-way conversation Bell Syst. Tech. J. 48 1969 2445 2472
28. A. Vinciarelli Capturing order in social interactions IEEE Signal Process. Mag. 26 5 2009 133 137
29. B. Schuller, G. Rigoll, M. Lang, Hidden Markov model-based speech emotion recognition, in: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing 2003 (ICASSP '03), vol. 2, April 2003, pp. 14.
30. D. Zhang, D. Gatica-Perez, S. Bengio, I. McCowan, G. Lathoud, Modeling individual and group actions in meetings with layered HMMs. IEEE Transactions on Multimedia 8(3) (2006) 509520
31. M. Zancanaro, B. Lepri, and F. Pianesi Automatic detection of group functional roles in face to face interactions ICMI '06: Proceedings of the Eighth International Conference on Multimodal Interfaces 2006 ACM New York, NY, USA 28 34 (Pubitemid 47113424)
32. K. Laskowski, M. Ostendorf, and T. Schultz Modeling vocal interaction for text-independent classification of conversation type Eighth ISCA/ACL SIGdial Workshop on Discourse and Dialogue 2007 194 201
33. K. Laskowski, M. Ostendorf, and T. Schultz Modeling vocal interaction for text-independent participant characterization in multi-party conversation SIGdial '08: Proceedings of the Ninth SIGdial Workshop on Discourse and Dialogue 2008 148 155
34. A. Vinciarelli, M. Pantic, H. Bourlard, and A. Pentland Social signals, their function, and automatic analysis: a survey IMCI '08: Proceedings of the 10th International Conference on Multimodal Interfaces 2008 ACM New York, NY, USA 61 68
35. John Grothendieck, Allen Gorin, Nash Borges, Social correlates of turn-taking behavior, in: IEEE International Conference on Acoustics, Speech and Signal Processing, 2009, pp. 47454748.
36. K. Scherer, T. Johnstone, T. Bänziger, Automatic verification of emotionally stressed speakers: the problem of individual differences, in: Proceedings of the International Workshop on Speech and Computer, 1998.
37. E. Shriberg Can prosody aid the automatic classification of dialog acts in conversational speech? Lang. Speech 41 4 1998 439 487
38. C. Gobl, and A. Chasaide The role of the voice quality in communicating emotions, mood and attitude Speech Commun. 40 2003 189 212
39. J. Liscombe, G. Riccardi, and D. Hakkani-Tur Using context to improve emotion detection in spoken dialog systems Proceedings of the Ninth European Conference on Speech Communication and Technology EUROSPEECH'05 vol. 1 2005 1845 1848 (Pubitemid 43908444)
40. R. Rienks, and D. Heylen Automatic dominance detection in meetings using easily obtainable features Revised Selected Papers of the Second Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms MLMI 2005, ser. Lecture Notes in Computer Science vol. 3869 2006 Springer-Verlag 76 86 (Pubitemid 43979727)
41. J. Dabbs, and R. Ruback Dimensions of group process: amount and structure of vocal interaction Adv. Exp. Psychol. 20 1987 123 169
42. H. Stern, S. Mahmoud, and W. Kin-Kwok A model for generating on-off patterns in conversational speech, including short silence gaps and the effects of interaction between parties IEEE Trans. Veh. Technol. 43 4 1994 1094 1100
43. A. Raux, and M. Eskenazi A finite-state turn-taking model for spoken dialog systems NAACL '09: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics 2009 Association for Computational Linguistics 629 637
44. D. Wyatt, T. Choudhury, J. Bilmes, and H. Kautz A privacy-sensitive approach to modeling multi-person conversations IJCAI'07: Proceedings of the 20th International Joint Conference on Artificial Intelligence 2007 1769 1775
45. R. Bogdan, D. Gatica-Perez, Inferring competitive role patterns in reality TV show through nonverbal analysis, Multimedia Tools and Applications, Special Issue on Social Media (2010) 120.
46. H. Hung, Y. Huang, G. Friedland, D. Gatica-Perez, Estimating dominance in multi-party meetings using speaker diarization, IEEE Transactions on Audio, Speech, and Language Processing (99) (2010) 11.
47. L. Rabiner A tutorial on hidden Markov models and selected applications in speech recognition Proc. IEEE 77 2 1989 257 286
48. M. Brand, N. Oliver, and S. Pentland Coupled hidden markov models for complex action recognition Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1997
49. C. Manning, and H. Schütze Foundations of Statistical Natural Language Processing 1999 The MIT Press Cambridge, MA
50. D. Zhang, D. Gatica-Perez, S. Bengio, and D. Roy Learning influence among interacting Markov chains NIPS 2005
51. L. Saul, and M. Jordan Mixed memory markov models: decomposing complex stochastic processes as mixtures of simpler ones Mach. Learn. 37 1 1999 75 87
52. C. Bishop, and M. Tipping A hierarchical latent variable model for data visualization IEEE Trans. Pattern Anal. Mach. Intell. 20 3 1998 281 293 (Pubitemid 128741316)
53. B.C.S. Basu, T. Choudhury, A. Pentland, Towards measuring human interactions in conversational settings, in: IEEE International Workshop on Cues in Communication (CUES 2001), Hawaii, CA, 2001.
54. D. McFarland Respiratory markers of conversational interaction J. Speech Lang. Hear. Res. 44 128 2001 43 48
55. K. Hird, and K. Kirsner The relationship between prosody and discourse in spontaneous discourse Brain Lang. 80 2002 536 555
56. D. Richardson, R. Dale, and N. Kirkham The art of conversation is coordination Psychol. Sci. 18 2007 407 413
57. A. Dempster, N. Laird, and andD. Rubin Maximum likelihood from incomplete data via the EM algorithm J. R. Stat. Soc. B 39 1977 1 38
58. A. Pesarin, M. Cristani, V. Murino, C. Drioli, A. Perina, and A. Tavano A statistical signature for automatic dialogue classification Proceedings of International Conference on Pattern Recognition (ICPR 2008) 2008
59. W. Pedrycz, and A. Gacek Temporal granulation and its application to signal analysis Inf. Sci. 143 14 2002 47 71 (Pubitemid 34506081)
60. R. Fernandez, and R. Picard Dialog act classification from prosodic features using support vector machines Proceedings of Speech Prosody 2002
61. J. Carletta Unleashing the killer corpus: experiences in creating the multi-everything ami meeting corpus Lang. Resour. Eval. 41 2 2007 181 190
62. T. Jaakkola, and D. Haussler Exploiting generative models in discriminative classifiers Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II 1999 MIT Press Cambridge, MA, USA 487 493