A Multimodal predictive model of successful debaters or how i learned to sway votes

MM 2015 - Proceedings of the 2015 ACM Multimedia Conference

Volumn , Issue , 2015, Pages 149-158

  Brilman, Maarten ^a   Scherer, Stefan ^b  

^a UNIVERSITY OF TWENTE   (Netherlands)

^b University of Southern California   (United States)

Author keywords

Information Fusion; Machine Learning; Multimodal; Nonverbal Behavior; Public Speaking

Indexed keywords

ARTIFICIAL INTELLIGENCE; EMPLOYMENT; INFORMATION FUSION; SOCIAL ASPECTS;

INTERPERSONAL SKILLS; MACHINE LEARNING APPROACHES; MULTI-MODAL; NONVERBAL BEHAVIOR; PREDICTIVE MODELING; PUBLIC SPEAKING; RESEARCH COMMUNITIES; SOCIAL ENVIRONMENT;

LEARNING SYSTEMS;

EID: 84962868678     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2733373.2806245     Document Type: Conference Paper

References (46)

1. P. Alku, T. Bäckström, and E. Vilkman. Normalized amplitude quotient for parametrization of the glottal flow. The Journal of the Acoustical Society of America, 112(2):701-710, 2002.
2. P. Alku, H. Strik, and E. Vilkman. Parabolic spectral parameter-A new method for quantification of the glottal ow. Speech Communication, 22(1):67-79, 1997.
3. J.-A. Bachorowski, M. J. Smoski, and M. J. Owren. The acoustic features of human laughter. Journal of the Acoustical Society of America, 110(3):1581-1597, 2001.
4. L. Batrinca, G. Stratou, A. Shapiro, L.-P. Morency, and S. Scherer. Cicero-Towards a multimodal virtual audience platform for public speaking training. In Proceedings of Intelligent Virtual Agents (IVA) 2013, pages 116-128. Springer, 2013.
5. B. Bozkurt, B. Doval, C. d'Alessandro, and T. Dutoit. Improved differential phase spectrum processing for formant tracking. pages 2421-2424, 2004.
6. C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.
7. L. Chen, G. Feng, J. Joe, C. W. Leong, C. Kitchen, and C. M. Lee. Towards automated assessment of public speaking skills using multimodal cues. In Proceedings of the 16th International Conference on Multimodal Interaction, pages 200-203. ACM, 2014.
8. G. Degottex, J. Kane, T. Drugman, T. Raitio, and S. Scherer. Covarep-A collaborative voice analysis repository for speech technologies. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014), pages 960-964, 2014.
9. M. R. DiMatteo, R. D. Hays, and L. M. Prince. Relationship of physicians' nonverbal communication skill to patient satisfaction, appointment noncompliance, and physician workload. Health Psychology, 5(6):581, 1986.
10. T. Drugman and A. Abeer. Joint robust voicing detection and pitch estimation based on residual harmonics. In Proceedings of Interspeech 2011, pages 1973-1976. ISCA, 2011.
11. J. A. Durlak. How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34(9):917-928, 2009.
12. G. Fant, J. Liljencrants, and Q. Lin. A four parameter model of glottal ow. KTH, Speech Transmission Laboratory, Quarterly Report, 4:1-13, 1985.
13. C. Gobl and A. N. Chasaide. Acoustic characteristics of voice quality. Speech Communication, 11(4):481-490, 1992.
14. A. Graves. Supervised sequence labelling with recurrent neural networks, volume 385. Springer, 2012.
15. M. Hanheide, M. Lohse, and A. Dierker. SALEM-statistical analysis of ELAN files in Matlab. Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality, pages 121-123, 2010.
16. H. M. Hanson and E. S. Chuang. Glottal characteristics of male speakers: Acoustic correlates and comparison with female data. The Journal of the Acoustical Society of America, 106(2):1064-1077, 1999.
17. L. V. Hedges. Distribution theory for glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2):107-128, 1981.
18. R. Hincks. Computer support for learners of spoken English. PhD thesis, Royal Institute of Technology, 2005.
19. J. B. Hirschberg and A. Rosenberg. Acoustic/prosodic and lexical correlates of charismatic speech. In Proceedings of Eurospeech 2005, pages 513-516, 2005.
20. M. E. Hoque, M. Courgeon, J.-C. Martin, B. Mutlu, and R. W. Picard. MACH: My automated conversation coach. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, pages 697-706. ACM, 2013.
21. J. Kane and C. Gobl. Identifying regions of non-modal phonation using features of the wavelet transform. In Proceedings of Interspeech 2011, pages 177-180. ISCA, 2011.
22. J. Kane and C. Gobl. Wavelet maxima dispersion for breathy to tense voice discrimination. IEEE Transactions on Audio, Speech, and Language Processing, 21(6):1170-1179, 2013.
23. M. Koppensteiner and K. Grammer. Motion patterns in political speech and their inuence on personality ratings. Journal of Research in Personality, 44(3):374-379, 2010.
24. K. Kurihara, M. Goto, J. Ogata, Y. Matsusaka, and T. Igarashi. Presentation sensei: A presentation training system using speech and image processing. In Proceedings of the 9th international conference on Multimodal interfaces, pages 358-365. ACM, 2007.
25. Q. Le and T. Mikolov. Distributed representations of sentences and documents. In Proceedings of The 31st International Conference on Machine Learning, pages 1188-1196, 2014.
26. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, 2013.
27. A.-T. Nguyen, W. Chen, and M. Rauterberg. Feedback system for presenters detects nonverbal expressions. SPIE Newsroom, 2012.
28. A.-T. Nguyen, W. Chen, and M. Rauterberg. Online feedback system for public speakers. In IEEE Symp. e-Learning, e-Management and e-Services, 2012.
29. S. Park, P. Shoemark, and L.-P. Morency. Toward crowdsourcing micro-level behavior annotations: The challenges of interface, training, and generalization. In Proceedings of the 18th International Conference on Intelligent User Interfaces (IUI '14), pages 37-46. ACM, 2014.
30. J. W. Pennebaker, C. K. Chung, M. Ireland, A. Gonzales, and R. J. Booth. The development and psychometric properties of LIWC2007, 2007.
31. T. Poster and P. Robinson. Real-Time recognition of affective states from nonverbal features of speech and its application for public speaking skill analysis. IEEE Transactions on Affective Computing, 2(2):66-78, 2011.
32. C. B. Pull. Current status of knowledge on public-speaking anxiety. Current opinion in psychiatry, 25(1):32-38, 2012.
33. S. Scherer, M. Glodek, F. Schwenker, N. Campbell, and G. Palm. Spotting laughter in natural multiparty conversations: A comparison of automatic online and offine approaches using audiovisual data. ACM Transactions on Interactive Intelligent Systems: Special Issue on Affective Interaction in Natural Environments, 2(1):4:1-4:31, 2012.
34. S. Scherer, Z. Hammal, Y. Yang, L.-P. Morency, and J. F. Cohn. Dyadic behavior analysis in depression severity assessment interviews. In Proceedings of the 16th International Conference on Multimodal Interaction, pages 112-119. ACM, 2014.
35. S. Scherer, G. Layher, J. Kane, H. Neumann, and N. Campbell. An audiovisual political speech analysis incorporating eye-Tracking and perception data. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), pages 1114-1120. ELRA, 2012.
36. L. M. Schreiber, D. P. Gregory, and L. R. Shibley. The development and test of the public speaking competence rubric. Communication Education, 61(3):205-233, 2012.
37. H. S. Shim, S. Park, M. Chatterjee, S. Scherer, K. Sagae, and L.-P. Morency. Acoustic and paraverbal indicators of persuasiveness in social multimedia. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2015.
38. E. Strangert and J. Gustafson. What makes a good speaker? Subject ratings, acoustic measurements and perceptual evaluations. In Proceedings of Interspeech 2008, pages 1688-1691. ISCA, 2008.
39. M. I. Tanveer, E. Lin, and M. E. Hoque. Rhema: A real-Time in-situ intelligent interface to help people with public speaking. In Proceedings of the 20th International Conference on Intelligent User Interfaces, pages 286-295. ACM, 2015.
40. Y. R. Tausczik and J. W. Pennebaker. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, 29(1):24-54, 2010.
41. Y.-l. Tian, T. Kanade, and J. F. Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97-115, 2001.
42. A. Tumasjan, T. O. Sprenger, P. G. Sandner, and I. M. Welpe. Predicting elections with twitter: What 140 characters reveal about political sentiment. In International AAAI Conference on Weblogs and Social Media, volume 10, pages 178-185, 2010.
43. F. Weninger, J. Bergmann, and B. Schuller. Introducing CURRENNT-The Munich open-source CUDA RecurREnt Neural Network Toolkit. Journal of Machine Learning Research, 16:547-551, 2015.
44. F. Weninger, J. Krajewski, A. Batliner, and B. Schuller. The voice of leadership: Models and performances of automatic analysis in online speeches. IEEE Transactions on Affective Computing, 3(4):496-508, 2012.
45. F. Weninger, P. Staudt, and B. Schuller. Words that fascinate the listener: Predicting affective ratings of on-line lectures. International Journal of Distance Education Technologies (IJDET), 11(2):110-123, 2013.
46. P. Wittenburg, H. Brugman, A. Russel, A. Klassmann, and H. Sloetjes. ELAN: A professional framework for multimodality research. In International Conference on Language Resources and Evaluation, pages 1556-1559, 2006.