Towards the automatic detection of spontaneous agreement and disagreement based on nonverbal behaviour: A survey of related cues, databases, and tools

Image and Vision Computing

Volumn 31, Issue 2, 2013, Pages 203-221

  Bousmalis, Konstantinos ^a   Mehu, Marc ^b   Pantic, Maja ^a,c  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF GENEVA   (Switzerland)

^c UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Agreement; Disagreement; Nonverbal behavior; Social signal processing

Indexed keywords

CONTRACTS; SIGNAL PROCESSING; TOOLS;

AUTOMATIC DETECTION; DISAGREEMENT; NON-VERBAL BEHAVIOURS; NONVERBAL BEHAVIOR; NUMBER OF METHODS; PAPER SURVEYS; SOCIAL SIGNAL PROCESSING; TEMPORAL PATTERN;

SURVEYS;

EID: 84886435226     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2012.07.003     Document Type: Article

References (223)

1. S. Cohen, A computerized scale for monitoring levels of agreement during a conversation, Univ. Pa. Working Pap. Linguist. 8 (1) (2003) 57-70.
2. A. Vinciarelli, M. Pantic, H. Bourlard, Social signal processing: survey of an emerging domain, Image Vision Comput. 27 (12) (2009) 1743-1759.
3. M. Pantic, R. Cowie, F. D'ericco, D. Heylen, M. Mehu, C. Pelachaud, I. Poggi, M. Schroder, A. Vinciarelli, Social Signal Processing: The Research Agenda, 2011, pp. 511-538.
4. M. Pantic, A. Nijholt, A. Pentland, T.S. Huang, Human-centred intelligent human- computer interaction (HCI2): how far are we from attaining it? J. Auton. Adapt. Commun. Syst. 1 (2) (2008) 168-187.
5. M. Pantic, A. Pentland, A. Nijholt, T.S.Huang, Human computing andmachine understanding of human behavior: A survey, Lect. Notes Comput. Sci. 4451 (2007) 47-71.
6. Z. Zeng, M. Pantic, G.I. Roisman, T.S. Huang, A survey of affect recognitionmethods: Audio, visual, and spontaneous expressions, IEEE Trans. Pattern Anal. Mach. Intell. 31 (1) (2009) 39-58 http://doi.ieeecomputersociety. org/10.1109/TPAMI.2008.52.
7. H. Gunes, M. Pantic, Automatic, dimensional and continuous emotion recognition, Int. J. Synth. Emotion 1 (1) (2010) 68-99.
8. D. Gatica-Perez, Automatic nonverbal analysis of social interaction in small groups: A review, Image Vision Comput. 27 (12) (2009) 1775-1787.
9. A. Vinciarelli, Capturing order in social interactions, IEEE Signal Process. Mag. 26 (2009) 133-137.
10. A. Kapoor, R.W. Picard, Multimodal affect recognition in learning environments, Proc. ACM Int'l Conf. on Multimedia, 2005, pp. 677-682.
11. D. Gatica-Perez, I. McCowan, D. Zhang, S. Bengio, Detecting group interest-level in meetings, Proc. IEEE Int'l Conf. on Acoustics, Speech and Signal Processing, 2005, pp. 489-492.
12. D. Gatica-Perez, Modeling Interest in Face-to-face Conversation from Multimodal Nonverbal Behavior, Academic Press, 2009. Ch. 15.
13. S. Asteriadis, P. Tzouveli, K. Karpouzis, S. Kollias, Estimation of behavioral user state based on eye gaze and head pose-application in an e-learning environment, Multimed. Tools Appl. 41 (3) (2009) 469-493.
14. K. Bousmalis, M. Mehu, M. Pantic, Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools, Proc. IEEE Int'l Conf. Affective Computing and Intelligent Interfaces, 2009, pp. 1-9.
15. I. Poggi, F. D'Errico, L. Vincze, Agreement and its multimodal communication in debates: A qualitative analysis, Cogn. Comput. (2010) 1-14.
16. H.M. Rosenfeld, M. Hancks, The relationship of verbal and nonverbal communication, The Nonverbal Context of Verbal Listener Responses, Walter de Gruyter, 1980, pp. 193-206.
17. P. Ekman, Human ethology, Emotional and Conversational Signals, Cambridge Univ. Press, 1979.
18. M. Argyle, Bodily Communication, 2nd edition Methuen & Co, 1988. Ch. 7.
19. J. Seiter, Does communicating nonverbal disagreement during an opponent's speech affect the credibility of the debater in the background? Psychol. Rep. 84 (1999) 855-861.
20. J.S. Seiter, H.Weger, Audience perceptions of candidates' appropriateness as a function of nonverbal behaviors displayed during televised political debates, J. Soc. Psychol. 145 (2) (2005) 225-236.
21. J.S. Seiter, H.J. Kinzer, H. Weger, Background behavior in live debates: The effects of the implicit ad hominem fallacy, Commun. Rep. 19 (1) (2006) 57-69. (Pubitemid 43663684)
22. A.M. Pomerantz, Second Assessments: A Study of Some Features of Agreements/disagreements, General sociology, University of California, Irvine, 1975.
23. A.M. Pomerantz, Structures of social action: studies in conversation analysis, studies in emotion and social interaction, Agreeing and Disagreeing with Assessments: Some Features of Preferred/dispreferred Turn Shapes, Cambridge University Press, 1984.
24. D.W. Cunningham, M. Kleiner, H.H. Bülthoff, C. Wallraven, The components of conversational facial expressions, Proc. Symp. on Applied Perception in Graphics and Visualization, 2004, pp. 143-150.
25. M. Nicolaou, H. Gunes, M. Pantic, Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space, IEEE Transactions on Affective Computing, 2011.
26. C. Darwin, The Expression of Emotions in Man and Animals, Oxford University Press, USA, 2002.
27. U. Hadar, T. Steiner, F.C. Rose, Head movement during listening turns in conversation, J. Nonverbal Behav. 9 (4) (1985) 214-228.
28. D. Morris, Bodytalk: A World Guide to Gestures, Jonathan Cape, 1994.
29. V. Manusov, A.R. Trees, "Are you kidding me?": The role of nonverbal cues in the verbal accounting process, J. Commun. 52 (3) (2002) 640-656. (Pubitemid 37160177)
30. D.B. Givens, The Nonverbal Dictionary of Gestures, Signs and Body Language Cue, Center for Nonverbal Studies Press, Sokane, WA, 2002.
31. I. Poggi, F. D'Errico, L. Vincze, Types of nods. the polysemy of a social signal, Proc. Int'l Conf. Language Resources and Evaluation, 2010, pp. 17-23.
32. L.J. Brunner, Smiles can be back channels, J. Pers. Soc. Psychol. 37 (5) (1979) 728-734.
33. A. Dittmann, Developmental factors in conversational behavior, J. Commun. 22 (4) (1972) 404-423.
34. P. Bull, Posture and gesture, The Encoding of Disagreement and Agreement, Pergamon Press, 1987, pp. 62-69, Ch. 5.
35. E. Keller, W. Tschacher, Prosodic and gestural expression of interactional agreement, Lect. Notes Comput. Sci. 4775 (2007) 85-98.
36. P. Ekman, W.V. Friesen, J.C. Hager, Facial Action Coding System Research Nexus, Salt Lake City, 2002.
37. W. Rinn, The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions, Psychol. Bull. 95 (1) (1984) 52-77.
38. N. Chovil, Discource-oriented facial displays in conversation, Res. Lang. Soc. Interact. 25 (1-4) (1991) 163-194.
39. R. Jakobson, Motor signs for yes and no, Lang. Soc. (1972) 91-96.
40. D. Hillard, M. Ostendorf, E. Shriberg, Detection of agreement vs. disagreement in meetings: Training with unlabeled data, Proc. Conf. North American Chapter of the Association for Computational Linguistics on Human Language Technology, 2003, pp. 34-36, http://dx.doi.org/10.3115/1073483.1073495.
41. M. Galley, K. McKeown, J. Hirschberg, E. Shriberg, Identifying agreement and disagreement in conversational speech: use of Bayesian networks to model pragmatic dependencies, Proc. Meeting Association for Computational Linguistics, 2004, pp. 669-676.
42. D. Greatbatch, Talk at work, On the Management of Disagreement Between News Interviewees, Cambridge University Press, 1992. Ch. 9.
43. P. Ekman, Telling Lies, Berkley Books, 1985.
44. J. Gottman, H. Markman, C. Notarius, The topography of marital conflict: A sequential analysis of verbal and nonverbal behavior, J. Marriage Fam. 39 (3) (1977) 461-477. (Pubitemid 8166174)
45. L. Cerrato, Linguistic functions of head nods, Proc. Conf. Multi-modal Communication, 2005, pp. 137-152.
46. S.J. Duncan, Some signals and rules for speaking turns in conversation, J. Pers. Soc. Psychol. 23 (1972) 283-292.
47. A. Vinciarelli, A. Dielmann, S. Favre, H. Salamin, Canal9: A database of political debates for analysis of social interactions, Proc. IEEE Int'l Conf. Affective Computing and Intelligent Interfaces, 2, 2009, pp. 96-99.
48. H. Rosenfeld, Conversational control functions of non-verbal behaviour, Non-verbal Behav. Commun. (1978) 291-338.
49. A. Kendon, Some uses of the head shake, Gesture 2 (2) (2002) 147-182.
50. P. Bull, Posture and gesture, The Decoding of Interest/Boredom and Disagreement/Agreement, Pergamon Press, 1987, pp. 70-84.
51. T. Chartrand, J. Bargh, The chameleon effect: The perception-behavior link and social interaction, J. Pers. Soc. Psychol. 76 (6) (1999) 893-910.
52. D. Tannen, Interpreting Interruption Conversation (1993) 53-83 (Ch. 2).
53. R. Ogden, Phonetics and social action in agreements and disagreements, J. Pragmat. 38 (10) (2006) 1752-1775. (Pubitemid 44227831)
54. F. Haumer, W. Donsbach, The rivalry of nonverbal cues on the perception of politicians by television viewers, J. Broadcast. Electron. Media 53 (2) (2009) 262-279.
55. K. Bousmalis, L.-P. Morency, M. Pantic, Modeling hidden dynamics of multimodal cues for spontaneous agreement and disagreement recognition, Proc. IEEE Conf. on Automatic Face and Gesture Recognition, 2011, pp. 746-752.
56. J. Carletta, Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus, Lang. Resour. Eval. J. 41 (2) (2007) 181-190.
57. S. Germesin, T. Wilson, Agreement detection in multiparty conversation, Proc. ACM Int'l Conf. Multimodal Interfaces, 2009, pp. 7-14.
58. D. Jayagopi, H. Hung, C. Yeo, D. Gatica-Perez, Modeling dominance in group conversations from nonverbal activity cues, IEEE Trans. Audio Speech Lang. Process. 17 (3) (2009) 501-513.
59. D. Jayagopi, D. Gatica-Perez, Discovering group nonverbal conversational patterns with topics, Proc. ACM Int'l Conf. Multimodal Interfaces, 2009.
60. H. Hung, D. Gatica-Perez, Estimating cohesion in small groups using audio-visual nonverbal behavior, IEEE Trans. Multimed. 12 (6) (2010) 563-575.
61. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, ICSI meeting corpus, Proc. IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing, 1, 2003, pp. 364-367.
62. H. Hung, G. Chittaranjan, The idiap wolf corpus: exploring group behaviour in a competitive role-playing game, Proc. ACM Int'l Conf. on Multimedia, 2010, pp. 879-882.
63. F. Pianesi, M. Zancanaro, B. Lepri, A. Cappelletti, A multimodal annotated corpus of consensus decision making meetings, Lang. Resour. Eval. 41 (2007) 409-429.
64. S. Hahn, R. Ladner, M. Ostendorf, Agreement/disagreement classification: exploiting unlabeled data using contrast classifiers, Proc. Human Language Technology Conf. of the NAACL, 2006, pp. 53-56.
65. E. Douglas-Cowie, R. Cowie, I. Sneddon, C. Cox, O. Lowry, M. McRorie, J. Martin, L. Devillers, S. Abrilian, A. Batliner, et al., The HUMAINE database: Addressing the collection and annotation of naturalistic and induced emotional data, Lect. Notes Comput. Sci. 4738 (2007) 483-500.
66. G. McKeown, M.F. Valstar, R. Cowie, M. Pantic, The SEMAINE corpus of emotionally coloured character interactions, Proc. Int'l Conf. Multimedia & Expo, 2010, pp. 1079-1084.
67. T. Kanade, J. Cohn, Y. Tian, Comprehensive database for facial expression analysis, Int'l Conf. Automatic Face and Gesture Recognition, 2000, pp. 46-53.
68. M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, J. Movellan, Fully automatic facial action recognition in spontaneous behavior, Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2006, pp. 223-230. (Pubitemid 44711354)
69. M. Pantic, M. Valstar, R. Rademaker, L. Maat, Web-based database for facial expression analysis, IEEE Conf. Multimedia and Expo, 2005, pp. 317-321. (Pubitemid 44668868)
70. Y. Tong, J. Chen, Q. Ji, A unified probabilistic framework for spontaneous facial action modeling and understanding, IEEE Trans. Pattern Anal. Mach. Intell. 32 (2) (2010) 258-273.
71. E. Douglas-Cowie, R. Cowie, M. Schroeder, The description of naturally occurring emotional speech, Proc. IEEE Int'l Congress of Phonetic Sciences, 2003, pp. 2877-2880.
72. Multiple aspects of discourse research lab, http://madresearchlab.org/ 2009.
73. J. Whitehill, C. Omlin, Haar features for facs au recognition, Proc. IEEE Int'l Conf. Face and Gesture Recognition, 2006, pp. 97-101. (Pubitemid 44711334)
74. S. Lucey, A. Ashraf, J. Cohn, Investigating spontaneous facial action recognition through an representations of the face, Face Recognition, 2007.
75. M. Pantic, I. Patras, Detecting facial actions and their temporal segments in nearly frontal-view face image sequences, IEEE Trans. Syst. Man Cybern. Part B 4 (2005) 3358-3363. (Pubitemid 41667331)
76. M.F. Valstar, M. Pantic, Fully automatic facial action unit detection and temporal analysis, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006, pp. 149-156.
77. M.F. Valstar, M. Pantic, Combined support vector machines and hidden markov models for modeling facial action temporal dynamics, Lect. Notes Comput. Sci. 4796 (2007) 118-127.
78. G. Littlewort, M.S. Bartlett, I. Fasel, J. Susskind, J. Movellan, Dynamics of facial expression extracted automatically from video, Image Vision Comput. 24 (6) (2006) 615-625. (Pubitemid 43867835)
79. E. Vural, M. Cetin, A. Ercil, G. Littlewort, M. Bartlett, J. Movellan, Automated drowsiness detection for improved driving safety, Proc. Int'l Conf. Automotive Technologies, 2008.
80. P. Yang, Q. Liu, D. Metaxas, Boosting coded dynamic features for facial action units and facial expression recognition, Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-6, http://dx.doi.org/10.1109/CVPR. 2007.383059.
81. P. Yang, Q. Liu, D.N. Metaxas, Boosting encoded dynamic features for facial expression recognition, Pattern Recognit. Lett. 30 (2) (2009) 132-139.
82. S. Koelstra, M. Pantic, Non-rigid registration using free-form deformations for recognition of facial actions and their temporal dynamics, Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2008.
83. S. Koelstra, M. Pantic, I. Patras, Dynamic texture based approach to recognition of facial actions and their temporal models, IEEE Trans. Pattern Anal. Mach. Intell. 99 (2010) 1940-1954.
84. Y. Tong, W. Liao, Q. Ji, Facial action unit recognition by exploiting their dynamic and semantic relationships, Pattern Analysis and Machine Intelligence, IEEE Trans. 29 (10) (2007) 1683-1699, http://dx.doi.org/10.1109/ TPAMI.2007.1094.
85. Y. Tong, W. Liao, Q. Ji, Affective information processing, Automatic Facial Action Unit Recognition by Modeling Their Semantic and Dynamic Relationships, Springer, London, 2009, pp. 159-180.
86. M. Pantic, I. Patras, Dynamics of facial expressions - recognition of facial actions and their temporal segments from face profile image sequences, IEEE Trans. Syst. Man Cybern. Part B 36 (2) (2006) 433-449.
87. M. Pantic, M. Bartlett, Machine analysis of facial expressions, K. Delac, M. Grgic (Eds.), Face Recognition, I-Tech Education and Publishing, 2007, pp. 377-416.
88. P. Ekman, Darwin, deception, and facial expression, Ann. N. Y. Acad. Sci. 1000 (2003) 205-221. (Pubitemid 38264836)
89. Y. Chang, M. Vieira, M. Turk, L. Velho, Automatic 3D facial expression analysis in videos, Proc. Int'l Workshop Analysis and Modelling of Faces and Gestures, 2005, pp. 293-307.
90. Y. Sun, L. Yin, Facial expression recognition based on 3D dynamic range model sequences, Proc. European Conf. on Computer Vision, 2008, pp. 58-71.
91. J. Sung, D. Kim, Real-time facial expression recognition using STAAM and layered GDA classifier, Image Vision Comput. 27 (2009) 1313-1325.
92. Y. Cheon, D. Kim, Natural facial expression recognition using differential-AAM and manifold learning, Pattern Recognit. 42 (2009) 1340-1350.
93. T. Wang, J. Lien, Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation, Pattern Recognit. 42 (2009) 962-977.
94. Y. Hu, Z. Zeng, L. Yin, X. Wei, J. Tu, T. Huang, A study of non-frontal-view facial expressions recognition, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1-4.
95. O. Rudovic, I. Patras, M. Pantic, Coupled gaussian process regression for pose-invariant facial expression recognition, Proc. European Conf. on Computer Vision, 2, 2010, pp. 350-363.
96. M. Pantic, Machine analysis of facial behaviour: naturalistic and dynamic behaviour, Philos. Trans. Royal Soc. B 364 (1535) (2009) 3505-3513.
97. S. Kawato, J. Ohya, Real-time detection of nodding and head-shaking by directly detecting and tracking the between-eyes, Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2000, pp. 40-45.
98. A. Kapoor, R.W. Picard, A real-time head nod and shake detector, Proc. ACM Int'l Conf. Perceptive User Interfaces, 15, 2001, pp. 1-5.
99. W. Tan, G. Rong, A real-time head nod and shake detector using HMMs, Expert Syst. Appl. 25 (3) (2003) 461-466.
100. S. Fujie, Y. Ejiri, K. Nakajima, Y. Matsusaka, T. Kobayashi, A conversation robot using head gesture recognition as para-linguistic information, Proc. IEEE Int'l Workshop Robot and Human Interactive Communication, 2004, pp. 159-164. (Pubitemid 40822661)
101. L.-P. Morency, C. Sidner, C. Lee, T. Darrell, Contextual recognition of head gestures, Proc. ACM Int'l Conf. Multimodal Interfaces, 2005, pp. 18-24. (Pubitemid 43218858)
102. L.-P. Morency, J. Whitehill, J. Movellan, Generalized adaptive view-based appearance model: integrated framework for monocular head pose estimation, Proc. Int'l Conf. Automatic Face and Gesture Recognition, 2008.
103. S. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, T. Darrell, Hidden conditional random fields for gesture recognition, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2, 2006, pp. 1521-1527. (Pubitemid 44931500)
104. L.-P. Morency, A. Quattoni, T. Darrell, Latent-dynamic discriminative models for continuous gesture recognition, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-8.
105. E. Murphy-Chutorian, M. Trivedi, Head pose estimation in computer vision: A survey, IEEE Trans. Pattern Anal. Mach. Intell. 31 (4) (2009) 607-626.
106. A. Erol, G. Bebis, M. Nicolescu, R.D. Boyle, X. Twombly, Vision-based hand pose estimation: A review, Comput. Vis. Image Underst. 108 (1-2) (2007) 52-73. (Pubitemid 47333390)
107. M.-H. Yang, N. Ahuja, M. Tabb, Extraction of 2d motion trajectories and its application to hand gesture recognition, IEEE Trans. Pattern Anal. Mach. Intell. 24 (8) (2002) 1061-1074.
108. L. Ding, A. Martinez, Modelling and recognition of the linguistic components in american sign language, Image and Vision Computing 27 (12) (2009).
109. M. Mahmoud, R. El-Kaliouby, A. Goneid, Towards communicative face occlusions: machine detection of hand-over-face gestures, Lect. Notes Comput. Sci. 5627 (2009) 481-490.
110. P. Scovanner, S. Ali, M. Shah, A 3-dimensional sift descriptor and its application to action recognition, Proc. Int'l Conf. Multimedia, 2007, pp. 357-360.
111. H. Wang, M.M. Ullah, A. Kläser, I. Laptev, C. Schmid, Evaluation of local spatio-temporal features for action recognition, Proc. British Machine Vision Conf, 2009.
112. I. Laptev, M. Marszalek, C. Schmid, B. Rozenfeld, Learning realistic human actions from movies, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1-8.
113. I. Laptev, P. Perez, Retrieving actions in movies, Proc. Int'l Conf. Computer Vision, 2007, pp. 1-8.
114. A. Gilbert, J. Illingworth, R. Bowden, Fast realistic multi-action recognition using mined dense spatio-temporal features, Proc. Int'l Conf. on Computer Vision, 2009, pp. 925-931.
115. D. Han, L. Bo, C. Sminchisescu, Selection and context for action recognition, Proc. Int'l Conf. on Computer Vision, 2009, pp. 1933-1940.
116. K. Mikolajczyk, H. Uemura, Action recognition with motion-appearance vocabulary forest, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1-8, http://dx.doi.org/10.1109/CVPR.2008.4587628.
117. M. Bregonzio, S. Gong, T. Xiang, Recognising action as clouds of space-time interest points, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 1-8.
118. H. Jhuang, T. Serre, L. Wolf, T. Poggio, A biologically inspired system for action recognition, Proc. Int'l Conf. on Computer Vision, 2007, pp. 1-8.
119. K. Reddy, J. Liu, M. Shah, Incremental action recognition using feature-tree, Proc. Int'l Conf. on Computer Vision, 2009.
120. K. Rapantzikos, Y. Avrithis, S. Kollias, Dense saliency-based spatiotemporal feature points for action recognition, Proc. Conf. Computer Vision and Pattern Recognition, 2009, pp. 1454-1461.
121. K. Schindler, L.V. Gool, Action snippets: how many frames does human action require? Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1-8.
122. S. Nowozin, G. Bakir, K. Tsuda, Discriminative sub-sequence mining for action classification, Proc. Int'l Conf. on Computer Vision, 1, 2007, pp. 1-8.
123. J. Niebles, L. Fei-Fei, A hierarchical model of shape and appearance for human action classification, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-8.
124. P. Dollár, V. Rabaud, G. Cottrell, S. Belongie, Behavior recognition via sparse spatio-temporal features, Proc. Int'l Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65-72. (Pubitemid 46178025)
125. C. Schüldt, I. Laptev, B. Caputo, Recognizing human actions: A local svm approach, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 3, 2004, pp. 32-36.
126. M. Marszałek, I. Laptev, C. Schmid, Actions in context, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 2929-2936.
127. L. Zelnik-Manor, M. Irani, Statistical analysis of dynamic actions, IEEE Trans. Pattern Anal. Mach. Intell. 28 (9) (2006) 1530-1535. (Pubitemid 44132659)
128. H. Ning, Y. Hu, T.S. Huang, Searching human behaviors using spatial-temporal words, Proc. Int'l Conf. Image Processing, 6, 2007, pp. 337-340.
129. E. Shechtman, M. Irani, Space-time behavior-based correlation-or-how to tell if two underlying motion fields are similar without computing them? IEEE Trans. Pattern Anal. Mach. Intell. 29 (11) (2007) 2045-2056. (Pubitemid 47423795)
130. P. Matikainen, M. Hebert, R. Sukthankar, Y. Ke, Fast motion consistency through matrix quantization, Proc. British Machine Vision Conf, 2008, pp. 1055-1064.
131. E. Shechtman, M. Irani, Matching local self-similarities across images and videos, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-8.
132. H.J. Seo, P. Milanfar, Detection of human actions from a single example, Proc. Int'l Conf. Computer Vision, 2009, pp. 1-8.
133. Y. Hu, L. Cao, F. Lv, S. Yan, Y. Gong, T.S. Huang, Action detection in complex scenes with spatial and temporal ambiguities, Proc. Int'l Conf. on Computer Vision, 2009, pp. 1-8.
134. J. Yuan, Z. Liu, Y. Wu, Discriminative subvolume search for efficient action detection, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 1-8.
135. I. Junejo, E. Dexter, I. Laptev, P. Pérez, Cross-view action recognition from temporal self-similarities, Proc. European Conf. on Computer Vision, 2, 2008, pp. 293-306.
136. O. Boiman, M. Irani, Detecting irregularities in images and in video, Int. J. Comput. Vis. 74 (1) (2007) 17-31. (Pubitemid 46673101)
137. M. Rodriguez, J. Ahmed, M. Shah, Action mach: A spatio-temporal maximum average correlation height filter for action recognition, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 8, 2006, pp. 1-8.
138. A. Oikonomopoulos, Spatiotemporal visual analysis of human actions, Ph.D. thesis, Imperial College London (2010).
139. R. Poppe, A survey on vision-based human action recognition, Image Vision Comput. 28 (2010) 976-990.
140. A. Oikonomopoulos, M. Pantic, I. Patras, Sparse B-spline polynomial descriptors for human activity recognition, Image Vision Comput. 27 (12) (2009) 1814-1825.
141. G. Willems, T. Tuytelaars, L.V. Gool, An efficient dense and scale-invariant spatio-temporal interest point detector, Lect. Notes Comput. Sci. 5303 (2008) 650-663.
142. A. Oikonomopoulos, I. Patras, M. Pantic, An implicit spatiotemporal shape model for human activity localisation and recognition, Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 3, 2009, pp. 27-33.
143. K. Rapantzikos, Y. Avrithis, S. Kollias, Spatiotemporal saliency for event detection and representation in the 3D wavelet doma potential in human action recognition, Proc. Int'l Conf. Image and Video Retrieval, 2007, pp. 294-301. (Pubitemid 350229639)
144. S. Ando, X. Wu, A. Suzuki, K. Wakabayahsi, H. Koike, Human pose estimation for image monitoring, NTT Tech. Rev. 5 (11) (2007) 1-8.
145. X. Zhao, H. Ning, Y. Liu, T. Huang, Discriminative estimation of 3D human pose using gaussian processes, Proc. Int'l Conf. on Pattern Recognition, 2008, pp. 1-4.
146. M.V. den Bergh, E. Koller-Meier, L.V. Gool, Real-time body pose recognition using 2D and 3D haarlets, Image Vision Comput. 83 (2009) 72-84.
147. M. Enzweiler, D. Gavrila, Integrated pedestrian classification and orientation estimation, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010, pp. 982-989.
148. Y. Huang, T. Huang, Model-based human body tracking, Proc. Int'l Conf Pattern Recognition, 1, 2002, pp. 552-555. (Pubitemid 44845058)
149. I. Mikić, M. Trivedi, E. Hunter, P. Cosman, Human body model acquisition and tracking using voxel data, Int. J. Comput. Vis. 53 (3) (2003) 199-223.
150. G. Mori, J. Malik, Recovering 3D human body configurations using shape contexts, IEEE Trans. Pattern Anal. Mach. Intell. 28 (7) (2006) 1052-1062.
151. R. Navaratnam, A. Thayananthan, P. Torr, R. Cipolla, Hierarchical part-based human body pose estimation, Proc. British Machine Vision Conf, 2005.
152. L. Taycher, G. Shakhanrovich, D. Demirdjian, T. Darrell, Conditional random people: Tracking humans with CRFs and grid filters, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1, 2006, pp. 222-229. (Pubitemid 44931334)
153. G. Shakhnarovich, P. Viola, T. Darrell, Fast pose estimation with parameter sensitive hashing, Proc. Int'l Conf. Computer Vision, 2, 2003, pp. 750-759.
154. E.-J. Ong, A. Micilotta, R. Bowden, A. Hilton, Viewpoint invariant exemplar-based 3D human tracking, Comput. Vis. Image Underst. 104 (2-3) (2006) 178-189. (Pubitemid 44573875)
155. A. Elgammal, C.-S. Lee, Inferring 3D body pose from silhouettes using activity manifold learning, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2, 2004, pp. 681-688.
156. A. Agarwal, B. Triggs, A local basis representation for estimating human pose from cluttered images, Lect. Notes Comput. Sci. 3851 (2006) 50-59.
157. L. Wang, W. Hu, T. Tan, Recent developments in human motion analysis, Pattern Recognit. 36 (3) (2003) 585-601.
158. J. Wang, S. Singh, Video analysis of human dynamics: A survey, Real-Time Imaging 9 (5) (2003) 321-346.
159. D. Gavrila, The visual analysis of human movement: A survey, Comput. Vis. Image Underst. 73 (1) (1999) 82-92.
160. J. Aggarwal, Q. Cai, Human motion analysis: A review, Comput. Vis. Image Underst. 73 (3) (1999) 428-440.
161. D.W. Hansen, Q. Ji, In the eye of the beholder: A survey of models for eyes and gaze, IEEE Trans. Pattern Anal. Mach. Intell. 32 (3) (2010) 478-500.
162. D.W. Hansen, J. Hansen, M. Nielsen, A. Johansen, M. Stegmann, Eye typing using markov and active appearance models, Proc. IEEE Workshop Applications on Computer Vision, 2003, pp. 132-136.
163. T. Ishikawa, S. Baker, I. Matthews, T. Kanade, Passive driver gaze tracking with active appearance models, Proc. World Congress Intelligent Transportation Systems, 2004, pp. 1-12.
164. D. Hansen, A. Pece, Eye tracking in the wild, Comput. Vis. Image Underst. 98 (1) (2005) 182-210.
165. J. Wang, E. Sung, R. Venkateswarlu, Estimating the eye gaze from one eye, Comput. Vis. Image Underst. 98 (1) (2005) 83-103.
166. O.Williams, A. Blake, R. Cipolla, Sparse and semi-supervised visual mapping with the s3p, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006, pp. 230-237. (Pubitemid 44931335)
167. H. Yamazoe, A. Utsumi, T. Yonezawa, S. Abe, Remote gaze estimation with a single camera based on facial-feature tracking without special calibration actions, Proc. Symp. Eye Tracking Research and Applications, 2008, pp. 140-145.
168. L. Morency, C. Christoudias, T. Darrell, Recognizing gaze aversion gestures in embodied conversational discourse, Proc. of the Int'l Conf. on Multi-modal Interfaces, 2006, pp. 287-294. (Pubitemid 47113460)
169. M. Voit, R. Stiefelhagen, Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios, Proc. ACM Int'l Conf. Multimodal Interfaces, 2008, pp. 173-180.
170. M. Voit, R. Stiefelhagen, Visual focus of attention in dynamic meeting scenarios, Machine Learning for Multimodal Interaction, Vol. 5237 of Lecture Notes in Computer Science, 2008, pp. 1-13.
171. S. Ba, J. Odobez, Recognizing visual focus of attention from head pose in natural meetings, IEEE Trans. Syst. Man and Cybern. Part B 39 (1) (2009) 16-33.
172. L.S. Kennedy, D.P.W. Ellis, Laughter detection in meetings, Proc. NIST Meeting Recognition Workshop, 2004.
173. N. Campbell, H. Kashioka, R. Ohara, No laughing matter, Proc. Europ. Conf. on Speech Commmunication and Technology, 2005, pp. 465-468. (Pubitemid 43908100)
174. K.P. Truong, D.A. van Leeuwen, Automatic discrimination between laughter and speech, Speech Commun. 49 (2) (2007) 144-158. (Pubitemid 46241514)
175. B. Schuller, F. Eyben, G. Rigoll, Static and dynamic modelling for the recognition of non-verbal vocalisations in conversational speech, Lect. Notes Comput. Sci. 5078 (2008) 99-110.
176. K. Laskowski, T. Schultz, Detection of laughter-in-interaction in multichannel close-talk microphone recordings of meetings, Lect. Notes Comput. Sci. 5237 (2008) 149-160.
177. M. Knox, N. Morgan, N. Mirghafori, Getting the last laugh: Automatic laughter segmentation in meetings, Proc. Interspeech, 2008, pp. 797-800.
178. R. Cai, L. Lu, H.-J. Zhang, L.-H. Cai, Highlight sound effects detection in audio stream, Proc. Int'l Conf. on Multimedia and Expo, 3, 2003, pp. 37-40.
179. A. Ito, W. Xinyue, M. Suzuki, S. Makino, Smile and laughter recognition using speech processing and face recognition from conversation video, Proc. Int'l Conf. on Cyberworlds, 2005, pp. 8-15.
180. B. Reuderink, M. Poel, K. Truong, R. Poppe, M. Pantic, Decision-level fusion for audio-visual laughter detection, Lect. Notes Comput. Sci. 5237 (2008) 137-148.
181. S. Petridis, M. Pantic, Audiovisual discrimination between speech and laughter: why and when visual information might help, IEEE Trans. Multimed. 13 (2) (2011) 216-234.
182. S. Petridis, M. Pantic, J. Cohn, Prediction-based classification for audiovisual discrimination between laughter and speech, Proc. IEEE Conf. on Automatic Face and Gesture Recognition, Santa Barbara, CA, USA, 2011, pp. 619-626.
183. S. Kumano, K. Otsuka, D. Mikami, J. Yamato, Recognizing communicative facial expressions for discovering interpersonal emotions in group meetings, Proc. ACM Int'l Conf. Multimodal Interfaces, 2009, pp. 99-106.
184. P. Boersma, Praat, a system for doing phonetics by computer, Glot Int. 5 (9/10) (2002) 341-345.
185. B. Schuller, R.Müller, F. Eyben, J. Gast, B. Hörnler, R.Wöllmer, G. Rigoll, A. Höthker, H. Konosu, Being bored? recognising natural interest by extensive audiovisual integration for real-life application, Image and Vision Computing 27 (12) (2009) 1760-1774.
186. S. Matos, S. Birring, I. Pavord, H. Evans, Detection of cough signals in continuous audio recordings using hidden markov models, IEEE Trans. Biomed. Eng. 53 (6) (2006) 1078-1083. (Pubitemid 43860731)
187. I. Gonzalez, I. Ravyse, H. Brouckxon, W. Verhelst, D. Jiang, H. Sahli, A visual silence detector constraining speech source separation, Proc. Int'l Conf. on Image and Graphics, 2009, pp. 463-470.
188. Y. Liu, E. Shriberg, A. Stolcke, D. Hillard, M. Ostendorf, M. Harper, Enriching speech recognition with automatic detection of sentence boundaries and disfluencies, IEEE Trans, Audio Speech Lang. Process. 14 (5) (2006) 1526-1540.
189. C.-C. Lee, S. Lee, S. Narayanan, An analysis of multimodal cues of interruption in dyadic spoken interactions, Proc. European Conf. Speech Communication and Technology, 2008, pp. 1678-1681.
190. M. Goto, K. Itou, S. Hayamizu, A real-time filled pause detection system for spontaneous speech recognition, Proc. European Conf. Speech Communication and Technology, 1999, pp. 227-230.
191. M. Gabrea, D. O'Shaughnessy, Detection of filled pauses in spontaneous conversational speech, Proc. Int'l Conf. Spoken Language Processing, 3, 2000, pp. 678-681.
192. C.-H. Wu, G.-L. Yan, Acoustic feature analysis and discriminative modeling of filled pauses for spontaneous speech recognition, J. VLSI Signal Process. 36 (2-3) (2004) 91-104.
193. F. Stouten, J. Duchateau, J.-P. Martens, P. Wambacq, Coping with disfluencies in spontaneous speech recognition: Acoustic detection and linguistic context manipulation, Speech Commun. 48 (11) (2006) 1590-1606. (Pubitemid 44740324)
194. K. Audhkhasi, K. Kandhway, O. Deshmukh, A. Verma, Formant-based technique for automatic filled-pause detection in spoken english, Proc. IEEE Int'l Conf. Acoustics, Speech and Signal Processing, 2009, pp. 4857-4860.
195. A. Bobick, J. Davis, The recognition of human movement using temporal templates, IEEE Trans. Pattern Anal. Mach. Intell. 23 (3) (2001) 257-267. (Pubitemid 32286632)
196. A. Pentland, Socially aware, computation and communication, Computer 38 (3) (2005) 33-40, http://dx.doi.org/10.1109/MC.2005.104.
197. T. Kim, A. Chang, A. Pentland, Meeting mediator: enhancing group collaboration with sociometric feedback, Proc. Conf. Computer Supported Collaborative Work, 2008, pp. 457-466.
198. K. Kim, M. Eckhardt, N. Bugg, R.W. Picard, The benefits of synchronized genuine smiles in face-to-face service encounters, Proc. Int'l Conf. on Computational Science and Engineering, 2009, pp. 801-808.
199. A. Madan, R. Caneel, A. Pentland, Voices of attraction, Proc. Int'l Conf. on Augmented Cognition, 2005.
200. A. Veenstra, H. Hung, Do they like me? Using video cues to predict desires during speed-dates, Int'l Conf. Computer Vision, 2011, pp. 838-845.
201. K. Kalimeri, B. Lepri, T. Kim, F. Pianesi, A. Pentland, Automatic modeling of dominance effects using granger causality, Human Behavior Understanding, Vol. 7065 of Lecture Notes in Computer Science, 2011, pp. 124-133.
202. R. Kaliouby, P. Robinson, Real-time inference of complex mental states from facial expressions and head gestures, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 3, 2004, p. 154.
203. M. Pantic, J. Rothkrantz, Automatic analysis of facial expressions: The state of the art, IEEE Trans. Pattern Anal. Mach. Intell. 22 (12) (2000) 1424-1445. (Pubitemid 32088469)
204. B. Fasel, J. Luettin, Automatic facial expression analysis: A survey, Pattern Recognit. 36 (1) (2003) 259-275.
205. Y. Tian, T. Kanade, J. Cohn, Facial expression analysis, S. Li, A. Jain (Eds.), Handbook of Face Recognition, Springer, 2004, pp. 247-275.
206. M.F. Valstar, H. Gunes, M. Pantic, How to distinguish posed from spontaneous smiles using geometric features, Proc. ACM Int'l Conf. Multimodal Interfaces, 2007, pp. 38-45.
207. S. Mitra, T. Acharya, Gesture recognition: A survey, IEEE Trans. Syst Man Cybern. Part C 37 (3) (2007) 311-324. (Pubitemid 46621535)
208. G.R.S. Murthy, R.S. Jadon, A review of vision based hand gestures recognition, Int. J. Info. Technol. Knowl. Manage. 2 (2) (2009) 405-410.
209. T.B. Moeslund, A. Hilton, V. Krüger, A survey of advances in vision-based human motion capture and analysis, Comput. Vis. Image Underst. 104 (2-3) (2006) 90-126. (Pubitemid 44573873)
210. V. Krüger, D. Kragic, A. Ude, C. Geib, The meaning of action: A review on action recognition and mapping, Adv. Robot. 21 (13) (2007) 1473-1501. (Pubitemid 47529843)
211. P. Turaga, R. Chellappa, V.S. Subrahmanian, O. Udrea, Machine recognition of human activities: A survey, IEEE Trans. Circ. Syst. Video Technol. 18 (11) (2008) 1473-1488.
212. L.-P. Morency, T. Darrell, Conditional sequence model for context-based recognition of gaze aversion, Proc. ACM Int'l Conf. Multimodal Interfaces, 2008, pp. 11-23. (Pubitemid 351333587)
213. A. Lockerd, F.M. Mueller, LAFCam: leveraging affective feedback camcorder, Proc. CHI, Human Factors in Computing Systems, 2002, pp. 574-575.
214. G. Guo, S. Li, Content-based audio classification and retrieval by support vector machines, IEEE Trans. Neural Netw. 14 (1) (2003) 209-215.
215. D. Baron, E. Shriberg, A. Stolcke, Automatic punctuation and disfluency detection in multi-party meetings using prosodic and lexical cues, Proc. Int'l Conf. Spoken Language Processing, 2002, pp. 949-952.
216. M. Marzinzik, B. Kollmeier, Speech pause detection for noise spectrum estimation by tracking power envelope dynamics, IEEE Trans. Speech Audio Process. 10 (2) (2002) 109-118. (Pubitemid 34295270)
217. S. Baron-Cohen, O. Golan, S. Wheelwright, J.J. Hill, Mind Reading: The Interactive Guide to Emotions, Jessica Kingsley, London, 2004.
218. T. Sheerman-Chase, E.-J. Ong, R. Bowden, Feature selection of facial displays for detection of non verbal communication in natural conversation, IEEE Int'l Conf. on Computer Vision Workshops, 2009, pp. 1985-1992.
219. K. Bousmalis, L.-P. Morency, S. Zafeiriou, M. Pantic, A discriminative nonparametric bayesian model: infinite hidden conditional random fields, Neural Information Processing Systems (NIPS) Workshop on Bayesian Nonparametrics, 2011.
220. W. Wang, S. Yaman, K. Precoda, C. Richey, G. Raymond, Detection of agreement and disagreement in broadcast conversations, Proc. Association for Computational Linguistics, 2011, pp. 374-378.
221. S. Strassel, C. Cieri, A. Cole, D. Di Persio, M. Liberman, X. Ma, M. Maamouri, K. Maeda, Integrated linguistic resources for language exploitation technologies, Proc. Int'l Conf. Language Resources and Evaluation, 2006, pp. 3052-3056.
222. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, M. Schroder, FEELTRACE: An instrument for recording perceived emotion in real time, Proc. ISCA Workshop on Speech and Emotion, 2000, pp. 19-24.
223. M. Kim, V. Pavlovic, Discriminative learning for dynamic state prediction, IEEE Trans, Pattern Anal. Mach. Intell. (2009) 1847-1861.