Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example

Journal of Nonverbal Behavior

Volumn 38, Issue 2, 2014, Pages 231-245

  Frauendorfer, Denise ^a   Schmid Mast, Marianne ^a   Nguyen, Laurent ^b   Gatica Perez, Daniel ^b  

^a UNIVERSITY OF NEUCHÂTEL   (Switzerland)

^b EPFL   (Switzerland)

Author keywords

Applicant nonverbal behavior; Automated extraction; Hiring decision; Job interview; Ubiquitous social sensing platform

Indexed keywords

EID: 84898544528     PISSN: 01915886     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10919-014-0173-5     Document Type: Article

References (40)

1. Anderson, N. R. (1992). Eight decades of employment interview research: A retrospective meta-review and prospective commentary. European Work and Organizational Psychologist, 2, 1-32. doi: 10. 1080/09602009208408532.
2. Anderson, N. R., & Shackleton, V. J. (1990). Decision making in the graduate selection interview: A field study. Journal of Occupational Psychology, 63, 63-76. doi: 10. 1111/j. 2044-8325. 1990. tb00510. x.
3. Ba, S., & Odobez, J.-M. (2011). Multiperson visual focus of attention from head pose and meeting contextual cues. IEEE Transaction on Pattern Analysis and Machine Intelligence, 3, 101-116.
4. Barrick, M. R., Shaffer, J. A., & DeGrassi, S. W. (2009). What you see may not be what you get: Relationships among self-presentations tactics and ratings of interview and job performance. Journal of Applied Psychology, 94, 1394-1411.
5. Basu, S. (2002). Conversational scene analysis. MIT Department of EECS, Cambridge, MA. http://alumni. media. mit. edu/~sbasu/papers. html.
6. Biel, J.-I., Aran, O., & Gatica-Perez, D. (2011). You are known by how you vlog: Personality impressions and nonverbal behavior in YouTube. Paper presented at the proceedings of international AAAI conference on weblogs and social media, Barcelona, Spain.
7. Burgoon, J. K. (1994). Nonverbal signals. In M. L. Knapp & G. R. Miller (Eds.), Handbook of interpersonal communication (pp. 229-285). Thousand Oaks, CA: Sage.
8. Burgoon, J. K., Birk, T., & Pfau, M. (1990). Nonverbal behaviors, persuasion, and credibility. Human Communication Research, 17, 140-169.
9. Campion, M. A., Cheraskin, L., & Stevens, M. J. (1994). Career-related antecedents and outcomes of job rotation. Academy of Management Journal, 37, 1518-1542.
10. Dovidio, J. F., Brown, C. E., Heltman, K., Ellyson, S. L., & Keating, C. F. (1988). Power displays between women and men in discussions of gender-linked tasks: A multichannel study. Journal of Personality and Social Psychology, 55(4), 580-587. doi: 10. 1037/0022-3514. 55. 4. 580.
11. Fillmore, C. J. (1979). On fluency. In D. Kempler & W. S. Y. Wang (Eds.), Individual differences in language ability and language behavior (pp. 85-102). New York: Academic Press.
12. Forbes, R. J., & Jackson, P. R. (1980). Non-verbal behaviour and the outcome of selection interviews. Journal of Occupational Psychology, 53, 65-72.
13. Funes, K., & Odobez, J.-M. (2012). Gaze estimation from multimodal kinect data. Paper presented at the IEEE conference in computer vision and pattern recognition, Providence, RI, USA.
14. Gatica-Perez, D., Guillaume, L., Odobez, J.-M., & McCowan, I. (2007). Audio-visual tracking of multiple speakers in meetings. IEEE Transaction on Audio Speech, and Language Processing, 15, 601-616.
15. Guerrero, L. K. (2005). Observer ratings of nonverbal involvement and immediacy. In V. Manusov (Ed.), The sourcebook of nonverbal measures. Mahwah, NJ: Lawrence Erlbaum Associates.
16. Hollandsworth, J. G., Kazelskis, R., Stevens, J., & Dressel, M. E. (1979). Relative contributions of verbal, articulative, and nonverbal communication to employment decisions in the job interview setting. Personnel Psychology, 32, 359-367.
17. Imada, A. S., & Hakel, M. D. (1977). Influence of nonverbal communication and rater proximity on impressions and decisions in simulated employment interviews. Journal of Applied Psychology, 62, 295-300.
18. Janz, T. (1982). Initial comparisons of patterned behavior description interviews versus unstructured interviews. Journal of Applied Psychology, 67, 577-580. doi: 10. 1037/0021-9010. 67. 5. 577.
19. Junker, H., Amft, O., Lukowicz, P., & Tröster, G. (2008). Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recognition, 41(6), 2010-2024. doi: 10. 1016/j. patcog. 2007. 11. 016.
20. Kleinke, C. L. (1986). Gaze and eye contact: A research review. Psychological Bulletin, 100, 78-100.
21. Knapp, M. L., & Hall, J. A. (2010). Nonverbal communication in human interaction (7th ed.). Wadsworth: Cengage Learning.
22. Lathoud, G., & McCowan, I. A. (2003). Location based speaker segmentation. Paper presented at the meeting of acoustics, speech, and signal processing.
23. Leathers, D. G. (1992). Successful nonvebal communication. London: Collier Macmillan.
24. Leigh, T. W., & Summers, J. O. (2002). An initial evaluation of industrial buyers' impressions of salespersons' nonverbal cues. Journal of Personal Selling & Sales Management, 22, 41-53.
25. Lu, H., Rabbi, M., Chittaranjan, G. T., Frauendorfer, D., Schmid Mast, M., Campbell, A. T., & Choudhury, T. (2012). Stresssense: Detecting stress in unconstrained acoustic environments using smartphones. Paper presented at the UbiComp, Pittsburgh, USA.
26. Marcos-Ramiro, A., Pizarro-Perez, D., Marron-Romera, M., Nguyen, L. S., & Gatica-Perez, D. (2013). Body communication cue extraction for conversational analysis. Paper presented at the IEEE international conference on automatic face and gesture recognition, Shanghai, China.
27. McGovern, T. V., & Tinsley, H. E. A. (1978). Interviewer evaluations of interviewee nonverbal behavior. Journal of Vocational Behavior, 13, 163-171. doi: 10. 1016/0001-8791(78)90041-6.
28. Mehrabian, A. (1972). Nonverbal communication. New Brunswick: AldineTransaction.
29. Motowidlo, S. J., Carter, G. W., Dunnette, M. D., Tippins, N., Werner, S., Burnett, J. R., et al. (1992). Studies of the structured behavioral interview. Journal of Applied Psychology, 77, 571-587. doi: 10. 1037/0021-9010. 77. 5. 571.
30. Nguyen, L. S., Odobez, J.-M., & Gatica-Perez, D. (2012). Using self-context for multimodal detection of head nods in face-to-face interactions. Paper presented at the international conference on multimodal interactions, Santa Monica, CA, USA.
31. Peterson, R. T. (2005). An examination of the relative effectiveness of training in nonverbal communication: Personal selling implications. Journal of Marketing Education, 27, 143-150. doi: 10. 1177/0273475305276627.
32. Pudil, P., Novovicova, J., & Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognition Letters, 15, 1119-1125.
33. Quené, H. (2007). On the just noticeable difference for tempo in speech. Journal of Phonetics, 35, 353-362.
34. Reis, H. T., & Charles, M. J. (2000). Handbook of research methods in social and personality psychology. Cambridge: Cambridge University Press.
35. Taute, H. A., Heiser, R. S., & McArthur, D. N. (2011). The effect of nonverbal signals on student role-play evaluations. Journal of Marketing Education, 33, 28-40. doi: 10. 1177/0273475310389153.
36. Tiedens, L. Z., & Fragale, A. R. (2003). Power moves: Complementarity in dominant and submissive nonverbal behavior. Journal of Personality and Social Psychology, 84(3), 558-568.
37. Vinciarelli, A., Pantic, M., Bourlard, H., & Pentland, A. (2008). Social signals, their function, and automatic analysis: A survey. Paper presented at the ICMI 2008, Chania, Crete, Greece.
38. Wardhaugh, R. (1985). How conversation works. New York: Basil Blackwell.
39. Wood, J. A. (2006). NLP revisited: Nonverbal communications and signals of trustworthiness. Journal of Personal Selling & Sales Management, 26, 197-204. doi: 10. 2753/pss0885-3134260206.
40. Woolbert, C. (1920). The effects of various modes of public reading. Journal of Applied Psychology, 16, 162-185.