Studying social interactions through immersive virtual environment technology: virtues, pitfalls, and future challenges

Frontiers in Psychology

Volumn 6, Issue , 2015, Pages

  Bombari, Dario ^a   Schmid Mast, Marianne ^a   Canadas, Elena ^a   Bachmann, Manuel ^b  

^a UNIVERSITY OF LAUSANNE   (Switzerland)

^b BERN UNIVERSITY OF APPLIED SCIENCES   (Switzerland)

Author keywords

avatars; copresence; immersive virtual environment; social interaction; virtual humans

Indexed keywords

EID: 85106142311     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00869     Document Type: Review

References (77)

1. Argyle M. Dean J. (1965). Eye-contact, distance and affiliation. Sociometry 28 289–304. 10.2307/2786027
2. Bailenson J. N. Beall A. C. Loomis J. M. Blascovich J. Turk M. (2005). Transformed social interaction, augmented gaze, and social influence in immersive virtual environments. Hum. Commun. Res. 31 511–537. 10.1111/j.1468-2958.2005.tb00881.x
3. Bailenson J. N. Blascovich J. Beall A. C. Loomis J. M. (2003). Interpersonal distance in immersive virtual environments. Pers. Soc. Psychol. Bull. 29 819–833. 10.1177/0146167203029007002
4. Bailenson J. N. Patel K. Nielsen A. Bajscy R. Jung S.-H. Kurillo G. (2008). The effect of interactivity on learning physical actions in virtual reality. Media Psychol. 11 354–376. 10.1080/15213260802285214
5. Bailenson J. N. Yee N. (2005). Digital chameleons: automatic assimilation of nonverbal gestures in immersive virtual environments. Psychol. Sci. 16 814–819. 10.1111/j.1467-9280.2005.01619.x
6. Batrinca L. Stratou G. Shapiro A. Morency L.-P. Scherer S. (2013). “Cicero - Towards a multimodal virtual audience platform for public speaking training,” in Intelligent Virtual Agents Vol. 8108 eds Aylett R. Krenn B. Pelachaud C. Shimodaira H. (Heidelberg: Springer Berlin Heidelberg) 116–128.
7. Baumeister R. F. Vohs K. D. Funder D. C. (2007). Psychology as the science of self-reports and finger movements: whatever happened to actual behavior? Perspect. Psychol. Sci. 2 396–403. 10.1111/j.1745-6916.2007.00051.x
8. Bente G. Eschenburg F. Aelker L. (2007). Effects of simulated gaze on social presence, person perception and personality attribution in avatar-mediated communication. Paper Presented at the PRESENCE 2007 Barcelona.
9. Biocca F. Delaney B. (1995). “Immersive virtual reality technology,” in Communication in the Age of Virtual Reality eds Biocca F. Levy M. (Hillsdale, NJ: Erlbaum) 57–124.
10. Blascovich J. (2002). “Social influence within immersive virtual environments,” in The Social Life of Avatars ed. Schroeder R. (London: Springer London).
11. Blascovich J. Loomis J. Beall A. C. Swinth K. R. Hoyt C. L. Bailenson J. N. (2002). Immersive virtual environment technology as a methodological tool for social psychology. Psychol. Inq. 13 103–124. 10.1207/s15327965pli1302_01
12. Branigan H. P. Pickering M. J. McLean J. F. Cleland A. A. (2007). Syntactic alignment and participant role in dialogue. Cognition 104 163–197. 10.1016/j.cognition.2006.05.006
13. Brown-Schmidt S. (2012). Beyond common and privileged: gradient representations of common ground in real-time language use. Lang. Cogn. Process. 27 62–89. 10.1080/01690965.2010.543363
14. Buck R. Losow J. I. Murphy M. M. Costanzo P. (1992). Social facilitation and inhibition of emotional expression and communication. J. Pers. Soc. Psychol. 63 962–968. 10.1037/0022-3514.63.6.962
15. Congdon S. P. Schober M. F. (2002). How examiners’ discourse cues affect scores on intelligence test. Paper Presented at the 43th Annual Meeting of the Psychonomic Society Kansas.
16. Dotsch R. Wigboldus D. H. J. (2008). Virtual prejudice. J. Exp. Soc. Psychol. 44 1194–1198. 10.1016/j.jesp.2008.03.003
17. Dyck M. Winbeck M. Leiberg S. Chen Y. Gur R. C. Mathiak K. (2008). Recognition profile of emotions in natural and virtual faces. PLoS ONE 3:e3628. 10.1371/journal.pone.0003628
18. Ekman P. Friesen W. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Palo Alto: Consulting Psychologists Press.
19. Fox J. Arena D. Bailenson J. N. (2009). Virtual reality. J. Media Psychol. 21 95–113. 10.1027/1864-1105.21.3.95
20. Freeman J. Avons S. E. Pearson D. E. Ijsselsteijn W. A. (1999). Effects of sensory information and prior experience on direct subjective ratings of presence. Presence Teleop. Virt. 8 1–13. 10.1162/105474699566017
21. Funder D. C. Colvin C. R. (1988). Friends and strangers: acquaintanceship, agreement, and the accuracy of personality judgment. J. Pers. Soc. Psychol. 55 149–158. 10.1037/0022-3514.55.1.149
22. Garau M. Slater M. Vinayagamoorthy V. Brogni A. Steed A. Sasse M. A. (2003). The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. Paper Presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Ft. Lauderdale, FL. 10.1145/642611.642703
23. Giannopoulos E. Wang Z. Peer A. Buss M. Slater M. (2010). Comparison of people’s responses to real and virtual handshakes within a virtual environment. Brain Res. Bull. 85 276–282. 10.1016/j.brainresbull.2010.11.012
24. Hall J. A. Coats E. J. LeBeau L. S. (2005). Nonverbal behavior and the vertical dimension of social relations: a meta-analysis. Psychol. Bull. 131 898–924. 10.1037/0033-2909.131.6.898
25. Harris S. R. Kemmerling R. L. North M. M. (2002). Brief virtual reality therapy for public speaking anxiety. Cyberpsychol. Behav. 5 543–550. 10.1089/109493102321018187
26. Hartanto D. Kampmann I. L. Morina N. Emmelkamp P. G. M. Neerincx M. A. Brinkman W.-P. (2014). Controlling social stress in virtual reality environments. PLoS ONE 9:e92804. 10.1371/journal.pone.0092804
27. Hatfield E. Cacioppo J. T. Rapson R. L. (1992). “Primitive emotional contagion,” in Emotion and Social Behavior. Review of Personality and Social Psychology ed. Clark M. S. (Thousand Oaks, CA: Sage Publications) 151–177.
28. Heeter C. (1992). Being there: the subjective experience of presence. Presence Teleop. Virt. 1 262–271.
29. Hess U. Blairy S. (2001). Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. Int. J. Psychophysiol. 40 129–141. 10.1016/S0167-8760(00)00161-6
30. Hess U. Fischer A. (2013). Emotional mimicry as social regulation. Pers. Soc. Psychol. Rev. 17 142–157. 10.1177/1088868312472607
31. Hoyt C. L. Blascovich J. Swinth K. R. (2003). Social inhibition in immersive virtual environments. Presence Teleop. Virt. 12 183–195. 10.1162/105474603321640932
32. Ijsselsteijn W. A. Lombard M. Freeman J. (2001). Toward a core bibliography of presence. Cyberpsychol. Behav. 4 317–321. 10.1089/109493101300117983
33. Jacobson D. (2001). Presence revisited: Imagination, competence, and activity in text-based virtual worlds. Cyberpsychol. Behav. 4 653–673. 10.1089/109493101753376605
34. Kelly O. Matheson K. Martinez A. Merali Z. Anisman H. (2007). Psychosocial stress evoked by a virtual audience: relation to neuroendocrine activity. Cyberpsychol. Behav. 10 655–662. 10.1089/cpb.2007.9973
35. Kilteni K. Bergstrom I. Slater M. (2013). Drumming in immersive virtual reality: the body shapes the way we play. IEEE Trans. Vis. Comput. Graph. 19 597–605. 10.1109/TVCG.2013.29
36. Kirschbaum C. Pirke K. M. Hellhammer D. H. (1993). The ‘Trier Social Stress Test’–a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology 28 76–81. 10.1159/000119004
37. Kober S. E. Kurzmann J. Neuper C. (2012). Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: an EEG study. Int. J. Psychophysiol. 83 365–374. 10.1016/j.ijpsycho.2011.12.003
38. Kuhlen A. K. Brennan S. E. (2013). Language in dialogue: when confederates might be hazardous to your data. Psychon. Bull. Rev. 20 54–72. 10.3758/s13423-012-0341-8
39. Landauer T. K. Dumais S. T. (1997). A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104 211–240. 10.1037/0033-295X.104.2.211
40. Latu I. M. Schmid Mast M. Lammers J. Bombari D. (2013). Successful female leaders empower women’s behavior in leadership tasks. J. Exp. Soc. Psychol. 49 444–448. 10.1016/j.jesp.2013.01.003
41. Lee K. M. (2004). Presence, explicated. Commun. Theor. 14 27–50. 10.1111/j.1468-2885.2004.tb00302.x
42. Llobera J. Spanlang B. Ruffini G. Slater M. (2010). Proxemics with multiple dynamic characters in an immersive virtual environment. ACM Trans. Appl. Perce. 8 1–12. 10.1145/1857893.1857896
43. Maister L. Slater M. Sanchez-Vives M. V. Tsakiris M. (2015). Changing bodies changes minds: owning another body affects social cognition. Trends Cogn. Sci. 19 6–12. 10.1016/j.tics.2014.11.001
44. Malatesta L. Raouzaiou A. Karpouzis K. Kollias S. (2009). Towards modeling embodied conversational agent character profiles using appraisal theory predictions in expression synthesis. Appl. Intell. 30 58–64. 10.1007/s10489-007-0076-9
45. Mantovani F. Castelnuovo G. (2003). “The sense of presence in virtual training: enhancing skills acquisition and transfer of knowledge through learning experience in virtual environments,” in Being There: Concepts, Effects and Measurement of User Presence in Synthetic Environments eds Riva G. Davide F. Ijsselsteijn W. A. (Amsterdam: Ios Press).
46. Mehl M. R. Pennebaker J. W. (2003). The sounds of social life: a psychometric analysis of students’ daily social environments and natural conversations. J. Pers. Soc. Psychol. 84 857–870. 10.1037/0022-3514.84.4.857
47. Milgram S. (1963). Behavioral study of obedience. J. Abnorm. Soc. Psychol. 67 371–378. 10.1037/h0040525
48. Mori M. (1970). Bukimi no tani [The uncanny valley]. Energy 7 33–35.
49. Palmer M. T. (1995). “Interpersonal communication and virtual reality: mediating interpersonal relationship,” in Communication in the Age of Virtual Reality eds Biocca F. Levy M. (Hillsdale, NJ: Erlbaum) 277–302.
50. Park K. M. Ku J. Choi S. H. Jang H. J. Park J. Y. Kim S. I. et al. (2011). A virtual reality application in role-plays of social skills training for schizophrenia: a randomized, controlled trial. Psychiatry Res. 189 166–172. 10.1016/j.psychres.2011.04.003
51. Patterson M. L. (1976). An arousal model of interpersonal intimacy. Psychol. Rev. 83 235–245. 10.1037/0033-295x.83.3.235
52. Patterson M. L. Webb A. Schwartz W. (2002). Passing encounters: patterns of recognition and avoidance in pedestrians. Basic Appl. Soc. Psychol. 24 57–66. 10.1207/s15324834basp2401_5
53. Peck T. C. Seinfeld S. Aglioti S. M. Slater M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious. Cogn. 22 779–787. 10.1016/j.concog.2013.04.016
54. Pelachaud C. (2009). Modelling multimodal expression of emotion in a virtual agent. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364 3539–3548. 10.1098/rstb.2009.0186
55. Perez-Marcos D. Solazzi M. Steptoe W. Oyekoya O. Frisoli A. Weyrich T. et al. (2012). A fully immersive set-up for remote interaction and neurorehabilitation based on virtual body ownership. Front. Neurol. 3:110. 10.3389/fneur.2012.00110
56. Pertaub D.-P. Slater M. Barker C. (2002). An experiment on public speaking anxiety in response to three different types of virtual audience. Presence Teleop. Virt. 11 68–78. 10.1162/105474602317343668
57. Poeschl S. Doering N. (2012). Designing virtual audiences for fear of public speaking training - an observation study on realistic nonverbal behavior. Stud. Health Technol. Inform. 181 218–222.
58. Qu C. Brinkman W.-P. Ling Y. Wiggers P. Heynderickx I. (2014). Conversations with a virtual human: synthetic emotions and human responses. Comput. Human Behav. 34 58–68. 10.1016/j.chb.2014.01.033
59. Qu C. Brinkman W.-P. Wiggers P. Heynderickx I. (2013). The effect of priming pictures and videos on a question–answer dialog scenario in a virtual environment. Presence Teleop. Virt. 22 91–109. 10.1162/PRES_a_00143
60. Roter D. L. Hall J. A. Aoki Y. (2002). Physician gender effects in medical communication: a meta-analytic review. JAMA 288 756–764. 10.1001/jama.288.6.756
61. Santos-Ruiz A. Peralta-Ramirez M. I. Garcia-Rios M. C. Muñoz M. A. Navarrete-Navarrete N. Blazquez-Ortiz A. (2010). Adaptation of the trier social stress test to virtual reality: psycho-physiological and neuroendocrine modulation. J. Cyber. Ther. Rehabil. 3 405–415.
62. Scherer K. R. (2001). “Appraisal considered as a process of multi-level sequential checking,” in Appraisal Processes in Emotion: Theory, Methods, Research eds Scherer K. R. Schorr A. Johnstone T. (New York and Oxford: Oxford University Press) 92–120.
63. Schmid Mast M. Gatica-Perez D. Frauendorfer D. Nguyen L. Choudhury T. (2015). Social sensing for psychology: automated interpersonal behavior assessment. Curr. Dir. Psychol. Sci. 24 154–160. 10.1177/0963721414560811
64. Schmid Mast M. Hall J. A. Roter D. L. (2008). Caring and dominance affect participants’ perceptions and behaviors during a virtual medical visit. J. Gen. Intern. Med. 23 523–527. 10.1007/s11606-008-0512-5
65. Schuemie M. J. van der Straaten P. Krijn M. van der Mast C. A. (2001). Research on presence in virtual reality: a survey. Cyberpsychol. Behav. 4 183–201. 10.1089/109493101300117884
66. Seyama J. I. Nagayama R. S. (2007). The uncanny valley: effect of realism on the impression of artificial human faces. Presence Teleop. Virt. 16 337–351. 10.1162/pres.16.4.337
67. Slater M. Antley A. Davison A. Swapp D. Guger C. Barker C. et al. (2006a). A virtual reprise of the Stanley Milgram obedience experiments. PLoS ONE 1:e39. 10.1371/journal.pone.0000039
68. Slater M. Guger C. Edlinger G. Leeb R. Pfurtscheller G. Antley A. et al. (2006b). Analysis of physiological responses to a social situation in an immersive virtual environment. Presence Teleop. Virt. 15 553–569. 10.1162/pres.15.5.553
69. Slater M. Steed A. (2002). “Meeting people virtually: experiments in shared virtual environments,” in The Social Life of Avatars ed. Schroeder R. (London: Springer London), 146–171.
70. Slater M. Wilbur S. (1997). A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence Teleop. Virt. 6 603–616.
71. Thalmann D. (2006). Populating virtual environments with crowds. Paper Presented at the Proceedings of the 2006 ACM International Conference on Virtual Reality Continuum and its Applications Hong Kong. 10.1145/1128923.1128925
72. Tinwell A. Grimshaw M. Nabi D. A. Williams A. (2011). Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Comput. Human Behav. 27 741–749. 10.1016/j.chb.2010.10.018
73. Topal J. Gergely G. Miklosi A. Erdohegyi A. Csibra G. (2008). Infants’ perseverative search errors are induced by pragmatic misinterpretation. Science 321 1831–1834. 10.1126/science.1161437
74. Vinayagamoorthy V. Steed A. Slater M. (2008). The impact of a character posture model on the communication of affect in an immersive virtual environment. IEEE Trans. Vis. Comput. Graph. 14 965–982. 10.1109/tvcg.2008.62
75. Wieser M. J. Pauli P. Grosseibl M. Molzow I. Mühlberger A. (2010). Virtual social interactions in social anxiety–the impact of sex, gaze, and interpersonal distance. Cyberpsychol. Behav. Soc. Netw. 13 547–554. 10.1089/cyber.2009.0432
76. Yee N. Bailenson J. (2007). The proteus effect: the effect of transformed self-representation on behavior. Hum. Commun. Res. 33 271–290. 10.1111/j.1468-2958.2007.00299.x
77. Zhang L. Yap B. (2012). Affect detection from text-based virtual improvisation and emotional gesture recognition. Adv. Human Comput. Interact. 2012:12. 10.1155/2012/461247