Enactive cinema paves way for understanding complex real-time social interaction in neuroimaging experiments

Frontiers in Human Neuroscience

Volumn , Issue NOVEMBER 2012, 2012, Pages

  Tikka, Pia ^a   Väljamäe, Aleksander ^b,c   de Borst, Aline W ^a   Pugliese, Roberto ^a   Ravaja, Niklas ^d,e   Kaipainen, Mauri ^f   Takala, Tapio ^a  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITY OF GRAZ   (Austria)

^c CIANT Prague   (Czech Republic)

^d UNIVERSITY OF HELSINKI   (Finland)

^e HELSINKI SCHOOL OF ECONOMICS   (Finland)

^f SÖDERTÖRN UNIVERSITY   (Sweden)

Author keywords

Brain computer interfaces; Enactive cinema; Generative storytelling; Implicit interaction; Neurofeedback; Real time fmri; Social neuroscience

Indexed keywords

ARTICLE; AUDIOVISUAL EQUIPMENT; BRAIN FUNCTION; BRAIN REGION; COPING BEHAVIOR; EGO DEVELOPMENT; EXPERIMENTAL DESIGN; FEEDBACK SYSTEM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; MOTIVATION; NEUROIMAGING; PSYCHOMOTOR ACTIVITY; PSYCHOPHYSIOLOGY; SOCIAL BEHAVIOR; SOCIAL INTERACTION; STIMULUS RESPONSE; TELEVISION VIEWING; VISUAL STIMULATION;

EID: 84933675520     PISSN: 16625161     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (74)

1. Alexander, O., Rogers, M., Lambeth, W., Chiang, J. Y., Ma, W. C., Wang, C. C., et al. (2010). The digital Emily project: achieving a photorealistic digital actor. IEEE Comput. Graph. Appl. 30, 20-31.
2. Arroyo-Palacios, J., and Romano, D. M. (2010). Bio-affective computer interface for game interaction. IJGCMS 2, 16-32.
3. Baess, P., Zhdanov, A., Mandel, A., Parkkonen, L., Hirvenkari, L., Mäkelä, J. P., et al. (2012). MEG dual scanning: a procedure to study real-time auditory interaction between two persons. Front. Hum. Neurosci. 6:83. doi: 10.3389/fnhum.2012.00083.
4. Bartels, A., and Zeki, S. (2003). Functional brain mapping during free viewing of natural scenes. Hum. Brain Mapp. 21, 75-85.
5. Bartels, A., and Zeki, S. (2004). The chronoarchitecture of the human brain-natural viewing conditions reveal a time-based anatomy of the brain. Neuroimage 22, 419-433.
6. Bartels, A., Zeki, S., and Logothetis, N. K. (2008). Natural vision reveals regional specialization to local motion and to contrast- invariant, global flow in the human brain. Cereb. Cortex 18, 705-717.
7. Baumgartner, T., Speck, D., Wettstein, D., Masnari, O., Beeli, G., and Jäncke, L. (2008). Feeling present in arousing virtual reality worlds: prefrontal brain regions differentially orchestrate presence experience in adults and children. Front. Hum. Neurosci. 2:8. doi: 10.3389/neuro.09.008.2008.
8. Cattaneo, L., and Rizzolatti, G. (2009). The mirror neuron system. Arch. Neurol. 66, 557-560.
9. Chaminade, T., Zecca, M., Blakemore, S. J., Takanishi, A., Frith, C. D., Micera, S., et al. (2010). Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures. PLoS ONE 5:e11577. doi: 10.1371/journal.pone.0011577
10. Cheetham, M., Suter, P., and Jäncke, L. (2011). The human likeness dimension of the "uncanny valley hypothesis": behavioral and functional MRI findings. Front. Hum. Neurosci. 5:126. doi: 10.3389/fnhum.2011.00126
11. Cox, R. W., Jesmanowicz, A., and Hyde, J. S. (1995). Real-time functional magnetic resonance imaging. Magn. Reson. Med. 33, 230-236.
12. Davidson, R. J. (2004). What does the prefrontal cortex "do" in affect: perspectives on frontal EEG asymmetry research. Biol. Psychol. 67, 219-233.
13. deCharms, R. C., Christoff, K., Glover, H. Pauly, J. M., Whitfield, S., and Gabrieli, J. D. (2004). Learned regulation of spatially localized brain activation using real-time fMRI. Neuroimage 21, 436-443.
14. Ehrsson, H. H., Wiech, K., Weiskopf, N., Dolan, R. J., and Passingham, R. E. (2007). Threatening a rubber hand that you feel is yours elicits a cortical anxiety response. Proc. Natl. Acad. Sci. U.S.A. 104, 9828-9833.
15. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. New Jersey, NJ: Lawrence Erlbaum Associates.
16. Grodal, T. (2009). Embodied Visions: Evolution, Emotion, Culture and Film. New York, NY: Oxford University Press.
17. Hamilton, J. P., Glover, G. H., Hsu, J. J., Johnson, R. F., and Gotlib, I. (2011). Modulation of subgen- ual anterior cingulate cortex activity with real-time neurofeedback. Hum. Brain Mapp. 32, 22-31.
18. Hari, R., and Kujala, M. V. (2009). Brain basis of human social interaction: from concepts to brain imaging. Physiol. Rev. 89, 453-479.
19. Hasson, U., Furman, O., Clark, D., Dudai, Y., and Davachi, L. (2008a). Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding. Neuron 57, 452-462.
20. Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., and Heeger, D. J. (2008b). Neurocinematics: the neuroscience of film. Projections 2, 1-26.
21. Hasson, U., Malach, R., and Heeger, D. J. (2010). Reliability of cortical activity during natural stimulation. Trends Cogn. Sci. 14, 40-48.
22. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., and Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science 303, 1634-1640.
23. Heller, A. S., Greischar, L. L., Honor, Anderle, M. J., and Davidson, R. J. (2011). Simultaneous acquisition of corrugator electromyography and functional magnetic resonance imaging: a new method for objectively measuring affect and neural activity concurrently. Neuroimage 58, 930-934.
24. Hoffman, H. G., Richards, T., Coda, Richards, A., and Sharar, S. R. (2003). The illusion of presence in immersive virtual reality during an fMRI brain scan. Cyberpsychol. Behav. 6, 127-131.
25. Hollmann, M., Rieger, J. W., Baecke, S., Lützkendorf, R., Müller, C., Adolf, D., et al. (2011). Predicting decisions in human social interactions using real-time fMRI and pattern classification. PLoS ONE 6:e25304. doi: 10.1371/journal.pone.0025304
26. Horton, D., and Wohl, R. R. (1956). Mass communication and para- social interaction: observation on intimacy at a distance. Psychiatry 19, 215-229.
27. Iacoboni, M., Molnar-Szakacs, I., Gallese, V., Buccino, G., Mazziotta, J. C., and Rizzolatti, G. (2005). Grasping the intentions of others with one's own mirror neuron system. PLoS Biol. 3:e79. doi: 10.1371/journal.pbio.0030079
28. Jääskeläinen, I. P., Koskentalo, K., Balk, M. H., Autti, T., Kauramaki, J., Pomren, C., et al. (2008). Inter-subject synchronization of prefrontal cortex hemodynamic activity during natural viewing. Open Neuroimag. J. 2, 14-19.
29. Johnston, S. J., Boehm, S. G., Healy, D., Goebel, R., and Linden, D. E. (2010). Neurofeedback: a promising tool for the self-regulation of emotion networks. Neuroimage 49, 1066-1072.
30. Kaipainen, M., Ravaja, N., Tikka, P., Vuori, R., Pugliese, R., Rapino, M., et al. (2011). Enactive systems and enactive media: embodied human - machine coupling beyond interfaces. Leonardo 44, 433-438.
31. Kaipainen, M., and Hautamäki, A. (2011). Epistemic pluralism and multi-perspective knowledge organization: explorative conceptualization of topical content domains. Knowl. Organ. 38, 503-514.
32. Kaipainen, M., Normak, P., Niglas, K., Kippar, J., and Laanpere, M. (2008). Soft ontologies, spatial representations and multi-perspective explorability. Expert Sys. 25, 474-483.
33. Kaplan, A. Y., Byeon, J.-G., Lim, J.-J., Jin, K.-S., and Park, B.-W. (2005). Unconscious operant conditioning in the paradigm of brain- computer interface based on color perception. Int. J. Neurosci. 115, 781-802.
34. Kuikkaniemi, K., Turpeinen, M., Laitinen, T., Saari, T., Kosunen, I., and Ravaja, N. (2010). "Influence of implicit and explicit biofeedback in first-person shooter games," in CHI '10: Proceedings of the 28 th International Conference on Human Factors in Computing Systems, (New York, NY: ACM Press), 859-868.
35. LaConte, S. M. (2011). Decoding fMRI brain states in real-time. Neuroimage 56, 440-454.
36. Lahnakoski, J. M., Glerean, E., Salmi, J., Jääskeläinen, I. P., Sams, M., Hari, R., et al. (2012). Naturalistic FMRI mapping reveals superior temporal sulcus as the hub for the distributed brain network for social perception. Front. Hum. Neurosci. 6:233. doi: 10.3389/fnhum.2012.00233
37. Lee, J. H., Kim, J., and Yoo, S. S. (2012). Real-time fMRI-based neurofeedback reinforces causality of attention networks. Neurosci. Res. 72, 347-354.
38. Lee, R., Dai, W., and Dix, W. (2010). A decoupled circular-polarized volume head coil pair for studying two interacting human brains with MRI. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2010, 6645-6648.
39. Lee, R., Dai, W., and Jones, J. (2011). Decoupled circular-polarized dualhead volume coil pair for studying two interacting human brains with dyadic fMRI. Magn. Reson. Med. 68, 1087-1096.
40. Ma, X., Zhang, H., Zhao, X., Yao, L., and Long, Z. (2012). Semiblind independent component analysis of fMRI based on real-time fMRI system. IEEE Trans. Neural Rehabil. Eng. doi: 10.1109/TNSRE.2012.2184303. [Epub ahead of print].
41. Manuck, S. B., Levenson, R. W., Hinrichsen, J. J., and Gryll, S. L. (1975). Role of feedback in voluntary control of heart rate. Percept. Mot. Skills 40, 747-752.
42. Mar, R. A., and Oatley, K. (2008). The function of fiction is the abstraction and simulation of social experience. Perspect. Psychol. Sci. 3, 173-192.
43. Mattout, J. (2012). Brain-computer interfaces: a neuroscience paradigm of social interaction? A matter of perspective. Front. Hum. Neurosci. 6:114. doi: 10.3389/fnhum.2012.00114
44. Montague, P. R., Berns, G. S., Cohen, J. D., McClure, S. M., Pagnoni, G., Dhamala, M., et al. (2002). Hyperscanning: simultaneous fMRI during linked social interactions. Neuroimage 16, 1159-1164.
45. Moser, E., Derntl, B., Robinson, S., Fink, B., Gur, R. C., and Grammar, K. (2007). Amygdala activation at 3T in response to human and avatar facial expressions of emotions. J. Neurosci. Methods 161, 126-133.
46. Mori, M. (1970). The Uncanny Valley. Energy 7, 33-35. [Republished in IEEE Robotics and Automation Magazine, June 2012, 98-100].
47. Neisser, U. (1976). Cognition and Reality: Principles and Implications of Cognitive Psychology. San Francisco, CA: W. H. Freeman and Co.
48. Nowlis, D. P., and Kamiya, J. (1970). The control of electroen- cephalographic alpha rhythms through auditory feedback and the associated mental activity. Psychophysiology 6, 476-484.
49. Parkinson, C., Sinnott-Armstrong, W., Koralus, P. E., Mendelovici, A, McGeer, V., and Wheatley, T. (2011). Is morality unified? evidence that distinct neural systems underlie moral judgments of harm, dishonesty, and disgust. J. Cogn. Neurosci. 23, 3162-3180.
50. Parsons, T. D., and Courtney, C. (2011). "Neurocognitive and psychophysiological interfaces for adaptive virtual environments," in Human-CenteredDesign ofE-Health Technologies: Concepts, Methods and Applications, eds M. Ziefle and C. Rocker (Hershey, PA: IGI Global), 208-233.
51. Posse, S., Fitzgerald, D., Gao, K., Habel, U., Rosenberg, D., Moore, G. J., et al. (2003). Realtime fMRI of temporolimbic regions detects amygdala activation during singletrial selfinduced sadness. Neuroimage 18, 760-768.
52. Pugliese, R., and Lehtonen, K. (2011). "A framework for motion based bodily enaction with virtual characters," in Proceedings of the 10th International Conference on Intelligent Virtual Agents (IVA'11) (Berlin, Heidelberg: Springer-Verlag), 162-168.
53. Pugliese, R., Tikka, P., and Kaipainen, M. (2012). Navigating story onto space: perspective-relative drive and combinatory montage of cinematic content. Stud. ArtArchit. (in press).
54. Ravaja, N. (2004). Contributions of psychophysiology to media research: review and recommendations. Media Psychol. 6, 193-235.
55. Redcay, E., Dodell-Feder, D., Pearrow, M. J., Mavros, P. L., Kleiner, M., Gabrieli, J. D. E., et al. (2010). Live face-to-face interaction during fMRI: a new tool for social cognitive neuroscience. Neuroimage 50, 1639-1647.
56. Rinman, M. L., Friberg, A., Bendiksen, Cirotteau, D., Dahl, S., Kjellmo, I, et al. (2004). "Ghost in the Cavean interactive collaborative game using non-verbal communication," in Lecture Notes in Computer Science, eds A. Camurri and G. Volpe (Berlin: Springer- Verlag), 549-556.
57. Riva, G., Raspelli, S., Algeri, D., Pallavicini, F., Gorini, A., Wiederhold, B. K., et al. (2010). Interreality in practice: bridging virtual and real worlds in the treatment of posttraumatic stress disorders. Cyberpsychol. Behav. Soc. Netw. 13, 55-65.
58. Rizzo, A. A., Schultheis, M. T., Kerns, K., and Mateer, C. (2004). Analysis of assets for virtual reality applications in neuropsychology. Neuropsychol. Rehabil. 14, 207-239.
59. Rizzolatti, G., and Arbib, M. A. (1998). Language within our grasp. Trends Neurosci. 21, 188-194.
60. Rizzolatti, G., and Fabbri-Destro, M. (2008). The mirror system and its role in social cognition. Curr. Opin. Neurobiol. 18, 179-184.
61. Sanchez-Vives, M. V., and Slater, M. (2005). From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6, 332-339.
62. Sarkheil, P., Zilverstand, A., Kilian- Huetten, N., Schneider, F., Mathiak, K., and Goebel, R. (2011). "Application of real-time fMRI-neurofeedback in treatment of emotional disorders," in Abstract at 17th Annual Meeting of the OHBM (Quebec, QC: Canada).
63. Saygin, A. P., Chaminade, T., Ishiguro, H., Driver, J., and Frith, C. (2011). The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc. Cogn. Affect. Neurosci. 7, 413-422.
64. Singer, T., Seymour, B., O'Doherty, J. P., Stephan, K. E., Dolan, R. J., and Frith, R. J. (2006). Empathic neural responses are modulated by the perceived fairness ofothers. Nature 439, 466-469.
65. Tan, E. S. (1996). Emotion and the Structure of the Narrative Film. Hillsdale, NJ: Lawrence Erlbaum.
66. Tikka, P. (2005). Enactive Cinema Installation "Obsession" Exhibited in the Museum of Contemporary Art Kiasma. Helsinki: Oblomovies and University of Art and Design.
67. Tikka, P. (2008). Enactive Cinema: Simulatorium Eisensteinense. Ph.D. dissertation, Helsinki: University of Art and Design Publication Series.
68. Tikka, P., Vuori, R., and Kaipainen, M. (2006). Narrative logic of enac- tive cinema: 'Obsession'. Digital Creativity 17, 205-212.
69. Varela, F., Thompson, E., and Rosch, E. (1991). Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press.
70. Veit, R., Singh, V., Sitaram, R., Caria, A., Rauss, K., and Birbaumer, N., (2011) Using real-time fMRI to learn voluntary regulation of the anterior insula in the presence of threat-related stimuli. Soc. Cogn. Affect. Neurosci. 7, 623-634.
71. Voyvodic, J. T., Glover, G. H., Greve, D., and Gadde, S. (2011). Automated real-time behavioral and physiological data acquisition and display integrated with stimulus presentation for FMRI. Front. Neuroinform. 5:27. doi: 10.3389/fninf.2011.00027
72. Weiskopf, N. (2011). Real-time fMRI and its application to neurofeedback. Neuroimage 62, 682-692.
73. Weiskopf, N., Mathiak, K., Bock, S. W., Scharnowski, F., Veit, R., Grodd, W., et al. (2004). Principles of a brain- computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 51, 966-970.
74. Wilms, M., Schilbach, L., Pfeiffer, U., Bente, G., Fink, G. R., and Vogeley, K. (2010). It's in your eyes-using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience. Soc. Cogn. Affect. Neurosci. 5, 98-107.