Augmenting the Instant Messaging Experience Through the Use of Brain-Computer Interface and Gestural Technologies

International Journal of Human-Computer Interaction

Volumn 29, Issue 3, 2013, Pages 178-191

  Kuber, Ravi ^a   Wright, Franklin P ^a  

^a UNIVERSITY OF MARYLAND BALTIMORE COUNTY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EMOTIONAL INFORMATION; EXPLORATORY STUDIES; FACIAL EXPRESSIONS; FACIAL GESTURES; INSTANT MESSAGES; INSTANT MESSAGING; INTER-PERSONAL COMMUNICATIONS; USER EXPERIENCE;

COMMUNICATION; MESSAGE PASSING; WORLD WIDE WEB;

BRAIN COMPUTER INTERFACE;

EID: 84873161965     PISSN: 10447318     EISSN: 15327590     Source Type: Journal    
DOI: 10.1080/10447318.2012.702635     Document Type: Article

References (50)

1. Birbaumer, N., Kubler, A., Ghanayim, N., Hinterberger, T., Perelmouter, J., Kaiser, J.,and Flor, H. 2000. The thought translation device (TTD) for completely paralyzed patients. IEEE Transactions on Rehabilitation Engineering, 8: 190-192.
2. Brave, S. and Dahley, A. 1997. inTouch: A medium for haptic interpersonal communication. Extended Abstracts on Human Factors in Computing Systems,: 363-364.
3. Cernea, D., Olech, P.S., Ebert, A. and Kerren, A. 2011. EEG-based measurement of subjective parameters in evaluations. Proceedings of the Fourteenth International Conference on Human Computer Interaction International,: 279-283.
4. Chanel, G., Kierkels, J.J.M., Soleymani, M. and Pun, T. 2009. Short-term emotion assessment in a recall paradigm. International Journal of Human-Computer Studies, 67: 607-627.
5. Chanel, G., Rebetez, C., Betrancourt, M. and Pun, T. 2008. Boredom, engagement and anxiety as indicators for adaptation to difficulty in games. Proceedings of the Twelfth International Conference on Entertainment and Media in the Ubiquitous Era,: 13-17.
6. Chanel, G., Rebetez, C., Bétrancourt, M. and Pun, R. 2009. Emotion assessment from physiological signals for adaptation of game difficulty. Systems, Man and Cybernetics, Part A: Systems and Humans,: 1-12.
7. Csikszentmihalyi, M. 1990. Flow: The psychology of optimal experience, New York, NY: Harper Perennial.
8. Derks, D., Bos, A. E. and von Grumbkow, J. 2008. Emoticons and online message interpretation. Social Science Computer Review, 26: 379-388.
9. El Kaliouby, R. and Robinson, P. 2004. FAIM: Integrating automated facial affect analysis in instant messaging. Proceedings of the Ninth International Conference on Intelligent User Interfaces,: 244-246.
10. Fabri, M. 2006. Emotionally expressive avatars for collaborative virtual environments (Unpublished doctoral thesis), Leeds,UK: Leeds Metropolitan University.
11. Fabri, M. and Moore, D. 2005. Is empathy the key? Effective communication via instant messaging. Proceedings of the Eleventh International Conference on Networking Entities.
12. Gibert, G., Pruzinec, M., Schultz, T. and Stevens, C. 2009. Enhancement of human computer interaction with facial electromyographic sensors. Proceedings of the Twenty-First Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7,: 421-424.
13. Gilleade, K., Dix, A. and Allanson, J. 2005. Affective videogames and modes of affective gaming: Assist me, challenge me, emote me. Proceedings of the Digital Games Research Association: Changing Views-Worlds in Play, June 16-20, 2005, Vancouver, Canada.
14. Gürkök, H. and Nijholt, A. 2012. Brain-computer interfaces for multimodal interaction: A survey and principles. International Journal of Human-Computer Interaction, 28: 292-307.
15. Holzapfel, H., Fuegen, C., Denecke, M. and Waibel, A. 2002. Integrating emotional cues into a framework for dialogue management. Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces,: 141-148.
16. Khalili, Z. and Moradi, M. H. 2009. Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of EEG. Proceedings of the 2009 International Joint Conference on Neural Networks,: 1571-1575.
17. Kim, K. H., Bang, S. W. and Kim, S. R. 2004. Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42: 419-427.
18. Ko, M., Bae, K., Oh, G. and Ryu, T. 2009. A study on new gameplay based on brain-computer interface. Proceedings of the Digital Games Research Association: Breaking New Ground: Innovation in Games, Play, Practice and Theory, September 1-4, 2009, London, UK.
19. Lewis, C. and Fabos, B. 2005. Instant messaging, literacies, and social identities. Reading Research Quarterly, 40: 470-501.
20. Li, Y., Zhang, J., Su, Y., Chen, W., Qi, Y., Zhang, J. and Zheng, X. 2009. P300 based BCI messenger. Proceedings of the International Conference on Complex Medical Engineering, 1-5.,
21. Lievesley, R., Wozencroft, M. and Ewins, D. 2011. The Emotiv EPOC neuroheadset: an inexpensive method of controlling assistive technologies using facial expressions and thoughts?. Journal of Assistive Technologies, 5: 67-82.
22. Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F. and Arnaldi, B. 2007. A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering, 4: R1-R13.
23. Luor, T., Lu, H. P., Wu, L. L. and Tao, Y. H. 2010. The effect of emoticon in simplex and complex task-oriented communication-An empirical study of instant messaging. Computers in Human Behavior, 26: 889-895.
24. Mandryk, R. L. and Atkins, M. S. 2007. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies, 65: 329-347.
25. McFarland, D. J. and Wolpaw, J. R. 2008. Brain-computer interface operation of robotic and prosthetic devices. Computer, 41(10): 48-52.
26. Morris DiMicco, J., Lakshmipathy, V. and Fiore, A. T. 2002. Conductive chat: Instant messaging with a skin conductivity channel. Proceedings of the Conference on Computer Supported Cooperative Work. pp. 193-194.
27. Mugler, E., Bensch, M., Halder, S., Rosenstiel, W., Bogdan, M., Birbaumer, N. and Kübler, A. 2008. Control of an Internet browser using the P300 event-related potential. International Journal of Bioelectromagnetism, 10: 56-63.
28. Nam, C. S., Li, Y. and Johnson, S. 2010. Evaluation of P300-based brain-computer interface (BCI) in real-world contexts. International Journal of Human-Computer Interaction, 26: 621-637.
29. Nam, C. S., Schalk, G. and Jackson, M. M. 2010. Current trends in brain computer interface (BCI) research and development. International Journal of Human-Computer Interaction, 27: 1-4.
30. Neviarouskaya, A. 2008. AffectIM: An avatar-based instant messaging system employing rule-based sensing, Tokyo,Japan: University of Tokyo. doctoral thesis
31. Neviarouskaya, A., Prendinger, H. and Ishizuka, M. 2007. Recognition of affect conveyed by text messaging in online communication. Proceedings of the 2nd International Conference on Online Communities and Social Computing,: 141-150.
32. Nijboer, F., Hoffmann, U., Morin, F., Carmien, S., Leon, E. and Koene, R. 2009. Affective brain-computer interfaces: Psychophysiological markers of emotion in healthy persons and in persons with Amyotrophic Lateral Sclerosis. Workshop on Affective Brain-Computer Interfaces and IEEE International Workshop on Social Signal Processing,: 1-11.
33. Nijboer, F., Sellers, E., Mellinger, J., Jordan, M., Matuz, T., Furdea, A,and Kubler, A. 2008. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clinical Neurophysiology, 19: 1909-1916.
34. Nijholt, A. 2009. BCI for games: A "state of the art" survey. Proceedings of the Seventh International Conference on Entertainment Computing,: 225-228.
35. Oakley, I. and O'Modhrain, M. S. 2002. ContactIM: Exploring asynchronous touch over distance. Proceedings of the Conference on Computer Supported Cooperative Work, November 16-20, 2002, New Orleans, LA.
36. Pennebaker, J. W., Booth, R. and Francis, M. 2007. Linguistic inquiry and word count: LIWC2007 operator's manual, University of Texas, Austin, Texas.
37. Petrantonakis, P. C. and Hadjileontiadis, L. J. 2009. On modeling user's EEG response during a human-computer interaction: A mirror neuron system-based approach. Proceedings of the Fourth European Conference of the International Federation for Medical and Biological Engineering,: 1241-1245.
38. Prendinger, H. and Ishizuka, M. 2005. The empathic companion: A character-based interface that addresses users' affective states. International Journal of Applied Artificial Intelligence, 19: 267-285.
39. Rebolledo-Mendez, G., Dunwell, I., Martinez-Miron, E., Vargas-Cerdan, M., Freitas, S., Liarokapis, F. and Garcia-Gaona, A. 2009. Assessing Neurosky's usability to detect attention levels in an assessment exercise. Proceedings of the Thirteenth International Conference on Human-Computer Interaction,: 149-158.
40. Rebsamen, B., Guan, C., Zhang, H., Wang, C., Teo, C. L., Ang, M. and Burdet, E. 2010. A brain-controlled wheelchair to navigate in familiar environments. Transactions on Neural Systems and Rehabilitation Engineering, 18: 590-598.
41. Sellers, E. W., Vaughan, T. M. and Wolpaw, J. R. 2010. A brain-computer interface for long-term independent home use. Amyotrophic Lateral Sclerosis, 11: 449-455.
42. Smith, R. W. and Gordon, S. A. 1996. Pragmatic issues in handling miscommunication: Observations of a spoken natural language dialog system. Workshop on Detecting, Repairing, and Preventing Human-Machine Miscommunication,: 21-28.
43. Takahashi, K. 2004. Remarks on emotion recognition from bio-potential signals. Proceedings of the Second International Conference on Autonomous Robots and Agents,: 186-190.
44. Tan, D. and Nijholt, A. 2010. "Chapter 1: Brain-computer interfaces and human-computer interaction". In Brain-computer interfaces, Edited by: Tan, D. and Nijholt, A. 3-19. London,UK: Springer Verlag.
45. Thompson, E. R. 2007. Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). Journal of Cross-Cultural Psychology, 38: 227-242.
46. Van Boxtel, A. 2010. Facial EMG as a tool for inferring affective states. Proceedings of Seventh Edition of the International Conference on Methods and Techniques in Behavioral Research,: 104-108.
47. Voida, A., Newstetter, W. C. and Mynatt, E. D. 2002. When conventions collide: The tensions of instant messaging attributed. Proceedings of the Conference of Human Factors in Computing Systems,: 187-194.
48. Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G. and Vaughan, T. M. 2002. Brain-computer interfaces for communication and control. Clinical Neurophysiology, 113: 767-791.
49. Yeo, Z. 2008. Emotional instant messaging with KIM. Extended Abstracts on Human Factors in Computing Systems,: 3729-3734.
50. Zander, T. O., Kothe, C., Welke, S. and Roetting, M. 2008. "Enhancing human-machine systems with secondary input from passive brain-computer interfaces". In Brain-computer interfaces, Edited by: Tan, D. and Nijholt, A. 181-199. London,UK: Springer.