Gaze-X: Adaptive, affective, multimodal interface for single-user office scenarios

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4451 LNAI, Issue , 2007, Pages 251-271

  Maat, Ludo ^a   Pantic, Maja ^b,c  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Affective computing; Anthropocentric interface design; Context sensing; Human sensing; Multimodal interfaces

Indexed keywords

ARTIFICIAL INTELLIGENCE; DECISION MAKING; ELECTRIC CURRENTS; EYE MOVEMENTS; FLOW INTERACTIONS; HUMAN ENGINEERING; INFORMATION MANAGEMENT; INTELLIGENT SYSTEMS; KNOWLEDGE MANAGEMENT; PROBLEM SOLVING; STANDARDS;

AFFECTIVE COMPUTING; ANTHROPOCENTRIC INTERFACE DESIGN; CONTEXT SENSING; EYE GAZE; FACIAL EXPRESSIONS; HUMAN COMPUTING; HUMAN SENSING; HYDERABAD , INDIA; INTERNATIONAL WORKSHOPS; JOINT CONFERENCE; MOUSE MOVEMENTS; MULTIMODAL HUMAN-COMPUTER INTERACTION; MULTIMODAL INTERFACES; SINGLE-USER; USABILITY STUDY;

HUMAN COMPUTER INTERACTION;

EID: 37848999283     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-72348-6_13     Document Type: Conference Paper

References (56)

1. Bartsch-Sporl, B., Lez, M., Hubner, A.: Case-based reasoning - survey and future directions. Lecture Notes in Artificial Intelligence, Vol. 1570 (1999) 67-89
2. Bianchi-Berthouze, N., Lisetti, C.L.: Modeling multimodal expression of user's affective subjective experience. User Modeling and User-Adapted Interaction, Vol. 12, No. 1 (2002) 49-84
3. Bolt, R.A.: Put-that-there: Voice and gesture at the graphics interface. Computer Graphics, Vol. 14, No. 3 (1980) 262-270 (Proc. ACM SIGGRAPH'80)
4. Browne, D., Norman, M., Riches, D.: Why Build Adaptive Interfaces? In: Browne, D., Totterdell, P., Norman, M. (eds.): Adaptive User Interfaces. Academic Press, London, UK (1990) 15-57
5. Caridakis, G., Malatesta, L., Kessous, L., Amir, N., Raouzaiou, A., Karpouzis, K.: Modeling Naturalistic Affective States via Facial and Vocal Expressions Recognition. Proc. Int'l Conf. Multimodal Interfaces (2006) 146-154
6. Cohen, P., Oviatt, S.L.: The role of voice input for human-machine communication. Proc. National Academy of Sciences, Vol. 92 (1995) 9921-9927
7. Conati, C.: Probabilistic assessment of user's emotions in educational games. Applied Artificial Intelligence, Vol. 16, No. 7-8 (2002) 555-575
8. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 23, No. 6 (2001) 681-685
9. Deng, B.L., Huang, X.: Challenges in adopting speech recognition. Communications of the ACM, Vol. 47, No. 1 (2004) 69-75
10. Dey, A.K., Abowd, G.D., Salber, D.: A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. J. Human-Computer Interaction, Vol. 16, No. 2-4 (2001) 97-166
11. Duric, Z., Gray, W.D., Heishman, R., Li, F., Rosenfeld, A., Schoelles, M.J., Schunn, C., Wechsler, H.: Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction. Proceedings of the IEEE, Vol. 90, No. 7 (2002) 1272-1289
12. El Kaliouby, R., Robinson, P.: Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures. Proc. Int'l Conf. Computer Vision & Pattern Recognition, Vol. 3 (2004) 154-
13. Gu, H., Ji, Q.: An automated face reader for fatigue detection. Proc. Int'l Conf. Face & Gesture Recognition (2006) 111-116
14. Gunes, H., Piccardi, M.: Affect Recognition from Face and Body: Early Fusion vs. Late Fusion. Proc. Int'l Conf. Systems, Man and Cybernetics (2005) 3437-3443
15. Hauptmann, A.G.: Speech and gestures for graphic image manipulation. Proc. ACM Int'l Conf. Human Factors in Computing Systems (1989) 241-245
16. Hoffman, D.L., Novak, T.P., Venkatesh, A.: Has the Internet become indispensable? Communications of the ACM, Vol. 47, No. 7 (2004) 37-42
17. Hudlicka, E.: To feel or not to feel: The role of affect in human-computer interaction. Int'l J. Human-Computer Studies, Vol. 59, No. 1-2 (2003) 1-32
18. Hudlicka, E., McNeese, M.D.: Assessment of user affective/belief states for interface adaptation. User Modeling & User-Adapted Interaction, Vol. 12, No. 1 (2002) 1-47
19. Jaimes, A., Sebe, N.: Multimodal human computer interaction: A survey. Proc. IEEE ICCV Int'l Workshop on HCI in conjunction with Int'l Conf. Computer Vision (2005)
20. Kapoor, A., Picard, R.W.: Multimodal affect recognition in learning environments. Proc. ACM Int'l Conf. Multimedia (2005) 677-682
21. Keltner, D., Ekman, P.: Facial expression of emotion. In: Lewis, M., Haviland-Jones, J.M. (eds.): Handbook of Emotions. The Guilford Press, New York (2000) 236-249
22. van Kuilenburg, H., Wiering, M., den Uyl, M.: A model-based method for automatic facial expression recognition. Lecture Notes in Artificial Intelligence, Vol. 3720 (2005) 194-205
23. Lisetti, C.L., Nasoz, F.: MAUI: A multimodal affective user interface. Proc. Int'l Conf. Multimedia (2002) 161-170
24. Lisetti, C.L., Schiano, D.J.: Automatic facial expression interpretation: Where human-computer interaction, AI and cognitive science intersect. Pragmatics and Cognition, Vol. 8, No. 1(2000)185-235
25. Maat, L., Pantic, M.: Gaze-X: Adaptive affective multimodal interface for single-user office scenarios. Proc. Int'l Conf. Multimodal Interfaces (2006) 171-178
26. Maglio, P.P., Barrett, R., Campbell, C.S., Selker, T.T.: SUITOR: An attentive information system. Proc. Int'l Cof. Intelligent User Interfaces (2000) 169-176
27. Marsic, I., Medl, A., Flanagan, J.: Natural communication with information systems. Proceedings of the IEEE, Vol. 88, No. 8 (2000) 1354-1366
28. Nielsen, J.: Multimedia and hypertext: The Internet and beyond. Academic Press, Cambridge, USA (1995)
29. Nock, H.J., Iyengar, G., Neti, C.: Multimodal processing by finding common cause. Communications of the ACM, Vol. 47, No. 1 (2004) 51-56
30. Oviatt, S. User-centred modelling and evaluation of multimodal interfaces. Proceedings of the IEEE, Vol. 91, No. 9 (2003) 1457-1468
31. Oviatt, S., Darrell, T., Flickner, M.: Multimodal Interfaces that Flex, Adapt, and Persist. Communications of the ACM, Vol. 47, No. 1 (2004) 30-33
32. Pantic, M., Grootjans, R.J., Zwitserloot, R.: Teaching Ad-hoc Networks using a Simple Agent Framework. Proc. Int'l Conf. Information Technology Based Higher Education and Training (2005) 6-11
33. Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behaviour: A survey. Proc. Int'l Conf. Multimodal Interfaces (2006) 239-248
34. Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal human-computer interaction. Proceedings of the IEEE, Vol. 91, No. 9 (2003) 1370-1390
35. Pantic, M., Rothkrantz, L.J.M.: Case-based reasoning for user-profiled recognition of emotions from face images. Proc. Int'l Conf. Multimedia and Expo (2004) 391-394
36. Pantic, M., Sebe, N., Cohn, J.F., Huang, T.S.: Affective multimodal human-computer interaction. Proc. ACM Int'l Conf. Multimedia (2005) 669-676
37. Pentland, A.: Looking at people: Sensing for ubiquitous and wearable computing. IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 22, No. 1 (2000) 107-119
38. Pentland, A.: Perceptual intelligence. Communications of the ACM, Vol. 43, No. 3 (2000) 35-44
39. Picard, R.W.: Affective Computing. The MIT Press, Cambridge, USA (1997)
40. Porta, M.: Vision-based user interfaces: methods and applications. Int'l J. Human-Computer Studies, Vol. 57, No. 1 (2002) 27-73
41. Preece, J., Schneiderman, B.: Survival of the fittest: Evolution of multimedia user interfaces. ACM Computing Surveys, Vol. 27, No. 4 (1995) 557-559
42. Prendinger, H., Ishizuka, M.: The empathic companion: A character-based interface that addresses users' affective states. Applied Artificial Intelligence, Vol. 19, No. 3-4 (2005) 267-285
43. Reeves, L.M., Lai, J., Larson, J.A., Oviatt, S., Balaji, T.S., Buisine, S., Collings, P., Cohen, P., Kraal, B., Martin, J.C., McTear, M., Raman, T.V., Stanney, K.M., Su, H., Wang, Q.Y.: Guidelines for multimodal user interface design. Communications of the ACM, Vol. 47, No. 1 (2004) 57-59
44. Ruttkay, Z., Pelachaud, C. (eds.): From brows to trust: Evaluating embodied conversational agents. Kluwer Academic Publishers, Norwell, USA (2004)
45. Schank, R.C.: Memory based expert systems. AFOSR.TR. 84-0814, Comp. Science Dept., Yale University (1984)
46. Sharma, R., Yeasin, M., Krahnstoever, N., Rauschert, I., Cai, G., Maceachren, A.M., Sengupta, K.: Speech-gesture driven multimodal interfaces for crisis management. Proceedings of the IEEE, Vol. 91, No. 9 (2003) 1327-1354
47. nd edition. John Wiley & Sons, Chichester, UK (2007)
48. Shneiderman, B.: Universal usability. Communications of the ACM, Vol. 43, No. 5 (2000) 85-91
49. Shneiderman, B.: CUU: Bridging the Digital Divide with Universal Usability. ACM Interactions, Vol. 8, No. 2 (2001) 11-15
50. Shoham, Y.: What we talk about when we talk about software agents. IEEE Intelligent Systems and Their Applications, Vol. 14, No. 2 (1999) 28-31
51. Tennenhouse, D.: Proactive computing. Communications of the ACM, Vol. 43, No. 5 (2000) 43-50
52. Turk, M.: Computer vision in the interface. Communications of the ACM, Vol. 47, No. 1 (2004) 61-67
53. Vo, M.T., Waibel, A.: Multimodal human-computer interaction. Proc. Int'l Symposium on Spoken Dialogue (1993)
54. Waibel, A., Vo, M.T., Duchnowski, P., Manke, S.: Multimodal Interfaces. Artificial Intelligence Review, Vol. 10, No. 3-4 (1995) 299-319
55. Zeng, Z., Hu, Y., Roisman, G.I., Wen, Z., Fu, Y., Huang, T.S.: Audio-Visual Emotion Recognition in Adult Attachment Interview. Proc. Int'l Conf. Multimodal Interfaces (2006) 139-145
56. Zhang, P., Li, N.: The importance of affective quality. Communications of the ACM, Vol. 48, No. 9 (2005) 105-108