E-learning and machine perception: in pursuit of human-like interaction in computer-based teaching systems

International Journal of Knowledge and Learning

Volumn 3, Issue 2-3, 2007, Pages 281-298

  Porta, Marco ^a  

^a UNIVERSITY OF PAVIA   (Italy)

Author keywords

emotion recognition; gesture recognition; human like interaction; machine perception; multimodal interfaces; natural interaction; perceptive e learning; perceptive interfaces; speech based interfaces; vision based interfaces

Indexed keywords

EID: 51849147270     PISSN: 17411009     EISSN: 17411017     Source Type: Journal    
DOI: 10.1504/ijkl.2007.015556     Document Type: Article

References (56)

1. Ahmad, A., Basir, O. and Hassanein, K. (2004) ‘Adaptive user interfaces for intelligent e-learning: issues and trends’, Proceedings of the Fourth International Conference on Electronic Business (ICEB 2004), 5–9 December, Beijing, China.
2. Andreassi, J.L. (2000) Psychophysiology: Human Behavior and Physiological Response, 4th edition, LEA, Inc.
3. Ang, J., Dhillon, R., Krupski, A., Shriberg, E. and Stolcke, A. (2002) ‘Prosody-based automatic detection of annoyance and frustration in human-computer dialog’, Proceedings of ICSLP-2002, Denver, Colorado, USA, September.
4. Bérard, F. (1999) ‘The perceptual window: head motion as a new input stream’, Proceedings of the 7th IFIP Conference on Human-Computer Interaction (INTERACT), Edinburgh, Scotland, pp.238–244.
5. Borghi, F., Lombardi, L. and Porta, M. (2005) ‘Basic hand gesture recognition for human-computer communication’, Proceedings of the 11th International Conference on Human-Computer Interaction, Las Vegas, Nevada, USA, 22–27 July, LEA Publ.
6. Buchholz, W.J. (1983) ‘Dissolving the granite complex’, in W. Buchholz (Ed). Communication Training and Consulting in Business, Industry, and Government, Urbana-Champaign, IL, USA: Association for Business Communication (ABC), University of Illinois.
7. Busso, C., Zhigang, D., Yildirim, S., Bulut, M., Min Lee, C., Kazemzadeh, A., Lee, S., Neumann, U. and Narayanan, S. (2004) ‘Analysis of emotion recognition using facial expressions, speech and multimodal information’, Proceedings of ACM 16th International Conference on Multimodal Interfaces (ICMI 2004), State College, PA, USA, October.
8. Cantoni, V., Cellario, M. and Porta, M. (2004) ‘Perspectives and challenges in e-learning: towards natural interaction paradigms’, Journal of Visual Languages and Computing, Elsevier Science Publishing, Vol. 15, pp.333–345.
9. Clark, R. (2003) ‘More than just eye candy: graphics for e-learning’, The e-Learning Developers’ Journal, 11 August.
10. Cole, R., Van Vuurel, S., Pellom, B., Hacioglu, K., Ma, J., Movellan, J., Schwartz, S., Wade-Stein, D., Ward, W. and Yan, J. (2003) ‘Perceptive animated interfaces: first steps toward a new paradigm for human-computer interaction’, Proceedings of the IEEE, Vol. 91, No. 9, pp.1391–1405.
11. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W. and Taylor, J.G. (2001) ‘Emotion recognition in human-computer interaction’, IEEE Signal Processing Magazine, Vol. 18, No. 1, pp.32–80.
12. Doswell, J.T. (2004) ‘Pedagogical embodied conversational agent’, Proceedings of the IEEE International Conference on Advanced Learning Technologies (ICALT’04), 30 August-1 September, pp.774–776.
13. Ekman, P. and Friesen, W.V. (1978) Facial Action Coding System (FACS): Manual, Palo Alto, CA, USA: Consulting Psychologists Press.
14. Fitzpatrick, T. (2004) ‘The eyes have it: Candidates’ eyes could be revealing’, Washington University in St. Louis News & Information, University News, Available at: http://news-info.wustl.edu/news/page/normal/4018.html.
15. Gladwell, M. (2005) Blink: The Power of Thinking Without Thinking, Little: Brown and Company.
16. Goleman, D. (1996) Emotional intelligence: Why it Can Matter More Than IQ, London: Boomsbury.
17. Grauman, K., Betke, M., Gips, J. and Bradski, G.R. (2001) ‘Communication via eye blinks - detection and duration analysis in real time’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Lihue, HI, December.
18. Gütl, C., Pivec, M., Trummer, C., García-Barrios, V.M., Mödritscher, F., Pripfl, J. and Umgeher, M. (2005) ‘AdeLE (Adaptive e-Learning with Eye-Tracking): theoretical background, system architecture and application scenarios’, European Journal of Open, Distance and E-Learning (EURODL), Vol. II, Available at: http://www.eurodl.org/materials/contrib/2005/Christian_Gutl.htm. Accessed on 27 September 2005.
19. Haag, A., Goronzy, S., Schaich, P. and Williams, J. (2004) ‘Emotion recognition using bio-sensors: first steps towards an automatic system’, Proceedings of Tutorial and Research Workshop on Affective Dialogue Systems, 14–16 June, Kloster Irsee, Germany.
20. Hedge, N. and MacDonald, D. (2005) ‘Affective considerations in distance e-learning’, Proceedings of the IASTED Conference on Education and Technology (ICET 2005), Calgary, Alberta, Canada, 4–6 July.
21. Hess, U. (2001) ‘The communication of emotions’, in A. Kaszniak (Ed). Emotions, Qualia, and Consciousness, Singapore: World Scientific, pp.397–409.
22. Ikehara, C.S. and Crosby, M.E. (2005) ‘Assessing cognitive load with physiological sensors’, Proceedings of the 38th Hawaii International Conference on System Sciences, 3–6 January, Big Island, HI, USA: IEEE Computer Society.
23. Jacob, R.J.K. (1991) ‘The use of eye movements in human-computer interaction techniques: what you look at is what you get’, ACM Transactions on Information Systems, Vol. 9, pp.152–169.
24. Jain, A.K. and Ross, A. (2004) ‘Multibiometric systems’, Communications of the ACM, Vol. 47, No. 1, pp.34–40.
25. Kapoor, A. and Picard, R.W. (2005) ‘Multimodal affect recognition in learning environments’, Proceedings of ACM International Conference on Multimedia (MM ’05), 6-11 November, Singapore.
26. Kirsh, D. (2002) ‘E-learning, metacognition and visual design’, Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet, L'Aquila, Italy, 29 July-4 August.
27. Lai, J. (2000) ‘Conversational interfaces’, Communications of the ACM, Vol. 43, No. 9, pp.24–27.
28. Langley, P. (1999) ‘User modeling in adaptive interfaces’, Proceedings of the 7th International Conference on User Modeling, 20–24 June, Banff, Canada.
29. Maglio, P.P. and Campbell, C.S. (2003) ‘Attentive agents’, Communications of the ACM, Vol. 46, No. 3, pp.47–51.
30. Mota, S. and Picard, R.W. (2003) ‘Automated posture analysis for detecting learner’s interest level’, Proceedings of Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction (CVPR HCI), June.
31. Nakayama, M. and Shimizu, Y. (2004) ‘Frequency analysis of task evoked pupillary response and eye-movement’, Proceedings of the 2004 Symposium on Eye Tracking Research and Applications (ETRA 2004), San Antonio, TX, USA, pp.71–76.
32. Nasoz, F., Alavrez, K., Lisetti, C.L. and Finkelstein, N. (2003) ‘Emotion recognition from physiological signals for presence technologies’, International Journal of Cognition, Technology, and Work – Special Issue on Presence, Vol. 6, No. 1.
33. Ong, S.C.W. and Ranganath, S. (2005) ‘Automatic sign language analysis: a survey and the future beyond lexical meaning’, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 6, pp.873–891.
34. Oviatt, S. (1999) ‘Ten myths of multimodal interaction’, Communications of the ACM, Vol. 42, No. 11, pp.74–81.
35. Pantic, M. and Rothkrantz, L.J.M. (2000) ‘Automatic analysis of facial expressions: the state of the art’, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 12, pp.1424–1445.
36. Pantic, M. and Rothkrantz, L.J.M. (2003) ‘Toward an affect-sensitive multimodal human-computer interaction’, Proceedings of the IEEE, Vol. 91.
37. Pentland, A. (2000a) ‘Looking at people: sensing for ubiquitous and wearable computing’, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 22, No. 1, pp.107–119.
38. Pentland, A. (2000b) ‘Perceptual intelligence’, Communications of the ACM, Vol. 43, pp.35–44.
39. Pentland, A. (2004) ‘Social dynamics: signals and behavior’, Proceedings of IEEE International Conference on Developmental Learning (ICDL), 20–22 October, San Diego, CA, USA.
40. Picard, R.W. (2001) ‘Building HAL: computers that sense, recognize, and respond to human emotion’, Society of Photo-Optical Instrumentation Engineers. Human Vision and Electronic Imaging, Vol. VI, part of IS&T/SPIE9s Photonic West.
41. Picard, R.W. and Klein, J. (2002) ‘Computers that recognize and respond to user emotions: theoretical and practical implications’, MIT Media Lab Tech Report 538.
42. Picard, R.W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover, T., Resnick, M., Roy, D. and Strohecker, C. (2004) ‘Affective learning - A manifesto’, BT Technology Journal, Vol. 22, No. 4, pp.253–269.
43. Porta, M. (2002) ‘Vision-based user interfaces: methods and applications’, International Journal of Human-Computer Studies, No. 57, pp.27–73.
44. Quinn, C.N. (2006) ‘Making it matter to the learner: e-motional e-learning’, Learning Solutions e-Magazine, 3 April.
45. Rist, T., André, E. and Muller, J. (1997) ‘Adding animated presentation agents to the interface’, Proceedings of the International Conference on Intelligent User Interfaces, Orlando, FL, USA, pp.79–86.
46. Sebe, N., Cohen, I. and Huang, T.S. (2005) ‘Multimodal emotion recognition’, Handbook of Pattern Recognition and Computer Vision, World Scientific, ISBN 981-256-105-6, January.
47. Takács, B. and Kiss, B. (2003) ‘The virtual human interface: a photorealistic digital human’, IEEE Computer Graphics and Applications, September/October, pp.38–45.
48. Toyama, K. (1998) ‘Look, Ma-no hands! hands-free cursor control with real-time 3D face tracking’, Proceedings of the Workshop on Perceptual User Interfaces (PUI ’98), San Francisco, CA, USA.
49. Turk, M. (1998) ‘Moving from GUIs to PUIs’, Technical Report MSR-TR-98-69 (Microsoft Research).
50. Turk, M. and Robertson, G. (2000) ‘Perceptual user interfaces’, Communications of the ACM, Vol. 43, No. 3, pp.33–34.
51. Ueno, M. (2005) ‘Animated pedagogical agent based on decision tree for e-learning’, Proceedings of the 5th IEEE International Conference on Advanced Learning Technologies (ICALT’05), 5–8 July, pp.188–192.
52. Vertegaal, R. (2003) ‘Attentive user interfaces’, Communications of the ACM, Vol. 46, No. 3, pp.30–33.
53. Zhai, S., Morimoto, C. and Ihde, S. (1999) ‘Manual and gaze input cascaded (MAGIC) pointing’, Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’99), Pittsburgh, 15–20 May, pp.246–253.
54. Zhang, Q., Imamiya, A., Go, K. and Mao, X. (2004) ‘A gaze and speech multimodal interface’, Proceedings of the 24th IEEE International Conference on Distributed Computing Systems Workshops (ICDCSW’04).
55. Zhang, Y., Silber, G. and Kambhamettu, C. (2002) ‘Facial expression driven tutorial system’, Proceedings of the 6th World Multi-conference on Systemics, Cybernetics and Informatics, Orlando, FL, July, Vol. IX, pp.287–292.
56. Zhao, W., Chellappa, R., Phillips, P.J. and Rosenfeld, A. (2003) ‘Face recognition: a literature survey’, ACM Computing Survey, Vol. 35, No. 4, pp.399–458.