Evaluating usability based on multimodal information: An empirical study

ICMI'06: 8th International Conference on Multimodal Interfaces, Conference Proceeding

Volumn , Issue , 2006, Pages 364-371

  Lin, Tao ^a   Imamiya, Atsumi ^a  

^a UNIVERSITY OF YAMANASHI   (Japan)

Author keywords

Eye tracking; Multimodal; Physiological measures; Usability

Indexed keywords

CORRELATION METHODS; INFORMATION ANALYSIS; INTERACTIVE COMPUTER SYSTEMS; PATTERN RECOGNITION; TRACKING (POSITION);

EYE TRACKING; MULTIMODAL INFORMATION; PHYSIOLOGICAL MEASURES;

USABILITY ENGINEERING;

EID: 34547184412     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1180995.1181063     Document Type: Conference Paper

References (47)

1. Hollnagel, E. From human factors to cognitive systems engineering: Human-machine interactions in the 21st century. In Kitamura, M., and Kimura, I. (eds.), Anzen-no-Tankyu (Researches on Safety). ERCPublishing, Tokyo, 2003.
2. Andreassi, J.L. Psychophysiology: Human Behavior and Physiological Response, 4th Edition. Lawrence Erlbaum Associates, Mahwah, NJ, 2000.
3. Picard, R.W. Affective Computing. MIT Press, Cambridge, MA, 1997.
4. Kramer, A.F. Physiological metrics of mental workload: A review of recent progress. In Damos, D.L. (ed.), Multiple-Task Performance, Taylor and Francis, London, 1991, 279-328.
5. Rowe, D.W., Sibert, J. and Irwin, D. Heart rate variability: Indicator of user state as an aid to human-computer interaction. In Proceeding of CHI1998, 480-487.1998.
6. Takahashi, K., Nakayama, M. and Shimizu, Y. The response of eye-movement and pupil size to audio instruction while viewing a moving target. In Proceedings of the Eye Tracking Research & Applications 2000.
7. Hopman, J.C.W., Kollee, L.A.A., Stoelinga, G.B.A., van Geijn, H.P. and van Ravenswaaij-Arts, C.M.A. Heart rate variability. Annals of Internal Medicine 1993, 118, 436-447.
8. Malik, M. Heart rate variability. Circulation 1996, 93, 1043-1065.
9. Pomeranz, B., and Macaulay, R.J.B., Caudill, M.A., et al. Assessment of autonomic function in humans by heart rate spectral analysis. Am J Physiol Heart Circ Physiol 1985, 248, 151-H153.
10. Pagani, M., Lombardi, F., Guzzetti, S., et al. Power spectral analysis of heart rate and arterial pressure variabilities as a maker of sympatho-vagal interaction in man and conscious dog. Cir Res 1986, 59, 178-93.
11. Ettema, J.H. and Ziclhuis, R.L. Physiological parameters of mental load. Ergonomics 1971, 14, 137-44.
12. Luczak, I.I. and Lauring, W.J. An analysis of heart rate variability. Ergonomics 1973, 16, 85-97.
13. Hyndman, B.W., and Gregory, J.R. Spectral analysis of sinus arrhythmia during mental loading. Ergonomics 1975, 18, 255-70.
14. Boutcher, S.H., Naugent, F.W., and Melaren, P.F., et al. Heart period variability of trained and untrained men at rest and during mental challenge. Psychophysiology 1998, 35, 16-22.
15. Hyde, C., and Izard, C.E. Cardiac rhythmicities and attention in young children. Psychophysiology 1997, 34, 547-52.
16. Vicente, K.J., Thornton, C., and Moray, N. Spectral analysis of sinus arrhythmia: A measure of mental effort. Human Factors 1987, 29, 171-182.
17. Tattersall, A.J., and Hockey, G.R J. Level of operator control and changes in heart rate variability during simulated flight maintenance. Human Factors 1995, 37, 682-698.
18. Hess, E.H. and Polt, J.M. Pupil size in relation to mental activity during simple problem solving. Science 1964, 132, 1190-1192.
19. Hoecks, B., and Levelt, W. Pupillary dilation as a measure of attention: A quantitative system analysis. Behavior Research Methods, Instruments, & Computers 1993, 25, 16-26.
20. Juris, M. and Velden, M. The pupillary response to mental overload. Physiological Psychology 1977, 5 (4), 421-424.
21. Kahneman, D. Pupillary responses in a pitch-discrimination task. Perception & Psychophysics 1967, 2, 101-105.
22. Nakayama, M., and Takahashi, K. The act of task difficulty and eye-movement frequency for the oculo-motor indices. In Proceedings of the Eye Tracking Research and Application 2002, 37-42.
23. Beatty, J. Task-evoked pupillary responses, processing load and the structure of processing resources. Psychological Bulletin 1982, 91 (2), 276-292.
24. Iqbal, S.T., Zheng, X.S., and Bailey, B.P. Task-evoked pupillary response to mental workload in human-computer interaction. In Proceeding of CHI 2004, ACM Press, 1477-1480.
25. Hess, E.H., and Petrovich, S.B. Pupillary behavior in communication. In Siegman, A.W., and Feldstein, S. (eds.), Nonverbal Behavior and Communication. Erlbaum, Hillsdale, NJ, 1987, 327-348.
26. Rayner, K. Eye movements and information processing: 20 years of research. Psychological Bulletin 1998, 124 (3), 372-422.
27. Noton, D., and Stark, L.W. Scanpath in saccadic eye movements while viewing and recognizing patterns. Vision Research 1971a, 11, 929-942.
28. Goldberg, H., and Kotval, X.P. Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics 1999, 24, 631-645.
29. Goldberg, J.H., Stimson, MJ, Lewenstein, M., Scott, N. and Wichansky, A. M. Eye tracking in web search tasks: Design implications. In Proceedings of the Symposium on ETRA 2002, March 2002.
30. Ellis, S., Candera, R., Misner, J., C.S., Lankford, C.P., and Hutshinson, T.E. Windows to the soul? What eye movements tell us about software usability? In Proceedings of the Usability Professionals' Association Conference, 1998, 151-178.
31. Cowen, L., Ball, L.J., and Delin, J. An eye movement analysis of web page usability. In Proceedings of the Annual HCI, 2002, 317-335
32. Altonen, A. Hyrskykari, A., and Räihä, K. 101 spots, or how do users read menus? In Proceedings of CHI 98 Human Factors in Computing Systems, ACM Press, 1998, 132-139.
33. Crosby, M.E., and Peterson, W.W. Using eye movements to classify search strategies. In Proceedings of the Human Factors Society 35th AnnualMeeting.1991, 1476-1480.
34. Andrews, T., and Coppola, D. Idiosyncratic characteristics of saccadic eye movements when viewing different visual environments. Vision Research 1999, 39, 2947-2953.
35. Lin, Y., Zhang, W.J., and Watson, L.G. Using eye movement parameters for evaluating human-machine interface frameworks under normal control operation and fault detection situation. International Journal of Human-Computer Studies 2003, 59 (6), 837-873.
36. Wickens, C.D., and Holland, J.G., Engineering Psychology and Human Performance, 3rd Edition. Longman, London, 2000.
37. Hébert. a,b,., Renée, a.b., Odrée Dionne-Fournelle. a.,. Martine Crête, a. and S. J. Lupien. Physiological stress response to video game playing: The contribution of built-in music. Life Sciences 2005, 76, 2371-2380.
38. Rubin, J. Handbook of Usability Testing. Wiley and Sons, New York, 1994.
39. Hart, S.G., and Staveland, L.E. Development of NASA-TLX (task load index): Results of experimental and theoretical research. In Hancock, P.A., and Meshakati, N. (eds.), Human Mental Workload. North-Holland, Amsterdam, 1988, 39-183.
40. Marshall, C., and Rossman, G.B. Designing Qualitative Research, 3rd Edition, Sage, Thousand Oaks, CA, 1999.
41. Wilson, G.M., and Sasse, M.A. Investigating the impact of audio degradations on users: Subjective vs. objective assessment methods. In Proceedings of OZCHI 2000: Interfacing Reality in the New Millennium, 2000, 135-142.
42. Wilson, G.M. Psychophysiological indicators of the impact of media quality on users. In Proceedings of CHI 2001 Doctoral Consortium, ACM Press, 2001, 95-96.
43. Ekman, P., Levenson, R.W., and Friesen, W.V. Autonomic nervous system activity distinguishes among emotions. Science 1983, 221 (4616), 1208-1210.
44. Hair, J.F., Anderson, R.E., Tatham, R.L., and Black, W.C. Multivariate Data Analysis, 5th Edition, Prentice-Hall, Englewood Cliffs, NJ, 1998.
45. Lin, T., Hu, W.H., Omata, M., and Imamiya, A. Do physiological data relate to traditional usability indexes? In Proceedings of the Australian Conference on Computer Human Interaction (OZCHI'05), 2005.
46. Meehan, M., Insko, B., Whitton, M., and Brooks, F. Physiological measures of presence in stressful virtual environments. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, 2002.
47. Lin, Y., Zhang, W. J. and Koubek, R. J. Effective Attention Allocation Behavior and its Measurement: A Preliminary Study, Interacting with Computers, 2004 Vol. 16, Issue 6, pp. 1195-1210.