Multimodal feedback as a solution to ocular disease-based user performance decrements in the absence of functional visual loss

International Journal of Human-Computer Interaction

Volumn 18, Issue 2, 2005, Pages 183-218

  Jacko, Julie A ^a   Moloney, Kevin P ^a   Kongnakorn, Thitima ^a   Barnard, Leon ^a   Edwards, Paula J ^a   Leonard, V Kathlene ^a   Sainfort, François ^a   Scott, Ingrid U ^b  

^a Georgia Institute of Technology   (United States)

^b UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; COMPUTER AIDED ANALYSIS; GRAPHICAL USER INTERFACES; HUMAN COMPUTER INTERACTION; PHOTODYNAMIC THERAPY; VISION;

AGE RELATED MACULAR DEGENERATION (AMD); OCULAR DISEASES; VISUAL ACUITY; VISUAL LOSS;

OPHTHALMOLOGY;

EID: 20444391749     PISSN: 10447318     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327590ijhc1802_4     Document Type: Article

References (84)

1. Akamatsu, M., MacKenzie, I. S., & Hasbroucq, T. (1995). A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics, 38, 816-827.
2. Alberti, W. E., Richard, G., & Sagerman, R. H. (2000). Age-related macular degeneration: Current treatment and concepts. New York: Springer-Verlag.
3. American Foundation for the Blind. (2000, October 16). Statistics and sources for professionals: Prevalence. Retrieved February 16, 2005, from http://www.afb.org/section.asp?documentid=1367
4. American Macular Degeneration Foundation. (2003, March 20). What is macular degeneration? Retrieved February 16, 2005, from http://www.macular.org/disease.html
5. Anderson, J. D. (1993). Sound and image together: Cross-modal confirmation. Wide Angle, 15, 30-43.
6. Bailey, I. L., & Lovie, J. E. (1976). New design principles for visual acuity letter charts. American Journal of Otometry & Physiological Optics, 53, 740-745.
7. Belz, S. M., & Robinson, G. M. (1999). A new class of auditory warning signals for complex systems: Auditory icons. Human Factors, 41, 608-618.
8. Biggs, S. J., & Srinivasan, M. A. (2002). Haptic interfaces. In K. M. Stanney (Ed.), Handbook of virtual environments: Design, implementation and applications (pp. 93-115). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
9. Biglan, A. W., Van Hasselt, V. B., & Simon, J. (1988). Visual impairment. In V. B. Van Hassek (Ed.),Handbook of developmental and physical disabilities (pp 471-502).NewYork:Pergamon.
10. Brewster, S. A. (1998). Sonically-enhanced drag and drop. Proceedings of the International Conference on Auditory Display (ICAD'98), Glasgow, Scotland, 1-7.
11. Brewster, S. A., & Crease, M. G. (1999). Correcting menu usability problems with sound. Behavior and Information Technology, 18, 165-177.
12. Brewster, S. A., Wright, P. C., & Edwards, A. D. N. (1994). The design and evaluation of an auditory-enhanced scrollbar. Proceedings of the 1994 ACM Conference on Human Factors in Computing Systems (CHI'94), Boston, 173-179.
13. Burdea, G. (1996). Force and touch feedback for virtual reality. New York: Wiley.
14. Center on an Aging Society. (2002, January 7). Visual impairments. Retrieved February 16, 2005, from http://ihcrp.georgetown.edu/agingsociety/pubhtml/visual/visual.html
15. Clemons, T. E., Chew, E. Y., Bressler, S. B., & McBee, W. (2003). National Eye Institute Visual Function Questionnaire in the Age-Related Eye Disease Study (AREDS): AREDS Report No. 10. Archives of Ophthalmology, 121, 211-217.
16. Diebert, E., Kraut, M., Kremen, S., & Hart, J. (1999). Neural pathways in tactile object recognition. Neurology, 52, 1413-1416.
17. Dix, A., Finlay, J., Abowd, G., & Beale, R. (1998). Human-computer interaction (2nd ed.). New York: Prentice Hall.
18. Driver, J. S., & Spence, C. J. (2000). Multisensory perception: Beyond modularity and convergence. Current Biology, 10, 731-735.
19. Dyson, B. J., & Quinlan, P. T. (2002). Within- and between-dimensional processing in the auditory modality. Journal of Experimental Psychology: Human Perception and Performance, 28, 1483-1498.
20. Eimer, M., & Driver, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch. Psychophysiology, 37, 697-705.
21. European Telecommunications Standards Institute [ETSI]. (2002). Human Factors (HF): Guidelines on the multimodality of icons, symbols, and pictograms (Rep. No. ETSI EG 202 048 v 1.1.1 (2002-08)). Sophia Antipolis, France: ETSI.
22. EyeMDLink.com. (2002, November 11). Photodynamic therapy for macular degeneration. Retrieved February 16, 2005, from http://www.eyemdlink.com/NewsArticle.asp? NewsID=39
23. Farnsworth, D. (1947). The Farnsworth Dichotomous Test for Color Blindness: Manual. New York: Psychological Corporation.
24. Farrell, F. E. (1991). Fitting physical screen parameters to the human eye. In J. A. J. Roufs (Ed.), Vision and visual dysfunction: The man-machine interface (Vol. 15; pp. 7-25). London: MacMillan Press.
25. Ferris, F. L., III, Kassoff, A., Bresnick, G. H., & Bailey, I. (1982). New visual acuity charts for clinical research. American Journal of Ophthalmology, 94, 91-96.
26. Feygin, D., Keehner, M., & Tendick, F. (2002). Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. Proceedings of the 10th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (Haptics 2002), Orlando, FL, 40-47.
27. Fitts, P. M., & Posner, M. I. (1967). Human performance. Belmont, CA: Brooks/Cole.
28. Fraser, J., & Gutwin, C. (2000). A framework of assistive pointers for low-vision users. Proceedings of the Fourth Annual International ACM/ SIGCAPH Conference on Assistive Technologies (ASSETS '00), Arlington, VA, 9-16.
29. Gaver, W. (1989). The SonicFinder: An interface that uses auditory icons. Human Computer Interaction, 4, 67-94.
30. Gerber, E. (2001). Who's surfing? Internet access and computer use by visually impaired youth and adults. Journal of Visual Impairment & Blindness, 95, 176-181.
31. Giard, M. H., & Peronnet, F. (1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473-490.
32. Hedge, A. (1999, September). Human information processing (course lecture). Retrieved February 16, 2005, from http://ergo.human.cornell.edu/studentdownloads/DEA325pdfs/hip.pdf
33. Inkpen, K. M. (2001). Drag-and-drop versus point-and-click mouse interaction styles for children. ACM Transactions on Computer-Human Interactions, 8, 1-33.
34. Jacko, J. A., Barnard, L., Kongnakorn, T., Maloney, K. P., Edwards, P. J., Emery, V. K., et al. (2004), Isolating the effects of visual impairment: Exploring the effects of AMD on the utility of multimodal feedback. CHI Letters, 6(1), 311-318.
35. Jacko, J. A., Barreto, A. B., Chu, Y. M., Scott, I. U., Rosa, R. H., & Pappas, C. J. (2000). Macular degeneration and visual search: What can we learn from eye movement analysis? Proceedings of the XIVth Triennial Congress of the International Ergonomics Association, 116-119.
36. Jacko, J. A., Barreto, A. B., Marmet, G. J., Chu, J. Y. M., Bautsch, H. S., Scott, I. U., et al. (2000). Low vision: The role of visual acuity in the efficiency of cursor movement. Proceedings of the Fourth International ACM Conference on Assistive Technologies (ASSETS 2000), 1-8.
37. Jacko, J. A., Barreto, A. B., Scott, I. U., Chu, J. Y. M., Bautsch, H. S., Marmet, G. J., et al. (2000). Using eye-tracking to investigate graphical elements for normally sighted and low vision users. ACM Eye Tracking-Research & Applications Symposium 2000, Palm Beach Gardens, FL, November 6-7, p. 112.
38. Jacko, J. A., Barreto, A. B., Scott, I. U., Chu, J. Y. M., Vitense, H. S., Conway, F. T., et al. (2002). Macular degeneration and visual icon use: Deriving guidelines for improved access. Universal Access in the Information Society, 1, 197-206.
39. Jacko, J. A., Barreto, A. B., Scott, I. U., Rosa, R. H., & Pappas, C. J. (2000), Using electroenceph-alogram to investigate stages of visual search in visually impaired computer users: Preattention and focal attention. International Journal of Human-Computer Interaction, 12, 135-150.
40. Jacko, J. A., Dixon, M. A., Rosa, R. H., Scott, I. U., & Pappas, C. J. (1999a). Linking visual capabilities of partially sighted computer users to psychomotor task performance. Proceedings of the 9th International Conference on Human-Computer Interaction, Munich, Germany, 975-979.
41. Jacko, J. A., Dixon, M. A., Rosa, R. H., Scott, I. U., & Pappas, C. J. (1999b). Visual profiles: A critical component of universal access. Proceedings of the Association for Computing Machinery Conference on Human factors in Computing Systems (ACH SIGCHI 99), 330-337.
42. Jacko, J. A., Rosa, R. H., Scott, I. U., Pappas, C. J., & Dixon, M. A. (2000). Visual impairment: The use of visual profiles in evaluations of icon use in computer-based tasks. International Journal of Human-Computer Interaction, 12, 151-164.
43. Jacko, J. A., Scott, I. U., Barreto, A. B., Bautsch, H. S., Chu, J. Y. M., & Fain, W. B. (2001). Iconic visual search strategies: A comparison of computer users with AMD versus computer users with normal vision. Proceedings of the 9th International Conference on Human-Computer Interaction, New Orleans, LA, 423-427.
44. Jacko, J. A., Scott, I. U., Sainfort, F., Barnard, L., Edwards, P. J., Emery, V. K., et al. (2003). Older adults and visual impairment: What do exposure times and accuracy tell us about performance gains associated with multimodal feedback? CHI Letters, 5, 33-40.
45. Jacko, J. A., Scott, I. U., Sainfort, F., Moloney, K. P., Kongnakorn, T., Zorich, B. S., et al. (2003). Effects of multimodal feedback on the performance of older adults with normal and impaired vision. Lecture Notes in Computer Science (LNCS), 2615, 3-22.
46. Jacko, J. A., & Sears, A. (1998). Designing interfaces for an overlooked user group: Considering the visual profiles of partially sighted users. Proceedings of ASSETS '98: The Third International ACM Conference on Assistive Technologies, 75-77.
47. Jacko, J. A., & Vitense, H. S. (2001) A review and reappraisal of information technologies within a conceptual framework for individuals with disabilities. Universal Access in the Information Society, 1, 56-76.
48. Joiner, R., Messer, D., Light, P., & Littleton, K. (1998). It is best to point for young children: A comparison of children's pointing and dragging. Computers in Human Behavior, 14, 513-529.
49. Kaufman, P. L., & Alm, A. (2003). Adler's physiology of the eye: Clinical application (10th ed.). St. Louis, MO: Mosby.
50. Kennett, S., Eimer, M., Spence, C., & Driver, J. (2001). Tactile-visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs. Journal of Cognitive Neuroscience, 13, 462-478.
51. King, A. J., & Calvert, G. A. (2001). Multisensory integration: Perceptual grouping by eye and ear. Current Biology, 11, R322-R325.
52. Klatzky, R. L., Lederman, S. J., & Matula, D. E. (1993). Haptic exploration in the presence of vision. Journal of Experimental Psychology: Human Perception and Performance, 19, 726-743.
53. Kohlrausch, A., & Van de Par, S. (1999). Auditory-visual interaction: From fundamental research in cognitive psychology to (possible) applications. Proceedings of IS&T/SPIE '99 Conference on Human Vision and Electronic Imaging IV, 34-44.
54. Lundberg, L., Johannesson, M., Isacson, D., & Borgquist, L. (1999). The relationship between health-state utilities and the SF-12 in a general population. Medical Decision Making, 19, 128-140.
55. MacKenzie, I. S., Kauppinen, T., & Silfverberg, M. (2001). Accuracy measures for evaluating computer pointing devices. Proceedings of the 2001 ACM Conference on Human Factors in Computing Systems (CHI'01), 9-16.
56. MacKenzie, I. S., Sellen, A., & Buxton, W. (1991). A comparison of input devices in elemental pointing and dragging tasks. Proceedings of CHI'91 Conference on Human Factors in Computing Systems, 161-166.
57. Macula.org. (2004). What is macular degeneration? Retrieved February 16, 2005, from http://www.macula.org/what_is/index.html
58. Macular Degeneration Partnership. (2004). What is AMD? Retrieved February 16, 2005, from http://amd.convio.net/site/PageServer?pagename=What_Is_AMD
59. Mangione, C. M., Berry, S., Spritzer, K., Janz, N. K., Klein, R., Owsley, C., et al. (1998). Identifying the content area for the 51-item National Eye Institute Visual Function Question-naire: Results from focus groups with visually impaired persons. Archives of Ophthalmology, 116, 227-233.
60. Mangione, C. M., Lee, P. P., Gutierrez, P. R., Spritzer, K., Berry, S., & Hays, R. D. (2001). Development of the 25-Item National Eye Institute Visual Function Questionnaire. Archives of Ophthalmology, 119, 1050-1058.
61. Marks, L. E. (1982). Bright sneezes and dark coughs, loud sunlight and soft moonlight. Journal of Experimental Psychology: Human Perception and Performance, 14, 726-743.
62. McGee, M. R. (1999). A haptically enhanced scrollbar: Force feedback as a means of reducing the problems associated with scrolling. Proceedings for the First PHANToM Users Research Symposium (PURS'99), 17-20.
63. Meredith, M. A., & Stein, B. E. (1986) Visual, auditory and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56, 640-662.
64. National Eye Institute. (2004, October). Age-related macular degeneration: What you should know. Retrieved February 16, 2005, from http://www.nei.nih.gov/health/maculardegen/armd_facts.asp#11
65. Neter, J., Kutner, M., Nachtsheim, C., & Wasserman, W. (1996). Applied linear statistical models (4th ed.). Boston: WCB McGraw-Hill.
66. Oakley, I., Brewster, S. A., & Gray, P. D. (2001). Solving multi-target haptic problems in menu interaction. Proceedings of the 2001 ACM Conference on Human Factors in Computing Systems (CHI'01), 357-358.
67. Oakley, I., McGee, M. R., Brewster, S., & Gray, P. (2000). Putting the feel in "look and feel." Proceedings of the 2000 ACM Conference on Human Factors in Computing Systems (CHI'00), 415-422.
68. Pelli, D. G., Robson, J. G., & Wilkins, A. J. (1988). The design of a new letter chart for measuring contrast sensitivity. Clinical Visual Science, 2, 187-199.
69. Popescu, G. V., Burdea, G. C., & Trefftz, H. (2002). Multimodal interaction modeling. In K. M. Stanney (Ed.), Handbook of virtual environments: Design, implementation and applications (pp. 435-454). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
70. Quillen, D. A. (1999). Common causes of vision loss in elderly patients. American family Physician, 60, 99-108.
71. Richard, P., Burdea, G., Gomez, D., & Coiffet, P. (1994). A comparison of haptic, visual and auditive force feedback for deformable virtual objects. Proceedings of ICAT'94 Conference, 49-62.
72. Rock, I., & Victor, J. (1964). Vision and touch: An experimentally created conflict between the two senses. Science, 143, 594-596.
73. Schepens Eye Research Institute. (2002). Eye disease information and resources: AMD fact sheet. Retrieved February 16, 2005, from http://www.theschepens.org/AMD_fact_sheet.htm
74. Scott, I. U., Feuer, W. J., & Jacko, J. A. (2002). Impact of visual function on computer task accuracy and reaction time in a cohort of patients with age-related macular degeneration. American Journal of Ophthalmology, 133, 350-357.
75. Seilen, A. J., Kurtenbach, G. P., & Buxton, W. A. S. (1990). The role of visual and kinaesthetic feedback in the prevention of mode errors. In D. Diaper, D. Gilmore, G. Cocton, & B. Shackel (Eds.), Human-Computer Interaction Interact 90 (pp. 667-673). Oxford, England: North-Holland.
76. Seilen, A. J., Kurtenbach, G. P., & Buxton, W. A. S. (1992). The prevention of mode errors through sensory feedback. Human-Computer Interaction, 7, 141-164.
77. Stanney, K. M., Samman, S., Reeves, L. M., Hale, K., Buff, W., Bowers, C., et al. (2004). A paradigm shift in interactive computing: Deriving multimodal design principles from behavioral and neurological foundations. International Journal of Human-Computer Interaction, 17, 229-257.
78. Storms, R. L. (2002). Auditory-visual cross modality interaction and illusions. In K. M. Stanney (Ed.), Handbook of virtual environments: Design, implementation, and applications (pp. 455-469). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
79. Tiffin, J., & Asher, E. J. (1948). The Purdue Pegboard: Norms and studies of reliability and validity. Journal of Applied Psychology, 32, 234-247.
80. VisionChannel.net. (2005, February 8). Macular degeneration. Retrieved February 16, 2005, from http://www.visionchannel.net/maculardegeneration/treatment.shtml
81. Vitense, H. S., Jacko, J. A., & Emery, V. K. (2003). Multimodal feedback: An assessment of performance and mental workload. Ergonomics, 46, 58-87.
82. Ware, J. E., Kosinski, M., & Keller, S. D. (1995). SF-22: How to score the SF-12 Physical and Mental Health Summary Scales (2nd ed.). Boston: Health Institute, New England Medical Center.
83. Welch, R., & Warren, D. H. (1986). Intersensory interactions. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance. Vol. I: Sensory processes and perception (pp. 25.1-25.36). New York: Wiley.
84. Wickens, C. (1992). Engineering psychology and human performance (2nd ed.). New York: HarperCollins.