A unified methodology for the evaluation of accessibility and usability of mobile applications

Universal Access in the Information Society

Volumn 9, Issue 4, 2010, Pages 337-356

  Billi, Marco ^a   Burzagli, Laura ^a   Catarci, Tiziana ^b   Santucci, Giuseppe ^b   Bertini, Enrico ^b   Gabbanini, Francesco ^a   Palchetti, Enrico ^a  

^a IFAC CNR   (Italy)

^b SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

Accessibility; Mobile computing; Usability

Indexed keywords

MOBILE COMPUTING; USABILITY ENGINEERING;

ACCESSIBILITY; ACCESSIBILITY GUIDELINES; MOBILE SYSTEMS; UNIFIED METHODOLOGY; UNIVERSAL ACCESS; USABILITY; USABILITY HEURISTICS; USABILITY OF MOBILE APPLICATIONS;

HEURISTIC METHODS;

EID: 78149371229     PISSN: 16155289     EISSN: 16155297     Source Type: Journal    
DOI: 10.1007/s10209-009-0180-1     Document Type: Article

References (65)

1. Bertini, E., Calì, A., Santucci, G.: Automatic interface generation through interaction, users, and devices modeling. In: Proceedings of 6th International Conference on Computer-Aided Design of User Interfaces CADUI2006, Bucharest, Romania, June 5-8, 2006.
2. Stephanidis, C., Savidis, A.: Universal Access in the Information Society: Methods, Tools and Interaction Technologies. Univers. Access Inf. Soci. J. 40-55, Springer-Verlag (2001).
3. Emiliani, P. L., Stephanidis, C.: Universal access to ambient intelligence environments: Opportunities and challenges for people with disabilities. IBM Sys. J. 44(3) (2005).
4. Gray, W. D., Salzman, M. C.: Damaged merchandise? A review of experiments that compare usability evaluation methods. Hum. Comput. Interact. 13(3), 203-261 (1998).
5. Lang, T.: Comparing website accessibility evaluation methods and learnings from usability evaluation methods http://www. peakusability. com. au/about-us/pdf/website_accessibility. pdf, December 2003.
6. ISO/IEC Guide 71Guidelines for standards developers to address the needs of older persons and persons with disabilities. International Organization for Standardization (ISO) (2001).
7. ISO 9241-171Ergonomics of human-system interaction-Part 171: Guidance on software accessibility (2006).
8. ISO 9241-20 Ergonomics of human-system interaction-Part 20: Accessibility guidelines for information/communication technology (ICT) equipment and services. (2006).
9. Wegge, K. P., Zimmermann, D.: Accessibility, usability, safety, ergonomics: concepts, models, and differences. Universal Access in HCI, Part I, HCII 2007, LNCS 4554, pp. 294-301, Springer-Verlag, (2007).
10. Henry, S. L.: Introduction to web Accessibility. http://www. w3. org/wai/intro/accessibility. php (2005).
11. Pernici, B. (ed.): Mobile Information Systems: Infrastructure and Design for Adaptivity and Flexibility, p. 237. Springer, Berlin (2006).
12. Petrie, H., Kheir, O.: The relationship between accessibility and usability of websites. CHI '07: In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 397-406, ACM Press, USA (2007).
13. Thatcher, J., Waddell, C. D., Henry, S. L., Swierenga, S., Urban, M. D., Burks, M., Regan, B., Bohman, P.: Constructing Accessible Web Sites. Glasshaus, San Francisco (2003).
14. Shneiderman, B.: Universal usability. Commun. ACM 43(5), 85-91 (2000).
15. Shneiderman, B.: Promoting universal usability with multi-layer interface design. In: Proceedings of the 2003 Conference on Universal Usability (CUU 2003).
16. Antona, M., Mourouzis, A., Stephanidis, C.: Towards a Walk- through Method for Universal Access Evaluation. Universal Access in HCI, Part I, HCII 2007, LNCS 4554, pp. 325-334, Springer-Verlag (2007).
17. Theofanos, M. F., Redish, J.: Bridging the gap: between accessibility and usability. ACM Interact. 10(6), 36-51 (2003). ACM Press.
18. Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1. 0 W3C Recommendation, (http://www. w3. org/TR/1999/WAI-WEBCONTENT-19990505). May 5, 1999.
19. Henry, S., Grossnicklem, M.: Just Ask: Accessibility in the User-Centered Design Process. Georgia Tech Research Corporation, Atlanta, Georgia, USA, 2004. On-line book: http://www. uiaccess. com/accessucd/index. html.
20. Brajnik, G.: A comparative test of web accessibility evaluation methods. In: Proceedings of ASSETS'08, ACM Press, Halifax, 13-15 Oct 2008.
21. Souza, R.: Design Accessible Sites Now, Forrester Report, December 2001 http://www. forrester. com/ER/Research/Report/Sum-mary/0, 1338, 11431, 00. html.
22. Zimmermann, G., Vanderheiden, G.: Accessible design and testing in the application development process: considerations for an integrated approach. Univer. Access Infor. Soci. Int. J. 7, 117-128 (2007). Springer-Verlag.
23. Manko, J., Dey, A. K., Hsieh, G., Kientz, J., Lederer, S., Ames, M.: Heuristic evaluation of ambient displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: CHI'03, ACM Press, pp. 169-176 (2003).
24. Baker, K., Greenberg, S., Gutwin, C.: Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration. In: Proceedings of the 8th IFIP Working Conference on Engineering for Human-Computer Interaction (EHCI'01). (May 11-13, Toronto, Canada), 2001.
25. Abowd, G. D., Mynatt, E. D.: Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput. Hum. Interact. (2000).
26. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-Computer Interaction, 3rd edn. Prentice Hall, Upper Saddle River (2004).
27. Abowd, G. D., Mynatt, E. D., Rodden, T.: The human experience. IEEE Perva. Comput. pp. 48-57, (2002).
28. Gabrielli, S., Mirabella, V., Kimani, S., Catarci, T.: Supporting Cognitive Walkthrough with Video Data: A Mobile Learning Evaluation Study. MobileHCI 05 Conf., Salzburg (Austria), September 2005.
29. Duh, H. B.-L., Tan, G. C. B. et Chen, V. H.-H.: Usability evaluation for mobile device: a comparison of laboratory and field tests. In: Proceedings of the 8th conference on human-computer interaction with mobile devices and services (Helsinki, Finland), pp. 181-186, ACM Press (2006).
30. Kaikkonen, A., Keklinen, A., Cankar, M., Kallio, T., Kankainen, A.: Usability testing of mobile applications: a comparison between laboratory and field testing. J. Usabil. Stud. 1(1), 4-17 (2005).
31. Kjeldskov, J., Stage, J.: New techniques for usability evaluation of mobile systems. IJHCS 60(5-6), 599-620 (2004).
32. Roto, V., Oulasvirta, A., Haikarainen, T., Lehmuskallio, H. et Nyyssnen, T.: Examining mobile phone use in the wild with quasi-experimentation. Helsinky Institute for Information Technology (HIIT), August 2004, Technical Report 2004-1.
33. Waterson, S., Landay, J. A. et Matthews, T.: In the lab and out in the wild: remote web usability testing for mobile devices. CHI'02 conference on Human Factors in Computing Systems extended abstracts (Minneapolis, Minnesota, USA), pp. 796-797, ACM Press (2002).
34. Po, S.: Mobile usability testing and evaluation. Master's Thesis, University of Melbourne, Australia (2003).
35. Vetere, F., Howard, S., Pedell, S., Balbo, S.: Walking through mobile use: Novel heuristics and their application. In: Proceedings of Conference of the Computer-Human Interaction Special Interest Group of the Human Factors and Ergonomics Society of Australia OZCHI2003 (2003).
36. Bertini, E., Gabrielli, S., Kimani, S., Catarci, T., Santucci, G.: Appropriating and Assessing Heuristics for Mobile Computing. In: Proceedings of the International Conference in Advanced Visual Interfaces (AVI) (2006).
37. Bertini, E., Catarci, T., Dix, A., Gabrielli, S., Kimani, S., Santucci, G.: Appropriating heuristic evaluation for mobile computing. Int. J. Mobile Hum. Comput. Interact. 1(1), 20-41 (2009).
38. Thorp, J., Henry, S. L.: Web Content Accessibility and Mobile Web: Making a Web Site Accessible Both for People with Disabilities and for Mobile Devices. http://www. w3. org/WAI/mobile/.
39. Rabin, J., McCathieNevile, C.: Mobile Web Best Practices 1. 0. W3C Recommendation 29 July 2008 http://www. w3. org/TR/mobile-bp/.
40. Smillie, D.: Web accessibility: is it just a merry-go-round?. In: Proceedings of the 2006 International Cross-Disciplinary Work- Shop on Web Accessibility (W4A). pp. 90-91. ACM Press, Edinburgh, 2006.
41. Harper, S., Yesilada, Y.: Building the mobile web: rediscovering acessibility? Univers. Access Infor. Soci. Int. J. 6, 219-220 (2007). Springer-Verlag.
42. Trewin, S. Physical usability and the mobile web. In: Proceedings of the 2006 international cross-disciplinary workshop on Web accessibility (W4A). pp. 109-112. ACM Press, Edinburgh, 2006.
43. Owen, S., Rabin, J.: "W3C mobileOK Basic Tests 1. 0-Can didate Recommendation" http://www. w3. org/TR/2007/CR-mobileOK-basic10-tests-20071130/30 November, 2007.
44. Garofalakis, J., Stefanis, V.: MokE: a tool for Mobile-ok evaluation of Web Content. In: Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), Beijing, China, pp. 57-64. ACM press, 2008.
45. Vigo, M., Aizpurua, A., Arrue, M., Abascal, J.: Evaluating Web Accessibility for Specific Mobile Devices. In: Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), Beijing, China, pp. 65-72. ACM press, 2008.
46. TAW mobileOK Basic checker, http://www. validadores. tawdis. net/mobileok/en/.
47. W3C mobileOK Checker http://www. validator. w3. org/mobile/.
48. Chandler, E., Dixon, E., Moniz Pereira, L., Espadinha, C.: A Comparison Study between a Heuristic Evaluation Technique and End User Trial for Mobile Phone Accessibility-Published by COST (2006).
49. Cockton, G., Lavery, D., Woolrych, A.: Inspection-based evaluations. In Jacko, J. J., Sears, A. (eds.). The Human-Computer Interaction Handbook (2003).
50. Nielsen, J.: Usability Engineering. Morgan Kaufmann Publishers, San Francisco (1994).
51. Caldwell, B., Chisholm, W., Vanderheiden, G., White, J.: Web Content Accessibility Guidelines 2. 0 W3C Working Draft, (http://www. w3. org/TR/2005/WD-WCAG20-20051123/). November 23, 2005.
52. Jacobs, I., Gunderson, J., Hansen, E.: User Agent Accessibility Guidelines 1. 0. W3C Recomendation, (http://www. w3. org/TR/UAAG10/). 17 December 2002.
53. Guidelines containing the technical requirements, the definitions of the different accessibility levels and the technical methodologies for the testing of Web site accessibility. Law 4 of 9/1/2004, Article 11, Paragraph 1, Letters a) and b) December 2004 (Version 3) http://www. pubbliaccesso. gov. it/biblioteca/documentazione/guidelines_study/.
54. Chuter, A.: Relationship Between Mobile Web Best Practices 1. 0 and Web Content Accessibility Guidelines, (http://www. w3. org/TR/mwbp-wcag/).
55. Po, S., Howard, S., Vetere, F., Skov, M. B. In: Brewster, S., Dunlop, M. (eds.). Heuristic Evaluation and Mobile Usability: Bridging the Realism Gap. In: Proceedings of Mobile HCI 2004, pp. 49-60, 2004.
56. Kjeldskov, J., Graham, C., Pedell, S., Vetere, F., Howard, S., Balbo, S., Davies, J.: Evaluating the usability of a mobile guide: The influence of location, participants and resources. Behav. Inf. Technol. 24(1), 51-65 (2005).
57. Bertini, E., Santucci, G.: Modeling user-system data exchange to design adaptive interfaces. In: Proceedings del International Workshop on Plastic Services for Mobile Devices 18 PSMD05, Rome, Italy, September 12, 2005, during the10th International IFIP Conference on Human-Computer Interaction Interact 2005.
58. MAIS Deliverable R. 7. 3. 4: Specifiche del Prototipo per la Generazione di Interfacce Utente, 2004.
59. MAIS Deliverable P. 7. 3. 8: Prototipo per la generazione di interfacce utente, 2005.
60. MAIS Deliverable R. 7. 3. 8: Specifications of the Second Level Prototype of the MaisDesigner, 2005.
61. Kjeldskov, J., Graham, C.: A Review of Mobile HCI Research Methods. In L. Chittaro (ed.). Proceedings of Mobile HCI 2003, pp. 317-335, Springer-Verlag, 2003.
62. Battestini, Flanagan, J. A.: "Modelling and Simulating Context Data in a Mobile Environment", Workshop on Context Awareness for Proactive Systems (CAPS), 2005, pp. 127-136, Helsinki, Finland.
63. De Waard, D., Brookhuis, K. A., Toffetti, K. A. (eds.): Developments in Human Factors in Transportation, Design, and Evaluation. Field Versus Laboratory Usability Testing: A First Comparison, pp. 205-212. Shaker Publishing, Maastricht (2006).
64. Nielsen, C.: Testing in the Field. In: Proceedings of APCHI 98, pp. 285-290, (1998).
65. Kjeldskov, J., Skov, M. B., Als, B. S., Høegh, R. B.: "Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field". In: Proceedings of 6th International Conference on Human Computer Interaction with Mobile Devices and Services, September 2004, University of Strathclyde, Glasgow, Scotland.