How much does expertise matter? A barrier walkthrough study with experts and non-experts

ASSETS'09 - Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility

Volumn , Issue , 2009, Pages 203-210

  Yesilada, Yeliz ^a   Brajnik, Giorgio ^b   Harper, Simon ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b UNIVERSITY OF UDINE   (Italy)

Author keywords

Evaluation; Expertise; Guideline; Web accessibility

Indexed keywords

ACCESSIBILITY EVALUATION; EVALUATION FRAMEWORK; EXPERT EVALUATION; HEARING IMPAIRED; LOW VISION; USER EVALUATIONS; USER GROUPS; WALKTHROUGHS; WEB ACCESSIBILITY; WEB CONTENT ACCESSIBILITY GUIDELINES; WEB PAGE;

AUDITION; FLUORINE CONTAINING POLYMERS; WORLD WIDE WEB;

QUALITY CONTROL;

EID: 72249085137     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1639642.1639678     Document Type: Conference Paper

References (25)

1. G. Brajnik. Barrier walkthrough: Heuristic evaluation guided by accessibility barriers. http://users.dimi.uniud. it/~giorgio.brajnik/projects/ bw/bw.html.
2. G. Brajnik. Web accessibility testing: When the method is the culprit. In In Proc. of ICCHP 2006, 2006.
3. B. Caldwell, M. Cooper, L.G. Reid, and G. Vanderheiden. Web Content Accessibility Guidelines 2.0 (WCAG 2.0). W3C, 2008. http://www.w3.org/TR/WCAG20/ .
4. M.B. Catani and D.W. Biers. Usability evaluation and prototype fidelity: Users and usability professionals. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1998.
5. J. Cegarra and J. Hoc. Cognitive styles as an explanation of experts' individual differences: A case study in computer-assisted troubleshooting diagnosis. Int. J. Hum.-Comput. Stud., 64(2):123-136, 2006.
6. G. Cockton and A. Woolrych. Understanding inspection methods: lessons from an assessment of heuristic evaluation. In People & Computers XV, pages 171-192. Springer, 2001.
7. DRC. Formal investigation report: web accessibility. Disability Rights Commission, www.drc-gb.org, April 2004. Visited Jan. 2006.
8. W. Gray and M. Salzman. Damaged merchandise: a review of experiments that compare usability evaluation methods. Human - Computer Interaction, 13(3):203-261, 1998.
9. T. S. Andre H. R. Hartson and R. C. Williges. Criteria for evaluating usability evaluation methods. Int. Journal of Human-Computer Interaction, 15(1):145-183, 2003.
10. S.L. Henry and M. Grossnickle. Just Ask: Accessibility in the User-Centered Design Process. Georgia Tech Research Corporation, Atlanta, Georgia, USA, 2004.
11. M. Hertzum and N. Jacobsen. The evaluator effect during first-time use of the cognitive walkthrough technique. In Proceedings of HCI International on Human-Computer Interaction: Ergonomics and User Interfaces-Volume I, pages 1063-1067, 1999.
12. M. Hertzum, N. Jacobsen, and R. Molich. Usability inspections by groups of specialists: perceived agreement in spite of disparate observations. In CHI '02 extended abstracts, pages 662-663. ACM, 2002.
13. M. Hertzum and N. Ebbe Jacobsen. The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human Computer Interaction, 13(4), 2001.
14. K. Hornbeak and E. Frokjear. A study of the evaluator effect in usability testing. Human-Computer Interaction,, 23(3):251- 277, 2008.
15. M. Ivory and M. Hearst. The state of the art in automating usability evaluation of user interfaces. ACM Computer Survey, 33(4):470-516, 2001.
16. N. Jacobsen, M. Hertzum, and B. John. The evaluator effect in usability tests. In CHI '98, pages 255-256. ACM, 1998.
17. T. Lang. Comparing website accessibility evaluation methods and learnings from usability evaluation methods. http://www.peakusability.com.au, Visited May 2008, 2003.
18. C. Ling and G. Salvendy. Effect of evaluators' cognitive style on heuristic evaluation: Field dependent and field independent evaluators. Int. J. Hum.-Comput. Stud., 67(4):382-393, 2009.
19. J. Mankoff, H. Fait, and T. Tran. Is your web page accessible?: a comparative study of methods for assessing web page accessibility for the blind. In CHI'05, pages 41-50. ACM, 2005.
20. J. Nielsen. Finding usability problems through heuristic evaluation. In CHI '92, pages 373-380. ACM, 1992.
21. H. Petrie and O. Kheir. The relationship between accessibility and usability of websites. In Proc. CHI 2007, pages 397-406, San Jose, CA, USA, 2007. ACM.
22. I. Curson S. Butler E. Kindlund D. Miller R. Molich, N. Bevan and J. Kirakowski. Comparative evaluation of usability tests. In In Proc. of the Usability Professionals Association Conference, 1998.
23. A. Sears. Heuristic walkthroughs: Finding the problems without the noise. International Journal of Human-Computer Interaction, 9:213-234, 1997.
24. Patrick E. Shrout and Joseph L. Fleiss. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2):420-428, Mar 1979.
25. A. Woolrych and G. Cockton. Assessing heuristic evaluation: mind the quality, not just the percentages. In In Proc. of HCI 2000, pages 35-36, 2000.