Developing guidelines for assessing visual analytics environments

Information Visualization

Volumn 10, Issue 3, 2011, Pages 212-231

  Scholtz, Jean ^a  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

Author keywords

Guidelines; Heuristics; User centered evaluation; Visual analytics

Indexed keywords

GUIDELINES; HEURISTICS; INTELLIGENCE ANALYSTS; SCIENCE AND TECHNOLOGY; SPECIFIC TASKS; USER STUDY; USER-CENTERED; VISUAL ANALYSIS; VISUAL ANALYTICS;

GRAPHICAL USER INTERFACES; INFORMATION SYSTEMS; PROFESSIONAL ASPECTS; QUALITY CONTROL; RESEARCH;

VISUALIZATION;

EID: 80054827882     PISSN: 14738716     EISSN: 14738724     Source Type: Journal    
DOI: 10.1177/1473871611407399     Document Type: Review

References (39)

1. Chinchor N and Pike W. The science of analytic reporting. Inform Visual 2009; 8: 286-293.
2. Mayhew D. The Usability Engineering Lifecycle. San Francisco: Morgan Kaufmann Publishers, 1999.
3. Hackos J and Redish J. User and Task Analysis for Interface Design. New York, NY: John Wiley and Sons, 1998.
4. Dumas J and Redish J. A Practical Guide to Usability Testing, Revised edn. Bristol: Intellect, 1999.
5. Nielsen J. Heuristic evaluation. In: Nielsen J and Mack RL (eds) Usability Inspection Methods. New York, NY: John Wiley & Sons, Inc., 1994; 25-62.
6. Butler K, Wishansky A, Laskowski SJ, Morse EL and Scholtz JC. Quantifying usability: The industry usability reporting project. The Proceedings of the Human Factors and Ergonomics Society Conference, Human Factors and Ergonomics Society: Santa Monica, CA, 2002; 7-13.
7. Butler K, Wishansky A, Laskowski S,Morse EL and Scholtz JC. The common industry format: A way for vendors and customers to talk about software usability. Proceedings of HCI International 2003, Vol. 1 (Human-Computer Interaction), Jacko, J. and Stephanidis, C. (eds): Mahwah, NJ: Lawrence Erlbaum Associates, 2003; 554-558.
8. Redish J. Expanding usability testing to evaluate complex systems. J Usability Studies 2007; 2(3): 102-111.
9. Plaisant C. The challenge of information visualization evaluation. Proceedings of the Working Conference on Advanced Visual Interfaces AVI'04, ACM Press: New York, 2004; 109-116.
10. Amar R and Stasko J. A knowledge task-based framework for design and evaluation of information visualizations. Proceedings of IEEE Information Visualization, IEEE Press: Los Alamitos, 2004; 143-149. (Pubitemid 40449139)
11. Baldonado MQW, Woodruff A and Kuchinsky A. Guidelines for using multiple views in information visualization. AVI'00 Proceedings of the Working Conference on Advanced Visual Interfaces, ACM Press: New York, NY, 2000; 110-119.
12. Carr D. Guidelines for designing information visualization applications. Proceedings of the 1999 Ericsson Conference on Usability Engineering ECUE'99 (Stockholm, Sweden), 1999; 1-3.
13. Craft B and Cairns P. Beyond guidelines: What can we learn from the information seeking mantra? Proceedings of Information Visualization, IEEE Press: Los Alamitos, CA, 2005; 110-118. (Pubitemid 44458502)
14. Shneiderman B. The eyes have it: A task by data type taxonomy of information visualizations. Proceedings of IEEE Symposium on Visual Languages '96, IEEE Press: Los Alamitos, CA, 1996; 336-343.
15. Trainer E, Quirk S, de Souza CRB and Redmiles DF. Analyzing a socio-technical visualization tool using usability inspection methods. The Proceedings of the IEEE Symposium on Visual Languages and Human Centric Computing, IEEE Computer Society: Washington, DC, 2008; 78-81.
16. Zuk T, Schlesier L, Neumann P, HancockMand Carpendale S. Heuristics for information visualization evaluation. The Proceedings of the 2006 AVI Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization BELIV '06 (Venice, Italy), ACM Press: New York, NY, 2006; ISBN:1-59593-562-2; 1-6.
17. Tory M and Möller T. Evaluating visualizations: Do expert reviews work. IEEE Comput. Graphics Appl. 2005; 25(5): 8-11. (Pubitemid 41555420)
18. Scholtz J, Cook K, Whiting MA, Lemon D and Greenblatt H. Visual analytics technology transition progress. Inform Visual 2009; 8: 294-301.
19. Grinstein G, Konecni S, Plaisant C, Scholtz J and Whiting MA. VAST 2010 challenge: Arms dealings and pandemics. Proceedings of Visual Analytics Science and Technology 2010 Conference, IEEE: Piscataway, NJ, 2010; 267-268.
20. Grinstein G, Plaisant C, Scholtz J and Whiting MA. VAST 2009 challenge: An insider threat. Proceedings of the IEEE Symposium on Visual Analytics Science and Technology VAST 2009, IEEE: Piscataway, NJ, 2009; 243-244.
21. Grinstein G, Plaisant C, Laskowski S, O'Connell T, Scholtz J and Whiting MA. VAST 2008 challenge: Introducing minichallenges. Proceedings of the IEEE Symposium on Visual Analytics Science and Technology, VAST 2008, IEEE: Piscataway, NJ, 2008; 195-196.
22. Grinstein G, Plaisant C, Laskowski S, O'Connell T, Scholtz J and Whiting MA. VAST 2007 contest-blue iguanodon. Proceedings of the IEEE Symposium on Visual Analytics Science and Technology, VAST 2007, IEEE: Piscataway, NJ, 2007; 231-232.
23. Grinstein G, Plaisant C, Connell T, Laskowski S, Scholtz J and Whiting MA. The VAST 2006 contest: A tale of Alderwood. Proceedings of IEEE Symposium on Visual Analytics Science and Technology, IEEE: Piscataway, NJ, 2006; 215-216.
24. Costello L, Grinstein G, Plaisant C and Scholtz J. Advancing user-centered evaluation of visual analytic environments through contests. Inform Visual 2009; 8: 230-238.
25. http://hcil.cs.umd.edu/localphp/hcil/vast/index.php/judging/ index, accessed March 3, 2010.
26. http://hcil.cs.umd.edu/localphp/hcil/vast/images/uploads/ Instructions%20for%20Reviewers.doc, accessed December 8, 2010.
27. http://hcil.cs.umd.edu/localphp/hcil/vast/index.php/taskdesc/ index, accessed December 6, 2010.
28. Zuk T and Carpendale S. Theoretical analysis of uncertainty visualizations. Proceedings of SPIE-IS&T Electronic Imaging, SPIE, (San Jose, CA), Society for Imaging Science and Technology and SPIE; 2006; 6060-6070.
29. Endsley M, Bolté B and Jones D. Designing for Situation Awareness. New York, NY: Taylor & Francis, 2003.
30. Research-Based Web Design and Usability Guidelines, www.usability.gov, accessed September 8, 2010 U.S. Government Printing Office, Internet: Bookstore.gpo.gov Washington, ISBN 0-16-076270-7.
31. Shneiderman B. Designing the User Interface, 3rd edn. Reading, MA: Addison Wesley Longman, Inc., 1998.
32. Weinschenk S and Yeo S. Guidelines for Enterprise-Wide GUI Design. New York, NY: John Wiley & Sons, Inc., 1995.
33. http://www.useit.com/papers/heuristic/heuristic-list.html, accessed September 10, 2010.
34. Pirolli P and Card S. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. Proceedings of 2005 International Conference on Intelligence Analysis (McLean, VA), 2005; http://analysis. mitre.org/proceedings/index.html, accessed June 6, 2011.
35. Forsell C and Johansson J. An heuristic set for evaluation in information visualization. (AVI '10) Proceedings of the International Conference on Advanced Visual Interfaces (Rome, Italy), ACM Press: New York, NY, 2010; 199-206.
36. Nielsen J. Enhancing the explanatory power of usability heuristics. (CHI '94) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Celebrating Interdependence (Boston, MA), ACM Press: New York, NY, 1994; 152-158.
37. http://www.fas.org/irp/dni/icd/icd-203.pdf, accessed March 20, 2010.
38. http://www.dni.gov/nic/PDF-GIF-otherprod/Iraq-NIE-Key-Ju dgments.pdf, accessed March 20, 2010.
39. Shneiderman B and Plaisant C. (2006) Strategies for evaluating information visualization tools: Multi-dimensional in-depth long-term case studies. BELIV '06 Proceedings of the 2006 AVI workshop on beyond time and errors: Novel evaluation methods for information visualization (Venice, Italy), ACM Press: New York, NY, 2006; doi:>10.1145/1168149. 1168158, 1-7.