A comparison of benchmark task and insight evaluation methods for information visualization

Information Visualization

Volumn 10, Issue 3, 2011, Pages 162-181

  North, Chris ^a   Saraiya, Purvi ^a   Duca, Karen ^b  

^a Virginia Polytechnic Institute and State University   (United States)

^b VIRGINIA BIOINFORMATICS INSTITUTE   (United States)

Author keywords

Benchmark tasks; Empirical evaluation; Graph visualization; Insight; Time series data analysis

Indexed keywords

BENCHMARK TASKS; EMPIRICAL EVALUATIONS; GRAPH VISUALIZATION; INSIGHT; TIME SERIES DATA ANALYSIS;

BIOINFORMATICS; DATA REDUCTION; INFORMATION ANALYSIS; INFORMATION SYSTEMS; TIME SERIES; TIME SERIES ANALYSIS; USER INTERFACES; VISUALIZATION;

DATA VISUALIZATION;

EID: 80054813833     PISSN: 14738716     EISSN: 14738724     Source Type: Journal    
DOI: 10.1177/1473871611415989     Document Type: Article

References (32)

1. Chen C and Czerwinski M. Empirical evaluation of information visualizations: An introduction. IJHCS 2006; 53: 631-635.
2. Plaisant C. The challenge of information visualization evaluation. Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '04). New York, NY: ACM, 2004, pp.109-116.
3. North C. Towards measuring visualization insight. IEEE Comput Graphics App 2006; 26(3): 6-9.
4. Saraiya P, North C and Duca K. An insight-based methodology for evaluating bioinformatics visualizations. IEEE Trans Vis Comput Graphics 2005; 11(4): 443-456. (Pubitemid 41049906)
5. Duggan D, Bittner B, Chen Y, Meltzer P and Trent J. Expression profiling using cDNA microarrays. Nat Genet 1999; 21: 11-14;
6. Saraiya P, North C and Duca K. Visualizing biological pathways: Requirements analysis, systems evaluation and research agenda. Info Vis 2005; 4(3): 191-205.
7. Saraiya P, Lee P and North C. Visualization of graphs with associated time series data. Proceedings of IEEE InfoVis 2005. IEEE. 2005, pp.225-232. (Pubitemid 44500060)
8. Carpendale S. Evaluating information visualizations. In: Kerren A, Stasko JT, Fekete J-D and North C (eds) Information Visualization, Lecture Notes In Computer Science. Vol. 4950, Berlin, Heidelberg: Springer-Verlag, 2008, pp.19-45.
9. Shneiderman B and Plaisant C. Strategies for evaluating information visualization tools: Multi-dimensional in-depth longterm case studies. Proceedings of the 2006 AVI workshop on Beyond time and errors: Novel evaluation methods for information visualization (BELIV '06). New York, NY: ACM, 2006, pp.1-7.
10. Bertini E, Perer A and Lam H. Beyond time and errors: Novel evaluation methods for information visualization (BELIV Workshop). Available at: Http://www.beliv.org/ (accessed 01 February 2011).
11. Chang R, Ziemkiewicz C, Green TM and Ribarsky W. Defining insight for visual analytics. IEEE Comput Graphics App 2009; 29(2): 14-17.
12. Plaisant C, Fekete JD and Grinstein G. Promoting insightbased evaluation of visualizations: From contest to benchmark repository. IEEE Trans Vis Comput Graphics 2008; 14(1): 120-134. (Pubitemid 350157885)
13. Smuc M, Mayr E, Lammarsch T, Aigner W, Miksch S and Gartner J. To score or not to score? Tripling insights for participatory design. IEEE Comput Graphics App 2009; 29(3): 29-38.
14. Golovchinsky G and Belkin NJ. Innovation and evaluation of information exploration interfaces: A CHI98 Workshop. SIGCHI Bull 1999; 31: 22-25.
15. Chen C and Yu Y. Empirical studies of information visualization: A meta-analysis. IJHCS 2000; 53: 851-866. (Pubitemid 32032824)
16. Kosara R, Healey C, Interrante V, Laidlaw D and Ware C. Thoughts on user studies: Why, how, and when. IEEE CG&A 2003; 23(4): 20-25.
17. House D, Interrante V, Laidlaw D, Taylor R and Ware C. Panel: Design and evaluation in visualization research. IEEE Visualization Conference '05. IEEE 2005, p.117. (Pubitemid 44508053)
18. Tory M and Möller T. Evaluating visualizations: Do expert reviews work? IEEE CG&A 2005; 25(5): 8-11. (Pubitemid 41555420)
19. Steves MP, Morse E, Gutwin C and Greenberg S. A comparison of usage evaluation and inspection methods for assessing groupware usability. Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work (GROUP '01), New York, NY: ACM, 2001, pp.125-134.
20. Brush AJ, Ames M and Davis J. A comparison of synchronous remote and local usability studies for an expert interface. CHI '04 extended abstracts on Human factors in computing systems (CHI EA '04). New York, NY: ACM, 2004, pp.1179-1182.
21. Bekker M, Baauw E and Barendregt W. A comparison of two analytical evaluation methods for educational computer games for young children. Cognition Tech Work 2008; 10(2): 129-140. (Pubitemid 351667222)
22. Jeffries R and Desurvire H. Usability testing vs heuristic evaluation: Was there a contest? SIGCHI Bull 1992; 24(4): 39-41.
23. Hartson HR and Andre TS. Criteria for evaluating usability methods. IJHCI 2001; 13(4): 373-410.
24. John B and Marks S. Tracking the effectiveness of usability evaluation methods. BIT 1997; 16(4/5): 188-202. (Pubitemid 127623939)
25. Doubleday A, Ryan M, Springett M and Sutcliffe A. A comparison of usability techniques for evaluating design. Proceedings of the 2nd conference on Designing interactive systems: Processes, practices, methods, and techniques (DIS '97). ACM: New York, NY, 1997, pp.101-110.
26. STKE, The Signal Transduction Knowledge Environment, Database of Cell Signalling. Science Signalling. Available at: Http://stke.sciencemag.org/cm/ (accessed 01 February 2011).
27. Cytoscape: An Open Source Platform for Complex Network Analysis and Visualization. Available at: Http://www.cytoscape. org/ (accessed 01 February 2011).
28. Barsky A, Munzner T, Gardy J and Kincaid R. Cerebral: Visualizing multiple experimental conditions on a graph with biological context. IEEE Trans Vis Comput Graphics 2008; 14: 1253-1260.
29. Saraiya P, North C, Lam V and Duca K. An insight-based longitudinal study of visual analytics. IEEE TVCG 2006; 12(6): 1511-1522. (Pubitemid 44523094)
30. Amar R, Eagan J and Stasko J. Low-level components of analytic activity in information visualization. Proceedings of IEEE InfoVis '05 (Minneapolis, MN) 2005, pp.111-117. (Pubitemid 44500045)
31. Seo J and Shneiderman B. Knowledge discovery in high-dimensional data: Case studies and a user survey for the rank-by-feature framework. IEEE Trans Vis Comput Graphics 2006; 12(3): 311-322.
32. Gualano RC, Hansen MJ, Vlahos R, et al. Cigarette smoke worsens lung inflammation and impairs resolution of influenza infection in mice. Respiratory Res 2008; 9(1): 53.