Evaluating information visualizations

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4950 LNCS, Issue , 2008, Pages 19-45

  Carpendale, Sheelagh ^a  

^a UNIVERSITY OF CALGARY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SYSTEMS; VISUALIZATION;

INFORMATION VISUALIZATION;

INFORMATION ANALYSIS;

EID: 50549104903     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-70956-5_2     Document Type: Conference Paper

References (85)

1. Amar, R.A., Stasko, J.T.: Knowledge Precepts for Design and Evaluation of Information Visualizations. IEEE Transactions on Visualization and Computer Graphic 11(4), 432-442 (2005)
2. Andrews, K.: Evaluating Information Visualisations. In: Proceedings of the 2006 AVI Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, pp. 1-5 (2006)
3. Auerbach, C. Qualitative Data: An Introduction to Coding and Analysis. University Press, New York (2003)
4. Baker, K., Greenberg, S., Gutwin, C. Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 96-105. ACM Press, New York (2002)
5. Baldonado, M., Woodruff, A., Kuchinsky, A.: Guidelines for Using Multiple Views in Information Visualization. In: Proeedings of the Conference on Advanced Visual Interfaces (AVI), pp. 110-119. ACM Press, New York (2000)
6. Barzun, J., Graff, H.: The Modern Researcher, 3rd edn. Harcourt Brace Jovanvich, New York (1977)
7. BELIV 2006, accessed http://www.dis.uniromal.it/~beliv06/ (February 4, 2008)
8. Bertin, J.: Semiology of Graphics (Translation: William J. Berg). University of Wisconsin Press (1983)
9. Bevan, N., Barnum, C. Cockton, G., Nielsen, J., Spool, J., Wixon, W.: The "Magic Number 5": Is It Enough for Web Testing? In: CHI Extended Abstracts, pp. 698-699. ACM Press, New York (2003)
10. Boyatzis, R.: Transforming Qualitative Information: Thematic Analysis and Code Development. Sage Publications, London (1998)
11. Brereton, M., McGarry, B.: An Observational Study of How Objects Support Engineering Design Thinking and Communication: Implications for the Design of Tangible Media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHP00), pp. 217-224. ACM Press, New York (2000)
12. Chen, C. Czerwinski, M.: Introduction to the Special Issue on Empirical Evaluation of Information Visualizations. International Journal of Human-Computer Studies 53(5), 631-635 (2000)
13. Corbin, J., Strauss, A.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 3rd edn. Sage Publications, Los Angeles (2008)
14. Creswell, J.: Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Sage Publications, London (1998)
15. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human Computer Interaction, 2nd edn. Prentice-Hall, Englewood Cliffs (1998)
16. Dumais, S., Cutrell, E., Chen, H.: Optimizing Search by Showing Results In Context. In: Proc. CHT'01, pp. 277-284. ACM Press, New York (2001)
17. Eberts, R.E.: User Interface Design. Prentice-Hall, Englewood Cliffs (1994)
18. Ellis, E., Dix, A.: An Explorative Analysis of User Evaluation Studies in Information Visualization. In: Proceedings of the Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization, BELIV (2006)
19. Ericsson, K., Simon, H.: Verbal Reports as Data. Psychological Review 87(3), 215-251 (1980)
20. Ericsson, K., Simon, H.: Verbal Reports on Thinking. In: Faerch, C., Kasper, G. (eds.) Introspection in Second Language Research, pp. 24-54. Multilingual Matters, Clevedon, Avon (1987)
21. Ericsson, K., Simon, H.: Protocol Analysis: Verbal Reports as Data, 2nd edn. MIT Press, Boston (1993)
22. Fall, J., Fall, A.: SELES: A Spatially Explicit Landscape Event Simulator. In: Proceedings of GIS and Environmental Modeling, pp. 104-112. National Center for Geographic Information and Analysis (1996)
23. Forlines, C., Shen, C., Wigdor, D., Balakrishnan, R.: Exploring the effects of group size and display configuration on visual search. In: Computer Supported Cooperative Work 2006 Conference Proceedings, pp. 11-20 (2006)
24. Garfinkel, H.: Studies in Ethnomethodology. Polity Press, Cambridge (1967)
25. Gonzalez, V., Kobsa, A.: A Workplace Study of the Adoption of Information Visualization systems. In: Proceedings of the International Conference on Knowledge Management, pp. 92-102 (2003)
26. Gorard, S.: Combining Methods in Educational Research. McGraw-Hill, New York (2004)
27. Greenberg, S., Buxton, B.: Usability Evaluation Considered Harmful (Some of the Time). In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2008)
28. Greene, J., Caracelli, V., Graham, W.: Toward a Conceptual Framework for Mixed-Method Evaluation Design. Educational Evaluation and Policy Analysis 11(3), 255-274 (1989)
29. Gutwin, C., Greenberg, S.: The Mechanics of Collaboration: Developing Low Cost Usability Evaluation Methods for Shared Workspaces. In: Proceedings WETICE, pp. 98-103. IEEE Computer Society Press, Los Alamitos (2000)
30. Healey, C.G.: On the Use of Perceptual Cues and Data Mining for Effective Visualization of Scientific Datasets. In: Proceedings of Graphics Interface, pp. 177-184 (1998)
31. Heer, J., Viegas, F., Wattenberg, M.: Voyagera and Voyeurs: Supporting Asynchronous Collaborative Information Visualization. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI'07), pp. 1029-1038. ACM Press, New York (2007)
32. Holtzblatt, K., Beyer, H.: Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann, San Francisco (1998)
33. Huck, S.W.: Reading Statistics and Research, 4th edn. Pearson Education Inc., Boston (2004)
34. Interrante, V.: Illustrating Surface Shape in Volume Data via Principal Direction-Driven 3D Line Integral Convolution. Computer Graphics, Annual Conference Series, pp. 109-116 (1997)
35. Isenberg, P., Carpendale, S.: Interactive Tree Comparison for Co-located Collaborative Information Visualization. IEEE Transactions on Visualization and Computer Graphics 12(5) (2007)
36. Isenberg, P., Tang, A., Carpendale, S.: An Exploratory Study of Visual Information Analysis. In: Proceedings of the Conference on Human Factors in Computing Systems (CHT08), ACM Press, New York (to appear, 2008)
37. Kay, J., Reiger, H., Boyle, M., Francis, G.: An Ecosystem Approach for Sustainability: Addressing the Challenge of Complexity. Futures 31(7), 721-742 (1999)
38. Kim, S., Hagh-Shenas, H., Interrante, V.: Conveying Shape with Texture: Experimental Investigations of Texture's Effects on Shape Categorization Judgments. IEEE Transactions on Visualization and Computer Graphics 10(4), 471-483 (2004)
39. Kleffner, D.A., Ramachandran, V.S.: On the Perception of Shape from Shading. Perception and Psychophysics 52(1), 18-36 (1992)
40. Kobsa, A.: User Experiments with Tree Visualization Systems. In: Proceedings of the IEEE Symposium on Information Visualization, pp. 9-26 (2004)
41. Kruger, R., Carpendale, S., Scott, S.D., Greenberg, S.: Roles of Orientation in Tabletop Collaboration: Comprehension, Coordination and Communication. Journal of Computer Supported Collaborative Work 13(5-6), 501-537 (2004)
42. Lewin, C. (ed.): Research Methods in the Social Sciences. Sage Publications, London (2004)
43. Lewis, C., Rieman, J.: Task-Centered User Interface Design: A Practical Introduction (1993)
44. Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology 140, 1-55 (1932)
45. Lofland, J., Lofland, L.: Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Wadsworth Publishing Company, CA, USA (1995)
46. Mankoff, J., Dey, A., Hsieh, G., Kientz, J., Lederer, S., Ames, A.: Heuristic Evaluation of Ambient Displays. In: Proceedings of CHI '03, pp. 169-176. ACM Press, New York (2003)
47. Mark, G., Kobsa, A., Gonzalez, V.: Do Four Eyes See Better Than Two? Collaborative Versus Individual Discovery in Data Visualization Systems. In: Proceedings of the IEEE Conference on Information Visualization (IV'02), July 2002, pp. 249-255. IEEE Press, Los Alamitos (2002)
48. Mazza, R., Berre, A.: Focus Group Methodology for Evaluating Information Visualization Techniques and Tools. In: Proceedings of the International Conference on Information Visualization IV (2007)
49. McCarthy, D.: Normal Science and Post-Normal Inquiry: A Context for Methodology (2004)
50. McGrath, J.: Methodology Matters: Doing Research in the Social and Behavioural Sciences. In: Readings in Human-Computer Interaction: Toward the Year 2000, Morgan Kaufmann, San Francisco (1995)
51. Moggridge, B.: Design Interactions. MIT Press, Cambridge (2006)
52. Morris, M.R., Ryall, K., Shen, C., Forlines, C., Vernier, F.: Beyond "Social Protocols": Multi-User Coordination Policies for Co-located Groupware. In: Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW, Chicago, IL, USA), CHI Letters, November 6-10, 2004, pp. 262-265. ACM Press, New York (2004)
53. Morse, E., Lewis, M., Olsen, K.: Evaluating Visualizations: Using a Taxonomie Guide. Int. J. Human-Computer Studies 53, 637-662 (2000)
54. Nielsen, J., Mack, R.: Usability Inspection Methods. John Wiley & Sons, Chichester (1994)
55. North, C.: Toward Measuring Visualization Insight. IEEE Computer Graphics and Applications 26(3), 6-9 (2006)
56. Patton, M.Q.: Qualitative Research and Evaluation Methods, 3rd edn. Sage Publications, London (2001)
57. Plaisant, C.: The Challenge of Information Visualization Evaluation. In: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 109-116 (2004)
58. Purchase, H.C., Hoggan, E., Görg, C.: How Important Is the "Mental Map"? - An Empirical Investigation of a Dynamic Graph Layout Algorithm. In: Kaufmann, M., Wagner, D. (eds.) GD 2006. LNCS, vol. 4372, pp. 184-195. Springer, Heidelberg (2007)
59. Purchase, H.C.: Effective Information Visualisation: A Study of Graph Drawing Aesthetics and Algorithms. Interacting with Computers 13(2), 477-506 (2000)
60. Purchase, H.C.: Performance of Layout Algorithms: Comprehension, Not Computation. Journal of Visual Languages and Computing 9, 647-657 (1998)
61. Brandenburg, F.J. (ed.): GD 1995. LNCS, vol. 1027. Springer, Heidelberg (1996)
62. Reilly, D., Inkpen, K.: White Rooms and Morphing Don't Mix: Setting and the Evaluation of Visualization Techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 111-120 (2007)
63. Saraiya, P., North, C., Duca, K.: An Insight-Based Methodology for Evaluating Bioinformatics Visualizations. IEEE Transactions on Visualization and Computer Graphics 11(4), 443-456 (2005)
64. Scott, S.D., Carpendale, S., Inkpen, K.: Territoriality in Collaborative Tabletop Workspaces. In: Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW, Chicago, IL, USA), CHI Letters, November 6-10, 2004, pp. 294-303. ACM Press, New York (2004)
65. Seidman, I.: Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences. Teachers' College Press, New York (1998)
66. Shneiderman, B.: The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In: Proceedings of the IEEE Symposium on Visual Languages, pp. 336-343. IEEE Computer Society Press, Los Alamitos (1996)
67. Shneiderman, B., Plaisant, C.: Strategies for Evaluating Information Visualization Tools: Multi-Dimensional In-Depth Long-Term Case Studies. In: Proceedings of the Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, BELIV (2006)
68. Spence, R.: Information Visualization, 2nd edn. Addison-Wesley, Reading (2007)
69. Spool, J., Schroeder, W.: Testing Web Sites: Five Users is Nowhere Near Enough. In: CHI '01 Extended Abstracts, pp. 285-286. ACM Press, New York (2001)
70. Strauss, A.L., Corbin, J.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, London (1998)
71. Tang, A., Tory, M., Po, B., Neumann, P., Carpendale, S.: Collaborative Coupling over Tabletop Displays. In: Proceedings of the Conference on Human Factors in Computing Systems (CHT06), pp. 1181-1290. ACM Press, New York (2006)
72. Tang, A., Carpendale, S.: An observational study on information flow during nurses' shift change. In: Proc. of the ACM Conf. on Human Factors in Computing Systems (CHI), pp. 219-228. ACM Press, New York (2007)
73. Tang, J.C.: Findings from observational studies of collaborative work. International Journal of Man-Machine Studies 34(2), 143-160 (1991)
74. Tory, M., Möller, T.: Evaluating Visualizations: Do Expert Reviews Work. IEEE Computer Graphics and Applications 25(5), 8-11 (2005)
75. Tufte, E.: The Visual Display of Quantitative Information. Graphics Press, Cheshire (1986)
76. Tufte, E.: Envisioning Information. Graphics Press, Cheshire (1990)
77. Tufte, E.: Visual Explanations. Images and Quantities, Evidence and Narrative. Graphics Press, Cheshire (1997)
78. Viegas, F.B., Wattenberg, M., van Ham, F., Kriss, J., McKeon, M.: Many Eyes: A Site for Visualization at Internet Scale. IEEE Transactions on Visualization and Computer Graphics (Proceedings Visualization / Information Visualization 2007) 12(5), 1121-1128 (2007)
79. Ware, C.: Information Visualization: Perception for Design, 2nd edn. Morgan Kaufmann, San Francisco (2004)
80. Wigdor, D., Shen, C., Forlines, C., Balakrishnan, R.: Perception of Elementary Graphical Elements in Tabletop and Multi-surface Environments. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI'07), pp. 473-482. ACM Press, New York (2007)
81. Willett, W., Heer, J., Agrawala, M.: Scented Widgets: Improving Navigation Cues with Embedded Visualizations. In: INFOVIS 2007. IEEE Symposium on Information Visualization (2007)
82. Yost, B., North, C.: The Perceptual Scalability of Visualization. IEEE Transactions on Visualization and Computer Graphics 12(5), 837-844 (2006)
83. Zuk, T.: Uncertainty Visualizations. PhD thesis. Department of Compute Science, University of Calgary (2007)
84. Zuk, T., Carpendale, S.: Theoretical Analysis of Uncertainty Visualizations. In: Proceedings of SPIE Conference Electronic Imaging, Vol. 6060: Visualization and Data Analysis (2006)
85. Zuk, T., Schlesier, L., Neumann, P., Hancock, M.S., Carpendale, S.: Heuristics for Information Visualization Evaluation. In: Proceedings of the Workshop BEyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV 2006), held in conjunction with the Working Conference on Advanced Visual Interfaces (AVI 2006), ACM Press, New York (2006)