User-centered evaluation of information visualization techniques: Making the HCI-infovis connection explicit

Handbook of Human Centric Visualization

Volumn , Issue , 2014, Pages 315-336

  Freitas, Carla M D S ^a   Pimenta, Marcelo S ^a   Scapin, Dominique L ^b  

^a FEDERAL UNIVERSITY OF RIO GRANDE DO SUL   (Brazil)

^b INRIA ROCQUENCOURT   (France)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN COMPUTER INTERACTION; INFORMATION ANALYSIS; INFORMATION SYSTEMS; PETROLEUM RESERVOIR EVALUATION; USABILITY ENGINEERING; VISUALIZATION;

INFORMATION VISUALIZATION; ON CURRENTS; QUALITY CRITERIA; USABILITY EVALUATION; USABILITY EVALUATION METHODS; USER EXPERIENCE; USER-CENTERED; VISUALIZATION TECHNIQUE;

QUALITY CONTROL;

EID: 84930368291     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-7485-2_12     Document Type: Chapter

References (79)

1. G. Albuquerque, M. Eisemann, and M. Marcus. Perception-Based Visual Quality Measures. Proceedings of IEEE Conference on Visual Analytics Science and Technology (Providence, RI, USA, October 23-28, 2011), IEEE, pp. 13-20, 2011.
2. K. Allendoerfer, S. Aluker, G. Panjwani, J. Proctor, D. Sturtz, M. Vukovic, and C. Chen. Adapting the Cognitive Walkthrough Method to Assess the Usability of a Knowledge Domain Visualization. Proceedings of IEEE Symposium on Information Visualization (Minneapolis, MN, USA, October 23-25, 2005), IEEE, pp. 26, 2005.
3. R. Amar and J. Stasko. A knowledge task-based framework for the design and evaluation of information visualizations. Proceedings of IEEE Symposium on Information Visualization (Austin, TX, USA, October 10-12, 2004), IEEE, pp. 143-149, 2004.
4. R. Amar, J. Eagan and J. Stasko. Low-Level Components of Analytic Activity in Information Visualization. Proceedings of IEEE Symposium on Information Visualization (Minneapolis, Minnesota, USA, October 23-25, 2005), IEEE, pp. 111-147, 2005
5. M. Bazire and R Brezillon. Understanding Context Before Using It. Proceedings of the 5th international conference on Modeling and Using Context (CONTEXT'05), Springer-Verlag, Berlin, Heidelberg, pp. 29-40, 2005.
6. BELIV: Beyond Time and Error: Novel Evaluation Methods for Information Visualization. Proceedings of Workshop of the Advanced Visual Interfaces Conference (Venice, Italy, May 23, 2006), ACM Digital Library, Eds. E. Bertini, C. Plaisant; G. Santucci, 2006
7. BELIV: Beyond Time and Error: Novel Evaluation Methods for Information Visualization. Proceedings of Workshop of the Conference on Human Factors in Computing Systems (Florence, Italy, April 5, 2008), ACM Digital Library, Eds. E. Bertini, C. Plaisant; G. Santucci, 2008.
8. BELIV: Beyond Time and Error: Novel Evaluation Methods for Information Visualization. Proceedings of Workshop of the Conference on Human Factors in Computing Systems (Atlanta, USA, April 10-11, 2010), ACM Digital Library, 2010, Eds. E. Bertini, A. Perer; H. Lam, 2010.
9. E. Bertini, H. Lam, and A. Perer. Special Issue on Evaluation for Information Visualization. In Information Visualization, volume 10, no. 3, July 2011. (online version available at http://ivi.sagepub.com/content/10/3)
10. E. Bertini and G. Santucci. Visual Quality Metrics. Proceedings of BELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 44-48, 2006.
11. N. Bevan. Cost benefits framework and case studies. In R.G. Bias and D.J. Mayhew (eds): Cost-Justifying Usability: An Update for the Internet Age. Morgan Kaufmann, 2005.
12. R.G. Bias, A. Druin, B.M. Wildemut, and S. Hirsch. Usability in practice: Avoiding pitfalls and seizing opportunities. In Proceedings of the Annual Meeting of the American Society of Information Science & Technology, volume 40, no. 1, pp. 432-433, 2003.
13. R. Brath. Concept Demonstration: Metrics for Effective Information Visualization. Proceedings of the IEEE Symposium on Information Visualization (Phoenix, AZ, USA, October 1997), IEEE, pp. 108-111, 1997.
14. S. Caffiau, D. L. Scapin, P. Girard, M. Baron, and F. Jambon. Increasing the expressive power of task analysis: systematic comparison and empirical assessment of tool-supported task models. In Interacting with Computers, volume 22, no. 6, pp. 569-593, November 2010.
15. S. K. Card, T. P. Moran, and A. Newell. The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993.
16. S. Carpendale. Evaluating Information Visualizations. In A. Kerren, J. Stasko, J-D. Fekete, C. North (eds.) Information Visualization, Lecture Notes in Computer Science, volume 4950, Berlin-Heilderberg, pp. 19-45, 2008.
17. J. Carrierre and R. Kazman. Interacting with Hierarchies: Beyond Cone Trees. In Proceedings of Information Visualization Symposium (Atlanta, GA, USA, October 30-31, 1995), IEEE, pp. 74-81, 1995.
18. R. Cava, P. R. G. Luzzardi, and C. D. S. Freitas. The Bifocal Tree: a Technique for the Visualization of Hierarchical Information Structures. Proceedings ofIHC 2002-5th Workshop on Human Factors in Computer Systems (Fortaleza, CE, Brazil, October 2002). SBC/UFCe, 2002
19. C. Chen and Y. Yu. Empirical studies of information visualization: A meta-analysis. In International Journal of Human-Computer Studies, volume 53, no. 5, pp. 851-866. 2000.
20. S. Conversy, S. Chatty, and C. Hurte. Visual scanning as a reference framework for interactive representation design. In Information Visualization, volume 10, no. 3, pp. 196-211, 2011.
21. P. Dourish. What we talk about when we talk about context. In Personal and Ubiquitous Computing, volume 8, no. 1, pp. 19-30, 2004.
22. G. Ellis and A. Dix. (2006) An Explorative Analysis of User Evaluation Studies. Proceedings of BELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 15-20, 2006.
23. N. Elmqvist, P. Dragicevic, and J-D. Fekete. Rolling the Dice: Multidimensional Visual Exploration using Scatterplot Matrix Navigation. In IEEE Transactions on Visualization and Computer Graphics, volume 14, no.6, pp. 1141-1148, 2008.
24. C. Forsell. A Guide to Scientific Evaluation in Information Visualization. Proceedings of the 14th International Conference on Information Visualisation (July 27-29, London, UK), IEEE, pp. 162-169, 2010.
25. C. Forsell and J. Johansson. An heuristic set for evaluation in information visualization. Proceedings of AVI 2010 (Rome, Italy, May 25-29, 2010), ACM Digital Library, pp. 199-206, 2010.
26. C.M.D.S. Freitas, P.R.G. Luzzardi, R.A. Cava, M.A.A. Winckler, M.S. Pimenta, and L.P. Nedel. Evaluating Usability of Information Visualization Techniques. In Proceedings of IHC 2002-5th Workshop on Human Factors in Computer Systems (Fortaleza, CE, Brazil, October 2002). SBC/UFCe, pp. 40-51, 2002
27. C.M.D.S. Freitas, M.S. Pimenta, and D. Scapin. User-Centered Evaluation of Information Visualization Techniques: Issues and Perspectives In Brazil/INRIA Colloquium: Cooperation, Advances and Challenges (Bento Goncalves, RS, Brazil, July 2009), INF/UFRGS, pp. 199-202, 2009.
28. J. Goldberg and J. Helfman. Eye tracking for visualization evaluation: Reading values on linear versus radial graphs. In Information Visualization, volume 10, no. 3, pp. 182-195, 2011.
29. J. Goldberg, J. Helfman. Eye Tracking on Visualizations: Progressive Extraction of Scanning Strategies. This book, chapter 13.
30. V. Gonzales and A. Kobsa. A Workplace Study of the Adoption of Information Visualization Systems. Proceedings of I-KNOW'03: Third Int'l Conf. Knowledge Management (2003), pp. 96-102, 2003.
31. H.R. Hartson, T.S. Andre and R.C. Williges. Criteria for evaluating usability evaluation methods. In International Journal of Human-Computer Interaction, volume13, pp. 373-410, 2001
32. E. Hetzler and A. Turner. Analysis Experiences Using Information Visualization. In IEEE Computer Graphics and Applications, volume 24, no. 5, pp. 22-26, 2004.
33. D. Hix and H.R. Hartson. Developing User Interfaces: Ensuring Usability through Product & Process. John Wiley & Sons Inc, 1993.
34. ISO 9241-11: 1998. Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11: Guidance on usability, 1998.
35. ISO/TR 16982: 2002. Ergonomics of human system interaction - Usability methods supporting human centered design, 2002.
36. O. Kulyk, R. Kosara, J. Urquiza, and I. Wassink. Human-Centered Aspects. In A. Kerren et al. (eds.): Human-Centered Visualization Environments, LNCS 4417, Springer-Verlag, Berlin Heidelberg, pp. 13-75, 2007.
37. H. Lam and T. Munzner. Increasing the utility of quantitative empirical studies for metaanalysis. Proceedings of BELIV '08 (Florence, Italy, April 5, 2008), ACM Digital Library, Article 2, 7 pages, 2008.
38. J. Lamping, R. Rao, and P. Pirolli. A focus C context technique based on hyperbolic geometry for visualizing large hierarchies. Proceedings of the CHI '95 (Denver, CO, May 07-11, 1995) ACM Press/Addison-Wesley Publishing Co., New York, USA, pp. 401-408, 1995.
39. B. Lee, C. Plaisant and C.S. Parr. Task Taxonomy for Graph Visualization. Proceedings of BELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 82-86, 2006.
40. S. M. McNee and B. Arnette. Productivity as a Metric for Visual Analytics: Reflections on E-Discovery. Proceedings of BELIV '08 (Florence, Italy, April 5, 2008), ACM Digital Library, Article 1, 2008.
41. T. Munzner. A Nested Model for Visualization Design and Validation. In IEEE Transactions on Visualization and Computer Graphics, volume 15, no. 6, pp. 921-928, 2006.
42. J. Nielsen. Heuristic Evaluation. In J. Nielsen and R.L. Mack (eds) Usability Inspection Methods. John Wiley & Sons, pp. 25-62, 1994.
43. D. Norman. Psychology of Everyday Action. In The Design of Everyday Things. New York: Basic Book, 1988.
44. C. North. Toward Measuring Visualization Insight. In IEEE Computer Graphics and Applications, volume 26, no. 3, pp. 6-9, 2006.
45. C. North, P. Saraiya, and K. Duca. A comparison of benchmark task and insight evaluation methods for information visualization. In Information Visualization, volume 10, no. 3, pp. 161-181, 2011
46. W. Peng, M.O. Ward, and E.A. Rundensteiner. Clutter Reduction in Multi-Dimensional Data Visualization Using Dimension Reordering. Proceedings of IEEE Symposium on Information Visualization (Austin, TX, USA, October 10-12, 2004), IEEE, pp. 89-96, 2004.
47. A. Perer and B. Shneiderman. Integrating Statistics and Visualization: Case Studies of Gaining Clarity during Exploratory Data Analysis. Proceedings of CHI 2008 (Florence, Italy, April 05-10, 2008) ACM Press, pp. 265-274, 2008.
48. P. Pirolli and R. Rao. Table lens as a tool for making sense of data. Proceedings of the AVI'96 (Gubbio, Italy, May 27-29, 1996), ACM, New York, USA, pp. 67-80, 1996.
49. C. Plaisant. The Challenge of Information Visualization Evaluation. Proceedings of AVI'04 (Gallipoli, Italy, May 25-28, 2004), ACM Press, pp. 109-116, 2004.
50. C. Plaisant, J-D. Fekete, and G. Grinstein. Promoting Insight-Based Evaluation of Visualizations: From Contest to Benchmark Repository. In IEEE Transactions on Visualization and Computer Graphics, volume 14, no. 1, pp. 120-134, 2008.
51. R. Rao and S. Card. The Table Lens: Merging Graphical and Symbolic Representations in an Interactive Focus C Context Visualization for Tabular Information. Proceedings of CHI'94 (Boston, Massachusetts, April 24-28, 1994), ACM Press, pp. 111-117, 1994.
52. Y. Rogers, H. Sharp, J. Preece. Interaction Design: Beyond Human-Computer Interaction. John-Wiley, 3rd edition, 2011.
53. P. Saraiya, C. North, and K. Duca. An insight-based methodology for evaluating bioinformatics visualizations. In IEEE Transactions on Visualization and Computer Graphics, volume 11, no. 4, pp. 443-456, July-August 2005
54. P. Saraiya, C. North, V. Lam, and K.A. Duca. An Insight-Based Longitudinal Study of Visual Analytics. In IEEE Transactions on Visualization and Computer Graphics, volume 12, no. 6, pp. 1511-1522, November/December 2006
55. D. L. Scapin and J.M.C. Bastien. Ergonomic Criteria for Evaluating the Ergonomic Quality of Interactive Systems. In Behaviour and Information Technology, volume 16, no. 4, pp. 220-231, 1997.
56. D. L. Scapin, B. Senach, B. Trousse, and M. Pallot. User Experience: Buzzword or New Paradigm? Proceedings of ACHI 2012 - The Fifth International Conference on Advances in Computer Human Interactions (Valencia, Spain, Jan 30-Feb 4, 2012).
57. D. L. Scapin, E. Law, and N. Bevan. Review, Report and Refine Usability Evaluation Methods. In 3rd COST294-MAUSE Project International Workshop (Athens, March 5, 2007). Technical Report available at http://141.115.28.2/cost294/upload/522.pdf
58. J. Scholtz. Developing guidelines for assessing visual analytics environments. In Information Visualization, volume 10, no. 3, pp. 212-231, 2011.
59. J. Seo and B. Shneiderman. Knowledge discovery in high dimensional data: Case studies and a user survey for the rank-by-feature framework. In IEEE Transactions on Visualization and Computer Graphics, volume12, no. 3, pp. 311-322, May/June, 2006.
60. B. Shneiderman. The Eyes Have It: A Task by Data Type Taxonomy for Information Visualization. Proceedings of the IEEE Symposium on Visual Languages (Boulder, Co., Sept. 3-6, 1996), IEEE, pp. 326-343, 1996.
61. B. Shneiderman and C. Plaisant. Strategies for Evaluating Information Visualization Tools: Multi-dimensional In-depth Long-term Case Studies Proceedings ofBELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 38-43, 2006.
62. M. Smuc, E. Mayr, T. Lammarsch, W. Aigner, S. Miksch, and J. Gartner. To score or not to score? Tripling insights for participatory design. In IEEE Computer Graphics and Applications, volume 29, no. 3, pp. 29-38, 2009.
63. R. Springmeyer, M. Blattner and N. L. Max. A characterization of the scientific data analysis process. In Proceedings of IEEE Visualization (San Francisco, October 23-26, 1990), pp. 235-242, 1990.
64. A.S. Spritzer and C.M.D.S. Freitas. A Physics-based Approach for Interactive Manipulation of Graph Visualizations. Proceedings of AVI'08 (Napoli, Italy, May 2008), ACM Press, pp. 271-278, 2008.
65. A.S. Spritzer and C.M.D.S. Freitas. Design and Evaluation of MagnetViz - a Graph Visualization Tool. In IEEE Transactions on Visualization and Computer Graphics, volume 18, no. 5, pp. 822-835, 2012.
66. J. Stasko. Evaluating Information Visualizations: Issues and Opportunities. Proceedings of Workshop of the Advanced Visual Interfaces Conference (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 5-8, 2006.
67. M. Tory and T. Moller. Human Factors in Visualization Research. In IEEE Transactions on Visualization and Computer Graphics, volume 10, no.1, pp. 72-84, 2004.
68. M. Tory and T. Moeller. Evaluating Visualizations: Do Expert Reviews Work? In IEEE Computer Graphics and Applications, volume 25, no. 5, pp. 8-11, 2005.
69. M. Tory. User Studies in Visualization: A Reflection on Methods. This book, chapter 16
70. E.R.A. Valiati, C.M.D.S. Freitas, and M.S. Pimenta. A Taxonomy of Tasks for Guiding the Evaluation of Multidimensional Visualizations. Proceedings of BELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 86-91, 2006
71. E.R.A. Valiati, M.S. Pimenta, and C.M.D.S. Freitas. Using Multi-dimensional In-depth Long Term Case Studies for Information Visualization Evaluation. Proceedings of BELIV'08 (Florence, Italy, April 5). ACM Digital Library, Article 9, 2008.
72. S. Wehrend and C. Lewis. A Problem-oriented Classification of Visualization Technique. Proceedings of IEEE Visualization (San Francisco, October 23-26, 1990), pp. 139-143, 1990.
73. M. Winckler, C.M.D.S. Freitas, and P. Palanque. Tasks and Scenario-based Evaluation of Information Visualization Techniques. Proceedings of 3rd International Workshop on Task Models and Diagrams for User Interface Design - TAMODIA'2004 (Prague, Czech Republic, 2004), ACM, pp. 165-172, 2004.
74. U. Wiss and D.A. Carr. An Empirical Study of Task Support in 3D Information Visualizations. Proceedings of International Conference on Information Visualisation, (London, England, 1999), IEEE, pp. 392-399, 1999.
75. J. Yang-Pelaez and W.C. Flowers. Information Content Measures of Visual Displays. Proceedings of IEEE Symposium on Information Visualization (Salt Lake City, Utah, 2000) IEEE, pp. 99-104, 2000.
76. J.S. Yi, Y. Kang, J. Stasko, and J. Jacko. Understanding and Characterizing Insights: How Do People Gain Insights Using Information Visualization? Proceedings of BELIV'08 (Florence, Italy, April 5). ACM Digital Library, Article 4, 2008.
77. M. Zhou and S.K. Feiner. Visual Task Characterization for Automated Visual Discourse Synthesis. Proceedings of CHI' 98 (Los Angeles, USA, April 18-23), ACM Press, pp. 392-399, 1998.
78. T. Zuk and S. Carpendale. Theoretical analysis of uncertainty visualizations. Proceedings of SPIE-IS&T Electronic Imaging, Vol. 6060: Visualization and Data Analysis 2006. (Robert F. Erbacher and Jonathan C. Roberts and Matti T. Gröhn and Katy Börner, Eds.) SPIE, page 606007, 2006.
79. T. Zuk, L. Schleiser, P. Neumann, M.S. Hancock, and S. Carpendale. Heuristics for Information Visualization Evaluation. Proceedings of BELIV'06 (Venice, Italy, May 23, 2006), ACM Digital Library, pp. 55-60, 2006.