A Case Study Using Visualization Interaction Logs and Insight Metrics to Understand How Analysts Arrive at Insights

IEEE Transactions on Visualization and Computer Graphics

Volumn 22, Issue 1, 2016, Pages 51-60

  Guo, Hua ^a   Gomez, Steven R ^a   Ziemkiewicz, Caroline ^b   Laidlaw, David H ^a  

^a BROWN UNIVERSITY   (United States)

^b APTIMA INC   (United States)

Author keywords

Correlation; Encoding; History; Measurement; Statistical analysis; Visual analytics

Indexed keywords

COMPUTER GRAPHICS; CORRELATION METHODS; ENCODING (SYMBOLS); HISTORY; MEASUREMENTS; SOFTWARE ENGINEERING; STATISTICAL METHODS;

ANALYSIS METHOD; APPLICATION SPECIFIC; BENEFITS AND COSTS; INTERACTION PATTERN; QUALITATIVE ANALYSIS; USER STUDY; VISUAL ANALYTICS; VISUALIZATION TOOLS;

VISUALIZATION;

EID: 84947104857     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2015.2467613     Document Type: Article

References (34)

1. VAST Challenge 2014. http://hcil2.cs.umd.edu/newvarepository/benchmarks.php#VAST2014. Accessed: 2015-03-24.
2. E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 19-26. ACM, 2006.
3. R. Amar, J. Eagan, and J. Stasko. Low-level components of analytic activity in information visualization. In Information Visualization, 2005. INFOVIS 2005. IEEE Symposium on, pages 111-117. IEEE, 2005.
4. I. Boyandin, E. Bertini, and D. Lalanne. A qualitative study on the exploration of temporal changes in flow maps with animation and small-multiples. In Computer Graphics Forum, volume 31, pages 1005-1014. Wiley Online Library, 2012.
5. E. T. Brown, A. Ottley, H. Zhao, Q. Lin, R. Souvenir, A. Endert, and R. Chang. Finding waldo: Learning about users from their interactions. Visualization and Computer Graphics, IEEE Transactions on, 20(12):1663-1672, 2014.
6. R. Chang, C. Ziemkiewicz, T. M. Green, and W. Ribarsky. Defining insight for visual analytics. Computer Graphics and Applications, IEEE, 29(2):14-17, 2009.
7. E. H. Chi, P. Pirolli, K. Chen, and J. Pitkow. Using information scent to model user information needs and actions and the web. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 490-497. ACM, 2001.
8. W. Dou, D. H. Jeong, F. Stukes, W. Ribarsky, H. R. Lipford, and R. Chang. Recovering reasoning processes from user interactions. IEEE Computer Graphics and Applications, (3):52-61, 2009.
9. J. D. Evans. Straightforward statistics for the behavioral sciences. Brooks/Cole, 1996.
10. S. Gomez, H. Guo, C. Ziemkiewicz, and D. Laidlaw. An insight- and task-based methodology for evaluating spatiotemporal visual analytics. In Visual Analytics Science and Technology (VAST), 2014 IEEE Conference on, pages 63-72, Oct 2014.
11. S. Gomez and D. Laidlaw. Modeling task performance for a crowd of users from interaction histories. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2465-2468. ACM, 2012.
12. D. Gotz, J. Lu, P. Kissa, N. Cao, W. H. Qian, S. X. Liu, and M. X. Zhou. Harvest: an intelligent visual analytic tool for the masses. In Proceedings of the first international workshop on Intelligent visual interfaces for text analysis, pages 1-4. ACM, 2010.
13. D. Gotz and M. X. Zhou. Characterizing users' visual analytic activity for insight provenance. Information Visualization, 8(1):42-55, 2009.
14. W. D. Gray, B. E. John, and M. E. Atwood. Project ernestine: Validating a goms analysis for predicting and explaining real-world task performance. Human-computer interaction, 8(3):237-309, 1993.
15. J. Heer, J. Mackinlay, C. Stolte, and M. Agrawala. Graphical histories for visualization: Supporting analysis, communication, and evaluation. Visualization and Computer Graphics, IEEE Transactions on, 14(6):1189-1196, 2008.
16. J. Heer and B. Shneiderman. Interactive dynamics for visual analysis. Queue, 10(2):30, 2012.
17. E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. The lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, pages 256-265. Morgan Kaufmann Publishers Inc., 1998.
18. Y.-a. Kang, C. Gorg, and J. Stasko. Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study. In Visual Analytics Science and Technology, 2009. VAST 2009. IEEE Symposium on, pages 139-146. IEEE, 2009.
19. A. Kerne and S. M. Smith. The information discovery framework. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques, pages 357-360. ACM, 2004.
20. Z. Liu and J. Heer. The effects of interactive latency on exploratory visual analysis. IEEE Transactions on Visualization and Computer Graphics, 2014.
21. C. North. Toward measuring visualization insight. Computer Graphics and Applications, IEEE, 26(3):6-9, 2006.
22. C. North, P. Saraiya, and K. Duca. A comparison of benchmark task and insight evaluation methods for information visualization. Information Visualization, 10(3):162-181, July 2011.
23. P. Pirolli and S. Card. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, volume 5, pages 2-4, 2005.
24. C. Plaisant, J. Fekete, and G. Grinstein. Promoting insight-based evaluation of visualizations: From contest to benchmark repository. Visualization and Computer Graphics, IEEE Transactions on, 14(1):120-134, 2008.
25. K. Reda, A. E. Johnson, J. Leigh, and M. E. Papka. Evaluating user behavior and strategy during visual exploration. In Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, pages 41-45. ACM, 2014.
26. D. M. Russell, M. J. Stefik, P. Pirolli, and S. K. Card. The cost structure of sensemaking. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems, pages 269-276. ACM, 1993.
27. P. Saraiya, C. North, and K. Duca. An insight-based methodology for evaluating bioinformatics visualizations. IEEE Transactions on Visualization and Computer Graphics, 11(4):443-456, July 2005.
28. B. Shneiderman. The eyes have it: A task by data type taxonomy for information visualizations. In Visual Languages, 1996. Proceedings., IEEE Symposium on, pages 336-343. IEEE, 1996.
29. B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multi-dimensional in-depth long-term case studies. In Proceedings of the 2006 AVI Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, BELIV '06, pages 1-7, New York, NY, USA, 2006. ACM.
30. J. J. Thomas. Illuminating the path:[the research and development agenda for visual analytics]. IEEE Computer Society, 2005.
31. A. Vande Moere, M. Tomitsch, C. Wimmer, B. Christoph, and T. Grechenig. Evaluating the effect of style in information visualization. Visualization and Computer Graphics, IEEE Transactions on, 18(12):2739-2748, 2012.
32. J. S. Yi, Y. ah Kang, J. T. Stasko, and J. A. Jacko. Toward a deeper understanding of the role of interaction in information visualization. Visualization and Computer Graphics, IEEE Transactions on, 13(6):1224-1231, 2007.
33. J. S. Yi, Y.-a. Kang, J. T. Stasko, and J. A. Jacko. Understanding and characterizing insights: how do people gain insights using information visualization? In Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization, page 4. ACM, 2008.
34. M. X. Zhou and S. K. Feiner. Visual task characterization for automated visual discourse synthesis. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 392-399. ACM Press/Addison-Wesley Publishing Co., 1998.