Patterns for visualization evaluation

ACM International Conference Proceeding Series

Volumn , Issue , 2012, Pages

  Elmqvist, Niklas ^a   Yi, Ji Soo ^b  

^a PURDUE UNIVERSITY   (United States)

^b SCHOOL OF INDUSTRIAL ENGINEERING   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PATTERN LANGUAGES; VISUAL SENSEMAKING;

DATA VISUALIZATION; SOFTWARE ENGINEERING; VISUALIZATION;

ERRORS;

EID: 84874876919     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2442576.2442588     Document Type: Conference Paper

References (49)

1. C. Alexander, S. Ishikawa, M. Silverstein, M. Jacobson, I. Fiksdahl-King, and S. Angel. A Pattern Language: Towns, Buildings, Construction. Oxford University Press, 1977.
2. C. Andrews, A. Endert, and C. North. Space to think: Large, high-resolution displays for sensemaking. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 55-64, 2010.
3. R. Arias-Hernandez, L. T. Kaastra, T. M. Green, and B. Fisher. Pair analytics: Capturing reasoning processes in collaborative visual analytics. In Proceedings of the Hawaii International Conference on System Sciences, pages 1-10, 2011.
4. D. Armstrong, A. Gosling, J. Weinman, and T. Marteau. The place of inter-rater reliability in qualitative research: An empirical study. Sciology, pages 597-606, 1997.
5. E. A. Bier, S. K. Card, and J. W. Bodnar. Entity-based collaboration tools for intelligence analysis. In Proceedings of IEEE Symposium on Visual Analytics Science & Technology, pages 99-106, 2008.
6. S. Björk and J. Holopainen. Patterns in Game Design. Charles River Media, 2004.
7. C. Callison-Burch. Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 286-295, 2009.
8. S. Carpendale. Evaluating information visualizations. In Information Visualization: Human-Centered Issues and Perspectives, number 4950 in Lecture Notes in Computer Science, pages 19-45. Springer, 2008.
9. J. W. Creswell. Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Sage Publications, 1997.
10. J. S. Downs, M. B. Holbrook, S. Sheng, and L. F. Cranor. Are your participants gaming the system?: screening Mechanical Turk workers. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 2399-2402, 2010.
11. T. Dwyer, B. Lee, D. Fisher, K. I. Quinn, P. Isenberg, G. G. Robertson, and C. North. A comparison of user-generated and automatic graph layouts. IEEE Transactions on Visualization and Computer Graphics, 15(6):961-968, 2009.
12. N. Elmqvist, P. Dragicevic, and J.-D. Fekete. Color lens: Adaptive color scale optimization for visual exploration. IEEE Transactions on Visualization and Computer Graphics, 17(6):795-807, 2011.
13. N. Elmqvist, N. Henry, Y. Riche, and J.-D. Fekete. Mélange: Space folding for multi-focus interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 1333-1342, 2008.
14. B. R. Gaines. Modeling and forecasting the information sciences. Information Science, 57-58:3-22, 1991.
15. E. Gamma. Design patterns - past, present & future. In S. Nanz, editor, The Future of Software Engineering. Springer, 2010.
16. E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-oriented Software. Addison Wesley, Boston, MA, 1994.
17. S. Ghani, N. Elmqvist, and J.-S. Yi. Perception of animated node-link diagrams for dynamic graphs. Computer Graphics Forum, 31(3):1205-1214, 2012.
18. J. Heer and M. Agrawala. Software design patterns for information visualization. IEEE Transactions on Visualization and Computer Graphics, 12(5):853-860, 2006.
19. J. Heer and M. Bostock. Crowdsourcing graphical perception: using Mechanical Turk to assess visualization design. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 203-212, 2010.
20. J. Heer, N. Kong, and M. Agrawala. Sizing the horizon: the effects of chart size and layering on the graphical perception of time series visualization. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 1303-1312, 2009.
21. J. Heer and G. Robertson. Animated transitions in statistical data graphics. IEEE Transactions on Visusalization and Computer Graphics, 13(6):1240-1247, 2007.
22. P. G. Ipeirotis. Analyzing the Amazon Mechanical Turk marketplace. XRDS: Crossroads, The ACM Magazine for Students, 17(2):16-21, 2010.
23. P. Isenberg, A. Tang, and S. Carpendale. An exploratory study of visual information analysis. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 1217-1226, 2008.
24. W. Javed, S. Ghani, and N. Elmqvist. PolyZoom: Multiscale and multifocus exploration in 2D visual spaces. In Proceedings of ACM Conference on Human Factors in Computing Systems, pages 287-296, 2012.
25. W. Javed, K. Kim, S. Ghani, and N. Elmqvist. Evaluating physical/virtual occlusion management techniques for horizontal displays. In Proceedings of INTERACT, pages 391-408, 2011.
26. D. H. Jeong, W. Dou, F. Stukes, W. Ribarsky, H. R. Lipford, and R. Chang. Evaluating the relationship between user interaction and financial visual analysis. In Proceedings of the IEEE Symposium on Visual Analytics Science & Technology, pages 83-90, 2008.
27. Y. Kang, C. Görg, and J. Stasko. Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study. In Proceedings of the IEEE Symposium on Visual Analytics Science & Technology, pages 139-146, 2009.
28. S.-H. Kim, Z. Dong, H. Xian, U. Benjavan, and J. S. Yi. Does an eye tracker tell us the truth about visualizations?: Findings while investigating visualizations for decision making. IEEE Transactions on Visualization and Computer Graphics, 2012.
29. R. Kosara and C. Ziemkiewicz. Do Mechanical Turks dream of square pie charts? In Proceedings of BEyond time and errors: novel evaLuation methods for Information Visualization, pages 63-70, 2010.
30. B. Kwon, W. Javed, S. Ghani, N. Elmqvist, J. S. Yi, and D. S. Ebert. Evaluating the role of time in investigative analysis of document collections. IEEE Transactions on Visualization and Computer Graphics, 2012.
31. H. Lam. A framework of interaction costs in information visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6):1149-1156, 2008.
32. H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale. Empirical studies in information visualization: Seven scenarios. IEEE Transactions on Visualization and Computer Graphics, 18(9):1520-1536, 2012.
33. H. Lam, T. Munzner, and R. Kincaid. Overview use in multiple visual information resolution interfaces. IEEE Transactions on Visualization and Computer Graphics, 13(6):1278-1285, 2007.
34. T. Munzner. Process and pitfalls in writing information visualization research papers. In A. Kerren, J. T. Stasko, J.-D. Fekete, and C. North, editors, Information Visualization: Human-Centered Issues and Perspectives, number 4950 in Lecture Notes in Computer Science. Springer, 2008.
35. T. Munzner. A nested process model for visualization design and validation. IEEE Trans. on Visualization and Computer Graphics, 15(6):921-928, 2009.
36. D. Nekrasovski, A. Bodnar, J. McGrenere, F. Guimbretiere, and T. Munzner. An evaluation of pan & zoom and rubber sheet navigation with and without an overview. In Proceedings of ACM Conference on Human Factors in Computing Systems, pages 11-20, 2006.
37. C. North. Toward measuring visualization insight. IEEE Computer Graphics & Appl., 26(3):6-9, 2006.
38. E. Pietriga, C. Appert, and M. Beaudouin-Lafon. Pointing and beyond: an operationalization and preliminary evaluation of multi-scale searching. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pages 1215-1224, 2007.
39. C. Plaisant. The challenge of information visualization evaluation. In Proceedings of the ACM Conference on Advanced Visual Interfaces, pages 109-116, 2004.
40. C. Plaisant, G. G. Grinstein, J. Scholtz, M. Whiting, T. O'Connell, S. J. Laskowski, L. Chien, A. Tat, W. Wright, C. Görg, Z. Liu, N. Parekh, K. Singhal, and J. T. Stasko. Evaluating visual analytics at the 2007 VAST symposium contest. IEEE Computer Graphics & Applications, 28(2):12-21, 2008.
41. G. Robertson, R. Fernandez, D. Fisher, B. Lee, and J. Stasko. Effectiveness of animation in trend visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6):1325-1332, 2008.
42. A. C. Robinson. Collaborative synthesis of visual analytic results. In Proceedings of the IEEE Symposium on Visual Analytics Science & Technology, pages 67-74, 2008.
43. J. Rogstadius, V. Kostakos, A. Kittur, B. Smus, J. Laredo, and M. Vukovic. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In Proceedings of the AAAI Conference on Weblogs and Social Media, 2011.
44. P. Saraiya, C. North, V. Lam, and K. A. Duca. An insight-based longitudinal study of visual analytics. IEEE Transactions on Visualization and Computer Graphics, 12(6):1511-1522, 2006.
45. A. D. Shaw, J. J. Horton, and D. L. Chen. Designing incentives for inexpert human raters. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work, pages 275-284, 2011.
46. J. T. Stasko, C. Görg, and Z. Liu. Jigsaw: supporting investigative analysis through interactive visualization. Information Visualization, 7(2):118-132, 2008.
47. H. E. Tinsley and D. J. Weiss. Interrater reliability and agreement of subjective judgments. Journal of Counseling Psychology, 22(4):358-374, 1975.
48. M. Tory and T. Möller. Evaluating visualizations: Do expert reviews work? IEEE Computer Graphics & Applications, 25:8-11, 2005.
49. F. van Ham and B. E. Rogowitz. Perceptual organization in user-generated graph layouts. IEEE Transactions on Visualization and Computer Graphics, 14(6):1333-1339, 2008.