Visualization evaluation for cyber security: Trends and future directions

ACM International Conference Proceeding Series

Volumn 10-November-2014, Issue , 2014, Pages 49-56

  Staheli, Diane ^a   Yu, Tamara ^a   Crouser, R Jordan ^a   Damodaran, Suresh ^a   Nam, Kevin ^a   O'Gwynn, David ^a   McKenna, Sean ^b   Harrison, Lane ^c  

^a MIT LINCOLN LABORATORY   (United States)

^b UNIVERSITY OF UTAH   (United States)

^c TUFTS UNIVERSITY   (United States)

Author keywords

Cyber security; Evaluation; Information visualization

Indexed keywords

INFORMATION ANALYSIS; INFORMATION SCIENCE; INFORMATION SYSTEMS; VISUALIZATION;

CYBER SECURITY; EVALUATION; EVALUATION METRICS; INFORMATION VISUALIZATION; RESEARCH EFFORTS; STATE OF THE ART; TOOLS AND TECHNIQUES;

NETWORK SECURITY;

EID: 84982795906     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2671491.2671492     Document Type: Conference Paper

References (44)

1. D. Afergan, E. M. Peck, E. T. Solovey, A. Jenkins, S. W. Hincks, E. T. Brown, R. Chang, and R. J. Jacob. Dynamic difficulty using brain metrics of workload. In Proc. 32nd annual ACM Conf. on Human factors in computing systems, pages 3797-3806. ACM, 2014.
2. W. Albert and T. Tullis. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes, 2013.
3. R. Ball, G. A. Fink, and C. North. Home-centric visualization of network traffic for security administration. In Proc. 2004 ACM workshop on Visualization and data mining for computer security, pages 55-64. ACM, 2004.
4. L. Barkhuus and J. A. Rode. From mice to men-24 years of evaluation in CHI. In Proc. SIGCHI Conf. on Human factors in computing systems. ACM, 2007.
5. S. Bateman, C. Gutwin, N. Osgood, and G. McCalla. Interactive usability instrumentation. In Proc. 1st ACM SIGCHI symposium on Engineering interactive computing systems, pages 45-54. ACM, 2009.
6. H. Beyer and K. Holtzblatt. Contextual design: defining customer-centered systems. Elsevier, 1997.
7. S. Carpendale. Evaluating information visualizations. In Information Visualization, pages 19-45. Springer, 2008.
8. Guide to a successful archive submission. Online, July 2014.
9. V. P. Committee. Paper submission guidelines: Paper types. Online, jul 2014.
10. W. Dou, D. H. Jeong, F. Stukes, W. Ribarsky, H. R. Lipford, and R. Chang. Recovering reasoning process from user interactions. IEEE Computer Graphics & Applications, 2009.
11. N. Elmqvist and J. S. Yi. Patterns for visualization evaluation. Information Visualization, page 1473871613513228, 2013.
12. G. A. Fink, C. L. North, A. Endert, and S. Rose. Visualizing cyber security: Usable workspaces. In Visualization for Cyber Security, 2009. VizSec 2009. 6th Int'l Workshop on, pages 45-56. IEEE, 2009.
13. C. Gates and S. Engle. Reecting on visualization for cyber security. In Intelligence and Security Informatics (ISI), 2013 IEEE Int'l Conf. on, pages 275-277. IEEE, 2013.
14. M. Gleicher. Why ask why?: considering motivation in visualization evaluation. In Proc. 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, page 10. ACM, 2012.
15. D. Gotz and M. X. Zhou. Characterizing users' visual analytic activity for insight provenance. Information Visualization, 8(1):42-55, 2009.
16. S. Greenberg and B. Buxton. Usability evaluation considered harmful (some of the time). In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 111-120. ACM, 2008.
17. L. Hao, C. G. Healey, and S. E. Hutchinson. Flexible web visualization for alert-based network security analytics. In Proc. Tenth Workshop on Visualization for Cyber Security, pages 33-40. ACM, 2013.
18. T. Isenberg, P. Isenberg, J. Chen, M. Sedlmair, and T. Moller. A systematic review on the practice of evaluating visualization. Visualization and Computer Graphics, IEEE Trans. on, 19(12):2818-2827, 2013.
19. B. Jackson, D. Coffey, L. Thorson, D. Schroeder, A. M. Ellingson, D. J. Nuckley, and D. F. Keefe. Toward mixed method evaluations of scientific visualizations and design process as an evaluation tool. In Proc. 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, page 4. ACM, 2012.
20. T. Jankun-Kelly, J. Franck, D. Wilson, J. Carver, D. Dampier, and J. E. SwanIi. Show me how you see: Lessons from studying computer forensics experts for visualization. In Visualization for Computer Security, pages 80-86. Springer, 2008.
21. H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale. Empirical studies in information visualization: Seven scenarios. Visualization and Computer Graphics, IEEE Trans. on, 18(9):1520-1536, 2012.
22. M. D. McNeese, K. Perusich, and J. R. Rentsch. Advancing socio-technical systems design via the living laboratory. In Proc. Human Factors and Ergonomics Society Annual Meeting, volume 44 Number 12, pages 2-610. SAGE Publications, 2000.
23. M. J. Muller. Participatory design: The third space in hci. Human-computer interaction: Development process, pages 165-185, 2003.
24. T. Munzner. Process and pitfalls in writing information visualization research papers. In Information visualization, pages 134-153. Springer, 2008.
25. T. Munzner. A nested model for visualization design and validation. Visualization and Computer Graphics, IEEE Trans. on, 15(6):921-928, 2009.
26. J. Nielsen. Usability inspection methods. In Conf. companion on Human factors in computing systems, pages 413-414. ACM, 1994.
27. C. North. Toward measuring visualization insight. Computer Graphics and Applications, IEEE, 26(3):6-9, 2006.
28. A. A. Ozok. Survey design and implementation in hci. Human-Computer Interaction: Development Process, page 253, 2009.
29. E. M. M. Peck, B. F. Yuksel, A. Ottley, R. J. Jacob, and R. Chang. Using fnirs brain sensing to evaluate information visualization interfaces. In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 473-482. ACM, 2013.
30. A. Perer and B. Shneiderman. Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis. In Proc. SIGCHI Conf. on Human Factors in computing systems, pages 265-274. ACM, 2008.
31. R. W. Picard and S. B. Daily. Evaluating a-ective interactions: Alternatives to asking what users feel. In CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches, pages 2119-2122, 2005.
32. W. A. Pike, J. Stasko, R. Chang, and T. A. O'Connell. The science of interaction. Information Visualization, 8(4):263-274, 2009.
33. C. Plaisant. The challenge of information visualization evaluation. In Proc. working Conf. on Advanced visual interfaces, pages 109-116. ACM, 2004.
34. J. Rasmussen, K. Ehrlich, S. Ross, S. Kirk, D. Gruen, and J. Patterson. Nimble cybersecurity incident management through visualization and defensible recommendations. In Proc. Seventh Int'l Symposium on Visualization for Cyber Security, pages 102-113. ACM, 2010.
35. N. Riche et al. Beyond system logging: human logging for evaluating information visualization. In Proc. BELIV 2010 Workshop, 2010.
36. J. Sauro and J. R. Lewis. Quantifying the user experience: Practical statistics for user research. Elsevier, 2012.
37. J. Scholtz. Beyond usability: Evaluation aspects of visual analytic environments. In Visual Analytics Science and Technology, 2006 IEEE Symposium On, pages 145-150. IEEE, 2006.
38. K. Sedig, P. Parsons, and A. Babanski. Towards a characterization of interactivity in visual analytics. JMPT, 3(1):12-28, 2012.
39. M. Sedlmair, M. Meyer, and T. Munzner. Design study methodology: Reections from the trenches and the stacks. Visualization and Computer Graphics, IEEE Trans. on, 18(12):2431-2440, 2012.
40. B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multi-dimensional in-depth long-term case studies. In Proc. 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization, pages 1-7. ACM, 2006.
41. B. Steichen, G. Carenini, and C. Conati. User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities. In Proc. 2013 Int Conf. on Intelligent user interfaces, pages 317-328. ACM, 2013.
42. M. Wattenberg and D. Fisher. Analyzing perceptual organization in information graphics. Information Visualization, 3(2):123-133, 2004.
43. L. E. Wood. Semi-structured interviewing for user-centered design. interactions, 4(2):48-61, 1997.
44. X. S. Zheng, I. Chakraborty, J. J.-W. Lin, and R. Rauschenberger. Correlating low-level image statistics with users-rapid aesthetic and affective judgments of web pages. In Proc. SIGCHI Conf. on Human Factors in Computing Systems, pages 1-10. ACM, 2009.