Beyond mouse and keyboard: Expanding design considerations for information visualization interactions

IEEE Transactions on Visualization and Computer Graphics

Volumn 18, Issue 12, 2012, Pages 2689-2698

  Lee, Bongshin ^a   Isenberg, Petra ^b   Riche, Nathalie Henry ^a   Carpendale, Sheelagh ^c  

^a MICROSOFT RESEARCH   (United States)

^b INRIA   (United States)

^c UNIVERSITY OF CALGARY   (Canada)

Author keywords

Design considerations; interaction; NUI (Natural User Interface); post WIMP

Indexed keywords

INFORMATION ANALYSIS; INFORMATION SYSTEMS; MAMMALS; USER INTERFACES; VISUALIZATION;

DESIGN CONSIDERATIONS; INFORMATION VISUALIZATION; INTERACTION; INTERACTION TECHNOLOGY; NATURAL INTERACTIONS; NATURAL USER INTERFACES; POST WIMPS; SPECIFIC INTERACTION;

DESIGN;

EID: 84867627334     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2012.204     Document Type: Article

References (82)

1. E.W. Anderson, K.C. Potter, L.E. Matzen, J.F. Shepherd, G.A. Preston, and C.T. Silva, A User Study of Visualization Effectiveness Using EEG and Cognitive Load, Computer Graphics Forum (Proc. EuroVis), vol. 30, no. 3, pp. 791-800, 2011.
2. Anoto - Start, http://www.anoto.com.
3. Apple - iPhone 4S - Ask Siri to help you get things done, http://www.apple.com/iphone/features/siri.html.
4. R. Ball and C. North, Realizing Embodied Interaction for Visual Analytics Through Large Displays, Computers & Graphics, vol. 31, no. 3, pp. 380-400, 2007. (Pubitemid 46726591)
5. D. Baur, F. Seiffert, M. Sedlmair, and S. Boring, The Streams of Our Lives: Visualizing Listening Histories in Context, IEEE TVCG (Proc. InfoVis), vol. 16, no. 6, pp. 1119-1128, 2010.
6. M. Beaudouin-Lafon, Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces, Proc. CHI, pp. 446-453, 2000.
7. M. Beaudouin-Lafon, Designing Interaction, Not Interfaces, Proc. AVI, pp. 15-22, 2004.
8. J. Bellegarda, Statistical Language Model Adaptation: Review and Perspectives, Speech Communication, vol. 42, pp. 93-108, 2004.
9. S. Boring, D. Baur, A. Butz, S. Gustafson, and P. Baudisch, Touch Projector: Mobile Interaction Through Video, Proc. CHI, pp. 2287- 2296, 2010.
10. P. Brandl, C. Richter, and M. Haller, NiCEBook - Supporting Natural Note Taking, Proc. CHI, pp. 599-608, 2010.
11. J. Browne, B. Lee, S. Carpendale, N. Riche, and T. Sherwood, Data Analysis on Interactive Whiteboards through Sketch-based Interaction, Proc. ITS, pp. 154-157, 2011.
12. S. Card, J. Mackinlay, and B. Shneiderman, Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann, 1999.
13. S. Carpendale, A Framework for Elastic Presentation Space, PhD Dissertation, Dept. of Computer Science, Simon Fraser University, Vancouver, Canada, 1999.
14. W.O. Chao, T. Munzner, and M. van de Panne, Poster: Rapid Pen- Centric Authoring of Improvisational Visualizations with NapkinVis. Posters Compendium InfoVis, 2010.
15. E.H. Chi and J. Riedl, An Operator Interaction Framework for Visualization Systems, Proc. InfoVis, pp. 63-70, 1998.
16. K. Cox, R.E. Grinter, S.L. Hibino, L.J. Jagadeesan, and D. Mantilla, A Multi-Modal Natural Language Interface to an Information Visualization Environment, J. Speech Technology, vol. 4, pp. 297-314, 2001. (Pubitemid 32968807)
17. A. van Dam. Post-WIMP user interfaces, CACM, vol. 40, no. 2, pp. 63- 67, 1997. (Pubitemid 127441124)
18. Data Science Revealed: A Data-Driven Glimpse into the Burgeoning New Field, http://www.emc.com/collateral/about/news/emc-datascience- study-wp.pdf.
19. P. Dourish, Where The Action Is: The Foundations of Embodied Interaction, MIT Press, 2001.
20. T. Dwyer, B. Lee, D. Fisher, K. Inkpen, P. Isenberg, G. Robertson, and C. North, Understanding Multi-touch Manipulation for Surface Computing, IEEE TVCG (Proc. InfoVis), vol. 25, no. 19, pp. 961-968, 2009.
21. N. Elmqvist, A. Vande Moere, H.-C. Jetter, D. Cernea, H. Reiterer, and T.J. Jankun-Kelly, Fluid Interaction for Information Visualization, Information Visualization, vol. 10, pp. 327-340, 2011.
22. M. Frisch, J. Heydekorn, and R. Dachselt. Investigating Multi-Touch and Pen Gestures for Diagram Editing on Interactive Surfaces, Proc. ITS, pp. 149-156, 2009.
23. D. Frohlich, Direct Manipulation and Other Lessons. In M. Helander, T. K. Landauer, and P. V. Prabhu, editors, Handbook of Human-Computer Interaction, pp. 463-488, 1997.
24. D.M. Frohlich, The History and Future of Direct Manipulation, Behaviour & Information Technology, vol. 12, no. 6, pp. 315-329, 1993.
25. M.J.F. Gales, Maximum Likelihood Linear Transformations for HMMBased Speech Recognition, Computer Speech & Language, vol. 12, pp. 75-98, 1998. (Pubitemid 128383747)
26. G. Goth, Brave NUI world, CACM, vol. 54, no. 12, pp. 14-16, 2011.
27. L. Grammel, M. Tory, and M.A. Storey, How Information Visualization Novices Construct Visualizations, IEEE TVCG (Proc. InfoVis), vol. 16, no. 6, pp. 943-952, 2010.
28. S. Greenberg, N. Marquardt, T. Ballendat, R. Diaz-Marino, and M. Wang. Proxemic Interactions: The New Ubicomp? ACM Interactions, vol. 18, no. 1, pp. 42-50, 2011.
29. C. Gutwin and S. Greenberg, A Descriptive Framework of Workspace Awareness for Real-Time Groupware, CSCW, vol. 11, no. 3-4, pp. 411- 446, 2002.
30. M. Haller, J. Leitner, T. Seifried, J. Wallace, S. Scott, C. Richter, P. Brandl, A. Gokcezade and S. Hunter, The NiCE Discussion Room: Integrating Paper and Digital Media to Support Co-Located Group Meetings, Proc. CHI, pp. 609-618, 2010.
31. M. Hancock, T. ten Cate, and S. Carpendale, Sticky Tools: Full 6DOF Force-Based Interaction for Multi-Touch Tables, Proc. ITS, pp. 145- 152, 2009.
32. J. Heer and B. Shneiderman, Interactive Dynamics for Visual Analysis, ACM Queue, vol. 10, no. 2, pp. 30-55, 2012.
33. M. Heilig, S. Huber, M. Demarmels, and H. Reiterer, ScatterTouch: A Multi Touch Rubber Sheet Scatter Plot Visualization for Co-located Data Exploration, Proc. ITS, pp. 264-264, 2010.
34. N. Henry, J.-D. Fekete, and M.J. McGuffin; NodeTrix: a Hybrid Visualization of Social Networks, IEEE TVCG (Proc. InfoVis), vol. 13, no. 6, pp. 1302-1309, 2007. (Pubitemid 350076603)
35. K. Hinckley, K. Yatani, M. Pahud, N. Coddington, J. Rodenhouse, A. Wilson, H. Benko, and B. Buxton, Pen + Touch = New Tools, Proc. UIST, pp. 27-36, 2010.
36. U. Hinrichs and S. Carpendale, Gestures in the Wild: Studying Multi- Touch Gesture Sequences on Interactive Tabletop Exhibits, Proc. CHI, pp. 3023-3032, 2011.
37. C. Holz and S. Feiner: Relaxed Selection Techniques for Querying Time-Series Graphs, Proc. UIST, pp. 213-222, 2009.
38. P. Isenberg and S. Carpendale, Interactive Tree Comparison for Colocated Collaborative Information Visualization, IEEE TVCG (Proc. InfoVis), vol. 13, no. 6, pp. 1232-1239, 2007. (Pubitemid 350076594)
39. P. Isenberg, A. Bezerianos, N. Henry, S. Carpendale, and J.-D. Fekete. CoCoNutTrix: Collaborative Retrofitting for Information Visualization, CG&A, vol. 29, no. 5, pp. 44-57, 2009.
40. P. Isenberg, D. Fisher, S.A. Paul, M.R. Morris, K. Inkpen, and M. Czerwinski, Collaborative Visual Analytics Around a Tabletop Display, IEEE TVCG, vol. 18, no. 5, pp. 689-702, 2012.
41. P. Isenberg, A. Tang, and S. Carpendale, An Exploratory Study of Visual Information Analysis, Proc. CHI, pp. 1217-1226, 2008.
42. R.J.K. Jacob, A. Girouard, L.M. Hirshfield, M.S. Horn, O. Shaer, E.T. Solovey, and J. Zigelbaum, Reality-Based Interaction: A Framework for Post-WIMP Interfaces, Proc. CHI, pp. 201-210, 2008
43. H.-C. Jetter, J. Gerken, M. Zöllner, H. Reiterer, and N. Milic-Frayling, Materializing the Query with Facet-Streams - A Hybrid Surface for Collaborative Search on Tabletops, Proc. CHI, pp. 3013-3022, 2011.
44. C.N. Klokmose and M. Beaudouin-Lafon, VIGO: Instrumental Interaction in Multi-Surface Environments, Proc. CHI, pp. 869-878, 2009.
45. R. Kosara, H. Hauser, and D.L. Gresh, An Interaction View on Information Visualization, Proc. Eurographics - State of the Art Reports, pp. 123-137, 2003.
46. A. Kretz, R. Huber, and M. Fjeld, Force Feedback Slider (FFS): Interactive Device for Learning System Dynamics, Proc. ICATL, pp. 457-458, 2005. (Pubitemid 44459424)
47. B. Kwon, W. Javed, N. Elmqvist, and J.-S. Yi, Direct Manipulation Through Surrogate Objects, Proc. CHI, pp. 627-636, 2011.
48. Z. Liu, N.J. Nersessian, and J.T. Stasko, Distributed Cognition as a Theoretical Framework for Information Visualization, IEEE TVCG (Proc. InfoVis), vol. 14, no. 6, pp. 1173-1180, 2008.
49. Many Eyes, http://www.many-eyes.com.
50. Microsoft Kinect for Windows | Develop for the Kinect | Kinect for Windows, http://www.microsoft.com/en-us/kinectforwindows.
51. J. Nielsen, Noncommand User Interfaces, CACM, vol. 36, no. 4, pp. 83- 99, 1993. (Pubitemid 23641151)
52. Nuance - Dragon Dictation: iPhone - Dragon Dictation for iPadTM, iPhoneTM and iPod touchTM is an easy-to-use voice recognition application, http://www.nuance.com/for-business/by-product/dragondictation- iphone/index.htm.
53. L. Olsen, F.F. Samavati, M.C. Sousa, and J.A. Jorge, Sketch-Based Modeling: A Survey, Computers & Graphics, vol. 33, pp. 85-103, 2009.
54. Perceptive Pixel, http://www.perceptivepixel.com.
55. W.A. Pike, J.T. Stasko, R. Chang, and T.A. O'Connell, The Science of Interaction, Information Visualization, vol. 8, no. 4, pp. 263-274, 2009.
56. N.H. Riche, K. Inkpen, J.T. Stasko, T. Gross, and M. Czerwinski, Supporting asynchronous collaboration in visual analytics systems, Proc. AVI, pp. 809-811, 2012.
57. K. Ryall, N. Lesh, T. Lanning, D. Leigh, H. Miyashita, and S. Makino, QueryLines: Approximate Query for Visual Browsing, Ext. Abst. CHI, pp. 1765-1768, 2005.
58. S. Schmidt, M. Nacenta, R. Dachselt, and S. Carpendale, A Set of Multitouch Graph Interaction Techniques, Proc. ITS, pp. 113-116, 2011.
59. C. Schwesig, I. Poupyrev, and E. Mori, Gummi: A Bendable Computer, Proc. CHI, pp. 263-270, 2004.
60. B. Shneiderman, Direct Manipulation: A Step Beyond Programming Languages, IEEE Computer, vol. 16, no. 8, pp. 57-69, 1983.
61. B. Shneiderman, The Eyes Have It: A Task By Data Type Taxonomy for Information Visualizations, Proc. VL, pp. 226-242, 1996.
62. B. Shneiderman and P. Maes, Direct Manipulation vs. Interface Agents, Interactions, vol. 4, no. 6, pp. 42-61, 1997.
63. D.C. Smith, C. Irby, R. Kimball, and E. Harslem, The Star User Interface: An Overview, Proc. AFIPS, pp. 515-528, 1982.
64. R. Spence, Information Visualization: Design for Interaction, Pearson, 2nd ed., 2007.
65. M. Spindler, S. Stellmach, and R. Dachselt, PaperLens: Advanced Magic Lens Interaction Above the Tabletop, Proc. ITS, pp. 69-76, 2009.
66. M. Spindler, C. Tominski, H. Schumann, and R. Dachselt, Tangible Views for Information Visualization, Proc. ITS, pp. 157-166. 2010.
67. Y. Sun, J. Leigh, A. Johnson, and S. Lee, Articulate: A Semi- Automated Model for Translating Natural Language Queries into Meaningful Visualizations, Proc. SG, LNCS, vol. 6133, pp. 184-195, 2010.
68. I.E. Sutherland, Sketchpad A Man-Machine Graphical Communication System, Proc. AFIPS Spring Joint Comp. Conf, pp. 329-346, 1963.
69. C. Swindells, K.E. MacLean, K.S. Booth, and M.J. Meitner, Exploring Affective Design for Physical Controls, Proc. CHI, pp. 933-942, 2007. (Pubitemid 47581682)
70. J.J. Thomas and K.A. Cook, Illuminating the Path, IEEE, 2005.
71. M. Tobiasz, P. Isenberg, and S. Carpendale, Lark: Coordinating Colocated Collaboration with Information Visualization, IEEE TVCG (Proc. InfoVis), vol. 15, no. 6, pp. 1065-1072, 2009.
72. F.B. Viégas, M. Wattenberg, M. McKeon, F. van Ham, and J. Kriss, Harry Potter and the Meat-Filled Freezer: A Case Study of Spontaneous Usage of Visualization Tools, Proc. HICSS, 2008.
73. M. Wattenberg, Baby Names, Visualization, and Social Data Analysis, Proc. InfoVis, pp. 1-7, 2005.
74. M. Wattenberg, Sketching a Graph to Query a Time-Series Database, Ext. Abs. CHI, pp. 381-382, 2001.
75. D. Wigdor and D. Wixon, Brave NUI World: Designing Natural User Interfaces for Touch and Gesture, Morgan Kaufmann, 2011.
76. C. Williamson and B. Shneiderman, The Dynamic HomeFinder: Evaluating Dynamic Queries in a Realestate Information Exploration System, Proc. Research and Development in Information Retrieval, pp. 338-346, 1992. (Pubitemid 23630479)
77. A.D. Wilson, TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction, Proc. ICMI, pp. 69-76, 2004. (Pubitemid 40371893)
78. A.D. Wilson, S. Izadi, O. Hilliges, A. Garcia-Mendoza, and D. Kirk, Bringing Physics to the Surface, Proc. UIST, pp. 67-76, 2008.
79. J.O. Wobbrock, M.R. Morris, and A.D. Wilson, User-Defined Gestures for Surface Computing, Proc. CHI, pp. 1083-1092, 2009.
80. Wordle - Beautiful Word Clouds, http://www.wordle.net.
81. J.S. Yi, Y.A. Kang, J.T. Stasko, and J.A. Jacko, Toward a Deeper Understanding of the Role of Interaction in Information Visualization, IEEE TVCG (Proc. InfoVis), vol. 13, no. 6, pp. 1224-1231, 2007. (Pubitemid 350076593)
82. H. Zhao, C. Plaisant, and B. Shneiderman, "I Hear the Pattern" - Interactive Sonification of Geographical Data Patterns, Ext. Abs. CHI, pp. 1905-1908, 2005.