Controlled user evaluations of information visualization interfaces for text retrieval: Literature review and meta-analysis

Journal of the American Society for Information Science and Technology

Volumn 59, Issue 6, 2008, Pages 1012-1024

  Julien, Charles Antoine ^a   Leide, John E ^a   Bouthillier, France ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DESIGN OF EXPERIMENTS; INFORMATION RETRIEVAL; USER INTERFACES; VISUALIZATION;

INFORMATION VISUALIZATION INTERFACES; META ANALYSIS; TEXT RETRIEVAL;

TEXT PROCESSING;

EID: 42249095470     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.20786     Document Type: Review

References (58)

1. Andriessen, J.H.E. (1996). The why, how and what to evaluate of interaction technology: A review and proposed integration. In P.J. Thomas (Ed.), CSCW requirements and evaluation (pp. 107-124). London: Springer.
2. Ashby, F.G., Isen, A.M., & Turken, A.U. (1999). A neuropsychological theory of positive effect and its influence on cognition. Psychological Review, 106, 529-550.
3. Barlow, T., & Neville, P. (2001). A comparison of 2D visualizations of hierarchies. In Proceedings of the IEEE Symposium on Information Visualization (pp. 131-138). San Diego, CA.
4. Becks, A., Seeling, C., & Minkenberg, R. (2002). Benefits of document maps for text access knowledge management: A comparative study. In Proceedings of the ACM Symposium on Applied Computing (pp. 621-626). New York: ACM Press.
5. Bederson, B.B., & Shneiderman, B. (2003). The craft of information visualization: Readings and reflections. Amsterdam: Kaufmann.
6. Blythe, M.A., Overbeeke, K., Monk, A.F., & Wright, P.C. (2003). Funology: From usability to enjoyment. Dordrecht, The Netherlands: Kluwer.
7. Brajnik, G., Mizzaro, S., & Tasso, C. (1996). Evaluating user interfaces to information retrieval systems: A case study on user support. In Proceedings of the annual International ACM SIGIR Conference (pp. 128-136). New York: ACM Press.
8. Byrd, D. (1999). A scrollbar-based visualization for document navigation. In Proceedings of the 4th ACM Conference on Digital Libraries (pp. 122-129). New York: ACM Press.
9. Carr, D. (1999). Guidelines for designing information visualization applications. In Proceedings of the Ericsson Conference on Usability Engineering (ECUE'99). Stockholm, Sweden.
10. Carroll, J.M., & Mazur, S.A. (1986). "LisaLearning." IEEE Computer, 19(11), 35-49.
11. Chen, C. (2004). Information visualization: Beyond the horizon (2nd ed.). London: Springer-Verlag.
12. Chen, C., & Czerwinski, M. (2000). Empirical studies of information visualization: A meta-analysis. Journal of Human-Computer Studies, 53(5), 851-866.
13. Chen, H., Houston, A.L., Sewell, R.R., & Schatz, B.R. (1998). Internet browsing and searching: User evaluations of category map and concept space techniques. Journal of the American Society of Information Science, 49(7), 582-603.
14. Chung, W., Chen, H., & Nunamaker, J.F. Jr. (2005). A visual framework for knowledge discovery on the Web: An empirical study of business intelligence exploration. Journal of Management Information Systems, 21(4), 57-84.
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
16. Cribbin, T., & Chen, C. (2001). Visual-spatial exploration of thematic spaces: A comparative study of three visualisation models. In Proceedings of the SPIE, Electronic Imaging 2001: Visual Data Exploration and Analysis VIII (pp. 199-209). San Jose, CA.
17. Dumais, S.T., & Belkin, N.J. (2005). The TREC interactive tracks: Putting the user into search. In E.M. Voorhees & D.K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval (pp. 123-152). Cambridge, MA: MIT Press.
18. Dumais, S.T., & Jones, W.P. (1985). A comparison of symbolic and spatial filing. In Proceedings of the ACM Conference of Human Factors in Computing Systems (pp. 127-130). New York: ACM Press.
19. Eibl, M. (2002). DEViD: A media design and software ergonomics integrating visualization for document retrieval. Information Visualization, 1(2), 139-157.
20. Grun, C., Gerken, J., Jetter, H.-C., Konig, W., & Reitener, H. (2005). Medio Vis - A user-centered library metadata browser. In Proceedings of the European Conference on Digital Libraries (ECDL) (pp. 174-185). Berlin: Springer-Verlag.
21. Hedman, A., Carr, D.A., & Nassla, H. (2004). Browsing thumbnails: A comparison of three techniques. In Proceedings of the International Conference on Information Technology Interfaces (ITI) (pp. 353-360), Cavtat, Croatia.
22. Jensen, J.J., & Skov, M.B. (2005). A review of research methods in children's technology design. In Proceedings of the Interaction Design and Children (pp. 80-87). New York: ACM Press.
23. Kobsa, A. (2001). An empirical comparison of three commercial information visualization systems. In Proceedings of the IEEE Symposium on Information Visualization (pp. 123-130). San Diego, CA.
24. Kobsa, A. (2004). User experiments with tree visualization systems. In Proceedings of the IEEE Symposium on Information Visualization (pp. 9-16). Austin, TX.
25. Koshman, S. (2005). Testing user interaction with a prototype visualization based information retrieval system. Journal of the American Society for Information Science and Technology, 56(8), 824-833.
26. Lethbridge, T.C., Elliott Sim, S., & Singer, J. (2005). Studying software engineers: Data collection techniques for software field studies. Empirical Software Engineering, 10, 311-341.
27. Lin, X. (1997). Map displays for information retrieval. Journal of the American Society for Information Science, 48, 40-54.
28. Marchionini, G. (2006). Exploratory search: From finding to understanding. Communications of the ACM, 49(4), 41-46.
29. Marcus, A., Comorski, D., & Sergeyev, A. (2005). Supporting the evolution of a software visualization tool through usability studies. In Proceedings of the International Workshop on Program Comprehension (pp. 307-316). St-Louis, MO.
30. Modjeska, D., & Chignell, M. (2003). Individual differences in exploration using desktop VR. Journal of the American Society for Information Science and Technology, 54(3), 216-228.
31. Modjeska, D., & Waterworth, J. (2000). Effects of desktop 3D world design on user navigation and search performance. In Proceedings of the IEEE Symposium on Information Visualization (pp. 215-220). IEEE.
32. Newby, G.B. (2002). Empirical study of a 3D visualization for information retrieval tasks. Journal of Intelligent Information Systems, 18, 31-53.
33. Newman, J. (2001). Validation, evaluation and virtual enterprise research. In Proceedings of the 10th IEEE Workshops on Enabling Technology: Infrastructure for Collaborative Enterprises (pp. 91-95). Los Alamitos, CA: IEEE Press.
34. Nielsen, J. (2003). User empowerment and the fun factor. In M.A. Blythe, K. Overbeeke, A.F. Monk, & P.C. Wright (Eds.), Funology: From usability to enjoyment (pp. 103-105). Dordrecht, The Netherlands: Kluwer.
35. Norman, D.A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books.
36. Osdin, R., Ounis, I., & White, R. (2002). Using hierarchical clustering and summarization approaches for Web retrieval: Glasgow at the TREC 2002 interactive track. In Proceedings of the Text REtrieval Conference (TREC). Gaithersburg, MP.
37. Perez, C., & de Antonio, A. (2004). 3D Visualization of text collections: An experimental study to assess the usefulness of 3D. In Proceedings of the IEEE Symposium on Information Visualization (pp. 317-323). Austin, TX.
38. Plaisant, C., Grosjean, J., & Bederson, B.B. (2002). Supporting exploration in large node link tree: Design evolution and empirical evaluation. In Proceedings of the IEEE Symposium on Information Visualization, (pp. 57-64). Boston, MA.
39. Risden, K., Czerwinski, M.P., Munzner, T., & Cook, D.B. (2000). An initial examination of ease of use for 2D and 3D information visualizations of web content. Journal of Human-Computer Studies, 53(5), 695-714.
40. Rivadeneira, W., & Bederson, B.B. (2003). A study of search result clustering interfaces: Comparing textual and zoomable user interfaces. (Report No. HCIL-2003-36). University of Maryland. College Park, MD.
41. Robertson, G., Czerwinski, M., Larson, K., Robbins, D., Thiel, D., & van Dantzich, M. (1998). Data mountain: Using spatial memory for document management. In Proceedings of the ACM Symposium: User Interface Software and Technology (UIST) (pp. 153-162). New York: ACM Press.
42. Rosenthal, R. (1991). Meta-analytic procedures for social research (Rev. ed.). Applied Social Research Methods Series 6. Newbury Park, CA: Sage.
43. Santos, B.S., Zamfir, F., Ferreira, C., Mealha, O., & Nunes, J. (2004). Visual application for the analysis of Web-based information systems usage: A preliminary usability evaluation. In Proceedings of the IEEE International Conference on Information Visualization (pp. 812-818). IEEE.
44. Sebrechts, M.M., Vasilakis, J., Miller, M.S., Cugini, J.V., & Laskowski, S.J. (1999). Visualization of search results: A comparative evaluation of text, 2D, and 3D interfaces. In Proceedings of the annual International ACM SIGIR Conference (pp. 3-10). New York: ACM Press.
45. Sengupta, A., & Dillon, A. (2006). Query by templates: Using the shape of information to search next-generation databases. IEEE Transactions on Professional Communication, 49(2), 128-144.
46. Shneiderman, B. (2003). The eyes have it: A task by data type taxonomy for information visualizations. In B.B. Bederson & B. Shneiderman (Eds.), The craft of information visualization: Readings and reflections (pp. 364-371). Amsterdam: Kaufmann.
47. Shneiderman, B. (2004). Designing for fun: How can we design user interfaces to be more fun? Interactions, 11(5), 48-50.
48. Sutcliffe, A.G., Ennis, M., & Hu, J. (2000). Evaluating the effectiveness of visual user interfaces for information retrieval. Journal of Human-Computer Studies, 53(5), 741-763.
49. Swan, R.C., & Allen, J. (1998). Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems. In Proceedings of the annual International ACM SIGIR Conference (pp. 171-181). New York: ACM Press.
50. Turetken, O., & Sharda, R. (2005). Clustering-based visual interfaces for presentation of Web search results: An empirical investigation. Information Systems Frontiers, 7(3), 273-297.
51. Veerasamy, A., & Belkin, N.J. (1996). Evaluation of a tool for visualization of information retrieval results. In Proceedings of the annual International ACM SIGIR Conference (pp. 85-92). New York: ACM Press.
52. Veerasamy, A., & Heikes, R. (1997). Effectiveness of a graphical display of retrieval results. In Proceedings of the annual International ACM SIGIR Conference (pp. 236-245). New York: ACM Press.
53. Westerman, S.J., Collins, J., & Cribbin, T. (2005). Browsing a document collection represented in two- and three-dimensional virtual information space. Journal of Human-Computer Studies, 62(6), 713-736.
54. Westerman, S.J., & Cribbin, T. (2000). Mapping semantic information in virtual space: Dimensions, variance and individual differences. Journal of Human-Computer Studies, 53, 765-787.
55. Whiteside, J., Jones, S., Levey, P., & Wixon, D. (1985). User performance with command, menu, and iconic interfaces. In Proceedings of the ACM Conference on Human Factors in Computing Systems (pp. 185-191). New York: ACM Press.
56. Wynekoop, J.L., & Congor, A.A. (1990). A review of computer aided software engineering research methods. In Proceedings of the IFIP TC8 WG8.2 Working Conference on the Information Systems Research Arena of the 1990s. Copenhagen, Denmark.
57. Yang, C.C., Chen, H., & Hong, K. (2002). Internet browsing: Visualizing category map by fisheye and fractal views. In Proceedings of the International Conference on Information Technology: Coding and Computing (pp. 34-39). IEEE. Las Vegas, NV.
58. Zhang, J., & Marchionini, G. (2005). Evaluation and evolution of a browse and search interface: Relation Browser++. In Proceedings of the National Conference on Digital Government Research (pp. 179-188). IEEE. Atlanta, GA