Model-driven formative evaluation of exploratory search: A study under a sensemaking framework

Information Processing and Management

Volumn 44, Issue 2, 2008, Pages 534-555

  Qu, Yan ^a   Furnas, George W ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Exploratory search; Information seeking; Model driven evaluation; Representation construction; Sensemaking

Indexed keywords

ALGORITHMS; DATA STRUCTURES; INFORMATION RETRIEVAL; SYSTEMS ENGINEERING;

EXPLORATORY SEARCH; INFORMATION SEEKING; SENSEMAKING;

SEARCH ENGINES;

EID: 39649121265     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2007.09.006     Document Type: Article

References (50)

1. Allan, J. (2003). HARD Track Overview in TREC 2003: High accuracy retrieval from documents. In Proceedings of the text retrieval conference (pp. 24-37).
2. Baldonado M.Q., and Winograd T. SenseMaker: An information-exploration interface supporting the contextual evolution of a user's interests. Proceedings of the SIGCHI 2007 (1997) 11-18
3. Basu, S., Banerjee, A., & Mooney, R. J. (2002). Semi-supervised clustering by seeding. In Proceedings of the nineteenth international conference on machine learning (ICML-2002) (pp. 19-26), Sydney, Australia, July 2002.
4. Bates M.J. The design of browsing and berrypicking techniques for the online search interface. Online Review 13 5 (1989) 407-424
5. Belkin N.J. Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information Science 5 (1980) 133-143
6. Belkin N.J., Cool C., Kelly D., Lin S., Park S.Y., Perez-Carballo J., et al. Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Inf. Process. Manage. 37 3 (2001) 403-434
7. Belkin, N. J., Kelly, D., Lee, H. J., Li, Y. L., & Muresan, G. etc. (2003). Rutgers' HARD and web interactive track experiments. In TREC12 Proceedings.
8. Belkin, N., Dumais, S., Scholtz, J., & Wilkinson, R. (2004). Evaluating interactive information retrieval systems: Opportunities and challenges. In CHI'04 extended abstracts on human factors in computing systems (pp. 1594-1595).
9. Belkin N.J., Oddy R., and Brooks H. ASK for information retrieval: part I& 2. Journal of Documentation 38 2,3 (1982) .(61-71; 145-164)
10. Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152.
11. Borlund P., and Ingwersen P. The development of a method for the evaluation of interactive information retrieval systems. Journal of Documentation 53 3 (1997) 225-250
12. Chen H., Houston A.L., Sewell R.R., and Schatz B.R. Internet browsing and searching: user evaluations of category map and concept space techniques. Journal of the American Society for Information Science 49 7 (1998) 582-603
13. Cleverdon, C. W. (1991). The significance of the Cranfield tests on index languages. In Proceedings of ACM SIGIR'91 (pp. 3-12).
14. Cleverdon, C. W., Mills, J., & Keen, M. (1966). Factors determining the performance of indexing systems. In ASLIB Cranfield project, Cranfield.
15. Cousins, S. B., Paepcke, A., Winograd, T., Bier, E. A., & Pier, K. (1997). The digital library integrated task environment (DLITE). In Proceedings of the second acm international conference on digital libraries (pp. 142-151).
16. Craswell, N., Hawking, D., Thoms, J., Trystan, U., Wilkinson, R., & Wu, M. (2002). TREC11 Web and interactive tracks at CSIRO. The 2002 Text Retrieval Conference (TREC2002). Washington, DC.
17. Cutting, D., Karger, D., Pedersen, J., & Tukey, J. (1992). Scatter-gather: A cluster-based approach to browsing large document collections. In Proceedings of SIGIR 1992 (pp. 318-329).
18. Dervin, B. (1992). From the mind's eye of the user: The sense-making qualitative-quantitative methodology. In J. D. Glazier & R. R. Powell, (Eds.), Qualitative research in information management (pp. 6-84).
19. Dervin B., and Nilan M. Information needs and uses. Annual Review of Information Science and Technology (ARIST) 21 (1986) 3-33
20. Dumais, S. T., & Belkin, N. J. (2005). The TREC interactive tracks: Putting the user into search. In: E. M. Voorhees & D. K. Harman (Eds.), TREC: Experiment and evaluation in information retrieval (pp. 123-153).
21. Dumais, S., & Chen, H. (2000). Hierarchical classification of Web content. In Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval (pp. 256-263).
22. Dumais, S., Cutrell, E., Cadiz, J., Jancke, G., Sarin, R., & Robbins, D. C. (2003). Stuff I've seen: A system for personal information retrieval and reuse. In Proceedings of the 26th annual international acm sigir conference on research and development in information retrieval (pp. 72-79).
23. Egan D.E., Remde J.R., Gomez L.M., Landauer T.K., Eberhardt J., and Lochbaum C.C. Formative design evaluation of superbook. ACM Transactions on Information Systems (TOIS) 7 1 (1989) 30-57
24. Furnas, G. W., & Rauch, S. J. (1998). Considerations for information environments and the NaviQue workspace. In Proceedings of the ACM conference on digital libraries (DL98) (pp. 79-88).
25. Hearst, M. A., & Karadi, C. (1997). Cat-a-Cone: An interactive interface for specifying searches and viewing retrieval results using a large category hierarchy. In Proceedings of the 20th annual international acm sigir conference on research and development in information retrieval (pp. 246-255).
26. Hendry D.G., and Harper D.J. An informal information-seeking environment. Journal Of American Society of Information Science (Special Issue on Human-Computer Interaction) 48 11 (1997) 1036-1048
27. Hutchins E. Cognition in the Wild (1996), MIT Press, Cambridge, MA
28. Ingwersen P. Cognitive perspectives of information retrieval interaction: elements of cognitive IR theory. Journal of Documentation 52 1 (1996) 3-50
29. Järvelin, K. (2002). Evaluating information retrieval systems under the challenges of interaction and multidimensional dynamic relevance. In H. Bruce, R. Fidel, P. Ingwersen, & Vakkari, P. (Eds.), Emerging frameworks and methods (pp. 253-270).
30. Kekäläinen, J., Marchionini, G., & Geisler, G. (2002). The open video digital library. D-Lib Magazine, 8, 12. www.dlib.org/dlib/december02/marchionini/12marchionini.html.
31. Kuhlthau C.C. Inside the search process: Information Seeking from the User's Perspective. Journal of American Society of Information Science 42 5 (1991) 361-371
32. Kules, B. (2006). Methods for evaluating changes in search tactics induced by exploratory search systems. Workshop on Evaluating Exploratory Search Systems (EESS 2006) at SIGIR 2006, Seattle, WA, August 2006.
33. Mackinlay, J. D., Rao, R., & Card, S. K. (1995). An organic user interface for searching citation links. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 67-73). Denver, Colorado, May 07-11, 1995.
34. Marchionini, G., & Geisler, G. (2002). The open video digital library. D-Lib Magazine, 8, 12. www.dlib.org/dlib/december02/marchionini/12marchionini.html.
35. Marchionini G. Exploratory search: From finding to understanding. Communications of the ACM 49 4 (2006) 41-46
36. Nigam K., Mccallum A.K., Thrun S., and Mitchell T. Text classification from labeled and unlabeled documents using EM. Machine Learning 39 (2000) 103-134
37. Pirolli P., and Card S.K. Information foraging. Psychological Review 106 (1999) 643-675
38. Pirolli P., Card S.K., and Van Der Wege M.M. The effects of information scent on visual search in the hyperbolic tree browser. ACM Transactions on Computer-Human Interaction 10 1 (2003) 20-53
39. Qu, Y., & Furnas, G. W. (2005). Source of structure in sensemaking. extended abstract of conference on human factors in computing systems (CHI'05), 1989-1992.
40. Qu, Y., & Furnas, G. W. (2006). What Do the Attributes of Exploratory Search Tell us about Evaluation? Workshop on Evaluating Exploratory Search Systems (EESS 2006) at SIGIR'06.
41. Qu Y. Sensemaking-Supporting Information Gathering System. CHI2003 Extended Abstract CD (2003) 906-907
42. Qu, Y., Furnas, G. W., & Walstrum, B. (2006). Supporting relationship exploration in categorical data. In Proceedings of the annual meeting of the American Society for Information Science and Technology (ASIST'06).
43. Russell D.M., Stefik M.J., Pirolli P., and Card S.K. The Cost Structure of Sensemaking. Proceedings of ACM INTERCHI'93 (1993) 269-276
44. Saracevic, T. (1995). Evaluation of evaluation in information retrieval. In E. A. Fox, P. Ingwersen, & Fidel, R., (Eds.), Proceedings of the 18th ACM sigir conference on research and development of information retrieval (pp. 138-146).
45. Schraefel m., Wilson M., Russell A., and Smith D.A. mSpace: improving information access to multimedia domains with multimodal exploratory search. Communications of the ACM 49 4 (2006) 47-49
46. White, R. W., Muresan, G., & Marchionini, G. (2006). Evaluating Exploratory Search Systems, Workshop on Evaluating Exploratory Search Systems (EESS 2006) at SIGIR'06.
47. White R.W., Kules B., Drucker S., and Schraefel M.C. Supporting exploratory search: Introduction. Communications of the ACM 49 4 (2006) 36-39
48. Wilson T.D. On user studies and information needs. Journal of Documentation 37 1 (1981) 3-15
49. Yee, K-P., Swearingen, K., Li, K., & Hearst, M. (2003). Faceted metadata for image search and browsing. In Proceedings of CHI'2003 (pp. 401-408).
50. Zhang J. The Nature of External Representation in Problem Solving. Cognitive Science 21 2 (1997) 179-217