Metrics for evaluating human information interaction systems

Interacting with Computers

Volumn 18, Issue 4, 2006, Pages 507-527

  Scholtz, Jean ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

Evaluation; Human information interaction; Information retrieval; Intelligence analysis; Metrics for utility; User centered

Indexed keywords

CUSTOMER SATISFACTION; DATA ACQUISITION; EVALUATION; HUMAN COMPUTER INTERACTION; INFORMATION DISSEMINATION; INFORMATION RETRIEVAL; INTELLIGENT CONTROL; SYSTEMS ANALYSIS;

HUMAN INFORMATION INTERACTION (HII); INTELLIGENCE ANALYSIS; METRICS FOR UTILITY; USER-CENTERED;

INFORMATION TECHNOLOGY;

EID: 33745145033     PISSN: 09535438     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.intcom.2005.10.004     Document Type: Article

References (24)

1. Advanced Research and Development Agency (ARDA) home page, http://www.ic-arda.org/, accessed May, 2004.
2. Blanford, A., Buchanan, G., 2002. Usability of digital libraries: a workshop at JCDL 2002-JCDL'02 workshop on usability of digital libraries. Joint Conference on Digital Libraries, Portland Oregon.
3. Covey, D., 2002. Usage and Usability Assessment: Library Practices and Concerns. Digital Library Federation, Council on Library and Information Resources, Washington-website http://www.clir.org/pubs/abstract/pub105abst.html.
4. Cowley, P., Nowell, L., Scholtz, J., 2005. Glassbox: An Instrumented Infrastructure for Supporting Human Interaction with Information Hawaii International Conference on System Science, Jan. 2005.
5. Craine, K., 2000. Designing a Document Strategy. MC2 Publishing.
6. Dahlback, N., Jonsson, A., Ahrenberg, L., 1993. Wizard of Oz studies-why and how. Proceedings from the 1993 International Workshop on Intelligent User Interfaces, Orlando, FL, pp. 193-200.
7. Gershon, N. 1995. Human Information Interaction, WWW4 Conference, December.
8. Grabo, C.M., 2005. Anticipating Surprise: Analysis for Strategic Warning. Center for Strategic Intelligence Research. Joint Military Intelligence College. 2003. University Press of America, Washington, DC.
9. Heuer, R., 1999. Psychology of intelligence analysis. Center for the Study of Intelligence. CIA.
10. Hill, L., Dolin, R., Frew, J., Kemp, R.B., Larsgaard, M., Montello, D.R., Rae, M., Simpson, J. 1997. User evaluation: summary of the methodologies and results for the Alexandria Digital Library, University of California at Santa Barbara. In: ASIS97 Digital Collections: Implications for Users, Funders, Developers and Maintainers-Proceedings of the 60th ASIS Annual Meeting, 34 pp. 225-243.
11. International Organization for Standards, ISO 9241. Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs), Part 11. Guidance on Usability Specification and Measures.
12. Krizan, L., 1999. Intelligence Essentials for Everyone. Joint Military Intelligence College, Occasional Paper Number Six, Washington, DC.
13. Lesk, M., 1997. Unpublished Paper, accessed Nov. 16, 2005. www.lesk.com/mlesk/diglib.html.
14. Lucas, P. 2000. Passive information access and the rise of human-information interaction, CHI, 2000, Invited Talk. CHI 2000. Extended Abstracts, pp. 202.
15. Mack, G., Lonergan, K., Hale, C., Scholtz, J., Steves, M., 2004. A framework for metrics in large, complex systems. Aerospace.
16. Marchionini G. 2000. Evaluating digital libraries: a longitudinal and multifaceted view. Library Trends. 49(2), 304-333. Online copy at http://www.ils.unc.edu/~march/perseus/lib-trends-final.pdf.
17. Medina, C. 2002. what to do when traditional models fail. Studies in Intelligence. 46(3), http://www.cia.gov/csi/studies/vol46no3/article03.html, accessed May, 2004.
18. National Research Council, More than Screen Deep, 1997. National Academy Press, p. 32.
19. NIMD home page, http://www.ic-arda.org/Novel_Intelligence/, accessed June 12, 2005.
20. Scholtz, J., Morse, E., Hewett, T., 2005. In depth observational studies of professional intelligence analysts. International Conference on Intelligence Analysis, May, MacLean, VA. Proceedings available at https://analysis.mitre.org/ [accessed June 12, 2005].
21. TREC Conference [http://trec.nist.gov/data/qa.html, accessed Feb. 16, 2004].
22. Voorhees, E., Buckland, L. (Eds.). 2003. Proceedings of the Eleventh Text REtrieval Conference (TREC 2002).
23. Warner, M., 2004. Wanted: a definition of intelligence. Studies in Intelligence, 46(3), http://www.cia.gov/csi/studies/vol46no3/article02.html, accessed May 2004.
24. White H., and McCain K. Bibliometrics. In: Williams M. (Ed). Annual Review of Information Science and Technology vol. 207 (1989), Information Today, Inc, Medford, NY 161-207