Users' evaluation of digital libraries (DLs): Their uses, their criteria, and their assessment

Information Processing and Management

Volumn 44, Issue 3, 2008, Pages 1346-1373

  Xie, Hong Iris ^a  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

Author keywords

Digital library evaluation; Digital library evaluation criteria; Digital library use; User perspectives

Indexed keywords

INFORMATION RETRIEVAL; KNOWLEDGE MANAGEMENT; REQUIREMENTS ENGINEERING; USER INTERFACES;

DIGITAL LIBRARY EVALUATION; DIGITAL LIBRARY EVALUATION CRITERIA; DIGITAL LIBRARY USE; USER PERSPECTIVES;

DIGITAL LIBRARIES;

EID: 40649120080     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2007.10.003     Document Type: Article

References (47)

1. Association of Research Libraries. (1995). Definition and purposes of a digital library. Retrieved July 17, 2002, from: http://www.arl.org/sunsite/definition.html.
2. Atilgan D., and Bayram O. An evaluation of faculty use of the digital library at Ankara University, Turkey. Journal of Academic Librianship 32 (2006) 86-93
3. Bertot J.C., Snead J.T., Jaeger P.T., and McClure C.R. Functionality, usability and accessibility: Iterative user-centered assessment strategies for digital libraries. Performance Measurement and Metrics 7 1 (2006) 17-28
4. Bishop A.P., Neumann L.J., Star S.L., Merkel C., Ignacio E., and Sandusky R.J. Digital libraries: Situating use in changing information infrastructure. Journal of the American Society for Information Science 51 (2000) 394-413
5. Blandford A., and Buchanan G. Workshop report: Usability of digital libraries @JCDL'02. SIGIR Forum 36 2 (2002) 83-89
6. Blandford, A. & Buchanan, G. (2003). Usability of digital libraries: A source of creative tensions with technical developments. TCDL Bulletin. Retrieved April 12, 2007 from: http://www.ieee-tcdl.org/Bulletin/v1n1/blandford/blandford.html.
7. Bollen J., and Luce R. Evaluation of digital library impact and user communities by analysis of usage patterns. D-Lib Magazine 8 5 (2002). http://www.dlib.org/dlib/june02/bollen/06bollen.html Retrieved March 15, 2005 from
8. Bollen J., Luce R., Vemulapalli S.S., and Xu W. Usage analysis for the identification of Research trends in digital libraries. D-Lib Magazine 9 5 (2003). http://www.dlib.org/dlib/may03/bollen/05bollen.html Retrieved March 15, 2005 from
9. Borgamn, C. L., & Larsen, R. (2003). ECDL 2003 Workshop report: Digital library evaluation - metrics, testbeds and processes. D-Lib Magazine, 9(9). Retrieved May 8, 2007 from http://www.dlib.org/dlib/september03/09inbrief.html#BORGMAN%20.
10. Borgman C.L. What are digital libraries? Competing visions. Information Processing and Management 35 (1999) 227-243
11. Borgman C.L., Leazer G.H., Gilliland-Swetland A.J., and Gazan R. Evaluating digital libraries for teaching and learning in undergraduate education: A case study of the Alexandria digital earth prototype (ADEPT). Library Trends 49 (2000) 228-250
12. Budhu M., and Coleman A. The design and evaluation of interactivities in a digital library. D-Lib Magazine 8 11 (2002). http://www.dlib.org/dlib/november02/coleman/11coleman.html Retrieved April 12, 2006 from
13. Buttenfield B. Usability evaluation of digital libraries. Science and Technology Libraries 17 3/4 (1999) 39-50
14. Carter D.S., and Janes J. Unobtrusive data analysis of digital reference questions and service at the Internet Public Library: An exploratory study. Library Trends 49 (2000) 251-265
15. Cherry J.M., and Duff W.M. Studying digital library users over time: A follow-up survey of early Canadiana online. Information Research 7 2 (2002). http://informationr.net/ir/7-2/paper123.html Retrieved March 12, 2005, from
16. Choudhury G.S., Hobbs B., Lorie M., and Flores N.E. A Framework for evaluating digital library services. D-Lib Magazine 8 7/8 (2002). http://www.dlib.org/dlib/july02/choudhury/07choudhury.html Retrieved March 8, 2007
17. Chowdhury G.G., and Chowdhury S. Introduction to digital libraries (2003), Facet Publisher, London
18. Dillon A. Evaluating on TIME: A framework for the expert evaluation of digital library interface usability. International Journal on Digital Libraries 2 2/3 (1999) 170-177
19. Fox E.A., Akscyn R.M., Furuta R.K., and Leggett J.J. Digital libraries. Communications of the ACM 38 4 (1995) 23-28
20. Fuhr N., Tsakonas G., Aalberg T., Agosti M., Hansen P., Kapidakis S., et al. Evaluation of digital libraries. International Journal on Digital Libraries 8 1 (2007) 21-38
21. Fuhr N., Hansen P., Mabe M., Micsik A., and Sølvberg I. Digital libraries: A generic classification and evaluation scheme. Lecture Notes in Computer Science 2163 (2001) 187-199
22. Heath, F., Kyrillidou, M., Webster, D., Choudhury, S., Hobbs, B., Lorie, M., et al. (2003). Emerging tools for evaluating digital library services: Conceptual adaptations of LibQUAL+ and CAPM. Article No. 170, 2003-06-09, Journal of Digital Information 4(2). Retrieved November 15, 2005, from http://jodi.tamu.edu/Articles/v04/i02/Heath/.
23. Hill L.L., Carver L., Larsgaard M., Dolin R., Smith T.R., Frew J., et al. Alexandria digital library: User evaluation studies and system design. Journal of the American Society for Information Science 51 (2000) 246-259
24. Jeng J. Usability assessment of academic digital libraries: Effectiveness, efficiency, satisfaction, and learnability. LIBRI 55 2-3 (2005) 96-121
25. Jeng J. What is usability in the context of the digital library and how can it be measured?. Information Technology and Libraries 24 2 (2005) 47-56
26. Kassim A.R.C., and Kochtanek T.R. Designing, implementing, and evaluating an educational digital library resource. Online Information Review 27 3 (2003) 160-168
27. Krippendorff K. Content analysis: An introduction to its methodology (2004), Sage Publications, Inc., Thousand Oaks, CA
28. Lancaster F.W. If you want to evaluate your library. 2nd ed. (1993), Urbana-Champaign, University of Illinois
29. Lesk M. Understanding Digital Libraries. 2nd ed. (2005), Morgan Kaufman, San Francisco
30. Marchionini G. Evaluation digital libraries: A longitudinal and multifaceted view. Library Trends 49 (2000) 304-333
31. Marchionini G., Plaisant C., and Komlodi A. Interfaces and tools for the library of congress national digital library program. Information Processing and Management 34 5 (1998) 535-555
32. Meyyappan N., Foo S., and Chowdhury G.G. Design and evaluation of a task-based digital library for the academic community. Journal of Documentation 60 4 (2004) 449-475
33. Monopoli M., Nicholas D., Georgiou P., and Korfiati M. A user-oriented evaluation of digital libraries: Case study: The "electronic journals" service of the library and information service of the University of Patras, Greece. Aslib Proceedings 54 2 (2002) 103-117
34. Nicholson S. Digital library archaeology. A conceptual framework for understanding library use through artifact-based evaluation. Library Quarterly 75 4 (2005) 496-520
35. Nielsen J. Usability Engineering (1993), Academic Press, San Diego, CA
36. Saracevic T. Digital library evaluation: Toward an evolution of concepts. Library Trends 49 (2000) 350-369
37. Saracevic, T. (2004). Evaluation of digital libraries: An Overview. Presented at the DELOS Workshop on the Evaluation of Digital Libraries. Retrieved April 12, 2007 from http://www.scils.rutgers.edu/~tefko/DL_evaluation_Delos.pdf.
38. Saracevic T., and Covi L. Challenges for digital library evaluation. Proceedings of the American Society for Information Science 37 (2000) 341-350
39. Saracevic T., and Kantor P. Studying the value of library and information services. II Methodology and taxonomy. Journal of the American Society for Information Science 48 (1997) 543-563
40. Shneiderman B. Designing the user interface strategies for effective human-computer interaction. 3rd ed. (1998), Addison-Wesley, Reading, MA
41. Su L.T. Evaluation measures for interactive information retrieval. Information Processing and Management 28 (1992) 503-516
42. Thong J.Y.L., Hong W.Y., and Tam K.Y. Understanding user acceptance of digital libraries: What are the roles of interface characteristics, organizational context, and individual differences?. International Journal of Human-Computer Studies 57 (2002) 215-242
43. Tsakonas, G., Kapidakis, S., Papatheodorou, C. (2004). Evaluation of user interaction in digital libraries. In Agosti, M., Fuhr, N. (Eds.), Notes of the DELOS WP7 Workshop on the Evaluation of Digital Libraries, Padua, Italy.
44. Van House, N. A., Butler, M. H., Ogle, V., & Schiff, L. (1996, February). User-centered iterative design for digital libraries: The Cypress experience. D-Lib Magazine, 2. Retrieved February 3, 2005, from http://www.dlib.org/dlib/february96/02vanhouse.html.
45. Xie H. Evaluation of Digital Libraries: Criteria and problems from users' perspectives. Library and information Science Research 28 (2006) 433-452
46. Xie, H., & Cool, C. (2006). Toward a better understanding of help seeking behavior: An evaluation of help mechanisms in two IR systems. In Dillon, A., Grove, A. (Eds.), Proceedings of the 69th ASIST Annual Meeting (Vol. 43). Retrieved February 1, 2007, from http://eprints.rclis.org/archive/00008279/01/Xie_Toward.pdf.
47. Yang S.C. An interpretive and situated approach to an evaluation of Perseus digital libraries. Journal of the American Society for Information Science and Technology 52 (2001) 1210-1223