Strategies for searching online finding aids: A retrieval experiment

Archivaria

Volumn 44, Issue , 1997, Pages 72-101

  Hutchinson, Tim ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33645036569     PISSN: 03186954     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (62)

1. See, for example, William Landis, "Archival Outreach on the World Wide Web, " Archival Issues 20, no. 1 (1995), pp. 129-47.
2. Abraham, "Net Worth. "
3. Version 1.0 of the EAD is due to be released in mid 1998. The EAD conforms to the Standard Generalized Markup Language, SGML (in principle, the relationship of EAD to SGML is the same as that of HyperText Markup Language, HTML, to SGML). The EAD standard is maintained by the Network Development and MARC Standards Office of the Library of Congress, in partnership with the Society of American Archivists. The official EAD web site is at .
4. Generally, catalogue records are shortened versions of the introductory sections of finding aids, together with controlled name and subject access points. For examples relating to the present study, see the references in the methodology section.
5. Carol Tenopir, "Full Text Database Retrieval Performance, " Online Review 9 (1985), pp. 149-64.
6. Rowley, "The Controlled Versus Natural, " p. 113.
7. Svenonius, "Unanswered Questions in the Design of Controlled Vocabularies, " Journal of the American Society for Information Science 37 (1986). p. 335.
8. Rowley, 'The Controlled Versus Natural, " pp. 116-17.
9. See, for example, C.W. Cleverdon, A Comparative Evaluation of Searching by Controlled Language and Natural Language in an Experimental NASA Data Base, Space Documentation Service, European Space Agency, 1977, cited in F.W. Lancaster, "The Perspective-Natural Language Versus Controlled Language: A New Examination, " Perspectives in Information Management 1 (London, 1989): "it was found that the natural language searches gave a significantly higher recall and differed little in precision from the controlled term searches. Cleverdon concluded. that it was the length of the abstract that was largely responsible" (p. 15).
10. This finding is supported in a literature review by Richard J. Cox.
11. Richard H. Lytle, "Intellectual Access to Archives: I. Provenance and Content Indexing Methods of Subject Retrieva1, "American Archivist 43 (Winter 1980), pp. 64-75.
12. Fernanda Ribeiro, "Subject Indexing and Authority Control in Archives: The Need for Subject Indexing in Archives and for an Indexing Policy Using Controlled Language, " Journal of the Society of Archivists 17, no. 1 (1996). pp. 27-54.
13. Avra Michelson, "Description and Reference in the Age of Automation, " American Archivist 50 (Spring 1987). pp. 192-208.
14. Helen Tibbo, "The Epic Struggle: Subject Retrieval from Large Bibliographic Databases, " American Archivist 57 (Spring 1994), pp. 310-26.
15. Lytle, "Intellectual Access I, " p. 64.
16. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
17. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
18. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
19. David A. Bearman and Richard H. Lytle, "The Power of the Principle of Provenance, " Archivaria 21 (Winter 1985-86). pp. 14-27.
20. Ribeiro, "Subject Indexing, " p. 35.
21. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
22. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
23. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
24. Michelson, "Description and Reference in the Age of Automation, " p. 194.
25. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
26. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
27. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
28. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
29. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
30. Tibbo, 'The Epic Struggle, " p. 317 (Tables 1 and 3).
31. Luciana Duranti, "Diplomatics: New Uses for an Old Science. (Part IV), " Archivaria 31 (Winter 1990). p. 14.
32. Stephen E. Wiberley, Jr., Robert A. Daugherty, and James A. Danowski, "User Persistence in Scanning Postings of a Computer-Driven Information System: LCS, " Library and Information Science Research 12 (October-December 1990), p. 352.
33. These findings aids are available on the World Wide Web, at
34. That is, if there is only one relevant collection for a particular question, then the recall score for that question may be only 0 per cent or 100 per cent. This is discussed in more detail in the Results and Analysis section.
35. Mandeville Special Collections Library home page, University of California, San Diego, .
36. E-mail message to author from Teri Rinne (Head, Public Services, Bancroft Library, University of California, Berkeley), 11 March 1997.
37. For details, see Appendix A.
38. The "relevant/possibly relevanthot relevant" triage was suggested by Joseph Janes, School of Information, University of Michigan (e-mail message to author, 27 February 1997). See also Ribeiro, "Subject Indexing, " in which four levels (very relevant [1.0], relevant [0.5], not very relevant [0.25], not relevant [O]) were used (see page 36).
39. For convenience of notation, "number of relevant collections" will refer to the total relevance score.
40. See, for example, the special section on relevance research in the Journal of the American Society for Information Science 45, no. 3 (April 1994).
41. Joseph W. Janes, "Other People's Judgements: A Comparison of Users' and Others' Judgements of Document Relevance, Topicality, and Utility, " Journal of the American Society for Information Science 45, no. 3 (April 1994). pp. 160-71.
42. The HTML versions of the finding aids are split into sections (title page, biographyhistory, scope and content, and container listing) and are named in a consistent manner.
43. The tag , which might have been a reasonable choice, was not used, since in many of Berkeley's finding aids the "Key to Arrangement" was quite extensive, and similar to a container listing.
44. In order to reduce the amount of typing needed (and thus to avoid errors, especially in terms of the tag names), a web form and CGI script were used to translate search requests such as correspondence and minutes [in introductory material] (correspondence within 1000000 words of minutes) in ( or or ). which could then be submitted to the Berkeley server. This form also made it possible to submit the same search for both method one and method two, without retyping, thus ensuring that the searches were identical. Also, DynaWeb has a non-standard interpretation of the boolean operator "and-it is translated to be a proximity operator, "within 20 words of".
45. See any basic text on information retrieval.
46. See, for example, Robert Fugmann, "Galileo and the Inverse Precision/Recall Relationship: Medieval Attitudes in Modem Information Science, " Knowledge Organization 21, no. 3 (1994). pp. 153-54.
47. Michael Buckland and Fredric Gey, "The Relationship between Precision and Recall, " Journal of the American Society for Information Science 45, no. 1 (January 1994). pp. 12-19.
48. See, for example, Mary Jo Pugh, "The Illusion of Omniscience: Subject Access and the Reference Archivist, " American Archivist 45 (Winter 1982). p. 38.
49. Louise Su, "The Relevance of Recall and Precision in User Evaluation, " Journal of the American Society for Information Science 45, no. 3 (April 1994), p. 207.
50. See, for example, Ribeiro, "Subject Indexing, " pp. 3940.
51. The results are summarized in the figures which appear throughout this section. Figure One summarizes the recall and precision scores for each method (see Appendices B and C for results for each question).
52. In the case of methods two and one, and methods two and three, a non-decrease in recall is inevitable, since the text searched with the first method is a subset of the text searched with the second method. That is, the introductory material of a finding aid (method two) is a subset of the entire finding aid (method one), and similarly for methods two and three.
53. For example, if the precision increases from twenty per cent to thirty per cent, that is a ten per cent increase in absolute terms.
54. Indeed, it is not entirely good practice to compute these measures to three significant figures when the original data (numbers of relevant collections) can only be read to one decimal place. It is also important to bear in mind that the figures being discussed here are averages.
55. Ribeiro, "Subject Indexing, " p. 52.
56. Wendy Duff has suggested that a reliability check could be carried out to help alleviate this problem.
57. In Janes, "Other People's Judgements, " the groups being compared included users, librarians, and information and library studies students.
58. Most tags available in the EAD include an "encodinganalog" attribute, which can be used to facilitate the creation of MARC records from EAD-encoded finding aids.
59. Traditionally, one assumption (borne out by user studies) has been that humanities researchers rely more on footnotes and word-of-mouth references to find primary source material.
60. See, for example, Bearman and Lytle, 'The Power of the Principle of Provenance".
61. Many archivists have called for wide-ranging user studies to try to address these questions.
62. Initial attempts to address this question include David Bearman, "User Presentation Language in Archives, " Archives and Museum Informatics 3, no. 4 (Winter 1989/90), pp. 3-7.