Google scholar search performance: Comparative recall and precision

Portal

Volumn 9, Issue 1, 2009, Pages 5-24

  Walters, William H ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 68149166454     PISSN: 15312542     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (60)

1. G.E. Gorman, "Giving Way to Google," Online Information Review 30, 2 (2006): 97-9;
2. Benjamin P. Norris, "Google: Its Impact on the Library," Library Hi Tech News 23, 9 (2006): 9-11;
3. Andrew K. Pace, "If You Can't Beat 'em, Join 'em," American Libraries 36, 8 (2005): 78-9;
4. Jeffrey Pomerantz, "Google Scholar and 100 Percent Availability of Information," Information Technology and Libraries 25, 2 (2006): 52-6;
5. Carol Tenopir, "Remaining Relevant Online," Library Journal 132, 10 (2007): 32.
6. Laura Bowering Mullen and Karen A. Hartman, "Google Scholar and the Library Web Site: The Early Response by ARL Libraries," College & Research Libraries 67, 2 (2006): 106-22;
7. Chris Neuhaus, Ellen Neuhaus, and Alan Asher, "Google Scholar Goes to School: The Presence of Google Scholar on College and University Web Sites," Journal of Academic Librarianship 34, 1 (2008): 39-51.
8. Janet Adlington and Chris Benda, "Checking Under the Hood: Evaluating Google Scholar for Reference Use," Internet Reference Services Quarterly 10, 3/4 (2005): 135-48;
9. Rebecca Donlan and Rachel Cooke, "Running With the Devil: Accessing Library-Licensed Full-Text Holdings Through Google Scholar," Internet Reference Services Quarterly 10, 3/4 (2005): 149-57;
10. Peter Jacsó, "As We May Search: Comparison of Major Features of the Web of Science, Scopus, and Google Scholar Citation-Based and Citation-Enhanced Databases," Current Science 89, 9 (2005): 1537-47, http://www.iisc.ernet.in/currsci/nov102005/1537.pdf (accessed September 28, 2008);
11. Jacsó, "Google Scholar: The Pros and the Cons," Online Information Review 29, 2 (2005): 208-14;
12. Jacsó, "Deflated, Inflated and Phantom Citation Counts," Online Information Review 30, 3 (2006): 297-309;
13. Philipp Mayr and Anne-Kathrin Walter, "An Exploratory Study of Google Scholar," Online Information Review 31, 6 (2007): 814-30;
14. Martin Myhill, "Google Scholar," Charleston Advisor 6, 4 (2005): 49-52;
15. Mick O'Leary, "Google Scholar: What's In It For You?" Information Today 22, 7 (2005): 35-9;
16. Joann M. Wleklinski, "Studying Google Scholar: Wall to Wall Coverage?" Online 29, 3 (2005): 22-6.
17. Burton Callicott and Debbie Vaughn, "Google Scholar vs. Library Scholar: Testing the Performance of Schoogle," Internet Reference Services Quarterly 10, 3/4 (2005): 82.
18. D.Yvonne Jones, "Biology Article Retrieval From Various Databases: Making Good Choices with Limited Resources," Issues in Science & Technology Librarianship 44 (Fall 2005), http://www.istl.org/05-fall/ refereed.html (accessed September 27, 2008).
19. Susan Gardner and Susanna Eng, "Gaga over Google? Scholar in the Social Sciences," Library Hi Tech News 22, 8 (2005): 43.
20. Rena Helms-Park, Pavlina Radia, and Paul Stapleton, "A Preliminary Assessment of Google Scholar as a Source of EAP Students' Research Materials," Internet and Higher Education 10, 1 (2007): 65-76.
21. William H. Walters, "Google Scholar Coverage of a Multidisciplinary Field," Information Processing & Management 43, 4 (2007): 1121-32.
22. For details, see Walters, "Later-Life Migration in the United States: A Review of Recent Research," Journal of Planning Literature 17, 1 (2002): 37-66. The 155 relevant documents include all the works cited in that review, except for those published before 1990 or after 2000; those published as books, book chapters, or dissertations; and those that are primarily bibliographic or editorial in nature (10 items).
23. All the full-text content available to GS is indexed even when that content cannot be viewed by the user due to licensing restrictions. Consequently, the GS search results do not vary in response to differences in institutional library holdings. According to a 2004 investigation, early versions of GS indexed only the first few pages of long fulltext documents.
24. See Gary Price, "Google Scholar Documentation and Large PDF Files," SearchEngineWatch.com (December 1, 2004), http://blog. searchenginewatch.com/blog/041201-105511 (accessed September 28, 2008). It is not clear whether GS now indexes long documents in their entirety.
25. Marian Burright, "Google Scholar: Science & Technology," Issues in Science & Technology Librarianship 45 (Winter 2006), http://www.istl.org/06-winter/databases2.html (accessed September 28, 2008);
26. Gardner and Eng; Chris Neuhaus et al., "The Depth and Breadth of Google Scholar: An Empirical Study," portal: Libraries and the Academy 6, 2 (2006): 127-41;
27. Roy Tennant, "Google, the Naked Emperor," Library Journal 130, 13 (2005): 29;
28. Tennant, "Is Metasearching Dead?" Library Journal 130, 12 (2005): 28; and Mayr and Walter.
29. Judit Bar-Ilan, "An Ego-Centric Citation Analysis of the Works of Michael O. Rabin Based on Multiple Citation Indexes," Information Processing & Management 42, 6 (2006): 1553-66;
30. Kathleen Bauer and Nisa Bakkalbasi, "An Examination of Citation Counts in a New Scholarly Communication Environment," D-Lib Magazine 11, 9 (2005), http://www.dlib.org/dlib/september05/bauer/09bauer.html (accessed September 28, 2008);
31. Jacsó, "As We May Search"; Jacsó, "Comparison and Analysis of the Citedness Scores in Web of Science and Google Scholar," in Digital Libraries: Implementing Strategies and Sharing Experiences, ed. Edward A. Fox et al. (New York: Springer, 2005), 360-9;
32. Jacsó, "Deflated, Inflated and Phantom Citation Counts."
33. Albert Chevan, "Holding On and Letting Go: Residential Mobility During Widowhood," Research on Aging 17, 3 (1995): 278-302;
34. Kevin E. McHugh and Robert C. Mings, "On the Road Again: Seasonal Migration to a Sunbelt Metropolis," Urban Geography 12, 1 (1991): 1-18.
35. Gary L. Hunt, "Equilibrium and Disequilibrium in Migration Modeling," Regional Studies 27, 4 (1993): 341-9;
36. Andrei Rogers and Alain Belanger, "The Importance of Place of Birth in Migration and Population Redistribution Analysis," Environment and Planning A 22, 2 (1990): 193-210.
37. David E. Clark and William J. Hunter, "The Impact of Economic Opportunity, Amenities and Fiscal Factors on Age-Specific Migration Rates," Journal of Regional Science 32, 3 (1992): 349-65;
38. John A. Fulton, Glenn V. Fuguitt, and Richard M. Gibson, "Recent Changes in Metropolitan-Nonmetropolitan Migration Streams," Rural Sociology 62, 3 (1997): 363-84.
39. Timothy D. Hogan and Stephen K. Happel, "1993-94 Winter Residents Important to AZ Economy," Arizona Business 41, 7 (1994): 1-4.
40. An earlier study (Walters, "Google Scholar Coverage") showed that 45 of the 144 relevant articles indexed by GS were included in Google Scholar only due to their appearance in the bibliographies of papers previously indexed by GS. Because all 155 relevant articles can be found in the bibliography of one particular paper (Walters, "Later-Life Migration"), it is conceivable that some relevant articles appear in GS solely because of their inclusion in that bibliography. That is, the publication of "Later-Life Migration" might have artificially inflated the number of relevant articles found by the GS keyword search undertaken for this analysis. However, further investigation revealed that of the 45 relevant articles taken by GS from the bibliographies of previously indexed papers, 41 had appeared in the bibliographies of one or more GS-indexed articles published prior to "Later-Life Migration." Of the four relevant articles that might have been indexed solely due to their inclusion in the bibliography of "Later-Life Migration," only two are returned by a GS keyword search for elderly migration.
41. Walters, "Google Scholar Coverage."
42. Gardner and Eng; Myhill; O'Leary; and Wleklinski.
43. Marilyn Christianson, "Ecology Articles in Google Scholar: Levels of Access to Articles in Core Journals," Issues in Science & Technology Librarianship 49 (Winter 2007), http://www.istl.org/07-winter/ refereed.html (accessed September 28, 2008).
44. See, for example, Chris Buckley and Janet Walz, "The TREC-8 Query Track," in Information Technology: The Eighth Text Retrieval Conference (TREC-8), ed. Ellen M. Voorhees and Donna K. Harman (Gaithersburg, MD: U.S. Department of Commerce, National Institute of Standards and Technology, 2000), 65-75, http://trec.nist.gov/pubs/trec8/papers/qtrack.pdf (accessed July 26, 2008).
45. Walters, "Google Scholar Coverage."
46. Because most academic libraries have educational missions that extend beyond the provision of information, the most effective databases for information retrieval are not necessarily the best for educational purposes. See Veronica Calderhead, "Reflections on Information Confusion in Chemistry Information Learning: The Meaning of the Shift from Library Instruction to Information Literacy," Research Strategies 16, 4 (1998): 285-99;
47. Steven J. Herro, "Bibliographic Instruction and Critical Thinking," Journal of Adolescent & Adult Literacy 43, 6 (2000): 554-8.
48. Glenn Haya, Else Nygren, and Wilhelm Widmark, "Metalib and Google Scholar: A User Study," Online Information Review 31, 3 (2007): 365-75.
49. See Bernard J. Jansen and Amanda Spink, "How Are We Searching the World Wide Web? A Comparison of Nine Search Engine Transaction Logs," Information Processing & Management 42, 1 (2006): 248-63;
50. Bernard J. Jansen, Amanda Spink, and Tefko Saracevic, "Real Life, Real Users, and Real Needs: A Study and Analysis of User Queries on the Web," Information Processing & Management 36, 2 (2000): 207-27;
51. Spink et al., "Searching the Web: The Public and Their Queries," Journal of the American Society for Information Science and Technology 52, 3 (2001): 226-34;
52. Dietmar Wolfram et al., "Vox Populi: The Public Searching of the Web," Journal of the American Society for Information Science and Technology 52, 12 (2001): 1073-4. The search engines evaluated by these authors did not include Google or GS.
53. Anselm Spoerri, "Using the Structure of Overlap Between Search Results to Rank Retrieval Systems Without Relevance Judgments," Information Processing & Management 43, 4 (2007): 1059-70.
54. Walters, "Later-Life Migration."
55. Pia Borlund, "The Concept of Relevance in IR," Journal of the American Society for Information Science and Technology 54, 10 (2003): 913-25;
56. Stefano Mizzaro, "Relevance: The Whole History," Journal of the American Society for Information Science 48, 9 (1997): 810-32;
57. Tefko Saracevic, "Relevance: A Review of and a Framework for the Thinking on the Notion in Information Science," Journal of the American Society for Information Science 26, 6 (1975): 321-43;
58. Saracevic, "Relevance: A Review of the Literature and a Framework for Thinking on the Notion in Information Science; Part II: Nature and Manifestations of Relevance," Journal of the American Society for Information Science and Technology 58, 13 (2007): 1915-33; Saracevic, "Relevance: A Review of the Literature and a Framework for Thinking on the Notion in Information Science; Part III: Behavior and Effects of Relevance," Journal of the American Society for Information Science and Technology 58, 13 (2007): 2126-44;
59. Arthur R. Taylor et al., "Relationships Between Categories of Relevance Criteria and Stage in Task Completion," Information Processing & Management 43, 4 (2007): 1071-84.
60. Walters, "Expertise and Evidence in the Assessment of Library Service Quality," Performance Measurement and Metrics 4, 3 (2003): 98-102.