Are wikipedia citations important evidence of the impact of scholarly articles and books?

Journal of the Association for Information Science and Technology

Volumn 68, Issue 3, 2017, Pages 762-779

  Kousha, Kayvan ^a   Thelwall, Mike ^a  

^a UNIVERSITY OF WOLVERHAMPTON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CITATION COUNTS; IMPACT EVIDENCES; RESEARCH EVALUATION; SCHOLARLY ARTICLES; SCHOLARLY RESEARCH; STAKEHOLDER INTEREST; WIKIPEDIA;

HUMAN; HUMANITIES; SCOPUS;

EID: 84992476972     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.23694     Document Type: Article

References (89)

1. Abramo, G., & D'Angelo, C.A. (2015). The VQR, Italy's second national research assessment: Methodological failures and ranking distortions. Journal of the Association for Information, Science and Technology, 66(11), 2202–2214.
2. Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26, 11–17. http://figshare.com/articles/Enriching_scholarly_content_with_article_level_discussion_and_metrics/105851.
3. Aibar, E., Lladós-Masllorens, J., Meseguer-Artola, A., Minguillón, J., & Lerga, M. (2015). Wikipedia at university: What faculty think and do about it. Electronic Library, 33(4), 668–683.
4. Ball, R., & Tunger, D. (2006). Science indicators revisited—Science Citation Index versus SCOPUS: A bibliometric comparison of both citation databases. Information Services & Use, 26(4), 293–301.
5. Bar-Ilan, J. (2004). The use of Web search engines in information science research. Annual Review of Information Science and Technology, 38(1), 231–288.
6. Bar-Ilan, J. (2008). Which h-index? A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271.
7. Bayliss, G. (2013). Exploring the cautionary attitude toward Wikipedia in higher education: Implications for higher education institutions. New Review of Academic Librarianship, 19(1), 36–57.
8. Bould, M.D., Hladkowicz, E.S., Pigford, A.E., Ufholz, L., Postonogova, T., Shin, E., & Boet, S. (2014). References that anyone can edit: Review of Wikipedia citations in peer reviewed health science literature. BMJ (Online), 348, g1585. doi: http://dx.doi.org/10.1136/bmj.g1585
9. Brazzeal, B. (2011). Citations to Wikipedia in chemistry journals: A preliminary study. Issues in Science and Technology Librarianship, 67. doi: 10.5062/F4057CV7
10. Chen, H.-L. (2009). The use and sharing of information from Wikipedia by high-tech professionals for work purposes. Electronic Library, 27(6), 893–905
11. Chen, H.-L. (2010). The perspectives of higher education faculty on Wikipedia. Electronic Library, 28(3), 361–373.
12. Chesney, T. (2006). An empirical examination of Wikipedia's credibility. First Monday, 11(11). http://journals.uic.edu/ojs/index.php/fm/article/view/1413/1331
13. Cohen, N. (2007). A history department bans citing Wikipedia as a research source. New York Times (21 February). Retrieved from http://www.nytimes.com/2007/02/21/education/21wikipedia.html
14. Denning, P., Horning, J., Parnas, D., & Weinstein, L. (2005). Wikipedia risks. Communications of the ACM, 48(12), 152.
15. Devgan, L., Powe, N., Blakey, B., & Makary, M. (2007). Wiki-Surgery? Internal validity of Wikipedia as a medical and surgical reference. Journal of the American College of Surgeons, 205(3), S76–S77. doi:10.1016/j.jamcollsurg.2007.06.190
16. de Winter, J.C.F., Zadpoor, A.A., & Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: A longitudinal study. Scientometrics, 98(2), 1547–1565.
17. Dooley, P.L. (2010). Wikipedia and the two-faced professoriate. Paper presented at the Proceedings of WikiSym 2010—the 6th International Symposium on Wikis and Open Collaboration, Gdansk; Poland; 7 July-9 July. doi:10.1145/1832772.1832803.
18. Eijkman, H. (2010). Academics and Wikipedia: Reframing web 2.0+as a disruptor of traditional academic power-knowledge arrangements. Campus-Wide Information Systems, 27(3), 173–185.
19. Elsevier (2013a). International Comparative Performance of the UK Research Base - 2013. Retrieved from https://www.gov.uk/government/publications/performance-of-the-uk-research-baseinternational-comparison-2013
20. Elsevier (2013b). Elsevier announces its Scopus book titles expansion program. Retrieved from http://www.elsevier.com/about/press-releases/science-and-technology/elsevier-announces-its-scopus-book-titles-expansion-program
21. Fairclough, R., & Thelwall, M. (2015). The influence of time and discipline on the magnitude of correlations between citation counts and quality scores. Journal of Informetrics, 9(3), 529–541.
22. Fallis, D. (2008). Toward an epistemology of Wikipedia. Journal of the American Society for Information Science and Technology, 59(10), 1662–1674.
23. Giles, J. (2005). Internet encyclopaedias go head to head. Nature, 438(7070), 900–901. doi:10.1038/438900a
24. Gorman, G.E. (2007). A tale of information ethics and Encyclopædias; or, is Wikipedia just another internet scam? Online Information Review, 31(3), 273–276.
25. Haigh, C.A. (2011). Wikipedia as an evidence source for nursing and healthcare students. Nurse Education Today, 31(2), 135–139.
26. Hale, S.A. (2015). Cross-language Wikipedia editing of Okinawa, Japan. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15) (pp. 1–8). New York: ACM Press.
27. Halfaker, A., & Taraborelli, D. (2015). Wikipedia scholarly article citations. Retrieved from http://dx.doi.org/10.6084/m9.figshare.1299540
28. Harzing, A. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.
29. Head, A.J., & Eisenberg, M.B. (2010). How today's college students use Wikipedia for course-related research. First Monday, 15(3).
30. Heilman, J.M., & West, A.G. (2015). Wikipedia and medicine: Quantifying readership, editors, and the significance of natural language. Journal of Medical Internet Research, 17(3). doi:10.2196/jmir.4069
31. Heilman, J.M., Kemmann, E., Bonert, M., Chatterjee, A., Ragar, B., Beards, G.M., … Laurent, M.R. (2011). Wikipedia: A key tool for global public health promotion. Journal of Medical Internet Research, 13(1), e14. doi:10.2196/jmir.1589
32. Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009). Junior physician's use of web 2.0 for information seeking and medical education: A qualitative study. International Journal of Medical Informatics, 78(10), 645–655.
33. Jacsó, P. (2011). Google Scholar duped and deduped - the aura of “robometrics”. Online Information Review, 35(1), 154–160.
34. Judd, T., & Kennedy, G. (2011). Expediency-based practice? Medical students' reliance on Google and Wikipedia for biomedical inquiries. British Journal of Educational Technology, 42(2), 351–360.
35. Jullien, N. (2012). What we know about Wikipedia. A review of the literature analyzing the project(s). Retrieved from https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2053597_code728676.pdf?abstractid=2053597&mirid=2
36. Khabsa, M., & Giles, C.L. (2014). The number of scholarly documents on the public web. PLoS One, 9(5), e93949. doi:10.1371/journal.pone.0093949.
37. Kim, J.Y., Gudewicz, T.M., Dighe, A.S., & Gilbertson, J.R. (2010). The pathology informatics curriculum wiki: Harnessing the power of user-generated content. Journal of Pathology Informatics, 1. doi:10.4103/2153-3539.65428
38. Knight, C., & Pryke, S. (2012). Wikipedia and the university, a case study. Teaching in Higher Education, 17(6), 649–659.
39. Kousha, K., & Thelwall, M. (2007a). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology, 58(7), 1055–1065.
40. Kousha, K., & Thelwall, M. (2008). Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses. Journal of the American Society for Information Science and Technology, 59(13), 2060–2069.
41. Kousha, K., & Thelwall, M. (2009). Google Book Search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537–1549.
42. Kousha, K., & Thelwall, M. (2014). An automatic method for extracting citations from Google Books. Journal of the Association for Information Science and Technology, 66(2), 309–320.
43. Kousha, K., & Thelwall, M. (2015a). An automatic method for assessing the teaching impact of books from online academic syllabi. Journal of the Association for Information Science and Technology. doi: 10.1002/asi.23542
44. Kousha, K., & Thelwall, M. (2015b). Patent citation analysis with Google. Journal of the Association for Information Science and Technology. doi: 10.1002/asi.23608
45. Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147–2164.
46. Kovic, I., Lulic, I., & Brumini, G. (2008). Examining the medical blogosphere: An online survey of medical bloggers. Journal of Medical Internet Research, 10(3), e28.
47. Kryl, D., Allen, L., Dolby, K., Sherbon, B., & Viney, I. (2012). Tracking the impact of research on policy and practice: Investigating the feasibility of using citations in clinical guidelines for research evaluation. BMJ Open, 2(2), e000897. doi: 10.1136/bmjopen-2012-000897
48. Labbé, C., & Labbé, D. (2013). Duplicate and fake publications in the scientific literature: How many SCIgen papers in computer science? Scientometrics, 94(1), 379–396.
49. Laurent, M.R., & Vickers, T.J. (2009). Seeking health information online: Does Wikipedia matter? Journal of the American Medical Informatics Association, 16(4), 471–479.
50. Leydesdorff, L., & Felt, U. (2012). Edited volumes, monographs and book chapters in the Book Citation Index (BKCI) and Science Citation Index (SCI, SoSCI, A&HCI). Journal of Scientometric Research, 1(1), 28–34.
51. Liao, H. (2014). The cultural politics of user-generated Encyclopedias: Comparing Chinese Wikipedia and Baidu. Oxford, UK: University of Oxford.
52. Lim, S. (2009). How and why do college students use Wikipedia? Journal of the American Society for Information Science and Technology, 60(11), 2189–2202.
53. Lin, J., & Fenner, M. (2014). An analysis of Wikipedia references across PLOS publications. Paper presented at the altmetrics14: Expanding impacts and metrics. An ACM Web Science Conference 2014 Workshop, Indiana University, IN. Retrieved from http://dx.doi.org/10.6084/m9.figshare.1048991
54. López-Cózar, E.D., Robinson-García, N., & Torres-Salinas, D. (2014). The Google Scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446–454.
55. Luyt, B., & Tan, D. (2010). Improving Wikipedia's credibility: References and citations in a sample of history articles. Journal of the American Society for Information Science and Technology, 61(4), 715–722.
56. Luzón, M.J. (2009). Scholarly hyperwriting: The function of links in academic weblogs. Journal of the American Society for Information Science and Technology, 60(1), 75–89.
57. MacRoberts, M.H., & MacRoberts, B.R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342–349.
58. MacRoberts, M.H., & MacRoberts, B.R. (1996). Problems of citation analysis. Scientometrics, 36(3), 435–444.
59. Meester, W. (2013). Towards a comprehensive citation index for the arts & humanities. Research Trends, 32. Retrieved from http://www.researchtrends.com/issue-32-march-2013/towards-a-comprehensive-citation-index-for-the-arts-humanities/
60. Mehegan, D. (2006). Bias, sabotage haunt Wikipedia's free world. The Boston Globe, February 12, 2006. Retrieved from http://www.boston.com/news/nation/articles/2006/02/12/bias_sabotage_haunt_wikipedias_free_world/
61. Meho, L.I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.
62. Mesgari, M., Okoli, C., Mehdi, M., Nielsen, F.Å., & Lanamäki, A. (2015). “The sum of all human knowledge”: A systematic review of scholarly research on the content of Wikipedia. Journal of the Association for Information Science and Technology, 66(2), 219–245. doi:10.1002/asi.23172
63. Moed, H.F. (2005). Citation analysis in research evaluation. New York, NY: Springer.
64. Narin, F., & Olivastro, D. (1992). Status report: Linkage between technology and science. Research Policy, 21(3), 237–249.
65. Nielsen, F.A. (2007). Scientific citations in Wikipedia. First Monday, 12(8). Retrieved from http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1997/1872
66. Okoli, C., Mehdi, M., Mesgari, M., Nielsen, F.Å., & Lanamäki, A. (2014). Wikipedia in the eyes of its beholders: A systematic review of scholarly research on Wikipedia readers and readership. Journal of the Association for Information Science and Technology, 65(12), 2381–2403.
67. Park, T.K. (2011). The visibility of Wikipedia in scholarly publications. First Monday, 16(8). Retrieved from http://www.ojphi.org/ojs/index.php/fm/article/view/3492
68. Priem, J., Piwowar, H., & Hemminger, B. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. Retrieved from http://arXiv.org/html/1203.4745v1
69. Rainie, L., & Tancer, B. (2007). Data memo. Retrieved from http://www.pewinternet.org/files/old-media//Files/Reports/2007/PIP_Wikipedia07.pdf.pdf
70. Schmoch, U. (1993). Tracing the knowledge transfer from science to technology as reflected in patent indicators. Scientometrics, 26(1), 193–211.
71. Scopus content coverage guide (2014). Retrieved from https://www.elsevier.com/__data/assets/pdf_file/0007/69451/sc_content-coverage-guide_july-2014.pdf
72. Shema, H., Bar-Ilan, J., & Thelwall, M. (2014). Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology, 65(5), 1018–1027.
73. Soules, A. (2015). Faculty perception of Wikipedia in the California state university system. New Library World, 116(3-4), 213–226.
74. Spoerri, A. (2007). What is popular on Wikipedia and why? First Monday, 12(4). Retrieved from http://ojs-prod-lib.cc.uic.edu/ojs/index.php/fm/article/view/1765/1645
75. Stankus, T., & Spiegel, S.E. (2010). Wikipedia, Scholarpedia, and references to books in the brain and behavioral sciences: A comparison of cited sources and recommended readings in matching free online Encyclopedia entries. Science and Technology Libraries, 29(1–2), 144–164.
76. Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143.
77. Teplitskiy, M., Lu, G., & Duede, E. (2015). Amplifying the impact of Open Access: Wikipedia and the diffusion of science. Retrieved from http://arxiv.org/abs/1506.07608
78. The Book Citation Index in Web of Science (2015). Thomson Reuters. http://wokinfo.com/products_tools/multidisciplinary/bookcitationindex/
79. Thelwall, M., & Kousha, K. (2008). Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals. Journal of the American Society for Information Science & Technology, 59(5), 805–815.
80. Thelwall, M., & Kousha, K. (2015). Web indicators for research evaluation. Part 1: Citations and links to academic articles from the Web. El Profesional De La Información, 24(5), 587–606.
81. Thelwall, M., & Maflahi, N. (2015). Guideline references and academic citations as evidence of the clinical value of health research. Journal of the Association for Information Science and Technology. doi: 10.1002/asi.23432
82. Thomas, G.R., Eng, L., de Wolff, J.F., & Grover, S.C. (2013). An evaluation of Wikipedia as a resource for patient education in nephrology. Seminars in Dialysis, 26(2), 159–163. doi:10.1111/sdi.12059
83. Thomson Reuters. (2015). Book citation index. Retrieved from http://wokinfo.com/products_tools/multidisciplinary/bookcitationindex/
84. Vaughan, L., & Shaw, D. (2003). Bibliographic and web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(14), 1313–1322.
85. Vaughan, L., & Shaw, D. (2005). Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology, 56(10), 1075–1087.
86. Web of Science core collection. (2015). Retrieved from http://wokinfo.com/products_tools/multidisciplinary/webofscience/
87. Wouters, P., & Costas, R. (2012). Users, narcissism and control: Tracking the impact of scholarly publications in the 21st century. In E. Archambault, Y. Gingras, & V. Larivière (Eds.), Proceedings of the 17th International Conference on Science and Technology Indicators (Vol. 2, pp. 487–497). Montreal: Science-Metrix and OST.
88. Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., & Franssen, T. (2015). The metric tide: Literature review (Supplementary report I to the independent review of the role of metrics in research assessment and management). The Higher Education Funding Council for England. doi: 10.13140/RG.2.1.5066.3520. Retrieved from http://blogs.lse.ac.uk/impactofsocialsciences/files/2015/07/2015_metrictideS1.pdf
89. Yasseri, T., Spoerri, A., Graham, M., & Kertész, J. (2014). The most controversial topics in Wikipedia: A multilingual and geographical analysis. In P. Fichman, N. Hara (Eds.), Global Wikipedia: International and cross-cultural issues in online collaboration (pp. 21–44). Lanham, MD: Rowman & Littlefield Education.