The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute

Journal of the Association for Information Science and Technology

Volumn 67, Issue 11, 2016, Pages 2778-2789

  Bornmann, Lutz ^a   Thor, Andreas ^b   Marx, Werner ^c   Schier, Hermann ^c  

^a MAX PLANCK INSTITUTE   (Germany)

^b LEIPZIG UNIVERSITY OF APPLIED SCIENCES   (Germany)

^c MAX PLANCK INSTITUT FÜR FESTKÖRPERFORSCHUNG   (Germany)

Author keywords

bibliometrics

Indexed keywords

INFORMATION ANALYSIS; INFORMATION SCIENCE; SOCIAL SCIENCES;

BIBLIOMETRICS; CONVERGENT VALIDITY; EXPLORATORY STUDIES; HUMANITIES AND SOCIAL SCIENCE; NORMALIZED SCORES; RESEARCH EVALUATION; RESEARCH INSTITUTES; RESEARCH PERFORMANCE;

BEHAVIORAL RESEARCH;

EID: 84960146093     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.23627     Document Type: Article

References (41)

1. Bornmann, L., & Marx, W. (2014). How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics, 98(1), 487–509.
2. Bornmann, L., Marx, W., Schier, H., Rahm, E., Thor, A., & Daniel, H.D. (2009). Convergent validity of bibliometric Google Scholar data in the field of chemistry. Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. Journal of Informetrics, 3(1), 27–35.
3. Bornmann, L., de Moya Anegón, F., & Mutz, R. (2013). Do universities or research institutions with a specific subject profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago ranking. Journal of the American Society for Information Science and Technology JASIST, 64(11), 2310–2316.
4. Bornmann, L., Bowman, B.F., Bauer, J., Marx, W., Schier, H., & Palzenberger, M. (2014). Standards for using bibliometrics in the evaluation of research institutes. In B. Cronin & C. Sugimoto (Eds.), Next Generation Metrics (pp. 201–223). Cambridge, MA: MIT Press.
5. Butler, L., & Visser, M.S. (2006). Extending citation analysis to non-source items. Scientometrics, 66(2), 327–343.
6. Chi, P.S. (2013). Do non-source items make a difference in the social sciences? Paper presented at the Proceedings of ISSI 2013—14th International Society of Scientometrics and Informetrics Conference.
7. Chi, P.S. (2014). Which role do non-source items play in the social sciences? A case study in political science in Germany. Scientometrics, 101(2), 1195–1213.
8. Colledge, L. (2014). Snowball Metrics Recipe Book. Amsterdam: Snowball Metrics Program Partners.
9. Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446–454.
10. Franceschet, M. (2010). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243–258.
11. Frandsen, T.F., & Nicolaisen, J. (2008). Intradisciplinary differences in database coverage and the consequences for bibliometric research. Journal of the American Society for Information Science and Technology JASIST, 59(10), 1570–1581.
12. Gorraiz, J., Purnell, P.J., & Glänzel, W. (2013). Opportunities for and limitations of the Book Citation Index. Journal of the American Society for Information Science and Technology JASIST, 64(7), 1388–1398.
13. Harzing, A.-W. (2013). A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel Prize winners. Scientometrics, 94(3), 1057–1075.
14. Harzing, A.-W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.
15. Hug, S.E., Ochsner, M., & Daniel, H.-D. (2013). Criteria for assessing research quality in the humanities—A delphi study among scholars of English literature, German literature and art history. Research Evaluation, 22(5), 369–383.
16. Hug, S.E., Ochsner, M., & Daniel, H.-D. (2014). A framework to explore and develop criteria for assessing research quality in the humanities. International Journal for Education Law and Policy, 10(1), 55–68.
17. Jacso, P. (2005). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208–214.
18. Jacso, P. (2009). Google Scholar's ghost authors. Library Journal, 134(18), 26–27.
19. Jacso, P. (2012). Google Scholar metrics for publications: The software and content features of a new open access bibliometric service. Online Information Review, 36(4), 604–619.
20. Köpcke, H., Thor, A., & Rahm, E. (2010). Evaluation of entity resolution approaches on real-world match problems. Paper presented at the 36th Intl. Conference on Very Large Databases (VLDB) / Proceedings of the VLDB Endowment.
21. Kousha, K., & Thelwall, M. (2007). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology JASIST, 58(7), 1055–1065.
22. Kousha, K., & Thelwall, M. (2008). Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines. Scientometrics, 74(2), 273–294.
23. Kousha, K., Thelwall, M., & Rezaie, S. (2010). Using the Web for research evaluation: The Integrated Online Impact indicator. Journal of Informetrics, 4(1), 124–135.
24. Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology JASIST, 62(11), 2147–2164. doi: 10.1002/asi.21608
25. Larsen, P.O., & von Ins, M. (2010). The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84(3), 575–603.
26. Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M., & Delgado López-Cózar, E. (2014). Does Google Scholar contain all highly cited documents (1950-2013)? Granada: EC3 Working Papers No. 19.
27. Marx, W., & Bornmann, L. (2015). On the causes of subject-specific citation rates in Web of Science. Scientometrics, 102(2), 1823–1827.
28. Meho, L.I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology JASIST, 58(13), 2105–2125.
29. Moed, H.F. (2005). Citation analysis in research evaluation. Dordrecht, The Netherlands: Springer.
30. Nederhof, A.J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100.
31. Ochsner, M., Hug, S.E., & Daniel, H.D. (2012). Indicators for research quality in the humanities: Opportunities and limitations. Bibliometrie-Praxis und Forschung, 1(4).
32. Ochsner, M., Hug, S.E., & Daniel, H.-D. (2013). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation, 22(2), 79–92 doi: 10.1093/reseval/rvs039
33. Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., & Delgado López-Cózar, E. (2014). About the size of Google Scholar: playing the numbers. Granada: EC3 Working Papers, 18: 23 July 2014.
34. Prins, A., Costas, R., van Leeuwen, T., & Wouters, P. (2014). Using Google Scholar in research evaluation of social science programs, with a comparison with Web of Science data. In E. Noyons (Ed.), Proceedings of the Science and Technology Indicators Conference 2014 Leiden “Context Counts: Pathways to Master Big and Little Data” (pp. 434–443). Leiden, the Netherlands: Centre for Science and Technology Studies.
35. Pudovkin, A.I., & Garfield, E. (2009). Percentile rank and author superiority indexes for evaluating individual journal articles and the author's overall citation performance. COLLNET Journal of Scientometrics and Information Management, 3(2), 3–10.
36. Rehn, C., Kronman, U., & Wadskog, D. (2007). Bibliometric indicators—Definitions and usage at Karolinska Institutet. Stockholm: Karolinska Institutet University Library.
37. Thor, A., & Rahm, E. (2007). MOMA—A mapping-based object matching system. In CIDR (Ed.), Third biennial conference on innovative data systems research (pp. 247–258). Asilomar, CA: Conference on Innovative Data Systems Research. Retrieved from www.cidrdb.org
38. Torres-Salinas, D., Robinson-Garcia, N., Campanario, J.M., & López-Cózar, E.D. (2014). Coverage, field specialisation and the impact of scientific publishers indexed in the Book Citation Index. Online Information Review, 38(1), 24–42.
39. Van Noorden, R. (2014). Scientists and the social network. Nature, 512(7513), 126–129.
40. Vinkler, P. (2010). The evaluation of research by scientometric indicators. Oxford, UK: Chandos Publishing.
41. Vinkler, P. (2012). The case of scientometricians with the “absolute relative” impact indicator. Journal of Informetrics, 6(2), 254–264.