To what extent does the Leiden manifesto also apply to altmetrics? A discussion of the manifesto against the background of research into altmetrics

Online Information Review

Volumn 40, Issue 4, 2016, Pages 529-543

  Bornmann, Lutz ^a   Haunschild, Robin ^b  

^a MAX PLANCK INSTITUTE   (Germany)

^b MAX PLANCK INSTITUT FÜR FESTKÖRPERFORSCHUNG   (Germany)

Author keywords

Altmetrics; Bibliometrics; Leiden manifesto; Scientometrics

Indexed keywords

SOCIAL NETWORKING (ONLINE);

ALTMETRICS; BIBLIOMETRIC; BIBLIOMETRICS; DESIGN/METHODOLOGY/APPROACH; LEIDEN MANIFESTO; RESEARCH EVALUATION; SCIENTOMETRICS;

COMPUTER APPLICATIONS;

BIBLIOMETRICS;

EID: 84981262524     PISSN: 14684527     EISSN: None     Source Type: Journal    
DOI: 10.1108/OIR-09-2015-0314     Document Type: Article

References (63)

1. Adie, E. (2014), "Taking the alternative mainstream", Profesional De La Informacion, Vol. 23 No. 4, pp. 349-351, doi: 10.3145/epi.2014.jul.01.
2. Anon (2012), "Alternative metrics", Nature Materials, Vol. 11 No. 11, p. 907.
3. Bar-Ilan, J., Shema, H. and Thelwall, M. (2014), "Bibliographic references in Web 2.0", in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-Dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 307-325.
4. Bik, H.M. and Goldstein, M.C. (2013), "An introduction to social media for scientists", PLoS Biology, Vol. 11 No. 4, doi: 10.1371/journal.pbio.1001535.
5. Bloch, C., Sørensen, M.P., Graversen, E.K., Schneider, J.W., Schmidt, E.K., Aagaard, K. and Mejlgaard, N. (2014), "Developing a methodology to assess the impact of research grant funding: a mixed methods approach", Evaluation and Program Planning, Vol. 43, pp. 105-117, doi: 10.1016/j.evalprogplan.2013.12.005.
6. Bornmann, L. (2011a), "Mimicry in science?", Scientometrics, Vol. 86 No. 1, pp. 173-177, doi: 10.1007/s11192-010-0222-8.
7. Bornmann, L. (2011b), "Scientific peer review", Annual Review of Information Science and Technology, Vol. 45, pp. 199-245.
8. Bornmann, L. (2013), "How to analyse percentile citation impact data meaningfully in bibliometrics: the statistical analysis of distributions, percentile rank classes and top-cited papers", Journal of the American Society for Information Science and Technology, Vol. 64 No. 3, pp. 587-595.
9. Bornmann, L. and Daniel, H.-D. (2007), "Multiple publication on a single research study: does it pay? The influence of number of research articles on total citation counts in biomedicine", Journal of the American Society for Information Science and Technology, Vol. 58 No. 8, pp. 1100-1107.
10. Bornmann, L. and Marx, W. (2014a), "How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons", Scientometrics, Vol. 98 No. 1, pp. 211-219.
11. Bornmann, L. and Marx, W. (2014b), "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations", Scientometrics, Vol. 98 No. 1, pp. 487-509, doi: 10.1007/s11192-013-1161-y.
12. Bornmann, L., Leydesdorff, L. and Mutz, R. (2013a), "The use of percentiles and percentile rank classes in the analysis of bibliometric data: opportunities and limits", Journal of Informetrics, Vol. 7 No. 1, pp. 158-165.
13. Bornmann, L., Mutz, R. and Daniel, H.-D. (2013b), "A multilevel-statistical reformulation of citation-based university rankings: the Leiden ranking 2011/2012", Journal of the American Society for Information Science and Technology, Vol. 64 No. 8, pp. 1649-1658.
14. Bornmann, L., Thor, A., Marx, W. and Schier, H. (2015), "The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized Google Scholar data for the publications of a research institute", Journal of the Association for Information Science and Technology (in press).
15. Chamberlain, S. (2013), "Consuming article-level metrics: observations and lessons", Information Standards Quarterly, Vol. 25 No. 2, pp. 4-13.
16. Chen, K.-H., Tang, M.-C., Wang, C.-M. and Hsiang, J. (2015), "Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan", Scientometrics, Vol. 102 No. 1, pp. 97-112, doi: 10.1007/s11192-014-1420-6.
17. Costas, R., Zahedi, Z. and Wouters, P. (2014), "Disentangling the meaning of 'altmetrics': content analysis of Web of Science scientific publications", available at: http://figshare.com/articles/Disentangling-the-meaning-of-altmetrics-content-analysis-of-Web-of-Science-scientific-publications/1041770 (accessed July 9, 2014).
18. Cronin, B. (2013), "The evolving indicator space (iSpace)", Journal of the American Society for Information Science and Technology, Vol. 64 No. 8, pp. 1523-1525, doi: 10.1002/asi.23041.
19. Darling, E.S., Shiffman, D., Côté, I.M. and Drew, J.A. (2013), "The role of Twitter in the life cycle of a scientific publication", PeerJ PrePrints, Vol. 1, doi: 10.7287/peerj. preprints.16v1.
20. Das, A. and Mishra, S. (2014), "Genesis of altmetrics or article-level metrics for measuring efficacy of scholarly communications: current perspectives", Journal of Scientometric Research, Vol. 3 No. 2, pp. 82-92.
21. Dinsmore, A., Allen, L. and Dolby, K. (2014), "Alternative perspectives on impact: the potential of alms and altmetrics to inform funders about research impact", PLoS Biology, Vol. 12 No. 11, doi: 10.1371/journal.pbio.1002003.
22. Fairclough, R. and Thelwall, M. (2015), "National research impact indicators from Mendeley readers", Journal of Informetrics, Vol. 9 No. 4, pp. 845-859, doi: 10.1016/j. joi.2015.08.003.
23. Fausto, S., Machado, F.A., Bento, L.F.J., Iamarino, A., Nahas, T.R. and Munger, D.S. (2012), "Research blogging: indexing and registering the change in science 2.0", PLoS ONE, Vol. 7 No. 12, doi: 10.1371/journal.pone.0050109.
24. Franceschini, F., Maisano, D. and Mastrogiacomo, L. (2015), "Errors in DOI indexing by bibliometric databases", Scientometrics, Vol. 102 No. 3, pp. 2181-2186, doi: 10.1007/s11192-014-1503-4.
25. Galloway, L.M., Pease, J.L. and Rauh, A.E. (2013), "Introduction to altmetrics for science, technology, engineering, and mathematics (STEM) librarians", Science & Technology Libraries, Vol. 32 No. 4, pp. 335-345, doi: 10.1080/0194262X.2013.829762.
26. Glänzel, W. and Gorraiz, J. (2014), "Usage metrics versus altmetrics: confusing terminology?", Scientometrics, Vol. 102 No. 3, pp. 2161-2164, doi: 10.1007/s11192-014-1472-7.
27. Hammarfelt, B. (2013), "An examination of the possibilities that altmetric methods offer in the case of the humanities (RIP)", paper presented at the Proceedings of ISSI 2013 - 14th International Society of Scientometrics and Informetrics Conference, Vienna, July 15-19.
28. Hammarfelt, B. (2014), "Using altmetrics for assessing research impact in the humanities", Scientometrics, Vol. 101 No. 2, pp. 1419-1430, doi: 10.1007/s11192-014-1261-3.
29. Haunschild, R. and Bornmann, L. (2016), "Normalization of Mendeley reader counts for impact assessment", Journal of Informetrics, Vol. 10 No. 1, pp. 62-73.
30. Haustein, S. (2014), "Readership metrics", in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-Dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 327-344.
31. Haustein, S., Larivière, V., Thelwall, M., Amyot, D. and Peters, I. (2014a), "Tweets vs Mendeley readers: how do these two social media metrics differ?", IT - Information Technology 2014, Vol. 56 No. 5, pp. 207-215.
32. Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M. and Larivière, V. (2014b), "Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature", Journal of the Association for Information Science and Technology, Vol. 65 No. 4, pp. 656-669, doi: 10.1002/asi.23101.
33. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H. and Terliesner, J. (2014c), "Coverage and adoption of altmetrics sources in the bibliometric community", Scientometrics, Vol. 101 No. 2, pp. 1145-1163, doi: 10.1007/s11192-013-1221-3.
34. Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. and Rafols, I. (2015), "Bibliometrics: the Leiden manifesto for research metrics", Nature, Vol. 520 No. 7548, pp. 429-431.
35. Kurtz, M.J. and Bollen, J. (2010), "Usage bibliometrics.", Annual Review of Information Science and Technology, Vol. 44, pp. 3-64.
36. Liu, C.L., Xu, Y.Q., Wu, H., Chen, S.S. and Guo, J.J. (2013), "Correlation and interaction visualization of altmetric indicators extracted from scholarly social network activities: dimensions and structure", Journal of Medical Internet Research, Vol. 15 No. 11, p. 17, doi: 10.2196/jmir.2707.
37. Liu, J. and Adie, E. (2013), "Five challenges in altmetrics: a toolmaker's perspective", Bulletin of the American Society for Information Science and Technology, Vol. 39 No. 4, pp. 31-34, doi: 10.1002/bult.2013.1720390410.
38. Marx, W. and Bornmann, L. (2015), "On the causes of subject-specific citation rates in Web of Science", Scientometrics, Vol. 102 No. 2, pp. 1823-1827.
39. Moed, H.F. (2005), Citation Analysis in Research Evaluation, Springer, Dordrecht.
40. Mohammadi, E. and Thelwall, M. (2014), "Mendeley readership altmetrics for the social sciences and humanities: research evaluation and knowledge flows", Journal of the Association for Information Science and Technology, Vol. 65 No. 8, pp. 1627-1638, doi: 10.1002/asi.23071.
41. Neylon, C. (2014), "Altmetrics: what are they good for?", available at: http://blogs.plos.org/opens/2014/10/03/altmetrics-what-are-they-good-for/#.VC8WETI0JAM.twitter (accessed October 6, 2014).
42. Neylon, C., Willmers, M. and King, T. (2014), Rethinking Impact: Applying Altmetrics to Southern African Research, International Development Research Centre, Ottawa.
43. NISO Alternative Assessment Metrics Project (2014), "NISO altmetrics standards project white paper", available at: www.niso.org/apps/group-public/document.php?document-id=132 95&wg-abbrev=altmetrics (accessed July 8, 2014).
44. Piwowar, H. (2013), "Altmetrics: value all research products", Nature, Vol. 493 No. 7431, p. 159.
45. Plume, A. and Kamalski, J. (2014), "Article downloads: an alternative indicator of national research impact and cross-sector knowledge exchange", Research Trends, No. 36, pp. 13-16.
46. Priem, J. (2014), "Altmetrics", in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-Dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 263-288.
47. Priem, J. and Hemminger, B.M. (2010), "Scientometrics 2.0: toward new metrics of scholarly impact on the social web", First Monday, Vol. 15 No. 7.
48. Priem, J., Taraborelli, D., Groth, P. and Neylon, C. (2010), "Altmetrics: a manifesto", available at: http://altmetrics.org/manifesto/ (accessed March 28).
49. Roemer, R.C. and Borchardt, R. (2013), "Institutional altmetrics & academic libraries", Information Standards Quarterly, Vol. 25 No. 2, pp. 14-19.
50. Rousseau, R. and Ye, F.Y. (2013), "A multi-metric approach for research evaluation", Chinese Science Bulletin, Vol. 58 No. 26, pp. 3288-3290, doi: 10.1007/s11434-013-5939-3.
51. Sanström, U. (2014), Bibliometric Evaluation of SEPA-Funded Large Research Programs 2003-2013, Swedish Environmental Protection Agency, Stockholm.
52. Schubert, A. and Braun, T. (1986), "Relative indicators and relational charts for comparative assessment of publication output and citation impact", Scientometrics, Vol. 9 Nos 5-6, pp. 281-291.
53. Shema, H., Bar-Ilan, J. and Thelwall, M. (2014), "Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics", Journal of the Association for Information Science and Technology, Vol. 65 No. 5, pp. 1018-1027, doi: 10.1002/asi.23037.
54. Taylor, M. (2013), "Towards a common model of citation: some thoughts on merging altmetrics and bibliometrics", Research Trends, No. 35, pp. 19-22.
55. Taylor, M. and Plume, A. (2014), "Party papers or policy discussions: an examination of highly shared papers using altmetric data", Research Trends, No. 36, pp. 17-20.
56. Thelwall, M. (2014), "Five recommendations for using alternative metrics in the future UK Research Excellence Framework", available at: http://blogs.lse.ac.uk/impactofsocialsciences/2014/10/23/alternative-metrics-future-uk-research-excellence-framework-thelwall/ (accessed January 2).
57. Vinkler, P. (2010), The Evaluation of Research by Scientometric Indicators, Chandos Publishing, Oxford.
58. Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S. and van Raan, A.F.J. (2011), "Towards a new crown indicator: some theoretical considerations", Journal of Informetrics, Vol. 5 No. 1, pp. 37-47, doi: 10.1016/j.joi.2010.08.001.
59. Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E.C.M., Tijssen, R.J.W., van Eck, N.J., van Leeuwen, T.N., van Raan, A.F.J., Visser, M.S. and Wouters, P. (2012), "The Leiden ranking 2011/2012: data collection, indicators, and interpretation", Journal of the American Society for Information Science and Technology, Vol. 63 No. 12, pp. 2419-2432.
60. Weller, K. (2015), "Social media and altmetrics: an overview of current alternative approaches to measuring scholarly impact.", in Welpe, I.M., Wollersheim, J., Ringelhan, S. and Osterloh, M. (Eds), Incentives and Performance, Springer International Publishing, pp. 261-276.
61. Zahedi, Z. and Eck, N.J.V. (2014), "Visualizing readership activity of Mendeley users using VOSviewer", available at: http://figshare.com/articles/Visualizing-readership-activity-of-Mendeley-users-using-VOSviewer/1041819 (accessed July 8, 2014).
62. Zahedi, Z., Costas, R. and Wouters, P. (2014a), "How well developed are altmetrics? A cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications", Scientometrics, Vol. 101 No. 2, pp. 1491-1513, doi: 10.1007/s11192-014-1264-0.
63. Zahedi, Z., Fenner, M. and Costas, R. (2014b), "How consistent are altmetrics providers? Study of 1000 PLOS ONE publications using the PLOS ALM, Mendeley and Altmetric.com APIs", available at: https://speakerdeck.com/mfenner/how-consistent-are-altmetrics-providersstudy-of-1000-plos-one-publications-using-the-plos-alm-mendeley-and-altmetric-dot-comapis (accessed July 8, 2014).