Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS and F1000Prime

Aslib Journal of Information Management

Volumn 67, Issue 3, 2015, Pages 305-319

  Bornmann, Lutz ^a  

^a MAX PLANCK INSTITUTE   (Germany)

Author keywords

Altmetrics; F1000; Facebook; Figshare; Mendeley; Twitter

Indexed keywords

LIBRARIES; PAPER; REGRESSION ANALYSIS;

ALTMETRICS; F1000; FACEBOOK; FIGSHARE; MENDELEY; TWITTER;

SOCIAL NETWORKING (ONLINE);

EID: 84929663411     PISSN: 20503806     EISSN: 20503814     Source Type: Journal    
DOI: 10.1108/AJIM-09-2014-0115     Document Type: Article

References (44)

1. Bar-Ilan, J., Shema, H. and Thelwall, M. (2014), “Bibliographic references in web 2.0.”, in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-Dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 307-325.
2. Bik, H.M. and Goldstein, M.C. (2013), “An introduction to social media for scientists”, PLoS Biology, Vol. 11 No. 4, e1001535.
3. Bornmann, L. (2012), “Measuring the societal impact of research”, EMBO Reports, Vol. 13 No. 8, pp. 673-676.
4. Bornmann, L. (2013), “What is societal impact of research and how can it be assessed? A literature survey”, Journal of the American Society of Information Science and Technology, Vol. 64 No. 2, pp. 217-233.
5. Bornmann, L. (2014a), “Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics”, Journal of Informetrics, Vol. 8 No. 4, pp. 895-903.
6. Bornmann, L. (2014b), “Is there currently a scientific revolution in scientometrics?”, Journal of the Association for Information Science and Technology, Vol. 65 No. 3, pp. 647-648.
7. Bornmann, L. (2014c), “Validity of altmetrics data for measuring societal impact: a study using data from Altmetric and F1000Prime”, Journal of Informetrics, Vol. 8 No. 4, pp. 935-950.
8. Bornmann, L. (in press-a), “Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics”, Scientometrics, available at: http://arxiv.org/abs/1407.8010 (accessed 19 March 2015).
9. Bornmann, L. (in press-b), “Inter-rater reliability and convergent validity of F1000Prime peer review”, Journal of the Association for Information Science and Technology, available at: http://arxiv.org/abs/1404.0359 (accessed 19 March 2015).
10. Bornmann, L. and Marx, W. (2014), “How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations”, Scientometrics, Vol. 98 No. 1, pp. 487-509.
11. Bornmann, L. andWilliams, R. (2013), “How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects”, Journal of Informetrics, Vol. 7 No. 2, pp. 562-574.
12. Claussen, J.E., Cooney, P.B., Defilippi, J.M., Fox, S.G., Glaser, S.M., Hawkes, E., Hutt, C., Jones, M. H., Kemp, I.M., Lerner, A., Midway, S.R., Nesbit, S., Osborne-Gowey, J., Roberts, R. and Steward, C. (2013), “Science communication in a digital age: social media and the American fisheries society”, Fisheries, Vol. 38 No. 8, pp. 359-362.
13. Darling, E.S., Shiffman, D., Côté, I.M. and Drew, J.A. (2013), “The role of Twitter in the life cycle of a scientific publication”, Ideas in Ecology and Evolution, Vol. 6 No. 1, pp. 32-43.
14. Dinsmore, A., Allen, L. and Dolby, K. (2014), “Alternative perspectives on impact: the potential of ALMs and altmetrics to inform funders about research impact”, PLoS Biology, Vol. 12 No. 11, e1002003.
15. Du, J., Tang, X. and Wu, Y. (2014), “The effects of research level and article type on the differences between citation metrics and F1000 recommendations”, available at: http://arxiv.org/abs/ 1501.01076 (accessed 16 January 2015).
16. F1000 (2012), “What is F1000?”, available at: http://f1000.com/about/whatis (accessed 25 October 2014).
17. Fenner, M. (2013), “What can article-level metrics do for you?”, PLoS Biology, Vol. 11 No. 10, e1001687.
18. Hahnel, M. (2013), “The reuse factor”, Nature, Vol. 502 No. 7471, p. 298.
19. Hardin, J. and Hilbe, J. (2012), Generalized Linear Models and Extensions, Stata Press, College Station, TX.
20. Hausman, J., Hall, B.H. and Griliches, Z. (1984), “Econometric models for count data with an application to the patents R and D relationship”, Econometrica, Vol. 52 No. 4, pp. 909-938.
21. Haustein, S. (2014), “Readership metrics”, in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-Dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 327-344.
22. Haustein, S. and Siebenlist, T. (2011), “Applying social bookmarking data to evaluate journal usage”, Journal of Informetrics, Vol. 5 No. 3, pp. 446-457.
23. Haustein, S., Larivière, V., Thelwall, M., Amyot, D. and Peters, I. (2014a), “Tweets vs Mendeley readers: How do these two social media metrics differ?”, It - Information Technology, Vol. 56 No. 5, pp. 207-215.
24. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H. and Terliesner, J. (2014b), “Coverage and adoption of altmetrics sources in the bibliometric community”, Scientometrics, Vol. 101 No. 2, pp. 1145-1163.
25. Hilbe, J.M. (2007), Negative Binomial Regression, Cambridge University Press, Cambridge.
26. Hosmer,D.W. and Lemeshow, S. (2000), Applied Logistic Regression, JohnWiley & Sons, Chichester.
27. Kreiman, G. and Maunsell, J.H.R. (2011), “Nine criteria for a measure of scientific output”, Frontiers in Computational Neuroscience, Vol. 5, Article no. 48.
28. Li, X. and Thelwall, M. (2012), “F1000, Mendeley and traditional bibliometric indicators”, Proceedings of the 17th International Conference on Science and Technology Indicators, Repro-UQAM, Montreal, pp. 541-551, available at: http://2012.sticonference.org/ Proceedings/vol2/Li_F1000_541.pdf (accessed 8 March 2015).
29. Li, X., Thelwall, M. and Giustini, D. (2012), “Validating online reference managers for scholarly impact measurement”, Scientometrics, Vol. 91 No. 2, pp. 461-471.
30. Long, J.S. and Freese, J. (2006), Regression Models for Categorical Dependent Variables Using Stata, Stata Press, College Station, TX.
31. Mahrt, M., Weller, K. and Peters, I. (2012), “Twitter in scholarly communication”, in Weller, K., Bruns, A., Burgess, J., Mahrt, M. and Puschmann, C. (Eds), Twitter and Society, Peter Lang, New York, NY, pp. 468-483.
32. Mohammadi, E. and Thelwall, M. (2013), “Assessing non-standard article impact using F1000 labels”, Scientometrics, Vol. 97 No. 2, pp. 383-395.
33. NISO Alternative Assessment Metrics Project (2014), “NISO altmetrics standards project white paper”, working paper, NISO Alternative Assessment Metrics Project, available at: www. niso.org/apps/group_public/document.php?document_id=13295&wg_abbrev=altmetrics (accessed 8 July 2014).
34. Priem, J. (2014), “Altmetrics”, in Cronin, B. and Sugimoto, C.R. (Eds), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance, MIT Press, Cambridge, MA, pp. 263-287.
35. Robinson-García, N., Torres-Salinas, D., Zahedia, Z. and Costas, R. (2014), “New data, new possibilities: exploring the insides of Altmetric.com”, El Profesional de la Información, Vol. 23 No. 4, pp. 359-366.
36. Stata Corporation (2013), Stata Statistical Software: Release 13, Stata Press, College Station, TX.
37. Taylor, M. and Plume, A. (2014), “Party papers or policy discussions: an examination of highly shared papers using altmetric data”, Research Trends, No. 36, pp. 17-20.
38. Thelwall, M. (2014), “Five recommendations for using alternative metrics in the future UK Research Excellence Framework”, The Impact Blog, available at: http://blogs.lse.ac.uk/ impactofsocialsciences/2014/10/23/alternative-metrics-future-uk-research-excellenceframework- thelwall/ (accessed 2 January 2015).
39. Waltman, L. and Costas, R. (2014), “F1000 recommendations as a potential new data source for research evaluation: a comparison with citations”, Journal of the Association for Information Science and Technology, Vol. 65 No. 3, pp. 433-445.
40. Wardle, D.A. (2010), “Do 'Faculty of 1000' (F1000) ratings of ecological publications serve as reasonable predictors of their future impact?”, Ideas in Ecology and Evolution, Vol. 3, pp. 11-15.
41. Wets, K., Weedon, D. and Velterop, J. (2003), “Post-publication filtering and evaluation: Faculty of 1000”, Learned Publishing, Vol. 16 No. 4, pp. 249-258.
42. Williams, R. (2012), “Using the margins command to estimate and interpret adjusted predictions and marginal effects”, The Stata Journal, Vol. 12 No. 2, pp. 308-331.
43. Wouters, P. and Costas, R. (2012), Users, Narcissism and Control - Tracking the Impact of Scholarly Publications in the 21st Century, SURFfoundation, Utrecht, available at: www.surf.nl/binaries/content/assets/surf/en/knowledgebase/2011/Users+narcissism+and +control.pdf (accessed 8 March 2015).
44. Zahedi, Z., Costas, R. and Wouters, P. (2014), “How well developed are altmetrics? A crossdisciplinary analysis of the presence of 'alternative metrics' in scientific publications”, Scientometrics, Vol. 101 No. 2, pp. 1491-1513.