Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime

Journal of Informetrics

Volumn 8, Issue 4, 2014, Pages 935-950

  Bornmann, Lutz ^a  

^a MAX PLANCK INSTITUTE   (Germany)

Author keywords

Altmetrics; Bibliometrics; F1000; Societal impact; Twitter

Indexed keywords

PAPER;

ALTMETRICS; BIBLIOMETRICS; F1000; SOCIETAL IMPACTS; TWITTER;

ECONOMIC AND SOCIAL EFFECTS;

EID: 84908669081     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2014.09.007     Document Type: Article

References (40)

1. Bar-Ilan J. JASIST 2001-2010. Bulletin of the American Society for Information Science and Technology 2012, 38(6):24-28. 10.1002/bult.2012.1720380607.
2. Bar-Ilan J., Shema H., Thelwall M. Bibliographic references in Web 2.0. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance 2014, 307-325. MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
3. Bornmann L. Measuring the societal impact of research. EMBO Reports 2012, 13(8):673-676.
4. Bornmann L. What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society of Information Science and Technology 2013, 64(2):217-233.
5. Bornmann L. Is there currently a scientific revolution in scientometrics?. Journal of the Association for Information Science and Technology 2014, 65(3):647-648.
6. Bornmann L. Inter-rater reliability and convergent validity of F1000Prime peer review. Journal of the Association for Information Science and Technology 2014, (in press).
7. Bornmann L., Leydesdorff L., Wang J. Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100). Journal of Informetrics 2013, 7(4):933-944. 10.1016/j.joi.2013.09.003.
8. Bornmann L., Marx W. How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons. Scientometrics 2014, 98(1):211-219.
9. Bornmann L., Williams R. How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects. Journal of Informetrics 2013, 7(2):562-574. 10.1016/j.joi.2013.02.005.
10. Darling E.S., Shiffman D., Côté I.M., Drew J.A. The role of Twitter in the life cycle of a scientific publication. PeerJ PrePrints 2013, 1:e16v11. 10.7287/peerj.preprints.16v1.
11. [F1000] What is F1000? 2012, Retrieved October 25, from http://f1000.com/about/whatis.
12. Grant J. Evaluating the outcomes of biomedical research on healthcare. Research Evaluation 1999, 8(1):33-38.
13. Hausman J., Hall B.H., Griliches Z. Econometric models for count data with an application to the patents R and D relationship. Econometrica 1984, 52(4):909-938.
14. Haustein S. Readership metrics. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance 2014, 327-344. MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
15. Hazelkorn E. Rankings and the reshaping of higher education. The battle for world-class excellence 2011, Palgrave Macmillan, New York, NY, USA.
16. Hilbe J.M. Negative binomial regression 2007, Cambridge University Press, Cambridge, UK.
17. Hosmer D.W., Lemeshow S. Applied logistic regression 2000, John Wiley & Sons, Inc., Chichester, UK. 2nd ed.
18. Kreiman G., Maunsell J.H.R. Nine criteria for a measure of scientific output. Frontiers in Computational Neuroscience 2011, 5. 10.3389/fncom.2011.00048.
19. Lewison G., Sullivan R. The impact of cancer research: How publications influence UK cancer clinical guidelines. British Journal of Cancer 2008, 98(12):1944-1950.
20. Loach T. Analysing attention 2014, Retrieved May 26, 2014, from. http://www.altmetric.com/blog/analysing-attention/.
21. Long J.S., Freese J. Regression models for categorical dependent variables using Stata 2006, Stata Press, Stata Corporation, College Station, TX, USA. 2.
22. Mahrt M., Weller K., Peters I. Twitter in scholarly communication. Twitter and society 2012, 399-410. Peter Lang, New York, NY, USA. K. Weller, A. Bruns, J. Burgess, M. Mahrt, C. Puschmann (Eds.).
23. Mohammadi E., Thelwall M. Assessing non-standard article impact using F1000 labels. Scientometrics 2013, 97(2):383-395. 10.1007/s11192-013-0993-9.
24. Mohammadi E., Thelwall M. Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology 2014, n/a-n/a. 10.1002/asi.23071.
25. Narin F., Hamilton K.S., Olivastro D. The increasing linkage between US technology and public science. Research Policy 1997, 26(3):317-330. 10.1016/s0048-7333(97)00013-9.
26. Priem J. Altmetrics. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance 2014, MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
27. Priem J., Costello K.L. How and why scholars cite on Twitter. Proceedings of the American Society for Information Science and Technology 2010, 47(1):1-4. 10.1002/meet.14504701201.
28. Priem J., Hemminger B.M. Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday 2010, 15(7).
29. Priem J., Taraborelli D., Groth P., Neylon C. Altmetrics: A manifesto 2010, Retrieved March 28, from. http://altmetrics.org/manifesto/.
30. Rodgers E.P., Barbrow S. A look at altmetrics and its growing significance to research libraries 2013, The University of Michigan University Library, Ann Arbor, MI, USA.
31. Stata statistical software: Release 13 2013, StataCorp, Stata Corporation, College Station, TX, USA.
32. Torres-Salinas D., Cabezas-Clavijo A., Jimenez-Contreras E. Altmetrics: New indicators for scientific communication in Web 2.0. Comunicar 2013, 41:53-60.
33. Vinkler P. The evaluation of research by scientometric indicators 2010, Chandos Publishing, Oxford, UK.
34. Waltman L., Costas R. F1000 recommendations as a potential new data source for research evaluation: A comparison with citations. Journal of the Association for Information Science and Technology 2014, 65(3):433-445. 10.1002/asi.23040.
35. Wang J. Citation time window choice for research impact evaluation. Scientometrics 2013, 94(3):851-872. 10.1007/s11192-012-0775-9.
36. Wets K., Weedon D., Velterop J. Post-publication filtering and evaluation: Faculty of 1000. Learned Publishing 2003, 16(4):249-258.
37. Williams R. Using the margins command to estimate and interpret adjusted predictions and marginal effects. The Stata Journal 2012, 12(2):308-331.
38. Williams R., Bornmann L. The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals. Measuring scholarly impact: Methods and practice 2014, (in press). Y. Ding, R. Rousseau, D. Wolfram (Eds.).
39. Wouters P., Costas R. Users, narcissism and control - Tracking the impact of scholarly publications in the 21st century 2012, SURF Foundation, Utrecht, The Netherlands.
40. Zubiaga A., Spina D., Martínez R., Fresno V. Real-time classification of twitter trends. Journal of the Association for Information Science and Technology 2014, n/a-n/a. 10.1002/asi.23186.