Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics

Journal of Informetrics

Volumn 8, Issue 4, 2014, Pages 895-903

  Bornmann, Lutz ^a  

^a MAX PLANCK INSTITUTE   (Germany)

Author keywords

Altmetrics; Broader impact; Scientometrics; Societal impact

Indexed keywords

SOCIAL NETWORKING (ONLINE);

ALTMETRICS; BIBLIOMETRICS; BROADER IMPACTS; PEER REVIEW; RESEARCH OUTPUTS; SCIENTOMETRICS; SOCIAL MEDIA PLATFORMS; SOCIETAL IMPACTS;

ECONOMIC AND SOCIAL EFFECTS;

EID: 84908672359     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2014.09.005     Document Type: Review

References (78)

1. Adie E. The grey literature from an altmetrics perspective - opportunity and challenges. Research Trends 2014, (37):23-25.
2. Adie E., Roe W. Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing 2013, 26(1):11-17. doi:10.1087/20130103.
3. American Psychological Association Publication manual of the American Psychological Association 2010, American Psychological Association (APA), Washington, DC, USA. 6th ed.
4. Bar-Ilan J., Shema H., Thelwall M. Bibliographic references in: Web 2.0. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance 2014, 307-325. MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
5. Bastow S., Dunleavy P., Tinkler J. The impact of the social sciences 2014, Sage, London, UK.
6. Bik H.M., Goldstein M.C. An Introduction to social media for scientists. PLoS Biology 2013, 11(4):e1001535. doi:10.1371/journal.pbio.1001535.
7. Bornmann L. Scientific peer review. Annual Review of Information Science and Technology 2011, 45:199-245.
8. Bornmann L. Measuring the societal impact of research. EMBO Reports 2012, 13(8):673-676.
9. Bornmann L. What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society of Information Science and Technology 2013, 64(2):217-233.
10. Bornmann L. Is there currently a scientific revolution in scientometrics?. Journal of the Association for Information Science and Technology 2014, 65(3):647-648.
11. Bornmann L., Daniel H.-D. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 2008, 64(1):45-80. doi:10.1108/00220410810844150.
12. Bornmann L., Leydesdorff L., Mutz R. The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits. Journal of Informetrics 2013, 7(1):158-165.
13. Bornmann L., Williams R. How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects. Journal of Informetrics 2013, 7(2):562-574. doi:10.1016/j.joi.2013.02.005.
14. Chamberlain S. Consuming article-level metrics: Observations and lessons. Information Standards Quarterly 2013, 25(2):4-13.
15. Claussen J.E., Cooney P.B., Defilippi J.M., Fox S.G., Glaser S.M., Hawkes E., et al. Science communication in a digital age: Social media and the American fisheries society. Fisheries 2013, 38(8):359-362. doi:10.1080/03632415.2013.816289.
16. Cronin B. The evolving indicator space (iSpace). Journal of the American Society for Information Science and Technology 2013, 64(8):1523-1525. doi:10.1002/asi.23041.
17. Darling E.S., Shiffman D., CÔté I.M., Drew J.A. The role of Twitter in the life cycle of a scientific publication. PeerJ PrePrints 2013, 1:e16v11. doi:10.7287/peerj.preprints.16v1.
18. Delgado López-Cózar E., Robinson-García N., Torres-Salinas D. The Google scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology 2014, 65(3):446-454. doi:10.1002/asi.23056.
19. Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res 2011, 13(4):e123.
20. Fausto S., Machado F.A., Bento L.F.J., Iamarino A., Nahas T.R., Munger D.S. Research blogging: Indexing and registering the change in science 2.0. PLoS ONE 2012, 7(12):e50109. doi:10.1371/journal.pone.0050109.
21. Fenner M. Letter from the guest content editor: Altmetrics have come of age. Information Standards Quarterly 2013, 25(2):3.
22. Fenner M. What can article-level metrics do for you?. PLoS Biol 2013, 11(10):e1001687. doi:10.1371/journal.pbio.1001687.
23. Frank C., Nason E. Health research: Measuring the social, health and economic benefits. Canadian Medical Association Journal 2009, 180(5):528-534. doi:10.1503/Cmaj.090016.
24. Galloway L.M., Pease J.L., Rauh A.E. Introduction to altmetrics for science, technology, engineering, and mathematics (STEM) Librarians. Science & Technology Libraries 2013, 32(4):335-345. doi:10.1080/0194262X. 2013.829762.
25. Gibbons M., Limoges C., Nowotny H., Schwartzman S., Scott P., Trow M. The new production of knowledge: The dynamics of science and research in contemporary societies 1994, SAGE, London, UK.
26. Greenhow C., Gleason B. Social scholarship: Reconsidering scholarly practices in the age of social media. British Journal of Educational Technology 2014, 45(3):392-402. doi:10.1111/bjet.12150.
27. Gunn W. Social Signals reflect academic impact: What it means when a scholar adds a paper to Mendeley. Information Standards Quarterly 2013, 25(2):33-39.
28. Hammarfelt B. Using altmetrics for assessing research impact in the humanities. Scientometrics 2014, 1-12. doi:10.1007/s11192-014-1261-3.
29. Haustein S. Readership metrics. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance 2014, 327-344. MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
30. Haustein S., Peters I. Using social bookmarks and tags as alternative indicators of journal content description. First Monday 2012, 17(11).
31. Haustein S., Peters I., Bar-Ilan J., Priem J., Shema H., Terliesner J. Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics 2014, 1-19. doi:10.1007/s11192-013-1221-3.
32. Haustein S., Peters I., Sugimoto C.R., Thelwall M., Larivière V. Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology 2014, 65(4):656-669. doi:10.1002/asi.23101.
33. Holmberg K., Thelwall M. Disciplinary differences in Twitter scholarly communication. Scientometrics 2014, 1-16. doi:10.1007/s11192-014-1229-3.
34. King D., Ramirez-Cano D., Greaves F., Vlaev I., Beales S., Darzi A. Twitter and the health reforms in the English National Health Service. Health Policy 2013, 110(2-3):291-297. doi:10.1016/j.healthpol.2013.02.005.
35. Konkiel S., Scherer D. New opportunities for repositories in the age of altmetrics. Bulletin of the American Society for Information Science and Technology 2013, 39(4):22-26. doi:10.1002/bult.2013.1720390408.
36. Kousha K., Thelwall M. Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology 2007, 58(7):1055-1065.
37. Kurtz M.J., Bollen J. Usage Bibliometrics. Annual Review of Information Science and Technology 2010, 44:3-64.
38. Li X., Thelwall M., Giustini D. Validating online reference managers for scholarly impact measurement. Scientometrics 2012, 91(2):461-471. doi:10.1007/s11192-011-0580-x.
39. Lin J., Fenner M. Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly 2013, 25(2):20-26. doi:10.3789/isqv25no2.2013.04.
40. Liu C.L., Xu Y.Q., Wu H., Chen S.S., Guo J.J. Correlation and interaction visualization of altmetric indicators extracted from scholarly social network activities: Dimensions and structure. Journal of Medical Internet Research 2013, 15(11):17. doi:10.2196/jmir.2707.
41. Liu J., Adie E. Five challenges in altmetrics: A toolmaker's perspective. Bulletin of the American Society for Information Science and Technology 2013, 39(4):31-34. doi:10.1002/bult.2013.1720390410.
42. Maflahi, N., & Thelwall, M. (in press). When are readers as good as citers for bibliometrics? Scopus vs. Mendeley for LIS journals. Journal of the Association for Information Science and Technology.
43. Mohammadi E., Thelwall M. Assessing non-standard article impact using F1000 labels. Scientometrics 2013, 97(2):383-395. doi:10.1007/s11192-013-0993-9.
44. Mohammadi E., Thelwall M. Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology 2014, 65(8):1627-1638. 10.1002/asi.23071.
45. Mohammadi E., Thelwall M., Haustein S., Larivière V. Who reads research articles? An altmetrics analysis of mendeley user categories1 2014, Retrieved http://www.academia.edu/6298635/Who_Reads_Research_Articles_An_Altmetrics_Analysis_of_Mendeley_User_Categories.
46. National Information Standards Organization Altmetrics standards project white paper 2014, National Information Standards Organization. (NISO), Baltimore, MD, USA.
47. Neylon C., Willmers M., King T. Rethinking impact: Applying altmetrics to southern African research 2014, International Development Research Centre, Ottawa, Canada.
48. Neylon C., Wu S. Article-level metrics and the evolution of scientific impact. Plos Biology 2009, 7(11). doi:e1000242.10.1371/journal.pbio.1000242.
49. Peters I., Beutelspacher L., Maghferat P., Terliesner J. Scientific bloggers under the altmetric microscope. Proceedings of the American Society for Information Science and Technology 2012, 49(1):1-4. doi:10.1002/meet.14504901305.
50. Piwowar H. Altmetrics: Value all research products. Nature 2013, 493(7431):159.
51. Piwowar H., Priem J. The power of altmetrics on a CV. Bulletin of the American Society for Information Science and Technology 2013, 39(4):10-13. doi:10.1002/bult.2013.1720390405.
52. Priem J. Altmetrics. Beyond bibliometrics: Harnessing multi-dimensional indicators of performance. 2014, MIT Press, Cambridge, MA, USA. B. Cronin, C.R. Sugimoto (Eds.).
53. Priem J., Hemminger B.M. Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday 2010, 15.(7).
54. Priem J., Parra C., Piwowar H., Waagmeester A. Uncovering impacts: CitedIn and total-impact, two new tools for gathering altmetrics. Paper presented at the iConference 2012 2011.
55. Priem J., Piwowar H., Hemminger B. Altmetrics in the wild: Using social media to explore scholarly impact 2012, Retrieved http://arxiv.org/html/1203.4745.
56. Priem J., Taraborelli D., Groth P., Neylon C. Altmetrics: A manifesto 2010, Retrieved http://altmetrics.org/manifesto/.
57. Puschmann C. (Micro)blogging science? Notes on potentials and constraints of new forms of scholarly communication. Opening science 2014, 89-106. Springer International Publishing, Cham, Switzerland. S. Bartling, S. Friesike (Eds.).
58. Rodgers E.P., Barbrow S. A look at altmetrics and its growing significance to research libraries 2013, The University of Michigan University Library, Ann Arbor, MI, USA.
59. Roemer R.C., Borchardt R. From bibliometrics to altmetrics: A changing scholarly landscape. College & Research Libraries News 2012, 73(10):596-600.
60. Roemer R.C., Borchardt R. Institutional altmetrics & academic libraries. Information Standards Quarterly 2013, 25(2):14-19.
61. Rousseau R., Ye F.Y. A multi-metric approach for research evaluation. Chinese Science Bulletin 2013, 58. doi:10.1007/s11434-013-5939-3.
62. Shema H., Bar-Ilan J., Thelwall M. Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology 2014, 65(5):1018-1027. doi:10.1002/asi.23037.
63. Shema H., Bar-Ilan J., Thelwall M. How is research blogged? A content analysis approach. Journal of the Association for Information Science and Technology 2014, (in press).
64. Sud P., Thelwall M. Evaluating altmetrics. Scientometrics 2014, 98(2):1131-1143. doi:10.1007/s11192-013-1117-2.
65. Taylor M. Exploring the boundaries: How altmetrics can expand our vision of scholarly communication and social impact. Information Standards Quarterly 2013, 25(2):27-32.
66. Taylor M. Towards a common model of citation: Some thoughts on merging altmetrics and bibliometrics. Research Trends 2013, (35):19-22.
67. Thelwall M. A brief history of altmetrics. Research Trends 2014, (37):3-4.
68. Thelwall M., Haustein S., Lariviere V., Sugimoto C.R. Do altmetrics work? Twitter and ten other social web services. PLoS ONE 2013, 8(5). doi:10.1371/journal.pone.0064841.
69. Torres-Salinas D., Cabezas-Clavijo A., Jimenez-Contreras E. Altmetrics: New indicators for scientific communication in Web 2.0. Comunicar 2013, 41:53-60.
70. Vaughan L., Shaw D. Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology 2005, 56(10):1075-1087. doi:10.1002/Asi.20199.
71. Vaughan L., Shaw D. A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics 2008, 74(2):317-330. doi:101007/s11192-008-0220-2.
72. Vinkler P. The evaluation of research by scientometric indicators 2010, Chandos Publishing, Oxford, UK.
73. Waltman L., Costas R. F1000 recommendations as a potential new data source for research evaluation: A Comparison with citations. Journal of the Association for Information Science and Technology 2014, 65(3):433-445. doi:10.1002/asi.23040.
74. Wang J. Citation time window choice for research impact evaluation. Scientometrics 2013, 94(3):851-872. doi:10.1007/s11192-012-0775-9.
75. Weller K., Dröge E., Puschmann C. Citation analysis in Twitter: Approaches for defining and measuring information flows within Tweets during scientific conferences. Making Sense of Microposts (MSM2011), CEUR workshop proceedings 2011, 1-12. M. Rowe, M. Stankovic, A.-S. Dadzie, M. Hardey (Eds.).
76. Weller K., Peters I. Citations in Web 2. 0. Science and the Internet 2012, 209-222. Düsseldorf University Press, Düsseldorf. Germany. A. Tokar, M. Beurskens, S. Keuneke, M. Mahrt, I. Peters, C. Puschmann, T. van Treeck, K. Weller (Eds.).
77. Wouters P., Costas R. Users, narcissism and control - tracking the impact of scholarly publications in the 21st century 2012, SURFfoundation, Utrecht, The Netherlands.
78. Zahedi Z., Costas R., Wouters P. How well developed are altmetrics? A cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications. Scientometrics 2014, 1-23. doi:10.1007/s11192-014-1264-0.