A large National Institute for Health Research (NIHR) Biomedical Research Centre facilitates impactful cross-disciplinary and collaborative translational research publications and research collaboration networks: a bibliometric evaluation study

Journal of Translational Medicine

Volumn 19, Issue 1, 2021, Pages

  Kiparoglou, Vasiliki ^a,b   Brown, Laurence A ^b,c   McShane, Helen ^a,d   Channon, Keith M ^a,b   Shah, Syed Ghulam Sarwar ^a,b  

^a JOHN RADCLIFFE HOSPITAL   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c UNIVERSITY OF OXFORD   (United Kingdom)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Author networks; Collaborative research; Research Institutions; Research outputs; Research productivity; Translational Research Organisations

Indexed keywords

ARTICLE; BIBLIOMETRICS; FUNDING; MAJOR CLINICAL STUDY; ORGANIZATION; PATENT; PRODUCTIVITY; TRANSLATIONAL RESEARCH; UNITED KINGDOM; WRITING; MEDICAL RESEARCH; PUBLICATION;

AUTHORSHIP; BIBLIOMETRICS; BIOMEDICAL RESEARCH; PUBLICATIONS; TRANSLATIONAL RESEARCH, BIOMEDICAL; UNITED KINGDOM;

EID: 85120036353     PISSN: None     EISSN: 14795876     Source Type: Journal    
DOI: 10.1186/s12967-021-03149-x     Document Type: Article

References (77)

1. Miyashita S, Katoh S, Anzai T, Sengoku S. Intellectual property management in publicly funded R&D program and projects: optimizing principal-agent relationship through transdisciplinary approach. Sustainability. 2020;12(23):9923. DOI: 10.3390/su12239923
2. D’Ippolito B, Miozzo M, Consoli D. Knowledge systematisation, reconfiguration and the organisation of firms and industry: the case of design. Res Policy. 2014;43(8):1334–52. DOI: 10.1016/j.respol.2014.03.013
3. Bodas Freitas IM, Verspagen B. The motivations, organisation and outcomes of university-industry interaction in the Netherlands. Maastricht Economic and Social Research and Training Centre on Innovation and Technology. Maastricht: United Nations University; 2009.
4. Goddard JG, Isabelle M. How do public laboratories collaborate with industry? New survey evidence from France. SSRN. Working Paper No. IMRI 2006/02. Paris, France: Paris Dauphine University; 2006.
5. Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1(1):35–43. 10.1177/135581969600100107. DOI: 10.1177/135581969600100107
6. Hanney S, Greenhalgh T, Blatch-Jones A, Glover M, Raftery J. The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews. Health Res Policy Syst. 2017;15(1):26–26. DOI: 10.1186/s12961-017-0191-y
7. D’Ippolito B, Rüling C-C. Research collaboration in Large Scale Research Infrastructures: collaboration types and policy implications. Res Policy. 2019;48(5):1282–96. DOI: 10.1016/j.respol.2019.01.011
8. Munoz DA, Nembhard HB, Kraschnewski JL. Quantifying complexity in translational research: an integrated approach. Int J Health Care Qual Assur. 2014;27(8):760–76. 10.1108/IJHCQA-01-2014-0002. DOI: 10.1108/IJHCQA-01-2014-0002
9. Qidwai W. Translational research and complexity of clinical practice: issues, challenges, and way forward. J Coll Physicians Surg Pak. 2016;26(6):453–4.
10. Long JC, Cunningham FC, Carswell P, Braithwaite J. Patterns of collaboration in complex networks: the example of a translational research network. BMC Health Serv Res. 2014;14(1):225. 10.1186/1472-6963-14-225. DOI: 10.1186/1472-6963-14-225
11. Trochim W, Kane C, Graham MJ, Pincus HA. Evaluating translational research: a process marker model. Clin Transl Sci. 2011;4(3):153–62. 10.1111/j.1752-8062.2011.00291.x. DOI: 10.1111/j.1752-8062.2011.00291.x
12. Sung NS, Crowley J, William F, Genel M, Salber P, Sandy L, Sherwood LM, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289(10):1278–87. 10.1001/jama.289.10.1278. DOI: 10.1001/jama.289.10.1278
13. Kim YH, Levine AD, Nehl EJ, Walsh JP. A bibliometric measure of translational science. Scientometrics. 2020;125(3):2349–82. 10.1007/s11192-020-03668-2. DOI: 10.1007/s11192-020-03668-2
14. van Steen J, Eijffinger M. Evaluation practices of scientific research in the Netherlands. Res Eval. 1998;7(2):113–22. 10.1093/rev/7.2.113. DOI: 10.1093/rev/7.2.113
15. Amdur RJ, Biddle C. Institutional review board approval and publication of human research results. JAMA. 1997;277(11):909–14. DOI: 10.1001/jama.1997.03540350059034
16. Geuna A, Martin BR. University research evaluation and funding: an international comparison. Minerva. 2003;41(4):277–304. 10.1023/B:MINE.0000005155.70870.bd. DOI: 10.1023/B:MINE.0000005155.70870.bd
17. Fire M, Guestrin C. Over-optimization of academic publishing metrics: observing Goodhart’s Law in action. GigaScience. 2019; 8(6): giz053. 10.1093/gigascience/giz053. DOI: 10.1093/gigascience/giz053
18. Frank C, Battista R, Butler L, Buxton M, Chappell N, Davies SC, et al. Making an impact, a preferred framework and indicators to measure returns on investment in health research. Ottawa: Canadian Academy of Health Sciences (CAHS); 2009.
19. Beaudet A. CAHS forum summary: the return on investments in health research: defining the best metrics. Montreal: Canadian Academy of Health Sciences; 2007.
20. Research Excellence Framework. About the REF. REF2014 research excellence framework. 2014. https://www.ref.ac.uk/2014/. Accessed 25 Nov 2020.
21. Parliament UK. Research assessment exercise: a re-assessment. Eleventh report of session 2003–04. Report No.: HC586. London: House of Commons; 2004.
22. Stern N. Building on success and learning from experience. An independent review of the research excellence framework. London: Department for Business, Energy & Industrial Strategy; 2016.
23. Research Excellence Framework. About the REF. REF 2021 research excellence framework. 2020. https://www.ref.ac.uk/about/. Accessed 25 Nov 2020.
24. Research Excellence Framework. Guidance on providing testimonies for REF 2021 impact case studies. 2020. https://www.ref.ac.uk/guidance/additional-guidance/. Accessed 25 Nov 2020.
25. National Institute for Health Research (NIHR). Our mission and core workstreams. Department of Health and Social Care, United Kingdom. 2020. https://www.nihr.ac.uk/about-us/our-mission/our-mission-and-strategic-workstreams.htm. Accessed 24 Feb 2021.
26. NIHR Oxford Biomedical Research Centre. About the NIHR Oxford Biomedical Research Centre. 2020. https://oxfordbrc.nihr.ac.uk/about-us-intro/. Accessed 4 Sep 2020.
27. Greenhalgh T, Ovseiko PV, Fahy N, Shaw S, Kerr P, Rushforth AD, et al. Maximising value from a United Kingdom Biomedical Research Centre: study protocol. Health Res Policy Syst. 2017;15(1):70. 10.1186/s12961-017-0237-1. DOI: 10.1186/s12961-017-0237-1
28. Hicks D, Wouters P, Waltman L, De Rijcke S, Rafols I. Bibliometrics: the Leiden Manifesto for research metrics. Nature. 2015;520(7548):429–31. DOI: 10.1038/520429a
29. Franceschini F, Maisano D. Structured evaluation of the scientific output of academic research groups by recent h-based indicators. J Informetr. 2011;5(1):64–74. DOI: 10.1016/j.joi.2010.08.003
30. Yao Q, Lyu P-H, Ma F-C, Yao L, Zhang S-J. Global informetric perspective studies on translational medical research. BMC Med Inform Decis Mak. 2013;13(1):77. 10.1186/1472-6947-13-77. DOI: 10.1186/1472-6947-13-77
31. Koya K, Chowdhury G. Metric-based vs peer-reviewed evaluation of a research output: lesson learnt from UK’s national research assessment exercise. PLoS ONE. 2017;12(7):e0179722–e0179722. DOI: 10.1371/journal.pone.0179722
32. Chen K, Zhang Y, Zhu G, Mu R. Do research institutes benefit from their network positions in research collaboration networks with industries or/and universities? Technovation. 2020;94–95:102002. DOI: 10.1016/j.technovation.2017.10.005
33. Marchand GC, Hilpert JC, Bragg KM, Cummings J. Network-based assessment of collaborative research in neuroscience. Alzheimer’s Dement Transl Res Clin Interv. 2018;4:433–43. DOI: 10.1016/j.trci.2018.08.006
34. Ramsden P. Describing and explaining research productivity. High Educ. 1994;28(2):207–26. 10.1007/BF01383729. DOI: 10.1007/BF01383729
35. Schneider M, Kane CM, Rainwater J, Guerrero L, Tong G, Desai SR, et al. Feasibility of common bibliometrics in evaluating translational science. J Clin Transl Sci. 2017;1(1):45–52. DOI: 10.1017/cts.2016.8
36. National Institute for Health Research (NIHR). NIHR research outputs and publications guidance. 2020. https://www.nihr.ac.uk/documents/nihr-research-outputs-and-publications-guidance/12250. Accessed 16 Dec 2020.
37. Todeschini R, Baccini A. Handbook of bibliometric indicators: quantitative tools for studying and evaluating research. Weinheim: Wiley; 2016. DOI: 10.1002/9783527681969
38. Huang J, Gates AJ, Sinatra R, Barabási A-L. Historical comparison of gender inequality in scientific careers across countries and disciplines. Proc Natl Acad Sci USA. 2020;117(9):4609. DOI: 10.1073/pnas.1914221117
39. van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–38. 10.1007/s11192-009-0146-3. DOI: 10.1007/s11192-009-0146-3
40. Bastian M, Heymann S, Jacomy M. Gephi: an open source software for exploring and manipulating networks. In: Proceedings of the Third International ICWSM Conference. San Jose: The AAAI Press; 2009. p. 361–2.
41. Kluyver T, Ragan-Kelley B, Pérez F, Granger BE, Bussonnier M, Frederic J, et al. Jupyter Notebooks—a publishing format for reproducible computational workflows. In: Loizides F, Schmidt B, editors., et al., Ebook: positioning and power in academic publishing: players, agents and agendas. Amsterdam: IOS Press; 2016. p. 87–90. 10.3233/978-1-61499-649-1-87. DOI: 10.3233/978-1-61499-649-1-87
42. Perez F, Granger BE. IPython: a system for interactive scientific computing. Comput Sci Eng. 2007;9(3):21–9. DOI: 10.1109/MCSE.2007.53
43. Reback J, McKinney W, den Bossche J, Jbrockmendel A, Augspurger T, Cloud P, et al. Pandas v0.25.3. 10.5281/zenodo.3524604. Zenodo. 2019. https://github.com/pandas-dev/pandas/tree/v0.25.3. Accessed 24 Dec 2020.
44. McKinney W. Python for data analysis. Sebastopol: O’Reilly; 2013.
45. van der Walt S, Cilbert C, Varoquaux G. The NumPy array: a structure for efficient numerical computation. Comput Sci Eng. 2011;13:22–30. DOI: 10.1109/MCSE.2011.37
46. Hagberg AA, Schult DA, Swart PJ. Exploring network structure, dynamics, and function using NetworkX. In: Varoquaux G, Vaught T, Millman J, editors. Proceedings of the 7th Python in Science conference (SciPy 2008). Pasadena: SciPy.org; 2008. p. 11–5.
47. Bokeh Development Team. The Bokeh visualization library. 2018. https://bokeh.org/. Accessed 12 Mar 2021.
48. Stevens JLR, Rudiger P, Bedner JA. HoloViews: building complex visualizations easily for reproducible science. In: Huff K, Bergstra J, editors. Proceedings of the 14th Python in Science Conference (SciPy 2015). Austin: SciPy.org; 2015. p. 59–66.
49. Kiparoglou V, Brown LA, McShane H, Channon KM, Shah SGS. Index of notebooks for OxBRC2 bibliometric analysis. San Francisco: GitHub Inc.; 2021. https://github.com/LozRiviera/BRC_Oxford2_bibliometrics/tree/published. Accessed 4 Oct 2021.
50. Brown LA, Kiparoglou V, McShane H, Channon KM, Shah SGS. Data and code for: a large National Institute for Health (NIHR) Biomedical Research Centre Facilitates Cross-Disciplinary and Collaborative Research Outputs: a bibliometric analysis. 2021. Zenodo. https://doi.org/10.5281/zenodo.5211298.
51. Klionsky DJ, Abdelmohsen K, Abe A, Abedin MJ, Abeliovich H, Arozena AA, et al. Guidelines for the use and interpretation of assays for monitoring autophagy (3rd edition). Autophagy. 2016;12(1):1–222. 10.1080/15548627.2015.1100356. DOI: 10.1080/15548627.2015.1100356
52. Regalado A. Multiauthor papers on the rise. Science. 1995;268(5207):25. DOI: 10.1126/science.7701334
53. The IST-3 Collaborative Group. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012;379(9834):2352–63. 10.1016/S0140-6736(12)60768-5. DOI: 10.1016/S0140-6736(12)60768-5
54. Thelwall M, Fairclough R. Geometric journal impact factors correcting for individual highly cited articles. J Informetr. 2015;9(2):263–72. DOI: 10.1016/j.joi.2015.02.004
55. Rosas SR, Kagan JM, Schouten JT, Slack PA, Trochim WMK. Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks. PLoS ONE. 2011;6(3):e17428. 10.1371/journal.pone.0017428. DOI: 10.1371/journal.pone.0017428
56. Yu F, Van AA, Patel T, Mani N, Carnegie A, Corbie-Smith GM, et al. Bibliometrics approach to evaluating the research impact of CTSAs: a pilot study. J Clin Transl Sci. 2020;4(4):336–44. DOI: 10.1017/cts.2020.29
57. Fox MF. Publication productivity among scientists: a critical review. Soc Stud Sci. 1983;13(2):285–305. 10.1177/030631283013002005. DOI: 10.1177/030631283013002005
58. Abramo G, D’Angelo CA, Di Costa F. Research collaboration and productivity: is there correlation? High Educ. 2009;57(2):155–71. 10.1007/s10734-008-9139-z. DOI: 10.1007/s10734-008-9139-z
59. VanderZanden A, Langlois EV, Ghaffar A, Bitton A, Fifield J, Hirschhorn LR. It takes a community: a landscape analysis of global health research consortia. BMJ Glob Health. 2019;4(Suppl 8):e001450. DOI: 10.1136/bmjgh-2019-001450
60. Stallings J, Vance E, Yang J, Vannier MW, Liang J, Pang L, et al. Determining scientific impact using a collaboration index. Proc Natl Acad Sci USA. 2013;110(24):9680. DOI: 10.1073/pnas.1220184110
61. Sahel J-A. Quality versus quantity: assessing individual research performance. Sci Transl Med. 2011;3(84):84cm13. DOI: 10.1126/scitranslmed.3002249
62. Siegel D, Baveye P, Sills J. Battling the paper glut. Science. 2010;329(5998):1466. DOI: 10.1126/science.329.5998.1466-a
63. Uddin S, Hossain L, Rasmussen K. Network effects on scientific collaborations. PLoS ONE. 2013;8(2):e57546–e57546. DOI: 10.1371/journal.pone.0057546
64. van Leeuwen T, Hoorens S, Grant J. Using bibliometrics to support the procurement of NIHR biomedical research centres in England. Res Eval. 2009;18(1):71–82. DOI: 10.3152/095820209X414178
65. Cole JR, Cole S. Social stratification in science. Chicago: The University of Chicago Press; 1973.
66. Shah SGS, Dam R, Milano MJ, Edmunds LD, Henderson LR, Hartley CR, et al. Gender parity in scientific authorship in a National Institute for Health Research Biomedical Research Centre: a bibliometric analysis. BMJ Open. 2021;11(3):e037935. DOI: 10.1136/bmjopen-2020-037935
67. Crossref. You are crossref. 2020. https://www.crossref.org/. Accessed 24 Dec 2020.
68. European Science Foundation. Why plan S. 2018. https://www.coalition-s.org/why-plan-s/. Accessed 24 Dec 2020.
69. Van Raan A. The influence of international collaboration on the impact of research results. Scientometrics. 2006;42(3):423–8. DOI: 10.1007/BF02458380
70. Dimensions Resources. Dimensions metrics API documentation. Dimensions. 2018. J Contrib. https://doi.org/10.6084/m9.figshare.5783694.v4.
71. The hidden REF. Submissions to REF2014. 2020. https://hidden-ref.org/submissions-to-ref2014/. Accessed 24 Dec 2020.
72. Sprague ER. ORCID. J Med Libr Assoc. 2017;105(2):207–8.
73. Digital Science & Research Solutions Ltd. GRID—global research identifier database. 2020. https://grid.ac/. Accessed 24 Dec 2020.
74. ROR (Research Organization Registry). Research Organization Registry Community. 2020. https://ror.org/. Accessed 24 Dec 2020.
75. Fenner M. Powering the PID Graph: announcing the DataCite GraphQL API. DataCite Blog. 2020. https://blog.datacite.org/powering-the-pid-graph/. Accessed 24 Dec 2020.
76. Hook DW, Porter SJ, Herzog C. Dimensions: building context for search and evaluation. Front Res Metr Anal. 2018. 10.3389/frma.2018.00023. DOI: 10.3389/frma.2018.00023
77. Python (3.7.6), Pandas (0.25.3), Networkx (2.4), Holoviews (1.12.7), Hvplot (0.5.2), Fuzzywuzzy (0.17.0), Habanero (0.7.2), Panel (0.7.0), Bokeh (1.4.0), ipython (7.11.1), Jupyter (1.0.0), notebook (6.0.2), numpy (1.17.5), scipy (1.3.2), selenium (3.141.0).