Conceptual frameworks and empirical approaches used to assess the impact of health research: An overview of reviews

Health Research Policy and Systems

Volumn 9, Issue , 2011, Pages

  Banzi, Rita ^a   Moja, Lorenzo ^a,b   Pistotti, Vanna ^a   Facchini, Andrea ^c,d   Liberati, Alessandro ^a,e,f  

^a MARIO NEGRI INSTITUTE FOR PHARMACOLOGICAL RESEARCH   (Italy)

^b UNIVERSITY OF MILAN   (Italy)

^c RIZZOLI ORTHOPAEDIC INSTITUTE   (Italy)

^d UNIVERSITY OF BOLOGNA   (Italy)

^e UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^f Agenzia Sanitaria e Sociale Regionale Emilia Romagna   (Italy)

Author keywords

Bibliometrics; Health research; Research governance; Research impact

Indexed keywords

ARTICLE; BIBLIOMETRICS; CLINICAL ASSESSMENT; COCHRANE LIBRARY; CONCEPTUAL FRAMEWORK; DECISION MAKING; EMPIRICAL RESEARCH; FUNDING; MEDICAL RESEARCH; MEDLINE; METHODOLOGY; POST HOC ANALYSIS; QUALITATIVE RESEARCH; SOCIOECONOMICS; STATISTICAL MODEL; SYSTEMATIC REVIEW; THEORETICAL STUDY;

EID: 79959458065     PISSN: None     EISSN: 14784505     Source Type: Journal    
DOI: 10.1186/1478-4505-9-26     Document Type: Article

References (55)

1. Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K, et al. Central challenges facing the national clinical research enterprise. JAMA 2003, 289(10):1278-1287. 10.1001/jama.289.10.1278, 12633190.
2. Agenzia sanitaria e sociale regionale dell'Emilia-Romagna Dossier ASSR N 182, La ricerca come attività istituzionale del Servizio sanitario regionale: principi generali e indirizzi operativi. 2009, Agenzia sanitaria e sociale regionale dell'Emilia-Romagna., http://asr.regione.emilia-romagna.it/wcm/asr/collana_dossier/doss182.htm
3. AIFA Research & Development Working Group Feasibility and challenges of independent research on drugs: the Italian medicines agency (AIFA) experience. Eur J Clin Invest 2010, 40(1):69-86. AIFA Research & Development Working Group.
4. Institute of Medicine National Priorities for Comparative Effectiveness Research Report Brief. 2009, Institute of Medicine., http://www.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspx
5. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009, 374(9683):86-89. 10.1016/S0140-6736(09)60329-9, 19525005.
6. UK Clinical Research Collaboration UK Health Research Analysis. 2006, UK Clinical Research Collaboration., http://www.ukcrc.org/publications/reports/
7. Woolf SH. The meaning of translational research and why it matters. JAMA 2008, 299(2):211-213. 10.1001/jama.2007.26, 18182604.
8. Wooding S, Hanney S, Buxton M, Grant J. Payback arising from research funding: evaluation of the Arthritis Research Campaign. Rheumatology (Oxford) 2005, 44(9):1145-1156.
9. Australian Society for Medical Research Access Economics. Exceptional returns: The value of investing in health R&D in Australia II. Canberra (Australia): Australian Society for Medical Research. 2008, Australian Society for Medical Research., http://www.asmr.org.au/ExceptII08.pdf
10. Funding First Exceptional returns: the economic value of America's investment in medical research. 2000, New York (NY): The Lasker Foundation, Funding First.
11. Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy 2003, 8(3):165-170. 10.1258/135581903322029520, 12869343.
12. Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health Technology Assessment Programme. Health Technol Assess 2007, 11(53):iii-iv. ix-xi, 1-180.
13. Williams V, Eiseman E, Landree E, Adamson D. Demonstrating and Communicating Research Impact Preparing NIOSH Programs for External Review. RAND Monograps, wwwrandorg 2009,
14. Higgins J, Altman D, . editors Chapter 22: Overviews of Reviews. Cochrane Handbook for Systematic Reviews of Interventions Wiley 2008, Higgins JPT, Green S, editors.
15. Soper B, Buxton M, Hanney S, Oortwijn W, Scoggins A, Steel N, Ling T. Developing the protocol for the evaluation of the health foundation's 'engaging with quality initiative' - an emergent approach. Implement Sci 2008, 3:46. 10.1186/1748-5908-3-46, 2612693, 18973650.
16. Fleurence RL. Setting priorities for research: a practical application of 'payback' and expected value of information. Health Econ 2007, 16(12):1345-1357. 10.1002/hec.1225, 17328053.
17. Townsend J, Buxton M, Harper G. Prioritisation of health technology assessment. The PATHS model: methods and case studies. Health Technol Assess 2003, 7(20):1-82. iii.
18. Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a critical review. Bull World Health Organ 2004, 82(10):733-739. 2623029, 15643793.
19. Frank C, Nason E. Health research: measuring the social, health and economic benefits. CMAJ 2009, 180(5):528-534. 10.1503/cmaj.090016, 2645444, 19164482.
20. Canadian Academy of Health Sciences Canadian Academy of Health Sciences Making an impact: A preferred framework and indicators to measure returns on investment in health research. 2009, Canadian Academy of Health Sciences., http://www.cahs-acss.ca/e/pdfs/ROI_FullReport.pdf
21. Coryn CLS, Hattie JA, Scriven M, Hartmann DJ. Models and Mechanisms for Evaluating Government-Funded Research: An International Comparison. American Journal of Evaluation 2007, 28(4):437-457.
22. Raftery J, Hanney S, Green C, Buxton M. Assessing the impact of England's National Health Service R&D Health Technology Assessment program using the "payback" approach. Int J Technol Assess Health Care 2009, 25(1):1-5. 10.1017/S0266462309090011, 20030885.
23. Hanney SR, Home PD, Frame I, Grant J, Green P, Buxton MJ. Identifying the impact of diabetes research. Diabet Med 2006, 23(2):176-184. 10.1111/j.1464-5491.2005.01753.x, 16433716.
24. Grant J, Cottrell R, Cluzeau F, Fawcett G. Evaluating "payback" on biomedical research from papers cited in clinical guidelines: applied bibliometric study. BMJ 2000, 320(7242):1107-1111. 10.1136/bmj.320.7242.1107, 27352, 10775218.
25. Linee Guida CIVR Comitato di Indirizzo per la Valutazione della Ricerca, CIVR Linee Guida per la valutazione della Ricerca. 2006, Linee Guida CIVR., http://vtr2006.cineca.it/documenti/linee_guida.pdf
26. Santori G, Valente R, Ghirelli R, Valente U. Evaluation of research products released during a national project funded by the Italian Ministry of Health. Transplant Proc 2008, 40(6):1820-1822. 10.1016/j.transproceed.2008.05.017, 18675060.
27. Kalucy L, Jackson-Bowers E, McIntyre E, Hordacre A, Reed R. Exploring the impact of primary health care reserach. Final Report Stage 2. 2009, http://www.phcris.org.au/activities/rip/
28. Kalucy L, Jackson-Bowers E, McIntyre E. Primary Health Care Research Impact Project. Final report stage 1. 2007, http://wwwphcrisorgau/activities/rip/indexphp
29. Stryer D, Tunis S, Hubbard H, Clancy C. The outcomes of outcomes and effectiveness research: impacts and lessons from the first decade. Health Serv Res 2000, 35(5 Pt 1):977-993. 1089179, 11130807.
30. Shah S, Ward JE. Outcomes from NHMRC public health research project grants awarded in 1993. Aust N Z J Public Health 2001, 25(6):556-560. 10.1111/j.1467-842X.2001.tb00325.x, 11824995.
31. Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS. Effect of a US National Institutes of Health programme of clinical trials on public health and costs. Lancet 2006, 367(9519):1319-1327. 10.1016/S0140-6736(06)68578-4, 16631910.
32. Kwan P, Johnston J, Fung AY, Chong DS, Collins RA, Lo SV. A systematic evaluation of payback of publicly funded health and health services research in Hong Kong. BMC Health Serv Res 2007, 7:121. 10.1186/1472-6963-7-121, 1952059, 17662157.
33. Ottoson JM, Green LW, Beery WL, Senter SK, Cahill CL, Pearson DC, Greenwald HP, Hamre R, Leviton L. Policy-contribution assessment and field-building analysis of the Robert Wood Johnson Foundation's Active Living Research Program. Am J Prev Med 2009, 36(2 Suppl):S34-43.
34. Gutman MA, Barker DC, Samples-Smart F, Morley C. Evaluation of Active Living Research progress and lessons in building a new field. Am J Prev Med 2009, 36(2 Suppl):S22-33.
35. Ruch-Ross H, Keller D, Miller N, Bassewitz J, Melinkovich P. Evaluation of community-based health projects: the healthy tomorrows experience. Pediatrics 2008, 122(3):e564-572. 10.1542/peds.2006-3273, 18762490.
36. Cutler DM, Kadiyala S. The economics of better health: the case of cardiovascular disease. 1999, New York (NY): The Lasker Foundation.
37. Shiel A, Di Ruggiero E. Assessing the return on Canada's public investment in population and public health research: methods and metrics. Return on investments in health research 2009, Ottawa (ON): Canadian Academy of Health Sciences., http://www.cahs-acss.ca/e/assessments/completedprojects.htm
38. Buxton M, Hanney S. How can payback from health services research be assessed?. J Health Serv Res Policy 1996, 1(1):35-43.
39. Buxton M, Hanney S. Evaluating the NHS research and development programme: will the programme give value for money?. J R Soc Med 1998, (91 Suppl 35):2-6.
40. Buxton M, Hanney S, Packwood T, Robert S. Assessing the benefits from North Thames R&D. HERG Report 25 1999, Uxbridge: Brunel University.
41. Hanney SR, Grant J, Wooding S, Buxton MJ. Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign'. Health Res Policy Syst 2004, 2(1):4. 10.1186/1478-4505-2-4, 503400, 15272939.
42. Development Assistance Committee Working Party on Aid Evaluation Glossary of Key Terms in Evaluation and Results Based Management. 2002, Paris, France: OECD, Development Assistance Committee Working Party on Aid Evaluation.
43. Research Excellence Framework (REF). , http://www.hefce.ac.uk/research/ref/
44. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst 2003, 1(1):2. 10.1186/1478-4505-1-2, 151555, 12646071.
45. Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact Framework. BMC Health Serv Res 2006, 6:134. 10.1186/1472-6963-6-134, 1635046, 17049092.
46. Landry R, Amara N, Lamari M. Climbing the Ladder of Research Utilization: Evidence from Social Science Research. Science Communication 2001, 396:442.
47. Weiss AP. Measuring the impact of medical research: moving from outputs to outcomes. Am J Psychiatry 2007, 164(2):206-214. 10.1176/appi.ajp.164.2.206, 17267781.
48. Jacob R, McGregor M. Assessing the impact of health technology assessment. Int J Technol Assess Health Care 1997, 13(1):68-80. 10.1017/S0266462300010242, 9119625.
49. Royal Netherlands Academy of Arts and Sciences The social impact of applied research towards a quality assessment system. 2002, Amsterdam, Royal Netherlands Academy of Arts and Sciences.
50. Van Ark G. Societal impact evaluation of research groups: The communication metaphor. Swedish Research Council Workshop on Economic Returns of Medical Research 2007, http://www.vr.se/download/18.34261071168fe6a62080001004/ZonMw+van+Ark+2.pdf
51. Kaplan R, Norton D. The balanced Scorecard-Measures that drive performance. Harvard Business Review 1992, 71-79. January-Febraury.
52. University Health Network Accountability to Our PatientsHow we measure up - the Balanced Scorecard. 2008, University Health Network., http://www.uhn.ca/Patients_&_Visitors/accountability/index.asp
53. Tomlinson S. The research assessment exercise and medical research. BMJ 2000, 320(7235):636-639. 10.1136/bmj.320.7235.636, 1117659, 10698889.
54. Mushkin S. Biomedical Research: costs and benefits. 1979, Cambridge, MA Ballinger.
55. NIH Cost savings resulting from NIH research support. 1993, Bethesda, MD: United States Department of Health and Human Services National Institute of Health, 2, NIH.