Developing a methodology to assess the impact of research grant funding: A mixed methods approach

Evaluation and Program Planning

Volumn 43, Issue , 2014, Pages 105-117

  Bloch, Carter ^a   Sørensen, Mads P ^a   Graversen, Ebbe K ^a   Schneider, Jesper W ^a   Schmidt, Evanthia Kalpazidou ^a   Aagaard, Kaare ^a   Mejlgaard, Niels ^a  

^a AARHUS UNIVERSITY   (Denmark)

Author keywords

Additionality; Grants; Impact analysis; Mixed methods; Research policy

Indexed keywords

COMPETITIVENESS; PERFORMANCE ASSESSMENT; PLANNING METHOD; POLICY APPROACH; RESEARCH PROGRAM;

ADDITIONALITY; ARTICLE; BIBLIOMETRICS; ECONOMICS; FINANCIAL MANAGEMENT; GRANTS; HUMAN; IMPACT ANALYSIS; INTERVIEW; METHODOLOGY; MIXED METHODS; POLICY; POLITICS; PROGRAM EVALUATION; QUALITATIVE RESEARCH; RESEARCH POLICY; STANDARD; STATISTICS;

ADDITIONALITY; GRANTS; IMPACT ANALYSIS; MIXED METHODS; RESEARCH POLICY;

BIBLIOMETRICS; HUMANS; INTERVIEWS AS TOPIC; POLITICS; PROGRAM EVALUATION; PUBLIC POLICY; QUALITATIVE RESEARCH; RESEARCH SUPPORT AS TOPIC;

EID: 84892449393     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2013.12.005     Document Type: Article

References (60)

1. Benavente J.M., Crespi G., Garone L.F., Maffioli A. The impact of national research funds: A regression discontinuity approach to the Chilean FONDECYT. Research Policy 2012, 41:1461-1475.
2. Bloch C., Graversen E.K., Pedersen H.S. Competitive research grants and their impact on career performance. Minerva 2014, (Forthcoming).
3. Bloch C., Sørensen M.P., Ravn T., Graversen E.K., Mortensen P.S., Faber S.T., et al. An evaluation of research project grants of the Danish Council for Independent Research. Main report and subreports 2011, Danish Agency for Science, Technology and Innovation, Copenhagen, (in Danish).
4. Blundell R., Costa Dias M. Alternative approaches to empirical evaluations in microeconomics. Portuguese Economic Journal 2002, 1(2):91-115.
5. Buisseret T.J., Cameron H.M., Georghiou L. What difference does it makeα Additionality in the public support of R&D in large firms. International Journal of Technology Management 1995, 10(4-6):587-600.
6. Brannen J. Mixed methods research: A discussion paper, NCRM methods review papers 2005, ESRC National Centre for Research Methods.
7. Bryman A. The debate about quantitative and qualitative research: A question of method or epistemologyα. The British Journal of Sociology 1984, 35(1):75-92.
8. Campbell D.T., Stanley J.C. Experimental and quasi-experimental designs for research. Handbook of research on teaching 1963, Rand McNally, Chicago. N.L. Gage (Ed.).
9. Caracelli V., Greene J. Crafting mixed-method evaluation designs. Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms 1997, Jossey Bass, San Francisco. J. Greene, V. Caracelli (Eds.).
10. Charmaz K. Constructing grounded theory: A practical guide through qualitative analysis 2006, Sage, London, Thousand Oaks, New Delhi.
11. Chudnovsky D., Lopez A., Rossi M.A., Ubfal D. Money for scienceα The impact of research grants on academic output. Fiscal Studies 2008, 29(1):75-87.
12. Corbin J., Strauss A.L. Basics of qualitative research: Grounded theory procedures and techniques 2008, Sage, Thousand Oaks, CA. 3rd ed.
13. Creswell J.W., Plano Clark V.L. Designing and conducting mixed methods research 2007, Sage, Thousand Oaks, CA.
14. Creswell J.W. Research design. Qualitative, quantitative, and mixed methods approaches 2006, Sage, Thousand Oaks, CA.
15. Creswell J.W., Plano Clark V., Gutmann M., Hanson W. Advanced mixed methods research designs. Handbook of mixed methods in social and behavioural research 2003, Sage, Thousand Oaks, CA. A. Tashakkori, C. Teddlie (Eds.).
16. Degn L., Thidemann Faber S., Ravn T. Evaluation of funding grants for research projects. Background report: The case/interview analysis 2011, (in Danish).
17. Dehejia R.H., Wahba S. Propensity score-matching for nonexperimental studies. The Review of Economics and Statistics 2002, 84(1):151-161.
18. Dehejia R.H., Wahba S. Casual effects in non-experimental studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association 1999, 94(448):1053-1062.
19. European Commission Frontier research: The European challenge. High-level expert group report 2005, European Commission, Brussels.
20. European Commission Europe 2020 flagship initiative - Innovation union, SEC (2010) 1161 2011, European Commission, Brussels.
21. Feller I. Mapping the frontiers of evaluation of public-sector R&D programs. Science and Public Policy 2007, 34(10):681-690.
22. Furman J.L., Murray F., Stern S. Growing stem cells: The impact of federal funding on the U.S. scientific frontier. Journal of Policy Analysis and Management 2012, 31(3):661-705.
23. Georghiou L. Impact of the framework programme on European industry, EUR 15907 EN 1994, Office for Official Publications of the European Communities, Luxembourg.
24. Georghiou L., Roessner D. Evaluating technology programs: Tools and methods. Research Policy 2000, 29:657-678.
25. Gibbs G. Qualitative data analysis - Explorations with Nvivo 2002, Open University Press, Berkshire.
26. Glaser B.G. Forcing the data. Doing grounded theory. Issues and discussions 1998, The Sociology Press, Mill Valley, CA.
27. Greene J.C., Caracelli V.J., Graham W.F. Toward a conceptual framework for mixed method evaluation designs. Educational Evaluation and Policy Analysis 1989, 11(3):255-274.
28. Heckman J.J. Sample selection bias as a specification error. Econometrica 1979, 47:153-161.
29. Heckman J.J. Micro data, heterogeneity, and the evaluation of public policy: Nobel lecture. The Journal of Political Economy 2001, 109:673-748.
30. Heckman J.J. The scientific model of causality. Sociological Methodology 2005, 35:1-97.
31. Heckman J.J., Ichimura H., Smith J., Todd P. Characterizing selection bias using experimental data. Econometrica 1998, 66(5):1017-1098.
32. Howe K.R. Against the quantitative-qualitative incompatibility thesis, or, Dogmas die hard. Educational Researcher 1988, 17:10-16.
33. Jacob B.A., Lefgren L. The impact of NIH postdoctoral training grants on scientific productivity. Research Policy 2011, 20:864-874. L.
34. Jaffe A.B. Building programme evaluation into the design of public research-support programmes. Oxford Review of Economic Policy 2002, 18(1):22-34.
35. Johnson R.B., Onwuegbuzie A.J. Mixed methods research: A research paradigm whose time has come. Educational Researcher 2004, 33(7):14-26.
36. Kelle U. Sociological explanations between micro and macro and the integration of qualitative and quantitative methods. Historical Social Research 2005, 30(1):95-117.
37. Lane J. Assessing the impact of science funding. Science 2009, 324:1273-1275.
38. Lane J., Bertuzzi S. Measuring the results of science investments. Science 2011, 331:678-680.
39. Leech N.L., Onwuegbuzie A.J. A typology of mixed methods research design. Quality & Quantity 2009, 43:265-275.
40. Lincoln Y.S. The making of a constructivist: A remembrance of transformations past. The paradigm dialog 1990, Sage, Newbury Park, CA. E.G. Guba (Ed.).
41. Macilwain C. What science is really worth. Nature 2010, 465:682-684.
42. Mark M.M., Henry G.T., Julnes G. Toward an integrative framework for evaluation practice. American Journal of Evaluation 1999, 20(2):177-198.
43. Maxcy S.J. Pragmatic threads in mixed methods research in the social sciences: The search for multiple modes of inquiry and the end of the philosophy of formalism. Handbook of mixed methods in social and behavioral research 2003, Sage, Thousand Oaks, CA. A. Tashakkori, C. Teddlie (Eds.).
44. Mohr L.B. The qualitative method of impact analysis. American Journal of Evaluation 1999, 20:69-83.
45. Morgan D.L. Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research 2007, 1:48-76.
46. Morse J.M. Principles of mixed methods and multimethod research design. Handbook of mixed methods in social and behavioral research 2003, Sage, Thousand Oaks, CA. A. Tashakkori, C. Teddlie (Eds.).
47. Mortensen P.S., Thomsen G.S., Kruuse J. Subreport - Bibliometric analysis of applicants to research project grants in 2002-2004 2011, Danish Agency for Science, Technology and Innovation, Copenhagen, (in Danish).
48. National Research Council Research universities and the future of America: Ten breakthrough actions vital to our nation's prosperity and security 2012, National Academies Press, Washington, DC.
49. OECD Performance-based funding for public research in tertiary education institutions 2010, OECD, Paris.
50. Patton M.Q. Qualitative evaluation and research methods 1990, Sage, Newbury Park, CA. 2nd ed.
51. Rosenbaum P.R., Rubin D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 1983, 70(1):41-55.
52. Rubin D.B. Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology 1974, 66:688-701.
53. Salter A.J., Martin B.R. The economic benefits of publicly funded basic research: A critical review. Research Policy 2001, 30:509-532.
54. Smith J.A., Todd P.E. Does matching overcome LaLonde's critique of nonexperimental estimatorsα. Journal of Econometrics 2005, 125(1-2):305-353.
55. Tashakkori A., Creswell J.W. Editorial: The new era of mixed methods. Journal of Mixed Methods Research 2007, 1:3-7.
56. Teddlie C., Tashakkori A. Status of mixed methods research in nursing. Handbook of mixed methods in social and behavioral research 2003, Sage, Thousand Oaks, CA. A. Tashakkori, C. Teddlie (Eds.).
57. Teddlie C., Tashakkori A. A general typology of research designs featuring mixed methods. Research in the Schools 2006, 13(1):12-28.
58. Teddlie C., Tashakkori A. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences 2009, Sage, Thousand Oaks, CA.
59. Ubfal D., Maffioli A. The impact of funding on research collaboration: Evidence from a developing country. Research Policy 2011, 40(9):1269-1279.
60. U.S.Department of Commerce The competitiveness and innovative capacity of the United States 2012, Department of Commerce, Washington, DC.