Cognizance and consultation of randomized controlled trials among ministerial policy analysts

Review of Policy Research

Volumn 29, Issue 5, 2012, Pages 625-644

  Bédard, Pierre Olivier ^a   Ouimet, Mathieu ^b  

^a UNIVERSITÉ LAVAL   (Canada)

^b Université Laval   (Canada)

Author keywords

Evidence informed policy making; Knowledge utilization; Nonlinear regression; Policy analysis; Randomized controlled trials

Indexed keywords

KNOWLEDGE; POLICY ANALYSIS; POLICY MAKING; REGRESSION ANALYSIS;

CANADA; QUEBEC [CANADA];

EID: 84864861312     PISSN: 1541132X     EISSN: 15411338     Source Type: Journal    
DOI: 10.1111/j.1541-1338.2012.00581.x     Document Type: Article

References (69)

1. Black, N. (1996). Why we need observational studies to evaluate the effectiveness of health care. BMJ, 312, 1215-1218.
2. Boswell, C. (2008). The political functions of expert knowledge: Knowledge and legitimation in European Union immigration policy. Journal of European Public Policy, 15(4), 471-488.
3. Cartwright, N. (2009a). Evidence-based policy: What's to be done about relevance? Philosophical Studies, 143(1), 127-136.
4. Cartwright, N. (2009b). What are randomised controlled trials good for? Philosophical Studies, 147(1), 59-70.
5. Cartwright, N. (2009c). What is this thing called "efficacy?" In C. Mantzavinos (Ed.), Philosophy of the social sciences. Philosophical theory and scientific practice (pp. 185-206). Cambridge: Cambridge University Press.
6. Cohen, W. M., & Levinthal, D. A. (1990). Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly, 35(1), 128-152.
7. Cohn, D. (2006). Jumping into the political fray: Academics and policy-making. IRPP Policy Matters, 7(3), 1-31.
8. Colebatch, H. (2006). What work makes policy? Policy Sciences, 39(4), 309-321.
9. Concato, J., Shah, N., & Horwitz, R. I. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342(25), 1887-1892.
10. Cook, T. (2003). Why have educational evaluators chosen not to do randomized experiments? Annals of the American Academy of Political and Social Science, 589(1), 114-149.
11. Cookson, R. (2005). Evidence-based policy making in health care: What it is and what it isn't. Journal of Health Services Research & Policy, 10(2), 118-121.
12. Davies, H. T. O., Nutley, S. M., & Smith, P. C. (2000). What works? Evidence-based policy and practice in public services. Bristol: Policy Press.
13. De Vito, C., Nobile, C. G., Furnari, G., Pavia, M., De Giusti, M., Angelillo, I. F., etal. (2009). Physicians' knowledge, attitudes and professional use of RCTs and meta-analyses: A cross-sectional survey. European Journal of Public Health, 19(3), 297-302.
14. Dobbins, M., Cockerill, R., & Barnsley, J. (2001). Factors affecting the utilization of systematic reviews. International Journal of Technology Assessment in Health Care, 17(2), 203-214.
15. Druckman, J. N., Green, D. P., Kuklinski, J. H., & Lupia, A. (2006). The growth and development of experimental research in political science. American Political Science Review, 100(4), 627-635.
16. Duflo, E., Glennerster, R., Kremer, M., Schultz, P. T., & Strauss, J. A. (2007). Using randomization in development economics research: A toolkit. Discussion Paper No. 6059. London: Centre for Economic Policy Research.
17. Durning, D., & Osuna, W. (1994). Policy analysts' roles and value orientations: An empirical investigation using q methodology. Journal of Policy Analysis and Management, 13(4), 629-657.
18. Farrelly, R. (2008). Policy on trial. Policy, 24(3), 7-12.
19. Farrington, D. P. (2003). A short history of randomized experiments in criminology. Evaluation Review, 27(3), 218-227.
20. Farrington, D. P., & Welsh, B. C. (2005). Randomized experiments in criminology: What have we learned in the last two decades? Journal of Experimental Criminology, 1(1), 9-38.
21. Gerring, J. (2011). How good is good enough? A multidimensional, best-possible standard for research design. Political Research Quarterly, 64(3), 625-636.
22. Geva-May, I., & Maslove, A. M. (2007). In between trends: Developments of public policy analysis and policy analysis instruction in Canada, the United Stated, and the European Union. In L. Dobuzinskis, M. Howlett, & D. Laycock (Eds.), Policy analysis in Canada. The state of the art (pp. 186-215). Toronto, ON: Toronto University Press.
23. Godin, B., & Gingras, Y. (2000). What is scientific and technological culture and how is it measured? A multidimensional model. Public Understanding of Science, 9, 43-58.
24. Gough, D., & Elbourne, D. (2002). Systematic research synthesis to inform policy, practice and democratic debate. Social Policy & Society, 1, 225-236.
25. Gow, J. I., & Sutherland, S. L. (2004). Comparison of Canadian master's programs in public administration, public management and public policy. Canadian Public Administration, 47(3), 379-405.
26. Green, D. P., & Gerber, A. (2003). The underprovision of experiments in political science. Annals of the American Academy of Political and Social Science, 589, 94-112.
27. Greenberg, D. H., & Shroder, M. (2004). Randomized Social Experiments eJournal. University of Maryland, Baltimore County, and U.S. Department of Housing and Urban Development. Retrieved from http://www.ssrn.com/update/ern/ern_random-social-experiments.html
28. Harford, T. (2010). Policy ideas need proper testing. Financial Times, March 18.
29. Howlett, M. (2009a). Policy advice in multi-level governance systems: Sub-national policy analysts and analysis. International Review of Public Administration, 3(3), 1-16.
30. Howlett, M. (2009b). Policy analytical capacity and evidence-based policy-making: Lessons from Canada. Canadian Public Administration, 52(2), 153-175.
31. Howlett, M., & Newman, J. (2010). Policy analysis and policy work in federal systems: Policy advice and its contribution to evidence-based policy-making in multi-level governance systems. Policy and Society, 29(2), 123-136.
32. Innvaer, S., Vist, G., Trommald, M., & Oxman, A. (2002). Health policy-makers' perceptions of their use of evidence: A systematic review. Journal of Health Services Research and Policy, 7, 239-244.
33. Jadad, A. R., & Enkin, M. W. (2007). Randomised controlled trials: Questions, answers and musings (2nd ed.). Oxford: Blackwell Publishing.
34. Khagram, S., & Thomas, C. W. (2010). Toward a platinum standard for evidence-based assessment by 2020. Public Administration Review, 70, 100-106.
35. King, G., Tomz, M., & Wittenberg, J. (2000). Making the most of statistical analyses: Improving interpretation and presentation. American Journal of Political Science, 44(2), 341-355.
36. Knott, J., & Wildavsky, A. (1980). If dissemination is the solution, what is the problem? Science Communication, 1(4), 537-578.
37. Kothari, A., MacLean, L., & Edwards, N. (2009). Increasing capacity for knowledge translation: Understanding how some researchers engage policy makers. Evidence & Policy, 5(1), 33-51.
38. Lavis, J. N. (2009). How can we support the use of systematic reviews in policymaking? PLoS Medicine, 6(11), 1-6.
39. Lavis, J. N., Davies, H. T. O., Oxman, A., Denis, J.-L., Golden-Biddle, K., & Ferlie, E. (2005). Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research & Policy, 10(3 Suppl. 1), 35-48.
40. Leigh, A. (2003). Randomised policy trials. Agenda, 10(4), 341-354.
41. Leigh, A. (2009). What evidence should social policymakers use? Australian Treasury Economic Roundup, 1, 27-43.
42. Levitt, S. D., & List, J. A. (2008). Field experiments in economics: The past, the present, and the future. NBER Working Paper 14356. Cambridge, MA: National Bureau of Economic Research.
43. Lomas, J. (1997). Improving research dissemination and uptake in the health sector: Beyond the sound of one hand clapping. Policy commentary C97-1. Hamilton, ON: McMaster University, Centre for Health Economics and Policy Analysis.
44. Lomas, J., & Brown, A. D. (2009). Research and advice giving: A functional view of evidence-informed policy advice in a Canadian ministry of health. Milbank Quarterly, 87(4), 903-926.
45. Mayer, I., van Daalen, E., & Bots, P. W. G. (2004). Perspectives on policy analysis: A framework for understanding and design. International Journal of Technology, Policy and Management, 4(1), 169-191.
46. Meltsner, A. (1976). Policy analysts in the bureaucracy. Berkeley, CA: University of California Press.
47. Mitton, C., Adair, C. E., McKenzie, E., Patten, S. B., & Perry, B. W. (2007). Knowledge transfer and exchange: Review and synthesis of the literature. Milbank Quarterly, 85, 729-768.
48. Morton, R. B., & Williams, K. C. (2010). Experimental political science and the study of causality: From nature to the lab. New York, NY: Cambridge University Press.
49. Mosteller, F., & Boruch, R. F. (2002). Evidence matters: Randomized trials in education research. Washington, DC: Brookings Institution Press.
50. Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence. How research can inform public services. Bristol: The Policy Press.
51. Orr, L. L. (1999). Social experiments: Evaluating public programs with experimental methods. Thousand Oaks, CA: Sage Publications.
52. Ouimet, M., Bédard, P.-O., Turgeon, J., Lavis, J. N., Gélineau, F., Gagnon, F., etal. (2010). Correlates of consulting research evidence among policy analysts in ministries: A cross sectional study. Evidence & Policy, 6(4), 433-460.
53. Ouimet, M., Landry, R., Ziam, S., & Bédard, P.-O. (2009). The absorption of research knowledge by public civil servants. Evidence & Policy, 5, 331-350.
54. Radaelli, C. M. (2009). Measuring policy learning: Regulatory impact assessment in Europe. Journal of European Public Policy, 16(8), 1145-1164.
55. Radin, B. A. (2000). Beyond Machiavelli. Policy analysis comes of age. Washington DC: Georgetown University Press.
56. Sanson-Fisher, R. W., Bonevski, B., Green, L. W., & D'Este, C. (2007). Limitations of the randomized controlled trial in evaluating population-based health interventions. American Journal of Preventive Medicine, 33(2), 155-161.
57. Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating interventions. Annual Review of Psychology, 60(1), 607-629.
58. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin Company.
59. Sherman, L. W. (2003). Misleading evidence and evidence-led policy: Making social science more experimental. Annals of the American Academy of Political and Social Science, 589(1), 6-19.
60. Stoker, G. (2010). Translating experiments into policy. Annals of the American Academy of Political and Social Science, 628, 47-58.
61. Stoker, G., & John, P. (2009). Design experiments: Engaging policy makers in the search for evidence about what works. Political Studies, 57(2), 356-373.
62. Stone, D. (2001). Policy paradox. The art of political decision making (3rd ed.). New York, NY: W. W. Norton & Company.
63. Todorova, G., & Durisin, B. (2007). Absorptive capacity: Valuing a reconceptualization. Academy of Management Review, 32(3), 774-786.
64. Tomz, M., Wittenberg, J., & King, G. (2001). CLARIFY: Software for interpreting and presenting statistical results. Cambridge, MA: Harvard University.
65. Torrance, H. (2008). Building confidence in qualitative research-Engaging the demands of policy. Qualitative Inquiry, 14(4), 507-527.
66. Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426-431.
67. Weiss, C. H., & Birckmayer, J. (2006). Social experimentation for public policy. In M. Moran, M. Rein, & R. E. Goodin (Eds.), The Oxford handbook of public policy (pp. 806-830). Oxford: Oxford University Press.
68. Wellstead, A. M., Stedman, R. C., & Lindquist, E. A. (2009). The nature of regional policy work in Canada's federal public service. Canadian Political Science Review, 3(1), 34-56.
69. Zahra, S. A., & George, G. (2002). Absorptive capacity: A review, reconceptualization, and extension. Academy of Management Review, 27(2), 185-203.