Evaluation of systems-oriented public health interventions: Alternative research designs

Annual Review of Public Health

Volumn 35, Issue , 2014, Pages 9-27

  Sanson Fisher, Robert W ^a   D'Este, Catherine A ^b   Carey, Mariko L ^a   Noble, Natasha ^a   Paul, Christine L ^a  

^a UNIVERSITY OF NEWCASTLE   (Australia)

^b UNIVERSITY OF NEWCASTLE   (Australia)

Author keywords

Cluster randomized controlled trial; Controlled prepost design; Interrupted time series; Multiple baseline design; Stepped wedge design

Indexed keywords

ACTION RESEARCH; CANCER SCREENING; CEREBROVASCULAR ACCIDENT; COMMUNITY ACQUIRED PNEUMONIA; CONFERENCE PAPER; EVIDENCE BASED MEDICINE; GENERAL PRACTITIONER; HEALTH CARE COST; HOSPITAL ADMISSION; HUMAN; LIFE EXPECTANCY; METHODOLOGY; PATIENT COMPLIANCE; PHYSICAL INACTIVITY; PRIMARY MEDICAL CARE; PRIORITY JOURNAL; PROSTATE CANCER; PUBLIC HEALTH PROBLEM; PUBLIC HEALTH SERVICE; RANDOMIZATION; RANDOMIZED CONTROLLED TRIAL (TOPIC); SMOKING; SOCIAL STATUS; STUDY DESIGN; COST BENEFIT ANALYSIS; EPIDEMIOLOGY; HEALTH BEHAVIOR; HEALTH DISPARITY; HEALTH PROMOTION; PUBLIC HEALTH; REPRODUCIBILITY;

CAUSALITY; COST-BENEFIT ANALYSIS; HEALTH BEHAVIOR; HEALTH PROMOTION; HEALTH STATUS DISPARITIES; HUMANS; PUBLIC HEALTH; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 84897046073     PISSN: 01637525     EISSN: 15452093     Source Type: Book Series    
DOI: 10.1146/annurev-publhealth-032013-182445     Document Type: Conference Paper

References (102)

1. Adler A. 1992. What Life Could Mean to You. Oxford, UK: Oneworld
2. Ajzen I. 1985. From intentions to actions: A theory of planned behaviour. In Action Control: From Cognition to Behavior, ed. J Kuhl, J Beckmann, pp. 11-40. Berlin: Springer-Verlag
3. Aust. Bur. Stat. 2006. Australian social trends, 2000, 4102.0. Last updatedMay 2. http://www.abs.gov.au/ausstats/abs@nsf/ 2f762f95845417aeca25706c00834efa/738d23457b86defbca2570ec000e2f5c! OpenDocument
4. Aust. Inst. Health Welf. 2011. 2010 National Drug Strategy Household Survey Report. Drug Stat. Ser. No. 25. Cat. no. PHE 145. Canberra: Aust. Inst. Health Welf. http://www.aihw.gov.au/WorkArea/DownloadAsset.aspx?id=10737421314
5. Aust. Inst. Health Welf. 2013. Public Health Expenditure in Australia 2008-2009. Canberra: Aust. Inst. HealthWelf. http://www.aihw.gov.au/public- health-expenditure/
6. Bandura A. 1986. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall
7. Baum F. 1998. The New Public Health: An Australian Perspective. Melbourne, Aust.: Oxford Univ. Press
8. Biglan A, Ary D, Koehn V, Levings D, Smith S, et al. 1996. Mobilizing positive reinforcement in communities to reduce youth access to tobacco. Am. J. Community Psychol. 24:625-38
9. Biglan A, Ary D,Wagenaar AC. 2000. The value of interrupted time-series experiments for community intervention research. Prev. Sci. 1:31-49
10. Biglan A, Henderson J, Humphrey D, Yasui M, Whisman R, et al. 1995. Mobilising positive reinforcement to reduce youth access to tobacco. Tob. Control 4:42-48
11. Brown CA, Lilford RJ. 2006. The stepped wedge trial design: A systematic review. BMC Med. Res. Methodol. 6:54-63
12. Brown CH, Wyman PA, Guo J, Pena J. 2006. Dynamic wait-listed designs for randomized trials: new designs for prevention of youth suicide. Clin. Trials 3:259-71
13. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, et al. 2000. Framework for design and evaluation of complex interventions to improve health. BMJ 321:694-96
14. Capelastegui A, Espana PP, Quintana JM, Gorordo I, Ortega M, et al. 2004. Improvement of processof- care and outcomes after implementing a guideline for themanagement of community-Acquired pneumonia: A controlled before-And-After design study. Clin. Infect. Dis. 39:955-63
15. CDC (Cent. Dis. Control Prev.). 2011. Vital signs: current cigarette smoking among adults aged ≥18 years-United States, 2005-2010. MMWR 60:1207-12
16. CDC (Cent. Dis. Control Prev.). 2012. Trends in current cigarette smoking among high school students and adults, United States, 1965-2011. Updated November 8, 2013. http://www.cdc.gov/tobacco/data-statistics/tables/trends/cig- smoking/index.htm
17. Cent. Evidence Based Medicine. 2013. Oxford Centre for Evidence-basedMedicine-levels of evidence (March 2009). Updated September 16. http://www.cebm.net/?o=1025
18. Corrigan O. 2003. Empty ethics: the problem with informed consent. Sociol. Health Illn. 25:768-92
19. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. 2008. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337:a1655
20. Crosbie J. 1993. Interrupted time-series analysis with brief single-subject data. J. Consult. Clin. Psychol. 61:966-74
21. DeVita VT Jr. 2005. The Framework Convention on Tobacco Control. Nat. Clin. Pract. Oncol. 2:171
22. Dickinson J,Wiggers J, Leeder S, Sanson-Fisher RW. 1989. General practitioners' detection of patients' smoking status. Med. J. Aust. 150:420-22, 425-26
23. Djulbegovic M, Beyth RJ, Neuberger MM, Stoffs TL, Vieweg J, et al. 2010. Screening for prostate cancer: systematic review and meta-Analysis of randomised controlled trials. BMJ 341:c4543
24. Donabedian A. 1988. The quality of health care: How can it be assessed? JAMA 260:1743-48
25. Donner A, Klar N. 2000. Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold Publ.
26. Eldridge S, Kerry S. 2012. A Practical Guide to Cluster Randomised Trials in Health Services Research. Chichester, UK:Wiley
27. Erbas B, Bui Q, Huggins R, Harper T, White V. 2006. Investigating the relation between placement of Quit antismoking advertisements and number of telephone calls to Quitline: A semiparametricmodelling approach. J. Epidemiol. Community Health 60:180-82
28. Ezzati M, Friedman AB, Kulkarni SC, Murray CJ. 2008. The reversal of fortunes: trends in county mortality and cross-county mortality disparities in the United States. PLoS Med. 5:e66
29. Freud S. 1974. The Standard Edition of the Complete Psychological Works of Sigmund Freud: Vol. 11. Great Britain: Hogarth
30. Freund M, Campbell E, Paul C, Sakrouge R, Lecathelinais C, et al. 2009. Increasing hospital-wide delivery of smoking cessation care for nicotine-dependent in-patients: A multi-strategic intervention trial. Addiction 104:839-49
31. Gambia Hepatitis Study Group. 1987. TheGambiaHepatitis Intervention Study. Cancer Res. 47:5782-87
32. Glasziou P, Vandenbroucke J, Chalmers I. 2004. Assessing the quality of research. BMJ 328:39-41
33. Green LW, Kreuter MW. 1991. Health Promotion Planning: An Educational and Environmental Approach. Mountain View, CA: Mayfield. 2nd ed.
34. Han PK, Coates RJ, Uhler RJ, Breen N. 2006. Decision making in prostate-specific antigen screening: National Health Interview Survey, 2000. Am. J. Prev. Med. 30:394-404
35. Hancock L, Sanson-Fisher R, Perkins J, Corkrey R, Burton R, Reid S. 2001. Effect of a community action intervention on cervical cancer screening rates in rural Australian towns: the CART Project. Prev. Med. 32:109-17
36. Hancock L, Sanson-Fisher R, Perkins J, Girgis A, Howley P, Schofield M. 2001. The effect of a community action intervention on adolescent smoking rates in rural Australian towns: the CART project. Prev. Med. 32:332-40
37. Hancock L, Sanson-Fisher R, Perkins J, McClintock A, Howley P, Gibberd R. 2001. Effect of a community action program on adult quit smoking rates in rural Australian towns: the CART Project. Prev. Med. 32:118-27
38. Hancock L, Sanson-Fisher R, Redman S, Burton R, Burton L, et al. 1996. Community action for cancer prevention: overview of the Cancer Action in Rural Towns (CART) project, Australia. Health Prom. Int. 11:277-90
39. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, et al. 2001. Current methods of the U.S. Preventive Services Task Force: A review of the process. Am. J. Prev. Med. 20:21-35
40. HawkinsN, Sanson-Fisher RW, Shakeshaft A,D'EsteC,Green LW. 2007. Themultiple baseline design for evaluating population-based research. Am. J. Prev. Med. 33:162-68
41. Hayes RJ, Moulton LH. 2009. Cluster Randomised Trials. London: CRC Press
42. Henstridge J, HomelR,Mackay P. 1997. The Long-Term Effects of Random Breath Testing in Four Australian States: A Time Series Analysis. CR 162. Canberra, Aust.: Fed. Off. Road Saf.
43. Heywood A, Sanson-Fisher R, Ring I, Mudge P. 1994. Risk prevalence and screening for cancer by general practitioners. Prev. Med. 23:152-59
44. Higgins JP, Altman DG, Gøtzsche PC, Juni P, Moher D, et al. 2011. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343:d5928
45. Higgins JPT, Green S, eds. 2009. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.2, updated September. Chichester, UK:Wiley
46. Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, et al. 2005. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff. 24:1103-17
47. Horrocks S, Anderson E, Salisbury C. 2002. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ 324:819-23
48. HusseyMA, Hughes JP. 2007. Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials 28:182-91
49. Ilic D, O'Connor D, Green S, Wilt TJ. 2006. Screening for prostate cancer. Cochrane Database Syst. Rev. 1:CD004720
50. Janz NK, Becker MH. 1984. The health belief model: A decade later. Health Educ. Behav. 11:1-47
51. Jeffreys M, Stevanovic V, Tobias M, Lewis C, Ellison-Loschmann L, et al. 2005. Ethnic inequalities in cancer survival in New Zealand: linkage study. Am. J. Public Health 95:834-37
52. Katsuta T, Saitoh A. 2012. Haemophilus influenzae type b vaccine. J. Infect. Dis. 206:1645-51
53. Killam WP, Tambatamba BC, Chintu N, Rouse D, Stringer E, et al. 2010. Antiretroviral therapy in antenatal care to increase treatment initiation in HIV-infected pregnant women: A stepped-wedge evaluation. AIDS 24:85-91
54. KitchenMS, Ransley JK, Greenwood DC, Clarke GP, ConnerMT, et al. 2009. Study protocol: A cluster randomised controlled trial of a school based fruit and vegetable intervention-Project Tomato. BMC Health Serv. Res. 9:101
55. Klar N, Donner A. 1997. The merits of matching in community intervention trials: A cautionary tale. Stat. Med. 16:1753-64
56. Kotz D, SpigtM, Arts IC, Crutzen R, Viechtbauer W. 2012. Researchers should convince policy makers to perform a classic cluster randomized controlled trial instead of a stepped wedge design when an intervention is rolled out. J. Clin. Epidemiol. 65:1255-56
57. Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. 2012. Use of the stepped wedge design cannot be recommended: A critical appraisal and comparison with the classic cluster randomized controlled trial design. J. Clin. Epidemiol. 65:1249-52
58. LeungWC. 2001. Balancing statistical and clinical significance in evaluating treatment effects. Postgrad. Med. J. 77:201-4
59. Macintyre S. 2003. Evidence based policy making: Impact on health inequalities still needs to be assessed. BMJ 326:5-6
60. Marmot M. 2005. Social determinants of health inequalities. Lancet 365:1099-104
61. Mdege ND, Man MS, Taylor (nee Brown) CA, Torgerson DJ. 2012. There are some circumstances where the stepped-wedge cluster randomized trial is preferable to the alternative: no randomized trial at all. Response to the commentary by Kotz and colleagues. J. Clin. Epidemiol. 65:1253-54
62. Mdege ND, Man MS, Torgerson DJ. 2011. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J. Clin. Epidemiol. 64:936-48
63. Mercer S, DeVinney B, Fine L, Green L, Dougherty D. 2007. Study designs for effectiveness and translation research: identifying trade-offs. Am. J. Prev. Med. 33:139-54
64. Michie S, van Stralen MM, West R. 2011. The behaviour change wheel: A newmethod for characterising and designing behaviour change interventions. Implement. Sci. 6:42-53
65. Middleton S, Levi C, Ward J, Grimshaw J, Griffiths R, et al. 2009. Fever, hyperglycaemia and swallowing dysfunction management in acute stroke: A cluster randomised controlled trial of knowledge transfer. Implement. Sci. 4:16-27
66. Middleton S, McElduff P, Ward J, Grimshaw J, Dale S, et al. 2011. Implementation of evidencebased treatment protocols to manage fever, hyperglycaemia, and swallowing dysfunction in acute stroke (QASC): A cluster randomised controlled trial. Lancet 378:1699-706
67. Murray DM. 1998. Design and Analysis of Group-Randomized Trials. New York: Oxford Univ. Press
68. Nestle M. 2005. Preventing childhood diabetes: the need for public health intervention. Am. J. Public Health 95:1497-99
69. Ozanne-Smith J, Ashby K, Newstead S, Stathakis V, Clapperton A. 2004. Firearm related deaths: the impact of regulatory reform. Inj. Prev. 10:280-86
70. Pan SY, Cameron C, DesMeules M, Morrison H, Craig CL, Jiang X. 2009. Individual, social, environmental, and physical environmental correlates with physical activity among Canadians: A cross-sectional study. BMC Public Health 9:21-33
71. Pawson R, Tilley N. 2006. Realist evaluation. DPRN Them. Meet. 2006 Rep. Eval. Amsterdam: Dev. Policy Rev. Netw. (DPRN). http://www.communitymatters.com. au/RE-chapter.pdf
72. Petticrew M, Whitehead M, Macintyre S, Graham H, Egan M. 2004. Evidence for public health policy on inequalities: 1: The reality according to policymakers. J. Epidemiol. Community Health 58:811-16
73. Reason P, Bradbury H, eds. 2008. The Sage Handbook of Action Research: Participative Inquiry and Practice. Thousand Oaks, CA: Sage
74. Reichardt CS,Mark MM. 2004. Quasi-experimentation. In Handbook of Practical Program Evaluation, ed. HH Wholey, JK Newcomer, pp. 126-49. San Francisco, CA: Jossey-Bass Inc.
75. Reid AL,WebbGR, Hennrikus D, Fahey PP, Sanson-Fisher RW. 1986. Detection of patients with high alcohol intake by general practitioners. BMJ Clin. Res. 293:735-37
76. Reid JL, Hammond D, Boudreau C, Fong GT, Siahpush M. 2010. Socioeconomic disparities in quit intentions, quit attempts, and smoking abstinence among smokers in four western countries: findings from the International Tobacco Control Four Country Survey. Nicotine Tob. Res. 12:S20-33
77. Rogers CR, Skinner BF. 1956. Some issues concerning the control of human behaviour. Science 124:1057- 66
78. Rosenthal MB, Frank RG, Li Z, Epstein AM. 2005. Early experience with pay-for-performance: from concept to practice. JAMA 294:1788-93
79. Rychetnik L, Frommer M, Hawe P, Shiell A. 2002. Criteria for evaluating evidence on public health interventions. J. Epidemiol. Community Health 56:119-27
80. Sanson-Fisher R, Bonevski B, Green L, D'Este C. 2007. Limitations of the randomised controlled trial in evaluating population-based health interventions. Am. J. Prev. Med. 33:155-61
81. Sanson-Fisher RW, Redman S, Oak S, Webb G. 1988. The burden of illness that is imposed by drug abuse and the allocation of research monies in the field: three views. Med. J. Aust. 149:134-38
82. Saunders RP, Evans MH, Joshi P. 2005. Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promot. Pract. 6:134-47
83. Schaap MM, Kunst AE, Leinsalu M, Regidor E, Ekholm O, et al. 2008. Effect of nationwide tobacco control policies on smoking cessation in high and low educated groups in 18 European countries. Tob. Control 17:248-55
84. ShadishWR,Cook TD,Campbell DT. 2002. Experimental and Quasi-ExperimentalDesigns forGeneralized Causal Inference. Boston/New York: Houghton Mifflin
85. Shakeshaft A,Doran C, Petrie D, Breen C, Havard A, et al. 2012. The Alcohol Action in Rural Communities (AARC) Project: Detailed Description of the Interventions and Their Costs. Sydney: Natl. Drug Alcohol Res. Cent., Univ. N.S.W.
86. Shakeshaft A,Doran C, Petrie D, Breen C, Havard A, et al. 2012. The Alcohol Action in Rural Communities (AARC) Project: Working with Communities to Select, Implement and Measure the Impact of Strategies to Reduce Alcohol-Related Harm. Deakin, Aust.: Found. Alcohol Res. Educ.
87. Shen D, Lu Z. 2006. Randomization in clinical trial studies. Pap. Po06 presented at PharmaSUG, May 21-24, Bonita Springs, Fla. http://www.lexjansen. com/pharmasug/2006/posters/po06.pdf
88. Swan J, Breen N, Coates RJ, Rimer BK, Lee NC. 2003. Progress in cancer screening practices in the United States: results from the 2000 National Health Interview Survey. Cancer 97:1528-40
89. Tabrizi SN, Brotherton JM, Kaldor JM, Skinner SR, Cummins E, et al. 2012. Fall in human papillomavirus prevalence following a national vaccination program. J. Infect. Dis. 206:1645-51
90. Thompson B, Coronado G, Snipes SA, PuschelK. 2003. Methodologic advances and ongoing challenges in designing community-based health promotion programs. Annu. Rev. Public Health 24:315-40
91. Tully J, Ninis N, Booy R, Viner R. 2000. The new system of review by multicentre research ethics committees: prospective study. BMJ 320:1179-82
92. Volmink J, Garner P. 2007. Directly observed therapy for treating tuberculosis. Cochrane Database Syst. Rev. (4):CD003343
93. Wakefield MA, Loken B, Hornik RC. 2010. Use of mass media campaigns to change health behaviour. Lancet 376:1261-71
94. Wang J, Etter JF. 2004. Administering an effective health intervention for smoking cessation online: the international users of Stop-Tabac. Prev. Med. 39:962-68
95. Wendy B. 2012. Vaccinationwith 3-dose paediatric rotavirus vaccine (RotaTeq): impact on the timeliness of uptake of the primary course of DTPa vaccine. Vaccine 30:5293-97
96. WHO (World Health Organ.). 2005. Preventing Chronic Diseases: A Vital Investment. Geneva:WHO
97. WHO (World Health Organ.). 2011. Global Status Report on Noncommunicable Diseases 2010: Description of the Global Burden of NCDs, Their Risk Factors and Determinants. Geneva:WHO
98. WHO Global Tuberc. Program. 1997. WHO Report on the Tuberculosis Epidemic 1997. Geneva:WHO
99. Wilbur J. 2008. Prostate cancer screening: the continuing controversy. Am. Fam. Physician 78:1377-84
100. Wolf A, Wender RC, Etzioni RB, Thompson IM, D'Amico AV, et al. 2010. American Cancer Society guideline for the early detection of prostate cancer: update 2010. CA Cancer J. Clin. 60:70-98
101. Zaza S, Briss PA, Harris KW, eds. 2005. The Guide to Community Preventive Services: What Works to Promote Health? New York: Oxford Univ. Press
102. Ziglio E. 1997. How to move towards evidence-based health promotion interventions. Promot. Educ. 4:29-33