Missing links: How descriptive validity impacts the policy relevance of randomized controlled trials in criminology

Journal of Experimental Criminology

Volumn 7, Issue 3, 2011, Pages 201-224

  Gill, Charlotte E ^a  

^a GEORGE MASON UNIVERSITY   (United States)

Author keywords

CONSORT; Descriptive validity; Evidence based policy; External validity; Internal validity; Methodological quality; Randomized controlled trials

Indexed keywords

EID: 80051674800     PISSN: 15733750     EISSN: 15728315     Source Type: Journal    
DOI: 10.1007/s11292-011-9122-z     Document Type: Article

References (73)

1. Altman, D. G., Schultz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., et al. (2001). The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Annals of Internal Medicine, 134(8), 663-694.
2. *Armstrong, T. A. (2002). The effect of environment on the behavior of youthful offenders: A randomized experiment. Journal of Criminal Justice, 30, 19-28.
3. *Armstrong, T. A. (2003). The effect of moral reconation therapy on the recidivism of youthful offenders: A randomized experiment.
4. *Banks, D., & Gottfredson, D. C. (2004). Participation in drug treatment court and time to rearrest. Justice Quarterly, 21(3), 637-658.
5. Barnes, G. C., Ahlman, L., Gill, C., Sherman, L. W., Kurtz, E., & Malvestuto, R. (2010). Low-intensity community supervision for low-risk offenders: A randomized, controlled trial. Journal of Experimental Criminology, 6, 159-189.
6. Berk, R. A., & Rossi, P. H. (1999). Thinking about program evaluation (2nd ed.). Thousand Oaks, CA: Sage.
7. *Biggam, F. H., & Power, K. G. (2002). A controlled, problem-solving, group-based intervention with vulnerable incarcerated young offenders. International Journal of Offender Therapy and Comparative Criminology, 46(6), 678-698.
8. *Bijleveld, C. (2007). Fare dodging and the strong arm of the law: An experimental evaluation of two different penalty schemes for fare evasion. Journal of Experimental Criminology, 3, 183-199.
9. *Blais, E., & Bacher, J. L. (2007). Situational deterrence and claim padding: Results from a randomized field experiment. Journal of Experimental Criminology, 3, 337-352.
10. Boruch, R. F. (1997). Randomized experiments for planning and evaluation: A practical guide. Thousand Oaks, CA: Sage.
11. Boruch, R., Snyder, B., & DeMoya, D. (2000). The importance of randomized field trials. Crime and Delinquency, 46(2), 156-180.
12. *Bottcher, J., & Ezell, M. E. (2005). Examining the effectiveness of boot camps: A randomized experiment with a long-term follow up. Journal of Research in Crime and Delinquency, 42(3), 309-332.
13. Braga, A. A., & Bond, B. J. (2008). Policing crime and disorder hot spots: A randomized controlled trial. Criminology, 46(3), 577-607.
14. Brown, S. E. (1989). Statistical power and criminal justice research. Journal of Criminal Justice, 17, 115-122.
15. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
16. Farrington, D. P. (2003a). Methodological quality standards for evaluation research. Annals of the American Academy of Political and Social Science, 587 (May), 49-58.
17. Farrington, D. P. (2003b). A short history of randomized experiments in criminology: A meager feast. Evaluation Review, 27(3), 218-227.
18. Farrington, D. P., Gottfredson, D. C., Sherman, L. W., & Welsh, B. C. (2006). The Maryland Scientific Methods Scale. In L. W. Sherman, D. P. Farrington, B. C. Welsh, & D. L. MacKenzie (Eds.), Evidence-based crime prevention (2nd ed., pp. 13-21). New York, NY: Routledge.
19. Farrington, D. P., & Welsh, B. C. (2005). Randomized experiments in criminology: What have we learned in the last two decades? Journal of Experimental Criminology, 1, 9-38.
20. *Feder, L., & Dugan, L. (2002). A test of the efficacy of court-mandated counseling for domestic violence offenders: The Broward experiment. Justice Quarterly, 19(2), 343-375.
21. *Feske, U. (2008). Treating low-income and minority women with posttraumatic stress disorder: A pilot study comparing prolonged exposure and treatment as usual conducted by community therapists. Journal of Interpersonal Violence, 23(8), 1027-1040.
22. *Giblin, M. J. (2002). Using police officers to enhance the supervision of juvenile probationers: An evaluation of the Anchorage CAN program. Crime and Delinquency, 48(1), 116-137.
23. Goldkamp, J. S. (2008). Missing the target and missing the point: 'Successful' random assignment but misleading results. Journal of Experimental Criminology, 4, 83-115.
24. *Goldkamp, J. S., & White, M. D. (2006). Restoring accountability in pretrial release: the Philadelphia pretrial release supervision experiments. Journal of Experimental Criminology, 2, 143-181.
25. *Gondolf, E. W. (2007). Culturally-focused batterer counseling for African-American men. Criminology and Public Policy, 6(2), 341-366.
26. *Gottfredson, D. C., & Exum, M. L. (2002). The Baltimore City drug treatment court: One-year results from a randomized study. Journal of Research in Crime and Delinquency, 39(3), 337-356.
27. *Gottfredson, D. C., Najaka, S. S., & Kearley, B. (2003). Effectiveness of drug treatment courts: Evidence from a randomized trial. Criminology and Public Policy, 2(2), 171-196.
28. *Gottfredson, D. C., Najaka, S. S., Kearley, B. W., & Rocha, C. M. (2006). Long-term effects of participation in the Baltimore City drug treatment court: Results from an experimental study. Journal of Experimental Criminology, 2, 67-98.
29. *Haapanen, R., & Britton, L. (2002). Drug testing for youthful offenders on parole: An experimental evaluation. Criminology and Public Policy, 1(2), 217-244.
30. *Kilmer, B. (2008). Does parolee drug testing influence employment and education outcomes? Evidence from a randomized experiment with noncompliance. Journal of Quantitative Criminology, 24, 93-123.
31. *Kinlock, T. W., Gordon, M. S., Schwartz, R. P., & O'Grady, K. E. (2008). A study of methadone maintenance for male prisoners: 3-month postrelease outcomes. Criminal Justice and Behavior, 35(1), 34-47.
32. *Labriola, M., Rempel, M., & Davis, R. C. (2008). Do batterer programs reduce recidivism? Results from a randomized trial in the Bronx. Justice Quarterly, 25(2), 252-282.
33. *Lane, J., Turner, S., Fain, T., & Seghal, A. (2005). Evaluating an experimental intensive juvenile probation program: Supervision and official outcomes. Crime and Delinquency, 51(1), 26-52.
34. Lauritsen, J. L. (2006). Assessing problematic research: How can academic researchers help improve the quality of anticrime program evaluations? Journal of Experimental Criminology, 2, 363-373.
35. *LeSure-Lester, G. E. (2002). An application of cognitive-behavior principles in the reduction of aggression among abused African American adolescents. Journal of Interpersonal Violence, 17(4), 394-402.
36. Lipsey, M., Petrie, C., Weisburd, D., & Gottfredson, D. (2006). Improving evaluation of anti-crime programs: Summary of a National Research Council report. Journal of Experimental Criminology, 2, 271-307.
37. *Little, M., Kogan, J., Bullock, R., & van der Laan, P. (2004). ISSP: An experiment in multi-systemic responses to persistent young offenders known to children's services. British Journal of Criminology, 44, 225-240.
38. Lösel, F., & Köferl, P. (1989). Evaluation research on correctional treatment in West Germany: A meta-analysis. In H. Wegener, F. Lösel, & J. Haisch (Eds.), Criminal behavior and the justice system: Psychological perspectives (pp. 334-355). New York, NY: Springer.
39. Lum, C., Koper, C. S., and Telep, C. W. (in press). The Evidence-Based Policing Matrix. Journal of Experimental Criminology.
40. Lum, C., & Yang, S.-M. (2005). Why do evaluation researchers in crime and justice choose non-experimental methods? Journal of Experimental Criminology, 1, 191-213.
41. MacKenzie, D. L. (2000). Evidence-based corrections: Identifying what works. Crime and Delinquency, 46(4), 457-471.
42. *MacKenzie, D. L., Bierie, D. L., & Mitchell, O. (2007). An experimental study of a therapeutic boot camp: Impact on impulses, attitudes and recidivism. Journal of Experimental Criminology, 3, 221-246.
43. *Marlowe, D. B., Festinger, D. S., Lee, P. A., Schepise, M. M., Hazzard, J. E. R., Merrill, J. C., et al. (2003). Are judicial status hearings a key component of drug court? During-treatment data from a randomized trial. Criminal Justice and Behavior, 30(2), 141-162.
44. *Marques, J. K., Wiederanders, M., Day, D. M., Nelson, C., & van Ommeren, A. (2005). Effects of a relapse prevention program on sexual recidivism: Final results from California's Sex Offender Treatment and Evaluation Project (SOTEP). Sexual Abuse: A Journal of Research and Treatment, 17(1), 79-107.
45. *McGarrell, E. F., & Hipple, N. K. (2007). Family group conferencing and re-offending among first-time juvenile offenders: The Indianapolis experiment. Justice Quarterly, 24(2), 221-246.
46. Moher, D., Jones, A., & Lepage, L. (2001). Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. Journal of the American Medical Association, 285(15), 1992-1995.
47. *Moynahan, L., & Strømgren, B. (2005). Preliminary results of Aggression Replacement Training for Norwegian youth with aggressive behavior and with a different diagnosis. Psychology, Crime & Law, 11(4), 411-419.
48. Perry, A. E. (2010). Descriptive validity and transparent reporting in randomised controlled trials. In A. R. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 333-352). New York, NY: Springer.
49. Perry, A. E., & Johnson, M. (2008). Applying the Consolidated Standards of Reporting Trials (CONSORT) to studies of mental health provision for juvenile offenders: A research note. Journal of Experimental Criminology, 4, 165-185.
50. Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6, 245-262.
51. Petrosino, A., Kiff, P., & Lavenberg, J. (2006). Randomized field experiments published in the British Journal of Criminology, 1960-1994. Journal of Experimental Criminology, 2, 99-111.
52. *Phillips, L. A. (2004). Evaluating the effects of a brief program for moral education in a county jail. Journal of Offender Rehabilitation, 39(2), 59-72.
53. Plint, A. C., Moher, D., Morrison, A., Schultz, K., Altman, D. G., Hill, C., et al. (2006). Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. The Medical Journal of Australia, 185(5), 263-267.
54. *Sacks, J. Y., Sacks, S., McKendrick, K., Banks, S., Schoeneberger, M., Hamilton, Z., et al. (2008). Prison therapeutic community treatment for female offenders: Profiles and preliminary findings for mental health and other variables (crime, substance use and HIV risk). Journal of Offender Rehabilitation, 46(3), 233-261.
55. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
56. Shepherd, J. P. (2003). Explaining feast or famine in randomized field trials: Medical science and criminology compared. Evaluation Review, 27(3), 290-315.
57. Sherman, L. W., Farrington, D. P., Welsh, B. C., & MacKenzie, D. L. (Eds.). (2006). Evidence-based crime prevention (2nd ed.). New York, NY: Routledge.
58. Sherman, L. W., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., & Bushway, S. (1997). Preventing crime: What works, what doesn't, what's promising. Washington, D. C.: United States Department of Justice, National Institute of Justice. Retrieved from http://www. ncjrs. gov/works.
59. Sherman, L. W., Strang, H., Angel, C., Woods, D., Barnes, G. C., Bennett, S., et al. (2005). Effects of face-to-face restorative justice on victims of crime in four randomized, controlled trials. Journal of Experimental Criminology, 1, 367-395.
60. *Stickle, W. P., Connell, N. M., Wilson, D. M., & Gottfredson, D. (2008). An experimental evaluation of teen courts. Journal of Experimental Criminology, 4, 137-163.
61. *Sullivan, C. M., Bybee, D. I., & Allen, N. E. (2002). Findings from a community-based program for battered women and their children. Journal of Interpersonal Violence, 17(9), 915-936.
62. *Taxman, F. S., & Thanner, M. (2006). Risk, need, and responsivity (RNR): It all depends. Crime and Delinquency, 52(1), 28-51.
63. *van Voorhis, P., Spruance, L. M., Ritchey, P. N., Listwan, S. J., & Seabrook, R. (2004). The Georgia Cognitive Skills experiment: A replication of Reasoning and Rehabilitation. Criminal Justice and Behavior, 31(3), 282-305.
64. *Vannoy, S. D., & Hoyt, W. T. (2004). Evaluation of an anger therapy intervention for incarcerated adult males. Journal of Offender Rehabilitation, 39(2), 39-57.
65. *Watt, K., Shepherd, J., & Newcombe, R. (2008). Drunk and dangerous: A randomised controlled trial of alcohol brief intervention for violent offenders. Journal of Experimental Criminology, 4, 1-19.
66. Weisburd, D. (2003). Ethical practice and evaluation of interventions in crime and justice: The moral imperative for randomized trials. Evaluation Review, 27(3), 336-354.
67. *Weisburd, D., Einat, T., & Kowalski, M. (2008). The miracle of the cells: An experimental study of interventions to increase payment of court-ordered financial obligations. Criminology and Public Policy, 7(1), 9-36.
68. Weisburd, D., Lum, C. M., & Petrosino, A. (2001). Does research design affect study outcomes in criminal justice? Annals of the American Academy of Social and Political Science, 578 (November), 50-70.
69. Weisburd, D., Morris, N. A., & Ready, J. (2008). Risk-focused policing at places: An experimental evaluation. Justice Quarterly, 25(1), 163-200.
70. Weisburd, D., Petrosino, A., & Mason, G. (1993). Design sensitivity in criminal justice experiments. Crime and Justice, 17, 337-379.
71. Weisburd, D., Wyckoff, L. A., Ready, J., Eck, J. E., Hinkle, J. C., & Gajewski, F. (2006). Does crime just move around the corner? A controlled study of spatial displacement and diffusion of crime control benefits. Criminology, 44(3), 549-592.
72. *White, M. D., Goldkamp, J. S., & Robinson, J. B. (2006). Acupuncture in drug treatment: Exploring its role and impact on participant behavior in the drug court setting. Journal of Experimental Criminology, 2, 45-65.
73. *Zhang, S. X., & Zhang, L. (2005). An experimental study of the Los Angeles County Repeat Offender Prevention Program: Its implementation and evaluation. Criminology and Public Policy, 4(2), 205-236.