Development and evaluation of an instrument for the critical appraisal of randomized controlled trials of natural products

BMC Complementary and Alternative Medicine

Volumn 9, Issue , 2009, Pages

  Jurgens, Tannis ^a   Whelan, Anne Marie ^a   MacDonald, Melissa ^a   Lord, Lindsay ^a  

^a DALHOUSIE UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL PRODUCT; BIOLOGICAL PRODUCT;

ARTICLE; CLINICAL RESEARCH; DELPHI STUDY; INSTRUMENT VALIDATION; METHODOLOGY; QUALITATIVE RESEARCH; RANDOMIZATION; VALIDATION STUDY; CONSENSUS; EVALUATION; EVIDENCE BASED MEDICINE; HERBAL MEDICINE; HUMAN; RANDOMIZED CONTROLLED TRIAL; STANDARD;

BIOLOGICAL PRODUCTS; CONSENSUS; DELPHI TECHNIQUE; EVIDENCE-BASED MEDICINE; HERBAL MEDICINE; HUMANS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN;

EID: 66849122563     PISSN: None     EISSN: 14726882     Source Type: Journal    
DOI: 10.1186/1472-6882-9-11     Document Type: Article

References (113)

1. Chan E Quality of efficacy research in Complementary and Alternative Medicine. Editorial JAMA 2008, 299(22):2685-2686. 10.1001/ jama.299.22.2685 18544730
2. Whelan AM Jurgens TM Lord L Evaluating the quality of randomized controlled trials that examine the efficacy of natural health products: a systematic review of critical appraisal instruments Evid Based Complement Alternat Med 2008. 18955310
3. Gagnier JJ Boon H Rochon P Moher D Barnes J Bombardier C CONSORT Group Reporting randomized, controlled trials of herbal interventions: An elaborated CONSORT statement Ann Intern Med 2006, 144(5):364-367. 16520478
4. Verhagen AP de Vet HC de Bie RA Kessels AG Boers M Bouter LM Knipschild PG The Delphi list: A criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus J Clin Epidemiol 1998, 51(12):1235-1241. 10.1016/ S0895-4356(98)00131-0 10086815
5. Altman DG Schulz KF Moher D Egger M Davidoff F Elbourne D Gotzsche PC Lang T CONSORT GROUP (Consolidated Standards of Reporting Trials) The revised CONSORT statement for reporting randomized trials: Explanation and elaboration Ann Intern Med 2001, 134(8):663-694. 11304107
6. Jadad AR Moore RA Carroll D Jenkinson C Reynolds DJ Gavaghan DJ McQuay HJ Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996, 17(1):1-12. 10.1016/ 0197-2456(95)00134-4 8721797
7. Low Dog T Clinical Trial Reviewer's Guidance and Checklist The Handbook of Clinically Tested Herbal Remedies Birmingham, New York: The Haworth Press, Inc Barrett M 2004, 1:141-148
8. Gardner DM Evidence-based decisions about herbal products for treating mental disorders J Psychiatry Neurosci 2002, 27(5):324-333. 161674 12271788
9. Turlik MA Kushner D Stock D Assessing the validity of published randomized controlled trials in podiatric medical journals J Am Podiatr Med Assoc 2003, 93(5):392-398. 13130087
10. Vamvakas EC Rationale, objectives, and interpretation of randomized controlled trials J Clin Apher 1997, 12(3):130-139. 10.1002/ (SICI)1098-1101(1997)12:3<130::AID-JCA6>3.0.CO;2-B 9365866
11. Heijden van der GJ Windt van der DA Kleijnen J Koes BW Bouter LM Steroid injections for shoulder disorders: A systematic review of randomized clinical trials Br J Gen Pract 1996, 46(406):309-316. 1239642 8762750
12. van Tulder MW Koes BW Bouter LM Conservative treatment of acute and chronic nonspecific low back pain. A systematic review of randomized controlled trials of the most common interventions Spine 1997, 22(18):2128-2156. 10.1097/00007632-199709150-00012 9322325
13. Weintraub M How to Critically Assess Clinical Drug Trials Drug Ther 1982:131-148
14. Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature Call for comments on a proposal to improve reporting of clinical trials in the biomedical literature Ann Intern Med 1994, 121(11):894-895. 7978706
15. Zelen M Guidelines for publishing papers on cancer clinical trials: responsibilities of editors and authors J Clin Oncol 1983, 1(2):164-169. 6668497
16. Nurmohamed MT Rosendaal FR Buller HR Dekker E Hommes DW Vandenbroucke JP Briet E Low-molecular-weight heparin versus standard heparin in general and orthopaedic surgery: A meta-analysis Lancet 1992, 340(8812):152-156. 10.1016/0140-6736(92)93223-A 1352573
17. Onghena P Van Houdenhove B Antidepressant-induced analgesia in chronic non-malignant pain: A meta-analysis of 39 placebo-controlled studies Pain 1992, 49(2):205-219. 10.1016/0304-3959(92)90144-Z 1535121
18. Pater JL Weir L Reporting the results of randomized trials of empiric antibiotics in febrile neutropenic patients - a critical survey J Clin Oncol 1986, 4(3):346-352. 3512784
19. Pua HL Lerman J Crawford MW Wright JG An evaluation of the quality of clinical trials in anesthesia Anesthesiology 2001, 95(5):1068-1073. 10.1097/00000542-200111000-00007 11684972
20. Reisch JS Tyson JE Mize SG Aid to the evaluation of therapeutic studies Pediatrics 1989, 84(5):815-827. 2797977
21. Rochon PA Gurwitz JH Cheung CM Hayes JA Chalmers TC Evaluating the quality of articles published in journal supplements compared with the quality of those published in the parent journal JAMA 1994, 272(2):108-113. 10.1001/jama.272.2.108 8015117
22. Rothwell PM External validity of randomised controlled trials: "to whom do the results of this trial apply?" Lancet 2005, 365(9453):82-93. 10.1016/S0140-6736(04)17670-8 15639683
23. Sackett DL How to read clinical journals: V: To distinguish useful from useless or even harmful therapy Can Med Assoc J 1981, 124(9):1156-1162. 1705333 7016293
24. Schulz KF Chalmers I Hayes JA Altman DG Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials JAMA 1995, 273(5):408-412. 10.1001/jama.273.5.408 7823387
25. Scottish Intercollegiate Guidelines Network http://www.sign.ac.uk/ methodology/checklists.html
26. Seals DR Tanaka H Manuscript peer review: A helpful checklist for students and novice referees Adv Physiol Educ 2000, 23(1):52-58. 10902527
27. Simon R Wittes RE Methodologic guidelines for reports of clinical trials Cancer Treat Rep 1985, 69(1):1-3. 3967252
28. Sindhu F Carpenter L Seers K Development of a tool to rate the quality assessment of randomized controlled trials using a Delphi technique J Adv Nurs 1997, 25(6):1262-1268. 10.1046/j.1365-2648.1997.19970251262.x 9181425
29. Stoddard GJ Ring WH How to evaluate study methodology in published clinical research J Intraven Nurs 1993, 16(2):110-117. 8478779
30. Strippoli GF Craig JC Schena FP The number, quality, and coverage of randomized controlled trials in nephrology J Am Soc Nephrol 2004, 15(2):411-419. 10.1097/01.ASN.0000100125.21491.46 14747388
31. The Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature Checklist of information for inclusion in reports of clinical trials Ann Intern Med 1996, 124(8):741-743. 8633835
32. The Standards of Reporting Trials Group A proposal for structured reporting of randomized controlled trials JAMA 1994, 272(24):1926-1931. 10.1001/jama.272.24.1926 7990245
33. Harrison JE Clinical trials in orthodontics II: Assessment of the quality of reporting of clinical trials published in three orthodontic journals between 1989 and 1998 J Orthod 2003, 30(4):309-15. discussion 297-8. 10.1093/ortho/30.4.309 14634169
34. What is...? series http://www.whatisseries.co.uk/whatis/
35. Huwiler-Muntener K Juni P Junker C Egger M Quality of reporting of randomized trials as a measure of methodologic quality JAMA 2002, 287(21):2801-2804. 10.1001/jama.287.21.2801 12038917
36. Imperiale TF McCullough AJ Do corticosteroids reduce mortality from alcoholic hepatitis? A meta-analysis of the randomized trials Ann Intern Med 1990, 113(4):299-307. 2142869
37. Juni P Altman DG Egger M Systematic reviews in health care: Assessing the quality of controlled clinical trials BMJ 2001, 323(7303):42-46. 1120670 11440947 10.1136/bmj.323.7303.42
38. Karassa FB Tatsioni A Ioannidis JP Design, quality, and bias in randomized controlled trials of systemic lupus erythematosus J Rheumatol 2003, 30(5):979-984. 12734892
39. Khan KS Daya S Jadad A The importance of quality of primary studies in producing unbiased systematic reviews Arch Intern Med 1996, 156(6):661-666. 10.1001/archinte.156.6.661 8629879
40. Kjaergard LL Frederiksen SL Gluud C Validity of randomized clinical trials in gastroenterology from 1964-2000 Gastroenterology 2002, 122(4):1157-1160. 10.1053/gast.2002.32390 11910365
41. Kjaergard LL Nikolova D Gluud C Randomized clinical trials in hepatology: Predictors of quality Hepatology 1999, 30(5):1134-1138. 10.1002/hep.510300510 10534332
42. Levine J Trial Assessment Procedure Scale Guide to Clinical Trials New York: Raven Press, Ltd Spilker B 1991:780-786
43. Liberati A Himel HN Chalmers TC A quality assessment of randomized control trials of primary treatment of breast cancer J Clin Oncol 1986, 4(6):942-951. 3711962
44. Liddle J Williamson M Irwig L Method for Evaluating Research Guideline Evidence (MERGE) Sydney: NSW Health Department 1996
45. Lionel ND Herxheimer A Assessing reports of therapeutic trials Br Med J 1970, 3(723):637-640. 1701692 4919020 10.1136/bmj.3.5723.637
46. Lohr KN Carey TS Assessing "best evidence": Issues in grading the quality of studies for systematic reviews Jt Comm J Qual Improv 1999, 25(9):470-479. 10481816
47. Lopex LR What are the essential factors in a critique of a research study for determination of quality and applicability? Fla Nurse 1987, 35(6):25. 3443195
48. Mahon WA Daniel EE A method for the assessment of reports of drug trials Can Med Assoc J 1964, 90:565-569. 1922379 14118699
49. Makela M Witt K How to read a paper: Critical appraisal of studies for application in healthcare Singapore Med J 2005, 46(3):108-14. quiz 115. 15735874
50. Meade MO Richardson WS Selecting and appraising studies for a systematic review Ann Intern Med 1997, 127(7):531-537. 9313021
51. Meinert CL Questions and factors to consider when reading a report from a clinical trial Clinical Trials: Design, Conduct, and Analysis New York: Oxford University Press Meinert CL 1986:272-276
52. Moher D Pham B Jones A Cook DJ Jadad AR Moher M Tugwell P Klassen TP Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998, 352(9128):609-613. 10.1016/S0140-6736(98)01085-X 9746022
53. Moher D Schulz KF Altman DG CONSORT GROUP (Consolidated Standards of Reporting Trials) The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials Ann Intern Med 2001, 134(8):657-662. 11304106
54. Mosteller F Gilbert JP McPeek B Reporting standards and research strategies for controlled trials. Agenda for the editor Control Clin Trials 1980, 1:37-58. 10.1016/S0197-2456(80)80006-7
55. NHMRC (National Health and Medical Research Council) How to review the evidence: Systematic identification and review of the scientific literature Canberra: NHMRC 2000
56. Public Health Resource Unit http://www.phru.nhs.uk/Doc_Links/ rct%20appraisal20%tool.pdf
57. DerSimonian R Charette LJ McPeek B Mosteller F Reporting on methods in clinical trials N Engl J Med 1982, 306(22):1332-1337. 7070458
58. Detsky AS Naylor CD O'Rourke K McGeer AJ L'Abbe KA Incorporating variations in the quality of individual randomized trials into meta-analysis J Clin Epidemiol 1992, 45(3):255-265. 10.1016/ 0895-4356(92)90085-2 1569422
59. Devereaux PJ Manns BJ Ghali WA Quan H Guyatt GH The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist Control Clin Trials 2002, 23(4):380-388. 10.1016/S0197-2456(02)00214-3 12161081
60. Dixon JS Smith A Evans SJ Reporting clinical trials Br J Rheumatol 1983, 22(3 Suppl):74-78. 6871595
61. Djulbegovic B Adams JR Lyman GH Lacevic M Hozo I Greenwich M Bennett CL Evaluation and appraisal of randomized controlled trials in myeloma Ann Oncol 2001, 12(11):1611-1617. 10.1023/A:1013181331874 11822763
62. Doig GS Interpreting and using clinical trials Crit Care Clin 1998, 14(3):513-524. 10.1016/S0749-0704(05)70014-2 9700445
63. Downs SH Black N The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions J Epidemiol Community Health 1998, 52(6):377-384. 1756728 9764259 10.1136/ jech.52.6.377
64. Ellenberg JH Clinical trials Neurol Clin 1990, 8(1):15-30. 2181263
65. Evans M Pollock AV A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection Br J Surg 1985, 72(4):256-260. 10.1002/bjs.1800720403 3986472
66. Evans M Pollock AV Trials on trial. A review of trials of antibiotic prophylaxis Arch Surg 1984, 119(1):109-113. 6689868
67. The Focus Project www.focusproject.org.uk The above URL is no longer active, for further information please contact the corresponding author.
68. Fowkes FG Fulton PM Critical appraisal of published research: introductory guidelines BMJ 1991, 302(6785):1136-1140. 1669795 2043787 10.1136/bmj.302.6785.1136
69. Gardner MJ Machin D Campbell MJ Use of check lists in assessing the statistical content of medical studies Br Med J (Clin Res Ed) 1986, 292(6523):810-812. 1339731 3082452 10.1136/bmj.292.6523.810
70. Gifford RH Feinstein AR A critique of methodology in studies of anticoagulant therapy for acute myocardial infarction N Engl J Med 1969, 280(7):351-357. 4883585
71. Gluud C Nikolova D Quality assessment of reports on clinical trials in the Journal of Hepatology J Hepatol 1998, 29(2):321-327. 10.1016/ S0168-8278(98)80021-4 9722217
72. Gotzsche PC Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis Control Clin Trials 1989, 10(1):31-56. 10.1016/ 0197-2456(89)90017-2 2702836
73. Grant A Reporting controlled trials Br J Obstet Gynaecol 1989, 96(4):397-400. 2751952
74. Grimes DA Schulz KF Methodology citations and the quality of randomized controlled trials in obstetrics and gynecology Am J Obstet Gynecol 1996, 174(4):1312-1315. 10.1016/S0002-9378(96)70677-4 8623862
75. Gross R Keller DR Appraising the evidence WMJ 1999, 98(2):44-48. 10235062
76. Gupta AK Ryder JE Bluhm R Johnson A Summerbell RC Onychomycosis: Quality of studies J Cutan Med Surg 2003, 7(4):312-316. 12879332
77. Guyatt GH Sackett DL Cook DJ Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group JAMA 1994, 271(1):59-63. 10.1001/jama.271.1.59 8258890
78. Guyatt GH Sackett DL Cook DJ Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group JAMA 1993, 270(21):2598-2601. 10.1001/jama.270.21.2598 8230645
79. Harris RP Helfand M Woolf SH Lohr KN Mulrow CD Teutsch SM Atkins D Methods Work Group, Third US Preventive Services Task Force Current methods of the US Preventive Services Task Force: A review of the process Am J Prev Med 2001, 20(3 Suppl) 21-35. 10.1016/ S0749-3797(01)00261-6 11306229
80. Essential Evidence Plus http://www.infopoems.com/product/ methods_validity.cfm#
81. Canadian Pharmacist's Letter http://www.pharmacistsletter.com
82. University of Glasgow http://www.gla.ac.uk/media/media_64037_en.pdf
83. Faculty of Medicine and Dentistry, University of Alberta http://www.ebm.med.ualberta.ca/therapy.pdf
84. Centre for Evidence Based Medicine http://www.cebm.net/index.aspx?o=1157
85. Ah-See KW Molony NC A qualitative assessment of randomized controlled trials in otolaryngology J Laryngol Otol 1998, 112(5):460-463. 10.1017/ S0022215100140770 9747475
86. Ajetunmobi O Critical appraisal of randomized clinical trials Making Sense of Critical Appraisal London: Arnold Ajetunmobi O 2002:122-145
87. Antczak AA Tang J Chalmers TC Quality assessment of randomized control trials in dental research. I. Methods J Periodontal Res 1986, 21(4):305-314. 10.1111/j.1600-0765.1986.tb01464.x 2942658
88. Avis M Reading research critically. II. An introduction to appraisal: assessing the evidence J Clin Nurs 1994, 3(5):271-277. 10.1111/ j.1365-2702.1994.tb00400.x 7834137
89. Balk EM Bonis PA Moskowitz H Schmid CH Ioannidis JP Wang C Lau J Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials JAMA 2002, 287(22):2973-2982. 10.1001/jama.287.22.2973 12052127
90. Bath FJ Owen VE Bath PM Quality of full and final publications reporting acute stroke trials: A systematic review Stroke 1998, 29(10):2203-2210. 9756604
91. Begg C Cho M Eastwood S Horton R Moher D Olkin I Pitkin R Rennie D Schulz KF Simel D Stroup DF Improving the quality of reporting of randomized controlled trials. The CONSORT statement JAMA 1996, 276(8):637-639. 10.1001/jama.276.8.637 8773637
92. Bigby M Gadenne AS Understanding and evaluating clinical trials J Am Acad Dermatol 1996, 34(4):555-90. quiz 591-3. 10.1016/ S0190-9622(96)80053-3 8601646
93. Bigby M Stern RS Bigby JA An evaluation of method reporting and use in clinical trials in dermatology Arch Dermatol 1985, 121(11):1394-1399. 10.1001/archderm.121.11.1394 4051526
94. Bland JM Jones DR Bennett S Cook DG Haines AP MacFarlane AJ Is the clinical trial evidence about new drugs statistically adequate? Br J Clin Pharmacol 1985, 19(2):155-160. 1463709 3872671
95. Brown CG Kelen GD Moser M Moeschberger ML Rund DA Methodology reporting in three acute care journals: Replication and reliability Ann Emerg Med 1985, 14(10):986-991. 10.1016/S0196-0644(85)80244-4 4037480
96. Brown CG Moser M Moeschberger M Kelen GD Controlled clinical trials in emergency medicine Am J Emerg Med 1985, 3(1):67-73. 10.1016/ 0735-6757(85)90016-6 3970757
97. Byers JF Beaudin CL Critical appraisal tools facilitate the work of the quality professional J Healthc Qual 2001, 23(5):35-8. 40-3. 11565170
98. Carneiro AV Critical appraisal of clinical studies: Practical tables Rev Port Cardiol 2005, 24(4):607-614. 15977782
99. cebmh http://cebmh.warne.ox.ac.uk/cebmh/downloads/education_resources/ overview.pdf
100. Chalmers I Adams M Dickersin K Hetherington J Tarnow-Mordi W Meinert C Tonascia S Chalmers TC A cohort study of summary reports of controlled trials JAMA 1990, 263(10):1401-1405. 10.1001/jama.263.10.1401 2304219
101. Chalmers TC Smith H Jr Blackburn B Silverman B Schroeder B Reitman D Ambroz A A method for assessing the quality of a randomized control trial Control Clin Trials 1981, 2(1):31-49. 10.1016/0197-2456(81)90056-8 7261638
102. Chan AW Altman DG Epidemiology and reporting of randomised trials published in PubMed journals Lancet 2005, 365(9465):1159-1162. 10.1016/ S0140-6736(05)71879-1 15794971
103. Chan KB Man-Son-Hing M Molnar FJ Laupacis A How well is the clinical importance of study results reported? An assessment of randomized controlled trials CMAJ 2001, 165(9):1197-1202. 81581 11706908
104. Cho MK Bero LA Instruments for assessing the quality of drug studies published in the medical literature JAMA 1994, 272(2):101-104. 10.1001/ jama.272.2.101 8015115
105. Colditz GA Miller JN Mosteller F How study design affects outcomes in comparisons of therapy. I: Medical Stat Med 1989, 8(4):441-454. 10.1002/ sim.4780080408 2727468
106. Schulz KF Subverting randomization in controlled trials JAMA 1995, 274(18):1456-1458. 10.1001/jama.274.18.1456 7474192
107. Noseworthy JH Ebers GC Vandervoort MK Farquhar RE Yetisir E Roberts R The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial Neurology 1994, 44:16-20. 8290055
108. Egger M Juni P Bartlett C Value of flow diagrams in reports of randomized controlled trials JAMA 2001, 285:1996-1999. 10.1001/ jama.285.15.1996 11308437
109. May GS Chir B Demets DL Friedman LM Furberg C Passamani E The randomized clinical trial: Bias in analysis Circulation 1981, 64:669-673. 7023743
110. Sackett DL Gent M Controversy in counting and attributing events in clinical trials New Eng J Med 1979, 301:1410-1412. 514321
111. Hasson F Keeney S McKenna H Research guidelines for the Delphi survey technique J Adv Nurs 2000, 32(4):1008-15. 10.1046/ j.1365-2648.2000.01567.x 11095242
112. Frost MH Reeve BB Liepa AM Stauffer JW Hays RD the Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health 2007, 10(Suppl 2):S94-S105. 10.1111/ j.1524-4733.2007.00272.x 17995479
113. Burns KEA Duffett M Kho ME Meade MO Adhikari NKJ Sinuff T Cook DJ Abenhaim L Sinuff T Cook DJ A. guide for the design and conduct of self-administered surveys of clinicians CMAJ 2008, 179(3):245-252. 2474876 18663204