Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers

Journal of Clinical Epidemiology

Volumn 66, Issue 9, 2013, Pages 982-993

  Hartling, Lisa ^a   Milne, Andrea ^a   Hamm, Michele P ^a   Vandermeer, Ben ^a   Ansari, Mohammed ^b   Tsertsvadze, Alexander ^b   Dryden, Donna M ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b OTTAWA HOSPITAL RESEARCH INSTITUTE   (Canada)

Author keywords

Cohort studies; Internal validity; Methodological quality; Reliability; Systematic reviews; Validity

Indexed keywords

ARTICLE; COHORT ANALYSIS; FOLLOW UP; INTERRATER RELIABILITY; KAPPA STATISTICS; META ANALYSIS (TOPIC); NEWCASTLE OTTAWA SCALE; PRIORITY JOURNAL; RATING SCALE;

COHORT STUDIES; INTERNAL VALIDITY; METHODOLOGICAL QUALITY; RELIABILITY; SYSTEMATIC REVIEWS; VALIDITY;

COHORT STUDIES; HUMANS; META-ANALYSIS AS TOPIC; OBSERVER VARIATION; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; REVIEW LITERATURE AS TOPIC;

EID: 84882846134     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2013.03.003     Document Type: Article

References (31)

1. P.T. Higgins, and S. Green Cochrane Handbook for Systematic Reviews of Interventions 2009 The Cochrane Collaboration London, UK 5.0.2 [updated September 2009]
2. H. Bastian, P. Glasziou, and I. Chalmers Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7 9 2010 e1000326
3. D. Moher, A.R. Jadad, G. Nichol, M. Penman, P. Tugwell, and S. Walsh Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists Control Clin Trials 16 1995 62 73
4. P. Juni, A. Witschi, R. Bloch, and M. Egger The hazards of scoring the quality of clinical trials for meta-analysis JAMA 282 1999 1054 1060
5. S. West, V. King, T.S. Carey, K.N. Lohr, N. McKoy, and S.F. Sutton Systems to rate the strength of scientific evidence Evid Rep Technol Assess (Summ) 47 2002 1 11
6. S.A. Olivo, L.G. Macedo, I.C. Gadotti, J. Fuentes, T. Stanton, and D.J. Magee Scales to assess the quality of randomized controlled trials: a systematic review Phys Ther 88 2008 156 175
7. Wells G, Shea B, O'Connell J, Robertson J, Peterson V, Welch V, et al. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analysis. Available at http://www.ohri.ca/programs/clinical- epidemiology/oxfordasp. Accessed April 13, 2013.
8. Wells G, Shea B, O'Connell J, Robertson J, Peterson V, Welch V, et al. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analysis [abstract]. Oxford, United Kingdom, July 3-5, 2000.
9. S.H. Downs, and N. Black The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions J Epidemiol Community Health 52 1998 377 384
10. Wells G, Brodsky L, O'Connell D, Robertson J, Peterson V, Welch V, et al. Evaluation of the Newcastle-Ottawa Scale (NOS): an assessment tool for evaluating the quality of non-randomized studies [abstract]. Barcelona, Spain, October 26-31, 2003.
11. S. Ip, M. Chung, G. Raman, P. Chew, N. Magula, and D. DeVine Breastfeeding and maternal and infant health outcomes in developed countries Evid Rep Technol Assess (Full Rep) 153 2007 1 186
12. F.A. McAlister, J. Ezekowitz, D.M. Dryden, N. Hooton, B. Vandermeer, and C. Friesen Cardiac resynchronization therapy and implantable cardiac defibrillators in left ventricular systolic dysfunction Evid Rep Technol Assess (Full Rep) 152 2007 1 199
13. P.L. Santaguida, C. Balion, D. Hunt, K. Morrison, H. Gerstein, and P. Raina Diagnosis, prognosis, and treatment of impaired glucose tolerance and impaired fasting glucose Evid Rep Technol Assess 128 2005 1 11
14. M. Egger, P. Juni, C. Bartlett, F. Holenstein, and J. Sterne How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study Health Technol Assess 7 2003 1 76
15. T.A. Furukawa, N. Watanabe, I.M. Omori, V.M. Montori, and G.H. Guyatt Association between unreported outcomes and effect size estimates in Cochrane meta-analyses JAMA 297 2007 468 470
16. R. Norman, L. Masters, C. Milner, J. Wang, and M. Davies Relative risk of conversion from normoglycaemia to impaired glucose tolerance or non-insulin dependent diabetes mellitus in polycystic ovarian syndrome Hum Reprod 16 2001 1995 1998
17. A. Wright, C. Holberg, L. Taussig, and F. Martinez Factors influencing the relation of infant feeding to asthma and recurrent wheeze in childhood Thorax 56 3 2001 192 197
18. Z. Arvanitakis, J.A. Schneider, R.S. Wilson, J.L. Bienias, J.F. Kelly, and D.A. Evans Statins, incident Alzheimer disease, change in cognitive function, and neuropathology Neurology 70 19 Pt 2 2008 1795 1802
19. A. Liebetrau Measures of association 1983 Sage Publications Newbury Park, CA
20. A. Agresti Categorical data analysis 2002 John Wiley & Sons New York, NY
21. J.R. Landis, and G.G. Koch The measurement of observer agreement for categorical data Biometrics 33 1977 159 174
22. J.A. Sterne, P. Juni, K.F. Schulz, D.G. Altman, C. Bartlett, and M. Egger Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research Stat Med 21 2002 1513 1524
23. S.D. McDonald, Z. Han, S. Mulla, and J. Beyene Overweight and obesity in mothers and risk of preterm birth and low birth weight infants: systematic review and meta-analyses BMJ 341 2010 c3428
24. D.D. Alexander, P.J. Mink, C.A. Cushing, and B. Sceurman A review and meta-analysis of prospective studies of red and processed meat intake and prostate cancer Nutr J 9 2010 50
25. N.K. Grote, J.A. Bridge, A.R. Gavin, J.L. Melville, S. Iyengar, and W.J. Katon A meta-analysis of depression during pregnancy and the risk of preterm birth, low birth weight, and intrauterine growth restriction Arch Gen Psychiatry 67 2010 1012 1024
26. K.R. Jacobson, D.B. Tierney, C.Y. Jeon, C.D. Mitnick, and M.B. Murray Treatment outcomes among patients with extensively drug-resistant tuberculosis: systematic review and meta-analysis Clin Infect Dis 51 2010 6 14
27. S. Janda, A. Young, J.M. Fitzgerald, M. Etminan, and J. Swiston The effect of statins on mortality from severe infections and sepsis: a systematic review and meta-analysis J Crit Care 25 4 2010 656 722
28. A. Stang Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses Eur J Epidemiol 25 9 2010 603 605
29. Schunemann HJ, Brozek J, Oxman AD. GRADE handbook for grading quality of evidence and strength of recommendation. Version 3.2 [update March 2009]. The Grade Working Group. Available at http://www.gradeworkinggroup.org/index.htm; Accessed April 13, 2013.
30. P.T. Higgins, and S. Green Cochrane Handbook for Systematic Reviews of Interventions 2011 The Cochrane Collaboration London, UK 5.1.0. [updated March 2011]
31. Berkman ND, Viswanathan M. Development of a tool to evaluate the quality of non-randomized studies of interventions or exposures. Bethesda, MD, September 15, 2009. Available at http://ahrq.gov/legacy/about/annualconf09/berkman- viswanathan/berkman-viswanathan.ppt.