Examiner alignment and assessment in clinical periodontal research

Periodontology 2000

Volumn 59, Issue 1, 2012, Pages 41-60

  Hefti, Arthur F ^a   Preshaw, Philip M ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CALIBRATION; DENTAL RESEARCH; EPIDEMIOLOGY; HUMAN; INFORMATION PROCESSING; OBSERVER VARIATION; PERIODONTICS; RANDOMIZED CONTROLLED TRIAL (TOPIC); REPRODUCIBILITY; STANDARD; STATISTICS;

BIAS (EPIDEMIOLOGY); CALIBRATION; DATA COLLECTION; DENTAL RESEARCH; HUMANS; OBSERVER VARIATION; PERIODONTAL INDEX; PERIODONTICS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REPRODUCIBILITY OF RESULTS;

EID: 84859834159     PISSN: 09066713     EISSN: 16000757     Source Type: Journal    
DOI: 10.1111/j.1600-0757.2011.00436.x     Document Type: Article

References (81)

1. ADA. Council on Dental Therapeutics. Guidelines for acceptance of chemotherapeutic products for the control of supragingival dental plaque and gingivitis. J Am Dent Assoc 1986: 112: 529-532.
2. Agresti A. A model for agreement between ratings on an ordinal scale. Biometrics 1988: 44: 539-548.
3. Agresti A. Modelling patterns of agreement and disagreement. Stat Methods Med Res 1992: 1: 201-218.
4. Ainamo J, Bay I. Problems and proposals for recording gingivitis and plaque. Int Dent J 1975: 25: 229-235.
5. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001: 134: 663-694.
6. Baker BO, Hardyck CD, Petrinovich LF. Weak measurements vs. strong statistics: an empirical critique of S. S. Stevens' proscriptions on statistics. Educ Psychol Meas 1966: 26: 291-309.
7. Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966: 19: 3-11.
8. Bartko JJ. Corrective note to the intraclass correlation coefficient as a measure of reliability. Psychol Rep 1974: 34: 418.
9. Becherer CF, Rateitschak KH, Hefti AF. Vergleichende Sondierung mit einer elektronischen und einer manuellen Parodontalsonde. Schweiz Monatsschr Zahnmed 1993: 103: 715-721.
10. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986: i: 307-310.
11. Chilton NW. Conference on clinical trials in periodontal diseases (May 21-23, 1985). J Clin Periodontol 1986: 13: 335-549.
12. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960: 20: 37-46.
13. Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 1968: 70: 213-220.
14. Cohen DW, Ship II. Clinical methods in periodontal diseases based on a conference held on May 20-23, 1967 at University of Pennsylvania School of Dental Medicine. J Periodontol 1967: 38: 571-795.
15. Cronbach LJ, Rajratnam N, Gleser GC. Theory of generalizability: a liberalization of reliability theory. Br J Statist Psych 1963: 16: 137-163.
16. Darroch JN, McCloud PI. Category distinguishability and observer agreement. Aust J Stat 1986: 28: 371-388.
17. Davies M, Fleiss JL. Measuring agreement for multinomial data. Biometrics 1982: 38: 1047-1051.
18. De Vet HCW. Observer reliability and agreement. In: Armitage P, Colton T (editors.), Encyclopedia Biostatistica, vol. 4. Chichester: John Wiley & Sons, Ltd., 1998: 3123-3128.
19. Donner A, Eliasziw M. Sample size requirements for reliability studies. Stat Med 1987: 6: 441-448.
20. Eliasziw M, Young SL, Woodbury MG, Fryday-Field K. Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example. Phys Ther 1994: 74: 777-788.
21. FDA. Guidance for Industry. Gingivitis: development and evaluation of drugs for treatment or prevention. Center for Drug Evaluation and Research. 2005. .
22. Fisher RA. Statistical Methods for Research Workers. New York: Hafner Publishing Company, 1973.
23. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971: 76: 378-382.
24. Fleiss JL, Chilton NW. The measurement of interexaminer agreement on periodontal disease. J Periodontal Res 1983: 18: 601-606.
25. Fleiss JL, Chilton NW. Conference on equivalency and superiority claims for products for gingivitis and periodontitis (May 13-16, 1991). J Periodontal Res 1992: 27: 305-456.
26. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973: 33: 613-619.
27. Fleiss JL, Levin B, Paik MC. Statistical Methods for Rates and Proportions, 3rd edn. Hoboken: John Wiley & Sons, Inc, 2003.
28. Fleiss JL, Park MH, Chilton NW, Alman JE, Feldman RS, Chauncey HH. Representativeness of the "Ramfjørd teeth" for epidemiologic studies of gingivitis and periodontitis. Community Dent Oral Epidemiol 1987: 15: 221-224.
29. Giraudeau B, Mary JY. Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Stat Med 2001: 20: 3205-3214.
30. Gjrup T. The kappa coefficient and the prevalence of a diagnosis. Method Inform Med 1988: 27: 184-186.
31. Haber M, Barnhart HX, Song J, Gruden J. Observer variability: a new approach in evaluating interobserver agreement. J Data Science 2005: 3: 69-83.
32. Hill EG, Slate EH, Wiegand RE, Grossi SG, Salinas CF. Study design for calibration of clinical examiners measuring periodontal parameters. J Periodontol 2006: 77: 1129-1141.
33. Hoyt WT, Kerns MD. Magnitude and moderators of bias in observer ratings: a meta-analysis. Psychol Methods 1999: 4: 403-424.
34. Imrey PB. Considerations in the statistical analysis of clinical trials in periodontitis. J Clin Periodontol 1986: 13: 517-532.
35. Imrey PB, Chilton NW, Pihlstrom BL, Proskin HM, Kingman A, Listgarten MA, Zimmerman SO, Ciancio SG, Cohen ME, D'Agostino RB, Fischman SL, Fleiss JL, Gunsolley JC, Kent RL, Killoy WJ, Laster LL, Marks RG, Varma AO. Recommended revisions to American Dental Association guidelines for acceptance of chemotherapeutic products for gingivitis control. J Periodontal Res 1994: 29: 299-304.
36. Kingman A, Albandar JM. Methodological aspects of epidemiological studies of periodontal diseases. Periodontol 2000 2002: 29: 11-30.
37. Klar N, Lipsitz SR, Ibrahim JG. An estimating equations approach for modelling kappa. Biometrical J 2001: 42: 45-58.
38. Klar N, Lipsitz SR, Parzen M, Leong T. An exact bootstrap confidence interval for κ in small samples. The Statistician 2002: 51: 467-478.
39. Kraemer HC, Periyakoil VS, Noda A. Kappa coefficients in medical research. Stat Med 2002: 21: 2109-2129.
40. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977: 33: 159-174.
41. Lang NP, Cullinan MP, Holborow DW, Heitz-Mayfield LJA. Examiner training: standardization and calibration in periodontal studies. In: Giannobile WV, editors, Clinical Research in Oral Health. Ames: John Wiley & Sons. 2010: 159-175.
42. Laster LL. The effect of subsampling sites within patients. J Periodontal Res 1985: 20: 91-96.
43. Laster LL. Discussion: analysis of data from clinical studies of localized juvenile periodontitis. J Clin Periodontol 1986: 13: 481-486.
44. Leuthesser L, Kohli CS, Harich KR. Brand equity: the halo effect measure. Eur J Marketing 1995: 29: 57-66.
45. Light RJ. Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol Bull 1971: 76: 365-377.
46. Lobene RR, Weatherford T, Ross NM, Lamm RA, Menaker L. A modified gingival index for use in clinical trials. Clin Prev Dent 1986: 8: 3-6.
47. Löe H, Silness J. Periodontal disease in pregnancy I. Prevalence and severity. Acta Odontol Scand 1963: 21: 533-551.
48. Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol 1987: 126: 161-169.
49. Marks RG, Conlon M, Ruberg SJ. Paradigm shifts in clinical trials enabled by information technology. Stat Med 2001: 20: 2683-2696.
50. Marks RG, Magnusson I, Taylor M, Clouser B, Maruniak J, Clark WB. Evaluation of reliability and reproducibility of dental indices. J Clin Periodontol 1993: 20: 54-58.
51. Massler M. The P-M-A index for the assessment of gingivitis. J Periodontol 1967: 38: 592-601.
52. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996: 1: 30-46.
53. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med 2001: 134: 657-662.
54. Molenberghs G, Fitzmaurice GM, Lipsitz SR. Efficient estimation of the intraclass correlation for a binary trait. J Agr Biol Envir St 1996: 1: 78-96.
55. Mühlemann HR, Mazor ZS. Gingivitis in Zürich school children. Helv Odontol Acta 1958: 2: 3-12.
56. Nelson JC, Pepe MS. Statistical description of interrater variability in ordinal ratings. Stat Methods Med Res 2000: 9: 475-496.
57. Oden NL. Estimating kappa from binocular data. Stat Med 1991: 10: 1303-1311.
58. Page RC, Armitage GC, DeRouen TA, Genco RJ, Hujoel PP, Jeffcoat MK, Kornman KS, Williams RC. Design and Conduct of Clinical Trials of Products Designed for the Prevention, Diagnosis and Therapy of Periodontitis. Chicago: American Academy of Periodontology, 1995.
59. Perkins SM, Becker MP. Assessing rater agreement using marginal association models. Stat Med 2002: 21: 1743-1760.
60. Polson AM. The research team, calibration, and quality assurance in clinical trials in periodontics. Ann Periodontol 1997: 2: 75-82.
61. Posner KL, Sampson PD, Caplan RA, Ward RJ, Cheney FW. Measuring interrater reliability among multiple raters: an example of methods for nominal data. Stat Med 1990: 9: 1103-1115.
62. Ramfjord SP. The Periodontal Disease Index (PDI). J Periodontol 1967: 38: 602-610.
63. Reichenheim ME. Confidence intervals for the kappa statistic. Stata J 2004: 4: 421-428.
64. Roethlisberger FJ, Dickson WJ. Management and the Worker. Cambridge: Harvard University Press, 1939.
65. Rosenthal R. Experimenter Effects in Behavioral Research. New York: Appleton-Century-Crofts, 1966.
66. Saal FE, Landy FJ. The mixed standard rating scale: an evaluation. Organ Behav Hum Dec 1977: 18: 19-35.
67. Saxer UP, Mühlemann HR. Motivation und Aufklärung. Schweiz Monatsschr Zahnheilk 1975: 85: 905-919.
68. Schour I, Massler M. Gingival disease in postwar Italy (1945). I. Prevalence of gingivitis in various age groups. J Am Dent Assoc 1947: 35: 475-482.
69. Schour I, Massler M. Prevalence of gingivitis in young adults (abstract). J Dent Res 1948: 27: 733-734.
70. Shoukri MM. Measures of Interobserver Agreement. Boca Raton: Chapman and Hall/CRC, 2004.
71. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res 1998: 7: 301-317.
72. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979: 86: 420-428.
73. Silness J, Löe H. Periodontal disease in pregnancy II. Correlation between oral hygiene and periodontal condition. Acta Odontol Scand 1964: 22: 121-135.
74. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005: 85: 257-268.
75. Stratford JM. Methods of assessing gingival and periodontal disease: a review. Int J Epidemiol 1975: 4: 79-86.
76. Thorndike EL. A constant error in psychological ratings. J Appl Psychol 1920: 4: 25-29.
77. Turesky S, Gilmore N, Glickman I. Reduced plaque formation by the chloromethyl analogue of victamin C. J Periodontol 1970: 41: 41-43.
78. Valet F, Guinot C, Mary JY. Log-linear non-uniform association models for agreement between two ratings on an ordinal scale. Stat Med 2007: 26: 647-662.
79. Vangeneugden T, Laenen A, Geys H, Renard D, Molenberghs G. Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics 2005: 61: 295-304.
80. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med 1998: 17: 101-110.
81. Yang MCK, Marks RG, Magnusson I, Clouser B, Clark WB. Reproducibility of an electronic probe in relative attachment level measurements. J Clin Periodontol 1992: 19: 541-548.