How reliable are reliability studies of fracture classifications? A systematic review of their methodologies

Acta Orthopaedica Scandinavica

Volumn 75, Issue 2, 2004, Pages 184-194

  Audigé, Laurent ^a   Bhandari, Mohit ^b   Kellam, James ^c  

^a AO Center   (Switzerland)

^b MCMASTER UNIVERSITY MEDICAL CENTRE   (Canada)

^c CAROLINAS MEDICAL CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA BASE; DISEASE CLASSIFICATION; EMBASE; FRACTURE; HUMAN; MEDLINE; METHODOLOGY; QUANTITATIVE ANALYSIS; RATING SCALE; RELIABILITY; REVIEW; STATISTICAL ANALYSIS; VALIDATION PROCESS;

FRACTURES, BONE; HUMANS; METHODS; REPRODUCIBILITY OF RESULTS;

EID: 2342505181     PISSN: 00016470     EISSN: None     Source Type: Journal    
DOI: 10.1080/00016470412331294445     Document Type: Review

References (63)

1. Altman D G. Inter-rater agreement. In: Practical statistics for medical research. Chapman & Hall / CRC London 1990: 403-9.
2. Andersen D J, Blair W F, Steyers C M, Jr., Adams B D, el Khouri G Y, Brandser E A. Classification of distal radius fractures: an analysis of interobserver reliability and intraobserver reproducibility. J Hand Surg (Am) 1996; 21 (4): 574-82.
3. Andersen E, Jorgensen L G, Hededam L T. Evans' classification of trochanteric fractures: an assessment of the interobserver and intraobserver reliability. Injury 1990; 21 (6): 377-8.
4. Andersen G R, Rasmussen J B, Dahl B, Solgaard S. Older's classification of Colles' fractures. Good intraobserver and interobserver reproducibility in 185 cases. Acta Orthop Scand 1991; 62 (5): 463-4.
5. Barton K L, Kaminsky C K, Green D W, Shean C J, Kautz S M, Skaggs D L. Reliability of a modified Gartland classification of supracondylar humerus fractures. J Pediatr Orthop 2001; 21 (1): 27-30.
6. Bernstein J, Adler L M, Blank J E, Dalsey R M, Williams G R, Iannotti J P. Evaluation of the Neer system of classification of proximal humeral fractures with computerized tomographic scans and plain radiographs. J Bone Joint Surg (Am) 1996; 78 (9): 1371-5.
7. Bernstein J, Monaghan B A, Silber J S, DeLong W G. Taxonomy and treatment - a classification of fracture classifications. J Bone Joint Surg (Br) 1997; 79 (5): 706-7.
8. Bjorgul K, Reikeraos O. Low interobserver reliability of radiographic signs predicting healing disturbance in displaced intracapsular fracture of the femoral neck. Acta Orthop Scand 2002; 73 (3): 307-10.
9. Bland J M, Altman D G. Statistics notes: Validating scales and indexes. BMJ 2002; 324 (7337): 606-7.
10. Blauth M, Bastian L, Knop C, Lange U, Tusch G. [Interobserver reliability in the classification of thoraco-lumbar spinal injuries]. Orthopäde 1999; 28 (8): 662-81.
11. Blundell C M, Parker M J, Pryor G A, Hopkinson-Woolley J, Bhonsle S S. Assessment of the AO classification of intracapsular fractures of the proximal femur. J Bone Joint Surg (Br) 1998; 80 (4): 679-83.
12. Brady O H, Garbuz D S, Masri B A, Duncan C P. The reliability and validity of the Vancouver classification of femoral fractures after hip replacement. J Arthroplasty 2000; 15 (1): 59-62.
13. Brage M E, Rockett M, Vraney R, Anderson R, Toledano A. Ankle fracture classification: a comparison of reliability of three X-ray views versus two. Foot Ankle Int 1998; 19 (8): 555-62.
14. Brien H, Noftall F, MacMaster S, Cummings T, Landells C, Rockwood P. Neer's classification system: a critical appraisal. J Trauma 1995; 38 (2): 257-60.
15. Brorson S, Bagger J, Sylvest A, Hobjartsson A. Improved interobserver variation after training of doctors in the Neer system. A randomised trial. J Bone Joint Surg (Br) 2002; 84 (7): 950-4.
16. Browner B D, Jupiter J B, Levine A M, Trafton P G. Skeletal trauma - fractures, dislocations, ligamentous injuries. W.B. Saunders Company, A Division of Harcourt Brace & Company Philadelphia, London, Toronto, Montreal, Sydney, Tokyo 1998.
17. Burstein A H. Fracture classification systems: do they work and are they useful? J Bone Joint Surg (Am) 1993; 75 (12): 1743-4.
18. Chan P S, Klimkiewicz J J, Luchetti W T, Esterhai J L, Kneeland J B, Dalinka M K, Heppenstall R B. Impact of CT scan on treatment plan and fracture classification of tibial plateau fractures. J Orthop Trauma 1997; 11 (7): 484-9.
19. Colton C L. Telling the bones. J Bone Joint Surg (Br) 1991; 73 (3): 362-4.
20. Cook R J. Kappa and its dependence on marginal rates. In: Encyclopedia of biostatistics (Eds. Armitage P and Colton T). Wiley New York 1998: 2166-8.
21. Craig W L, III, Dirschl D R. Effects of binary decision making on the classification of fractures of the ankle. J Orthop Trauma 1998; 12 (4): 280-3.
22. Dirschl D R, Adams G L. A critical assessment of factors influencing reliability in the classification of fractures, using fractures of the tibial plafond as a model. J Orthop Trauma 1997; 11 (7): 471-6.
23. Fleiss J L. Statistical methods for rates and proportions, 2nd ed. In: John Wiley & Sons New York 1981: 217-8.
24. Flinkkilä T, Nikkola-Sihto A, Kaarela O, Paakko E, Raatikainen T. Poor interobserver reliability of AO classification of fractures of the distal radius. Additional computed tomography is of minor value. J Bone Joint Surg (Br) 1998; 80 (4): 670-2.
25. Frandsen P A, Andersen E, Madsen F, Skjodt T. Garden's classification of femoral neck fractures. An assessment of inter-observer variation. J Bone Joint Surg (Br) 1988; 70 (4): 588-90.
26. Gehrchen P M, Nielsen J O, Olesen B. Poor reproducibility of Evans' classification of the trochanteric fracture. Assessment of 4 observers in 52 cases. Acta Orthop Scand 1993; 64 (1): 71-2.
27. Gehrchen P M, Nielsen J O, Olesen B, Andresen B K. Seinsheimer's classification of subtrochanteric fractures. Poor reproducibility of 4 observers' evaluation of 50 cases. Acta Orthop Scand 1997; 68 (6): 524-6.
28. Gjørup T. The kappa coefficient and the prevalence of a diagnosis. Methods Inf Med 1988; 27 (4): 184-6.
29. Illarramendi A, Gonzalez D, V, Segal E, De Carli P, Maignon G, Gallucci G. Evaluation of simplified Frykman and AO classifications of fractures of the distal radius. Assessment of interobserver and intraobserver agreement. Int Orthop 1998; 22 (2): 111-5.
30. Johnstone D J, Radford W J, Parnell E J. Interobserver variation using the AO/ASIF classification of long bone fractures. Injury 1993; 24 (3): 163-5.
31. Kreder H J, Hanel D P, McKee M, Jupiter J, McGillivary G, Swiontkowski M F. Consistency of AO fracture classification for the distal radius. J Bone Joint Surg (Br) 1996; 78 (5): 726-31.
32. Kristiansen B, Andersen U L, Olsen C A, Varmarken J E. The Neer classification of fractures of the proximal humerus. An assessment of interobserver variation. Skeletal Radiol 1988; 17 (6): 420-2.
33. Landis J R, Koch G G. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (1): 159-74.
34. Lichtenhahn P, Fernandez D L, Schatzker J. [Analysis of the "user friendliness" of the AO classification of fractures]. Helv Chir Acta 1992; 58 (6): 919-24.
35. Lindsjö U. Classification of ankle fractures: the Lauge-Hansen or AO system? Clin Orthop 1985; (199): 12-6.
36. Martin J S, Marsh J L. Current classification of fractures. Rationale and utility. Radiol Clin North Am 1997; 35 (3): 491-506.
37. Martin J S, Marsh J L, Bonar S K, DeCoster T A, Found E M, Brandser E A. Assessment of the AO/ASIF fracture classification for the distal tibia. J Orthop Trauma 1997; 11 (7): 477-83.
38. McAdams T R, Blevins F T, Martin T P, DeCoster T A. The role of plain films and computed tomography in the evaluation of scapular neck fractures. J Orthop Trauma 2002; 16 (1): 7-11.
39. Morgan S J, Groshen S L, Itamura J M, Shankwiler J, Brien W W, Kuschner S H. Reliability evaluation of classifying radial head fractures by the system of Mason. Bull Hosp Jt Dis 1997; 56 (2): 95-8.
40. Nielsen J O, Dons-Jensen H, Sorensen H T. Lauge-Hansen classification of malleolar fractures. An assessment of the reproducibility in 118 cases. Acta Orthop Scand 1990; 61 (5): 385-7.
41. Oner F C, Ramos L M, Simmermacher R K, Kingma P T, Diekerhof C H, Dhert W J, Verbout A J. Classification of thoracic and lumbar spine fractures: problems of reproducibility. A study of 53 patients using CT and MRI. Eur Spine J 2002; 11 (3): 235-45.
42. Oskam J, Kingma J, Klasen H J. Interrater reliability for the basic categories of the AO/ASIF's system as a frame of reference for classifying distal radial fractures. Percept Mot Skills 2001; 92 (2): 589-94.
43. Pervez H, Parker M, Pryor G, Lutchman L, Chirodian N. Classification of trochanteric fracture of the proximal femur: a study of the reliability of current systems. Injury 2002; 33 (8): 713-5.
44. Pewsner D, Battaglia M, Bucher H, Minder C, Grossenbacher F, Egger M. The Bayes Library of Diagnostic Studies and Reviews. Basel Institute for Clinical Epidemiology, Institute for Social and Preventive Medicine University of Berne Basel, Berne 2002.
45. Rasmussen S, Madsen P V, Bennicke K. Observer variation in the Lauge-Hansen classification of ankle fractures. Precision improved by instruction. Acta Orthop Scand 1993; 64 (6): 693-4.
46. Rockwood C A, Green D P, Bucholz R W, Heckman J D. Rockwood and Green's Fractures in Adults. Lippincott-Raven Philadelphia, New York 1996.
47. Schipper I B, Steyerberg E W, Castelein R M, van Vugt A B. Reliability of the AO/ASIF classification for pertrochanteric femoral fractures. Acta Orthop Scand 2001; 72 (1): 36-41.
48. Shrout P E. Measurement reliability and agreement in psychiatry. Stat Methods Med Res 1998; 7 (3): 301-17.
49. Sidor M L, Zuckerman J D, Lyon T, Koval K, Cuomo F, Schoenberg N. The Neer classification system for proximal humeral fractures. An assessment of interobserver reliability and intraobserver reproducibility. J Bone Joint Surg (Am) 1993; 75 (12): 1745-50.
50. Siebenrock K A, Gerber C. The reproducibility of classification of fractures of the proximal end of the humerus. J Bone Joint Surg (Am) 1993; 75 (12): 1751-5.
51. Silber J S, Flynn J M, Katz M A, Ganley T J, Koffler K M, Drummond D S. Role of computed tomography in the classification and management of pediatric pelvic fractures. J Pediatr Orthop 2001; 21 (2): 148-51.
52. Sjödén G O, Movin T, Guntner P, Aspelin P, Ahrengart L, Ersmark H, Sperber A. Poor reproducibility of classification of proximal humeral fractures. Additional CT of minor value. Acta Orthop Scand 1997; 68 (3): 239-42.
53. Sjödén G O, Movin T, Aspelin P, Guntner P, Shalabi A. 3D-radiographic analysis does not improve the Neer and AO classifications of proximal humeral fractures. Acta Orthop Scand 1999; 70 (4): 325-8.
54. Svanholm H, Starklint H, Gundersen H J, Fabricius J, Barlebo H, Olsen S. Reproducibility of histomorphologic diagnoses with special reference to the kappa statistic. APMIS 1989; 97 (8): 689-98.
55. Swiontkowski M F, Sands A K, Agel J, Diab M, Schwappach J R, Kreder H J. Interobserver variation in the AO/OTA fracture classification system for pilon fractures: is there a problem? J Orthop Trauma 1997; 11 (7): 467-70.
56. Thomsen N O, Overgaard S, Olsen L H, Hansen H, Nielsen S T. Observer variation in the radiographic classification of ankle fractures. J Bone Joint Surg (Br) 1991; 73 (4): 676-8.
57. Thomsen N O, Jensen C M, Skovgaard N, Pedersen M S, Pallesen P, Soe-Nielsen N H, Rosenklint A. Observer variation in the radiographic classification of fractures of the neck of the femur using Garden's system. Int Orthop 1996; 20 (5): 326-9.
58. Thomsen N O, Olsen L H, Nielsen S T. Kappa statistic in the assessment of observer variation: the significance of multiple observers classifying ankle fractures. J Orthop Sci 2002; 7 (2): 163-6.
59. Uebersax J. Kappa Coefficients. http://ourworld.compuserve.com/homepages/ jsuebersax/kappa.htm2002a.
60. Uebersax J. Statistical Methods for Rater Agreement. http://ourworld. compuserve.com/homepages/jsuebersax/agree.htm2002b.
61. Visutipol B, Chobtangsin P, Ketmalasiri B, Pattarabanjird N, Varodompun N. Evaluation of Letournel and Judet classification of acetabular fracture with plain radiographs and three-dimensional computerized tomographic scan. J Orthop Surg 2000; 8 (1): 33-7.
62. Wainwright A M, Williams J R, Carr A J. Interobserver and intraobserver variation in classification systems for fractures of the distal humerus. J Bone Joint Surg (Br) 2000; 82 (5): 636-42.
63. Yacoubian S V, Nevins R T, Sallis J G, Potter H G, Lorich D G. Impact of MRI on treatment plan and fracture classification of tibial plateau fractures. J Orthop Trauma 2002; 16 (9): 632-7.