Fracture risk assessment: State of the art, methodologically unsound, or poorly reported?

Current Osteoporosis Reports

Volumn 10, Issue 3, 2012, Pages 199-207

  Collins, Gary S ^a   Michaëlsson, Karl ^b  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b UPPSALA UNIVERSITY   (Sweden)

Author keywords

Calibration; Clinical prediction models; Discrimination; Fracture risk assessment; FRAX; Garvan; Hip fracture; Osteoporosis; QFracture; Validation

Indexed keywords

ADULT; AGED; ANALYTIC METHOD; ARTICLE; BONE DENSITY; BOOTSTRAPPING; CLINICAL PRACTICE; CONTROLLED STUDY; FEMALE; FRAGILITY FRACTURE; FRAX; GARVAN; HUMAN; MAJOR CLINICAL STUDY; MALE; MATHEMATICAL MODEL; METHODOLOGY; PREDICTION; QFRACTURE; RECEIVER OPERATING CHARACTERISTIC; RISK ASSESSMENT; RISK FACTOR; VALIDATION STUDY;

HIP FRACTURES; HUMANS; OSTEOPOROTIC FRACTURES; PROGNOSIS; REVIEW LITERATURE AS TOPIC; RISK ASSESSMENT; RISK FACTORS; VALIDATION STUDIES AS TOPIC;

EID: 84872190358     PISSN: 15441873     EISSN: 15442241     Source Type: Journal    
DOI: 10.1007/s11914-012-0108-1     Document Type: Article

References (100)

1. Johnell O, Kanis JA. An estimate of the worldwide prevalence and disability associated with osteoporotic fractures. Osteoporos Int. 2006;17(12):1726-33.
2. Kanis JA, Oden A, McCloskey EV, Johansson H, Wahl DA, Cooper C. A systematic review of hip fracture incidence and probability of fracture worldwide. Osteoporos Int. 2012;in press.
3. Cummings SR, Melton LJ. Epidemiology and outcomes of osteoporotic fractures. Lancet. 2002;359:1761-67.
4. Kanis JA, Johnell O. Requirements for DXA for the management of osteoporosis in Europe. Osteoporos Int. 2005;16:229-38.
5. Feinstein AR. Clinical judgment revisited: the distraction of quantitative models. Ann Intern Med. 1994;120:799-805.
6. Meads C, Ahmed I, Riley RD. A systematic review of breast cancer incidence risk prediction models with meta-Analysis of their performance. Breast Canc Res Treat. 2012;132(2):365-77.
7. Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat. 1998;52(1-3):289-303.
8. Mallett S, Royston P, Dutton S, Waters R, Altman DG. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010;8:20.
9. Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103. Highlights the general poor level of methodological conduct and reporting in studies developing clinical prediction models.
10. Perel P, Edwards P, Wentz R, Roberts I. Systematic review of prognostic models in traumatic brain injury. BMC Med Inform Decis Mak. 2006;6:38.
11. Steurer J, Haller C, Hauselmann H, Brunner F, Bachmann LM. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One. 2011;6(5):e19994.
12. Kanis JA, Johnell O, Oden A, Johansson H, McCloskey E. FRAX and the assessment of fracture probability in men and women from the UK. Osteoporos Int. 2008;19(4):385-97.
13. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: Validating a prognostic model. BMJ. 2009;338:b605. Provides an excellent overview of the general issues in evaluating the performance of clinical prediction models.
14. Moons KGM, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009;338:b606. Describes the issues in evaluating impact of clinical prediciton models on clinician behavior and patient outcomes.
15. Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338(feb23 1):b375-b75.
16. Royston P, Moons KGM, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ 2009;338(mar31 1):b604-b04.
17. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201-9. (Pubitemid 46780691)
18. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2008.
19. Vickers AJ. Prediction models in cancer care. CA: a cancer journal for clinicians. 2011;61(5):315-26.
20. Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010;37(1):31-8. Describes a new method to evalaute the clinical usefulness of a clinical prediction model without imposing one single treatment threshold.
21. Wallace E, Smith SM, Perera-Salazar R, Vaucher P, McCowan C, Collins G, et al. Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs). BMC Med Inform Decis Mak. 2011;11:62.
22. Toll DB, Janssen KJM, Vergouwe Y, Moons KGM. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085-94.
23. Moons KG, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio) marker. Heart. 2012;Mar 7([Epub ahead of print]).
24. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453-73.
25. Bleeker SE, Moll HA, Steyerberg EW, Donders ART, Derksen-Lubsen G, Grobbee DE, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826-32.
26. Peek N, Arts DG, Bosman RJ, van der Voort PH, de Keizer NF. External validation of prognostic models for critically ill patients required substantial sample sizes. J Clin Epidemiol. 2007;60 (5):491-501.
27. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012.
28. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-38. Methodological experts in clinical prediction give an overview of traditional and new methods to evaluate clinical prediction models.
29. Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298-301.
30. Collins GS, Altman DG. An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ. 2010;340:c2442.
31. Mallett S, Royston P, Waters R, Dutton S, Altman DG. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21.
32. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565-74.
33. Gelman A. Scaling regression inputs by dividing by two standard deviations. Stat Med. 2008;27(15):2865-73.
34. Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64(12):1463-64.
35. Peduzzi P, Concato J, Feinsten AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis.2. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48(12):1503-12.
36. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373-79. (Pubitemid 27028925)
37. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165(6):710-8. (Pubitemid 46348374)
38. Vergouwe Y, Steyerberg EW, Eijkemans MJC, Habbema JDF. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58(5):475-83.
39. Ettema RG, Peelen LM, Schuurmans MJ, Nierich AP, Kalkman CJ, Moons KG. Prediction models for prolonged intensive care unit stay after cardiac surgery: systematic review and validation study. Circulation. 2010;122(7):682-89.
40. Burton A, Altman DG. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. Br J Cancer. 2004;91(1):4-8. (Pubitemid 39013036)
41. Janssen KJ, Donders AR, Harrell Jr FE, Vergouwe Y, Chen Q, Grobbee DE, et al. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol. 2010;63(7):721-7.
42. Janssen KJM, Vergouwe Y, Donders ART, Harrell Jr FE, Chen Q, Grobbee DE, et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem. 2009;55(5):994-1001.
43. Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Meth. 2009;9:57.
44. Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data withing prognostic modelling studies: a simulation study. BMC Med Res Meth. 2010;10:7.
45. Moons KG, Donders RA, Stijnen T, Harrell Jr FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59(10):1092-101.
46. Vergouwe Y, Royston P, Moons KGM, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol. 2010;63(2):205-14.
47. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30(4):377-99.
48. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338 (jun29 1):b2393-b93.
49. Kanis JA, Oden A, Johnell O, Johansson H, De Laet C, Brown J, et al. The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18(8):1033-46. This article describes the development of FRAX and the indivdual studies used in the development and validation of FRAX.
50. National Osteoporosis Foundation. Clinician's guide to prevention and treatment of osteoporosis (available at http://www.nof. org/sites/default/files/ pdfs/NOF-ClinicianGuide2009-v7.pdf). 2010.
51. National Osteoporosis Guideline Group (NOGG). Osteoporosis: clinical guidelines for prevention and treatment. Executive Summary. 2010.
52. Papaioannou A, Morin S, Cheung AM, Atkinson S, Brown JP, Feldman S, et al. 2010 clinical practice guidelines for the diagnosis and management of osteoporosis in Canada: summary. CMAJ. 2010;182(17):1864-73.
53. Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Intern Med. 2007;146(6):450-53. A discussion on the importance of reproducible research.
54. Peng RD. Reproducible research and biostatistics. Biostatistics. 2009;10(3):405-08.
55. Mayor S. Ethics code for professional medical writers emphasises transparency and completeness of research reporting. BMJ. 2010;341:c7025.
56. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24.
57. Simera I, Altman DG. Writing a research article that is "fit for purpose": EQUATOR Network and reporting guidelines. Evid Based Med. 2009;14(5):132-34.
58. Steurer J, Haller C, Häuselmann H, Brunner F, Bachmann LM. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One 2011;6(5):e19994. Systematic review of clinical prediction model for osteoporotic fracture, highlighting weaknesses in the development and validation of prediction models.
59. Kanis JA, Oden A, Johansson H, McCloskey E. Pitfalls in the external validation of FRAX. Osteoporos Int. 2012;23(2):423-31.
60. Collins GS, Mallett S, Altman DG. Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores. BMJ. 2011;342: d3651.
61. Siontis GC, Tzoulaki I, Ioannidis JP. Predicting death: an empirical evaluation of predictive tools for mortality. Arch Intern Med. 2011;171(19):1721-6.
62. Silverman SL, Calderon AD. The utility and limitations of FRAX: a US perspective. Curr Osteoporos Rep. 2010;8(4):192-7.
63. van Geel TA, van den Bergh JP, Dinant GJ, Geusens PP. Individualizing fracture risk prediction. Maturitas. 2010;65(2):143-8.
64. Sennerby U, Melhus H, Gedeborg R, Byberg L, Garmo H, Ahlbom A, et al. Cardiovascular diseases and risk of hip fracture. JAMA. 2009;302(15):1666-73.
65. Leslie WD, Lix LM, Johansson H, Oden A, McCloskey E, Kanis JA. Independent clinical validation of a Canadian FRAX tool: fracture prediction and model calibration. J Bone Miner Res. 2010;25(11):2350-8.
66. Hippisley-Cox J, Coupland C. Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractureScores. BMJ. 2009;339: b4229. Describes the development and internal validation of QFracture in a very large cohort of general practice patients in the United Kingdom.
67. Cummins NM, Poku EK, Towler MR, O'Driscoll OM, Ralston SH. clinical risk factors for osteoporosis in Ireland and the UK: a comparison of FRAX and QFractureScores. Calcif Tissue Int. 2011;89(2):172-77.
68. Hippisley-Cox J, Coupland C. Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective cohort study. BMJ. 2012;344:e3427.
69. Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV. Development of a nomogram for individualizing hip fracture risk in men and women. Osteoporos Int. 2007;18(8):1109-17. Describes the development of the Garvan prediction model.
70. Nguyen ND, Frost SA, Center JR, Eisman JA, Nguyen TV. Development of prognostic nomograms for individualizing 5-year and 10-year fracture risks. Osteoporos Int. 2008;19 (10):1431-44.
71. Langsetmo L, Nguyen TV, Nguyen ND, Kovacs CS, Prior JC, Center JR, et al. Independent external validation of nomograms for predicting risk of low-Trauma fracture and hip fracture. CMAJ. 2011;183(2):E107-E14.
72. Pluskiewicz W, Adamczyk P, Franek E, Leszczynski P, Sewerynek E,Wichrowska H, et al. Ten-year probability of osteoporotic fracture in 2012 Polish women assessed by FRAX and nomogram by Nguyen et al.-Conformity between methods and their clinical utility. Bone. 2010;46(6):1661-67.
73. Sandhu SK, Nguyen ND, Center JR, Pocock NA, Eisman JA, Nguyen TV. Prognosis of fracture: evaluation of predictive accuracy of the FRAX algorithm and Garvan nomogram. Osteoporos Int. 2010;21(5):863-71.
74. van Geel TA, Nguyen ND, Geusens PP, Center JR, Nguyen TV, Dinant GJ, et al. Development of a simple prognostic nomogram for individualising 5-year and 10-year absolute risks of fracture: a population-based prospective study among postmenopausal women. Ann Rheum Dis. 2011;70(1):97-7.
75. Watts NB. The Fracture Risk Assessment Tool (FRAX(R)): applications in clinical practice. J Womens Health (Larchmt). 2011;20(4):525-31.
76. Shepstone L, Fordham R, Lenaghan E, Harvey I, Cooper C, Gittoes N, et al. A pragmatic randomised controlled trial of the effectiveness and cost-effectiveness of screening older women for the prevention of fractures: rationale, design and methods for the SCOOP study. Osteoporos Int. 2012.
77. Wells G, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Risedronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD004523.
78. Wells GA, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Alendronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD001155.
79. Wells GA, Cranney A, Peterson J, Boucher M, Shea B, Robinson V, et al. Etidronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane Database Syst Rev. 2008;23(1):CD003376.
80. McClung M, Boonen S, Törring O, Roux C, Rizzoli R, Bone H, et al. Effect of denosumab treatment on the risk of fractures in subgroups of women with postmenopausal osteoporosis. J Bone Miner Res. 2011;Epub ahead of print.
81. Bolland MJ, Grey A. Disparate outcomes from applying U.K. and U.S. osteoporosis treatment guidelines. J Clin Endocrinol Metab. 2010;95(4):1856-60.
82. Compston J, Cooper A, Cooper C, Francis R, Kanis JA, Marsh D, et al. Guidelines for the diagnosis and management of osteoporosis in postmenopausal women and men from the age of 50 years in the UK. Maturitas. 2009;62(2):105-08.
83. Dawson-Hughes B. National Osteoporosis Foundation Guide C. A revised clinician's guide to the prevention and treatment of osteoporosis. J Clin Endocrinol Metab. 2008;93 (7):2463-65. (Pubitemid 352008504)
84. Compston J. Pathophysiology of atypical femoral fractures and osteonecrosis of the jaw. Osteoporos Int. 2011;22(12):2951-61.
85. Michaëlsson K, Schilcher J, Aspenberg P. Comment on Compston: Pathophysiology of atypical femoral fractures and osteonecrosis of the jaw. Osteoporos Int 2011;Jan 31. [Epub ahead of print].
86. Schilcher J, Michaelsson K, Aspenberg P. Bisphosphonate use and atypical fractures of the femoral shaft. N Engl J Med. 2011;364(18):1728-37.
87. Siris ES, Chen YT, Abbott TA, Barrett-Connor E, Miller PD, Wehren LE, et al. Bone mineral density thresholds for pharmacological intervention to prevent fractures. Arch InternMed. 2004;164(10):1108-12.
88. Stone KL, Seeley DG, Lui LY, Cauley JA, Ensrud K, Browner WS, et al. BMD at multiple sites and risk of fracture of multiple types: long-Term results from the Study of Osteoporotic Fractures. J Bone Miner Res. 2003;18(11):1947-54.
89. Ankerst DP, Koniarski T, Liang Y, Leach RJ, Feng Z, Sanda MG, et al. Updating risk prediction tools: a case study in prostate cancer. Biom J. 2012;54(1):127-42.
90. Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61(1):76-86.
91. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23(16):2567-86.
92. van Houwelingen HC, Thorogood J. Construction, validation and updating of a prognostic model for kidney graft-survival. Stat Med. 1995;14(18):1999-2008.
93. Calltorp J, Adami HO, Aström H, Fryklund L, Rossner S, Trolle Y, et al. Country profile: Sweden. Lancet. 1996;347(9001):587-94.
94. Gedeborg R, Engquist H, Berglund L, Michaelsson K. Identification of incident injuries in hospital discharge registers. Epidemiology. 2008;19(6):860-67.
95. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.
96. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41-4. (Pubitemid 36110721)
97. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-Analyses: the PRISMA statement. BMJ. 2009;339:b2535.
98. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-57.
99. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM. REporting recommendations for tumour MARKer prognostic studies (REMARK). Br J Cancer. 2005;93(4):387-91.
100. Collins GS. Opening up multivariable prediction models: Consensus-based guidelines for transparent reporting: BMJ Blogs. 2011.