The diagnostic accuracy of the patient health questionnaire-2 (PHQ-2), patient health questionnaire-8 (PHQ-8), and patient health questionnaire-9 (PHQ-9) for detecting major depression: Protocol for a systematic review and individual patient data meta-analyses

Systematic Reviews

Volumn 3, Issue 1, 2014, Pages

  Thombs, Brett D ^a,b,l   Benedetti, Andrea ^a,c   Kloda, Lorie A ^a   Levis, Brooke ^a,b   Nicolau, Ioana ^b   Cuijpers, Pim ^d   Gilbody, Simon ^e,f   Ioannidis, John P A ^g,h   McMillan, Dean ^e,f   Patten, Scott B ^i,j   Shrier, Ian ^a,b   Steele, Russell J ^a,b   Ziegelstein, Roy C ^k  

^a MCGILL UNIVERSITY   (Canada)

^b MCGILL UNIVERSITY   (Canada)

^c MCGILL UNIVERSITY HEALTH CENTRE   (Canada)

^d UNIVERSITY OF AMSTERDAM   (Netherlands)

^e UNIVERSITY OF HULL   (United Kingdom)

^f UNIVERSITY OF YORK   (United Kingdom)

^g STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^h STANFORD UNIVERSITY   (United States)

ⁱ UNIVERSITY OF CALGARY   (Canada)

^j UNIVERSITY OF CALGARY   (Canada)

^k JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^l NONE   (Canada)

more..

Author keywords

Depression; Diagnostic test accuracy; Individual patient data meta analysis; Patient health questionnaire; PHQ 2; PHQ 8; PHQ 9; Screening; Systematic review

Indexed keywords

ARTICLE; CLINICAL PRACTICE; DIAGNOSTIC ACCURACY; DIAGNOSTIC TEST ACCURACY STUDY; HUMAN; MAJOR DEPRESSION; META ANALYSIS; PATIENT HEALTH QUESTIONNAIRE 2; PATIENT HEALTH QUESTIONNAIRE 8; PATIENT HEALTH QUESTIONNAIRE 9; PRIORITY JOURNAL; PUBLICATION; QUALITY CONTROL; RECEIVER OPERATING CHARACTERISTIC; SCORING SYSTEM; SEMI STRUCTURED INTERVIEW; SENSITIVITY AND SPECIFICITY; SYSTEMATIC REVIEW; VALIDATION PROCESS; DEPRESSIVE DISORDER, MAJOR; INFORMATION PROCESSING; METHODOLOGY; PREDICTIVE VALUE; QUESTIONNAIRE;

DATASETS AS TOPIC; DEPRESSIVE DISORDER, MAJOR; HUMANS; PREDICTIVE VALUE OF TESTS; QUESTIONNAIRES; RESEARCH DESIGN;

EID: 84927921914     PISSN: None     EISSN: 20464053     Source Type: Journal    
DOI: 10.1186/2046-4053-3-124     Document Type: Article

References (81)

1. Preventive Services Task Force US. Screening for depression in adults: U.S. preventive services task force recommendation statement. Ann Intern Med 2009, 151:784-792.
2. National Collaborating Center for Mental Health: The NICE Guideline on The Management and Treatment of Depression in Adults (Updated Edition). London (UK): National Institute for Health and Clinical Excellence; 2010.
3. National Collaborating Centre for Mental Health: Depression in Adults With a Chronic Physical Health Problem: The NICE Guideline on Treatment and Management. Leicester (UK): The British Psychological Society & The Royal College of Psychiatrists; 2010.
4. Evans DL, Charney DS, Lewis L, Golden RN, Gorman JM, Krishnan KR, Nemeroff CB, Bremner JD, Carney RM, Coyne JC, Delong MR, Frasure-Smith N, Glassman AH, Gold PW, Grant I, Gwyther L, Ironson G, Johnson RL, Kanner AM, Katon WJ, Kaufmann PG, Keefe FJ, Ketter T, Laughren TP, Leserman J, Lyketsos CG, McDonald WM, McEwen BS, Miller AH, Musselman D, et al. Mood disorders in the medically ill: scientific review and recommendations. Biol Psychiatry 2005, 58:175-189. 10.1016/j.biopsych.2005.05.001, 16084838.
5. Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B. Depression, chronic diseases, and decrements in health: results from the World Health Surveys. Lancet 2007, 370:851-858. 10.1016/S0140-6736(07)61415-9, 17826170.
6. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet 2006, 367:1747-1757. 10.1016/S0140-6736(06)68770-9, 16731270.
7. Mathers CD, Lopez AD, Murray CJL. The burden of disease and mortality by condition: data, methods, and results for 2001. Global Burden of Disease and Risk Factors 2006, Washington (DC): The International Bank for Reconstruction and Development/The World Bank Group, Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJL, Chapter 3.
8. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, Charlson FJ, Norman RE, Flaxman AD, Johns N, Burstein R, Murray CJ, Vos T. Global burden of disease attributable to mental and substance use disorders: findings from the global burden of disease study 2010. Lancet 2013, 382:1575-1586. 10.1016/S0140-6736(13)61611-6, 23993280.
9. Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet 2009, 374:609-619. 10.1016/S0140-6736(09)60879-5, 19640579.
10. Duhoux A, Fournier L, Nguyen CT, Roberge P, Beveridge R. Guideline concordance of treatment for depressive disorders in Canada. Soc Psychiatry Psychiatr Epidemiol 2009, 44:385-392. 10.1007/s00127-008-0444-8, 18946624.
11. Duhoux A, Fournier L, Gauvin L, Roberge P. What is the association between quality of treatment for depression and patient outcomes? A cohort study of adults consulting in primary care. J Affect Disord 2013, 151:265-274. 10.1016/j.jad.2013.05.097, 23876194.
12. Mojtabai R, Olfson M. Proportion of antidepressants prescribed without a psychiatric diagnosis is growing. Health Aff (Millwood) 2011, 30:1434-1442. 10.1377/hlthaff.2010.1024, 21821561.
13. Mojtabai R. Clinician-identified depression in community settings: concordance with structured-interview diagnoses. Psychother Psychosom 2013, 82:161-169. 10.1159/000345968, 23548817.
14. Mental Health Commission of Canada Changing Directions, Changing Lives: The Mental Health Strategy For Canada 2012, Calgary, AB: Author, Mental Health Commission of Canada.
15. MacMillan HL, Patterson CJ, Wathen CN, Feightner JW, Bessette P, Elford RW, Feig DS, Langley J, Palda VA, Patterson C, Reeder BA, Walton R., Canadian Task Force on Preventive Health Care Screening for depression in primary care: recommendation statement from the Canadian Task Force on Preventive Health Care. CMAJ 2005, 172:33-35. 10.1503/cmaj.1030823, 543939, 15632399, Canadian Task Force on Preventive Health Care.
16. Thombs BD, Arthurs E, El-Baalbaki G, Meijer A, Ziegelstein RC, Steele R. Risk of bias from inclusion of already diagnosed or treated patients in diagnostic accuracy studies of depression screening tools: a systematic review. BMJ 2011, 343:d4825. 10.1136/bmj.d4825, 3191850, 21852353.
17. U.S. Preventive Services Task Force Screening for depression: recommendations and rationale. Ann Intern Med 2002, 136:760-764. U.S. Preventive Services Task Force.
18. O'Connor EA, Whitlock EP, Gaynes B, Beil TL. Screening for Depression in Adults and Older Adults in Primary Care: An Updated Systematic Review. Evidence Synthesis No. 75, AHRQ Publication No. 10-05143-EF-1 2009, Agency for Healthcare Research and Quality: Rockville, Maryland.
19. O'Connor EA, Whitlock EP, Beil TL, Gaynes BN. Screening for depression in adult patients in primary care settings: a systematic evidence review. Ann Intern Med 2009, 151:793-803. 10.7326/0003-4819-151-11-200912010-00007, 19949145.
20. Thombs BD, Ziegelstein RC. Does depression screening improve depression outcomes in primary care?. BMJ 2014, 348:g1253. 10.1136/bmj.g1253, 24496211.
21. Thombs BD, Ziegelstein RC, Roseman M, Kloda LA, Ioannidis JP. There are no randomized controlled trials that support the United States preventive services task force guideline on screening for depression in primary care: a systematic review. BMC Med 2014, 12:13. 10.1186/1741-7015-12-13, 3922694, 24472580.
22. Allaby M. Screening for Depression: A Report for the UK National Screening Committee (Revised report) 2010, UK National Screening Committee: UK.
23. Joffres M, Jaramillo A, Dickinson J, Lewin G, Pottie K, Shaw E, Connor Gorber S, Tonelli M., Canadian Task Force on Preventive Health Care Recommendations on screening for depression in adults. CMAJ 2013, 185:775-782. 3680556, 23670157, Canadian Task Force on Preventive Health Care.
24. National Institute for Clinical Excellence Guideline on Cancer Services: Improving Supportive and Palliative Care for Adults with Cancer 2010, London (UK): National Institute for Clinical Excellence, National Institute for Clinical Excellence.
25. Network NCC. Distress Management. NCCN Clinical Practice Guidelines in Oncology 2008, NCCN: Fort Washington, PA.
26. Holland JC, Andersen B, Breitbart WS, Buchmann LO, Compas B, Deshields TL, Dudley MM, Fleishman S, Fulcher CD, Greenberg DB, Greiner CB, Handzo GF, Hoofring L, Hoover C, Jacobsen PB, Kvale E, Levy MH, Loscalzo MJ, McAllister-Black R, Mechanic KY, Palesh O, Pazar JP, Riba MB, Roper K, Valentine AD, Wagner LI, Zevon MA, McMillian NR, Freedman-Cass DA. Distress management. J Natl Compr Canc Netw 2013, 11:190-209.
27. Lichtman JH, Bigger JT Jr, Blumenthal JA, Frasure-Smith N, Kaufmann PG, Lesperance F, Mark DB, Sheps DS, Taylor CB, Froelicher ES, American Heart Association Prevention Committee of the Council on Cardiovascular Nursing, American Heart Association Council on Clinical Cardiology, American Heart Association Council on Epidemiology and Prevention, American Heart Association Interdisciplinary Council on Quality of Care and Outcomes Research, American Psychiatric Association: Depression and coronary heart disease: recommendations for screening, referral, and treatment: a science advisory from the American Heart Association Prevention Committee of the Council on Cardiovascular Nursing, Council on Clinical Cardiology, Council on Epidemiology and Prevention, and Interdisciplinary Council on Quality of Care and Outcomes Research: endorsed by the American Psychiatric Association. Circulation 2008, 118:1768-1775.
28. Colquhoun DM, Bunker SJ, Clarke DM, Glozier N, Hare DL, Hickie IB, Tatoulis J, Thompson DR, Tofler GH, Wilson A, Branagan MG. Screening, referral and treatment for depression in patients with coronary heart disease. Med J Aust 2013, 198:483-484. 10.5694/mja13.10153, 23682890.
29. Canadian Diabetes Association Clinical Practice Guidelines Expert Committee of the Canadian Diabetes Advisory Board: Canadian Diabetes Association 2013 clinical practice guidelines for the prevention and management of diabetes in Canada. Can J Diabetes 2013, 37(suppl 1):S1-S212.
30. Strategy TCS. Canadian best practice recommendations for stroke care. CMAJ 2008, 179:E1-E93.
31. ACOG Committee on Practice Bulletins--Obstetrics ACOG practice bulletin: clinical management guidelines for obstetrician-gynecologists number 92, April 2008 (replaces practice bulletin number 87, November 2007). Use of psychiatric medications during pregnancy and lactation. Obstet Gynecol 2008, 111:1001-1020. 10.1097/AOG.0b013e31816fd910, 18378767, ACOG Committee on Practice Bulletins--Obstetrics.
32. Thombs BD, de Jonge P, Coyne JC, Whooley MA, Frasure-Smith N, Mitchell AJ, Zuidersma M, Eze-Nliam C, Lima BB, Smith CG, Soderlund K, Ziegelstein RC. Depression screening and patient outcomes in cardiovascular care: a systematic review. JAMA 2008, 300:2161-2171. 10.1001/jama.2008.667, 19001627.
33. Thombs BD, Jewett LR, Knafo R, Coyne JC, Ziegelstein RC. Learning from history: a commentary on the American Heart Association Science advisory on depression screening. Am Heart J 2009, 158:503-505. 10.1016/j.ahj.2009.07.032, 19781405.
34. Ziegelstein RC, Thombs BD, Coyne JC, de Jonge P. Routine screening for depression in patients with coronary heart disease. Never mind. J Am Coll Cardiol 2009, 54:886-890. 10.1016/j.jacc.2009.01.082, 2749208, 19712796.
35. Thombs BD, Roseman M, Coyne JC, de Jonge P, Delisle VC, Arthurs E, Levis B, Ziegelstein RC. Does evidence support the American Heart Association's recommendation to screen patients for depression in cardiovascular care? An updated systematic review. PLoS One 2013, 8:e52654. 10.1371/journal.pone.0052654, 3538724, 23308116.
36. Meijer A, Roseman M, Milette K, Coyne JC, Stefanek ME, Ziegelstein RC, Arthurs E, Leavens A, Palmer SC, Stewart DE, de Jonge P, Thombs BD. Depression screening and patient outcomes in cancer: a systematic review. PLoS One 2011, 6:e27181. 10.1371/journal.pone.0027181, 3215716, 22110613.
37. Altman DG, Bland JM. Diagnostic tests. 1: sensitivity and specificity. BMJ 1994, 308:1552. 10.1136/bmj.308.6943.1552, 2540489, 8019315.
38. Li J, Fine JP. Assessing the dependence of sensitivity and specificity on prevalence in meta-analysis. Biostatistics 2011, 12:710-722. 10.1093/biostatistics/kxr008, 4042906, 21525421.
39. Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med 1997, 16:981-991. 10.1002/(SICI)1097-0258(19970515)16:9<981::AID-SIM510>3.0.CO;2-N, 9160493.
40. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143:29-36. 10.1148/radiology.143.1.7063747, 7063747.
41. Swets JA. Measuring the accuracy of diagnostic systems. Science 1988, 240:1285-1293. 10.1126/science.3287615, 3287615.
42. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ 1994, 309:102. 2540558, 8038641.
43. Meader N, Mitchell AJ, Chew-Graham C, Goldberg D, Rizzo M, Bird V, Kessler D, Packham J, Haddad M, Pilling S. Case identification of depression in patients with chronic physical health problems: a diagnostic accuracy meta-analysis of 113 studies. Br J Gen Pract 2011, 61:e808-e820. 10.3399/bjgp11X613151, 3223779, 22137418.
44. Meader N, Moe-Byrne T, Llewellyn A, Mitchell AJ. Screening for poststroke major depression: a meta-analysis of diagnostic validity studies. J Neurol Neurosurg Psychiatry 2014, 85:198-206. 10.1136/jnnp-2012-304194, 23385849.
45. Manea L, Gilbody S, McMillan D. Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): a meta-analysis. CMAJ 2012, 184:E191-E196. 10.1503/cmaj.110829, 3281183, 22184363.
46. Brennan C, Worrall-Davies A, McMillan D, Gilbody S, House A. The hospital anxiety and depression scale: a diagnostic meta-analysis of case-finding ability. J Psychosom Res 2010, 69:371-378. 10.1016/j.jpsychores.2010.04.006, 20846538.
47. Hewitt C, Gilbody S, Brealey S, Paulden M, Palmer S, Mann R, Green J, Morrell J, Barkham M, Light K, Richards D. Methods to identify postnatal depression in primary care: an integrated evidence synthesis and value of information analysis. Health Technol Assess 2009, 13:1-145. 147-230.
48. Kroenke K. Enhancing the clinical utility of depression screening. CMAJ 2012, 184:281-282. 10.1503/cmaj.112004, 3281149, 22231681.
49. Guidelines and Protocols Advisory Committee, British Columbia Medical Association: Depression (MDD) - Diagnosis and Management. British Columbia: British Columbia Ministry of Health; 2004.
50. American College of Physicians Depression Care Guide http://depression.acponline.org/ps/main.
51. Suicide Prevention Resource Center http://www.sprc.org/sites/sprc.org/files/MacArthur%20Depression-Primary%20Care_toolkit.pdf.
52. Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ 2010, 340:c221. 10.1136/bmj.c221, 20139215.
53. Stewart LA, Tierney JF, Clarke M. Chapter 18: Reviews of individual patient data. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011] 2011, Higgins JPT, Green S, The Cochrane Collaboration, Available from http://www.cochrane-handbook.org.
54. Ahmed I, Sutton AJ, Riley RD. Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey. BMJ 2012, 344:d7762. 10.1136/bmj.d7762, 22214758.
55. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001, 16:606-613. 10.1046/j.1525-1497.2001.016009606.x, 1495268, 11556941.
56. Kroenke K, Spitzer RL, Williams JB. The patient health questionnaire-2: validity of a two-item depression screener. Med Care 2003, 41:1284-1292. 10.1097/01.MLR.0000093487.78664.3C, 14583691.
57. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009, 114:163-173. 10.1016/j.jad.2008.06.026, 18752852.
58. Deeks JJ, Wisniewski S, Davenport C. Chapter 4: Guide to the contents of a Cochrane Diagnostic Test Accuracy Protocol. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0.0 2013, Deeks JJ, Bossuyt PM, Gatsonis C, The Cochrane Collaboration, Available from: http://srdta.cochrane.org/.
59. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011] 2011, Higgins JPT, Green S, The Cochrane Collaboration, Available from http://www.cochrane-handbook.org.
60. Moher D, Liberati A, Tetzlaff J, Altman DG., PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009, 6:e1000097. 10.1371/journal.pmed.1000097, 2707599, 19621072, PRISMA Group.
61. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009, 6:e1000100. 10.1371/journal.pmed.1000100, 2707010, 19621070.
62. Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline?. J Clin Epidemiol 2003, 56:943-955. 10.1016/S0895-4356(03)00110-0, 14568625.
63. Mann R, Gilbody SM. Should methodological filters for diagnostic test accuracy studies be used in systematic reviews of psychometric instruments? a case study involving screening for postnatal depression. Syst Rev 2012, 1:9. 10.1186/2046-4053-1-9, 3351740, 22588034.
64. Ingui BJ, Rogers MA. Searching for clinical prediction rules in MEDLINE. J Am Med Inform Assoc 2001, 8:391-397. 10.1136/jamia.2001.0080391, 130084, 11418546.
65. Wong SS, Wilczynski NL, Haynes RB, Ramkissoonsingh R., Hedges Team Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE. AMIA Annu Symp Proc 2003, 2003:728-732. Hedges Team.
66. First MB, Spitzer RL, Gibbon M, Williams J. Structured Clinical Interview for DSM-IV Axis I Disorders - Patient Edition (SCID-I/P, Version 2.0) 1996, New York: Biometrics Research Department, New York State Psychiatric Institute.
67. Wittchen HU. Reliability and validity studies of the WHO-Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res 1994, 28:57-84. 10.1016/0022-3956(94)90036-1, 8064641.
68. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM., QUADAS-2 Group QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011, 155:529-536. 10.7326/0003-4819-155-8-201110180-00009, 22007046, QUADAS-2 Group.
69. Riley RD, Dodd SR, Craig JV, Thompson JR, Williamson PR. Meta-analysis of diagnostic test studies using individual patient data and aggregate data. Stat Med 2008, 27:6111-6136. 10.1002/sim.3441, 18816508.
70. Smits N, Smit F, Cuijpers P, De Graaf R. Using decision theory to derive optimal cut-off scores of screening instruments: an illustration explicating costs and benefits of mental health screening. Int J Methods Psychiatr Res 2007, 16:219-229. 10.1002/mpr.230, 18188835.
71. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002, 21:1539-1558. 10.1002/sim.1186, 12111919.
72. Neuhaus JM, Kalbfleisch JD. Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics 1998, 54:638-645. 10.2307/3109770, 9629647.
73. Simmonds MC, Higgins JP. Covariate heterogeneity in meta-analysis: criteria for deciding between meta-regression and individual patient data. Stat Med 2007, 26:2982-2999. 10.1002/sim.2768, 17195960.
74. Berlin JA, Santanna J, Schmid CH, Szczech LA, Feldman HI., Anti-Lymphocyte Antibody Induction Therapy Study Group Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med 2002, 21:371-387. 10.1002/sim.1023, 11813224, Anti-Lymphocyte Antibody Induction Therapy Study Group.
75. Lambert PC, Sutton AJ, Abrams KR, Jones DR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol 2002, 55:86-94. 10.1016/S0895-4356(01)00414-0, 11781126.
76. Schmid CH, Stark PC, Berlin JA, Landais P, Lau J. Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. J Clin Epidemiol 2004, 57:683-697. 10.1016/j.jclinepi.2003.12.001, 15358396.
77. Steyerberg EW. Clinical Prediction Models 2009, New York: Springer.
78. Steyerberg EW, Harrell FE, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001, 54:774-781. 10.1016/S0895-4356(01)00341-9, 11470385.
79. White IR, Royston P. Imputing missing covariate values for the Cox model. Stat Med 2009, 28:1982-1998. 10.1002/sim.3618, 2998703, 19452569.
80. Riley RD, Takwoingi Y, Trikalinos T, Guha A, Biswas A, Ensor J, Morris RK, Deeks JJ. Meta-analysis of test accuracy studies with multiple and missing thresholds: a multvariate-normal model. J Biomed Biostat 2014, 5:100196.
81. Hamza TH, Arends LR, van Houwelingen HC, Stijnen T. Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds. BMC Med Res Methodol 2009, 9:73. 10.1186/1471-2288-9-73, 2787531, 19903336.