Validation of online psychometric instruments for common mental health disorders: A systematic review

BMC Psychiatry

Volumn 16, Issue 1, 2016, Pages

  van Ballegooijen, Wouter ^a,b   Riper, Heleen ^a,b   Cuijpers, Pim ^a,b   van Oppen, Patricia ^a   Smit, Johannes H ^a  

^a VU UNIVERSITY MEDICAL CENTER   (Netherlands)

^b VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Anxiety; Depression; Internet; Measurement; Psychometrics; Systematic review

Indexed keywords

AGORAPHOBIA; AGORAPHOBIC COGNITION QUESTIONNAIRE; ARTICLE; AUTOMUTILATION; BECK ANXIETY INVENTORY; BODY SENSATION QUESTIONNAIRE; CES D; DEPRESSION ANXIETY STRESS SCALE; DEPRESSION ASSESSMENT; DIAGNOSTIC ERROR; DIAGNOSTIC TEST ACCURACY STUDY; DSM-5; GENERALIZED ANXIETY DISORDER; HOSPITAL ANXIETY AND DEPRESSION SCALE; HUMAN; ICD-10; INTERNAL CONSISTENCY; INVENTORY OF STATEMENT ABOUT SELF INJURY; MAJOR DEPRESSION; MENTAL DISEASE; MENTAL STRESS; MOBILITY INVENTORY; MONTGOMERY ASBERG DEPRESSION RATING SCALE; OBSESSIVE COMPULSIVE DISORDER; ONLINE SYSTEM; PANIC; PATIENT WORRY; POSTTRAUMATIC STRESS DISORDER; PSYCHOLOGIC TEST; PSYCHOMETRY; QUESTIONNAIRE; RECEIVER OPERATING CHARACTERISTIC; SENSITIVITY AND SPECIFICITY; SOCIAL INTERACTION ANXIETY SCALE; SOCIAL PHOBIA; SOCIAL PHOBIA SCALE; SUICIDAL IDEATION; SYSTEMATIC REVIEW; TEST RETEST RELIABILITY; VALIDATION PROCESS; WEB BASED DEPRESSION AND ANXIETY TEST; WEB BROWSER; WEB SCREENING QUESTIONNAIRE; ANXIETY DISORDERS; COMPUTER ASSISTED DIAGNOSIS; DEPRESSION; DEPRESSIVE DISORDER; INTERNET; MENTAL DISORDERS; MENTAL HEALTH; PROCEDURES; PSYCHOLOGICAL RATING SCALE; REPRODUCIBILITY; STANDARDS; UTILIZATION;

ANXIETY DISORDERS; DEPRESSION; DEPRESSIVE DISORDER; DIAGNOSIS, COMPUTER-ASSISTED; HUMANS; INTERNET; MENTAL DISORDERS; MENTAL HEALTH; PSYCHIATRIC STATUS RATING SCALES; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; SURVEYS AND QUESTIONNAIRES;

EID: 84959133657     PISSN: None     EISSN: 1471244X     Source Type: Journal    
DOI: 10.1186/s12888-016-0735-7     Document Type: Article

References (58)

1. National Institute for Health and Clinical Excellence (NICE). Common mental health disorders: Identification and pathways to care. London: NICE; 2011.
2. Alfonsson S, Maathz P, Hursti T. Interformat reliability of digital psychiatric self-report questionnaires: a systematic review. J Med Internet Res. 2014;16(12), e268. doi: 10.2196/jmir.3395.
3. Hedman E, Ljotsson B, Ruck C, Furmark T, Carlbring P, Lindefors N, et al. Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation. Comput Hum Behav. 2010;26(4):736-40.
4. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, Ost LG, et al. Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput Hum Behav. 2007;23(3):1421-34.
5. Cuijpers P, Boluijt P, Van SA. Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires. Eur Child AdolescPsychiatry. 2008;17(1):32-8. doi: 10.1007/s00787-007-0631-2.
6. Rogers KD, Young A, Lovell K, Campbell M, Scott PR, Kendal S. The British Sign Language versions of the Patient Health Questionnaire, the Generalized Anxiety Disorder 7-item Scale, and the Work and Social Adjustment Scale. J Deaf Stud Deaf Educ. 2013;18(1):110-22. doi: 10.1093/deafed/ens040.
7. van Ballegooijen W, Riper H, Donker T, Martin Abello K, Marks I, Cuijpers P. Single-item screening for agoraphobic symptoms: validation of a web-based audiovisual screening instrument. PLoS One. 2012;7(7):e38480. doi: 10.1371/journal.pone.0038480.
8. Farvolden P, McBride C, Bagby RM, Ravitz P. A Web-based screening instrument for depression and anxiety disorders in primary care. J Med Internet Res. 2003;5(3), e23. doi: 10.2196/jmir.5.3.e23.
9. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535. doi: 10.1136/bmj.b2535.
10. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651-7. doi: 10.1007/s11136-011-9960-1.
11. Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009;18(8):1115-23. doi: 10.1007/s11136-009-9528-5.
12. World Health Organization. ICD-10 Version:2015. 2015. http://apps.who.int/classifications/icd10/browse/2015/en. Accessed January 2015.
13. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, (DSM-5®). American Psychiatric Pub.; 2013.
14. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539-49. doi: 10.1007/s11136-010-9606-8.
15. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737-45. doi: 10.1016/j.jclinepi.2010.02.006.
16. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10:22. doi: 10.1186/1471-2288-10-22.
17. Mokkink LB, Terwee CB, Gibbons E, Stratford PW, Alonso J, Patrick DL, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. 2010;10:82. doi: 10.1186/1471-2288-10-82.
18. Donker T, Van SA, Marks I, Cuijpers P. A brief Web-based screening questionnaire for common mental disorders: development and validation. J Med Internet Res. 2009;11(3):e19. doi: 10.2196/jmir.1134.
19. McCue P, Martin C, Buchanan T, Rodgers J, Scholey A. An investigation into the psychometric properties of the Hospital Anxiety and Depression Scale in individuals with chronic fatigue syndrome. Psychol Health Med. 2003;8(4):425-39. doi: 10.1080/1354850310001604568.
20. Zlomke KR. Psychometric properties of internet administered versions of Penn State Worry Questionnaire (PSWQ) and Depression, Anxiety, and Stress Scale (DASS). Comput Hum Behav. 2009;25(4):841-3.
21. Carlbring P, Forslin P, Ljungstrand P, Willebrand M, Strandlund C, Ekselius L, et al. Is the Internet-administered CIDI-SF equivalent to a clinician-administered SCID interview? Cogn Behav Ther. 2002;31(4):183-9.
22. Andersson G, Kaldo-Sandstrom V, Strom L, Stromgren T. Internet administration of the Hospital Anxiety and Depression Scale in a sample of tinnitus patients. J Psychosom Res. 2003;55(3):259-62.
23. McCue P, Buchanan T, Martin CR. Screening for psychological distress using internet administration of the Hospital Anxiety and Depression Scale (HADS) in individuals with chronic fatigue syndrome. Br J Clin Psychol. 2006;45(Pt 4):483-98. doi: 10.1348/014466505X82379.
24. Thoren ES, Andersson G, Lunner T. The use of research questionnaires with hearing impaired adults: online vs. paper-and-pencil administration. BMC Ear Nose Throat Disord. 2012;12:12. doi: 10.1186/1472-6815-12-12.
25. Whitehead L. Methodological issues in Internet-mediated research: a randomized comparison of internet versus mailed questionnaires. J Med Internet Res. 2011;13(4):e109. doi: 10.2196/jmir.1593.
26. Drake G, Csipke E, Wykes T. Assessing your mood online: acceptability and use of Moodscope. Psychol Med. 2013;43(7):1455-64. doi: 10.1017/S0033291712002280.
27. Faries DE, Houston JP, Sulcs EM, Swindle RW. A cross-validation of the provisional diagnostic instrument (PDI-4). BMC Fam Pract. 2012;13:104. doi: 10.1186/1471-2296-13-104.
28. Zimmerman M, Martinez JH. Web-based assessment of depression in patients treated in clinical practice: reliability, validity, and patient acceptance. J Clin Psychiatry. 2012;73(3):333-8. doi: 10.4088/JCP.10m06519.
29. Birnholz JL, Young MA. Differential item functioning for lesbians, bisexual, and heterosexual women in the center for epidemiological studies depression scale. Assessment. 2012;19(4):502-5. doi: 10.1177/1073191112440032.
30. Donker T, Van SA, Marks I, Cuijpers P. Brief self-rated screening for depression on the Internet. J AffectDisord. 2010;122(3):253-9. doi: 10.1016/j.jad.2009.07.013.
31. Fortson BL, Scotti JR, Del Ben KS, Chen YC. Reliability and validity of an Internet traumatic stress survey with a college student sample. J Trauma Stress. 2006;19(5):709-20. doi: 10.1002/jts.20165.
32. Leykin Y, Torres LD, Aguilera A, Munoz RF. Factor structure of the CES-D in a sample of Spanish- and English-speaking smokers on the Internet. Psychiatry Res. 2011;185(1-2):269-74. doi: 10.1016/j.psychres.2010.04.056.
33. Yu SC, Yu MN. Comparison of Internet-based and paper-based questionnaires in Taiwan using multisample invariance approach. CyberpsycholBehav. 2007;10(4):501-7. doi: 10.1089/cpb.2007.9998.
34. Hedman E, Ljotsson B, Blom K, E-A S, Kraepelien M, Ruck C, et al. Telephone versus internet administration of self-report measures of social anxiety, depressive symptoms, and insomnia: psychometric evaluation of a method to reduce the impact of missing data. J Med Internet Res. 2013;15(10):e229. doi: 10.2196/jmir.2818.
35. Holländare F, Askerlund AM, Nieminen A, Engstrom I. Can BDI-II and MADRS-S be transferred to online use without affecting their psychometric properties? E-J Appl Psychol. 2008;4(2):63-5.
36. Holländare F, Andersson G, Engstrom I. A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J Med Internet Res. 2010;12(5):e49. doi: 10.2196/jmir.1392.
37. Thorndike FP, Carlbring P, Smyth FL, Magee JC, Gonder-Frederick L, Ost LG, et al. Web-based measurement: Effect of completing single or multiple items per webpage. Comput Hum Behav. 2009;25(2):393-401.
38. Donker T, Van SA, Marks I, Cuijpers P. Quick and easy self-rating of Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res. 2011;188(1):58-64. doi: 10.1016/j.psychres.2011.01.016.
39. Austin DW, Carlbring P, Richards JC, Andersson G. Internet administration of three commonly used questionnaires in panic research: Equivalence to paper administration in Australian and Swedish samples of people with panic disorder. Int J Test. 2006;6(1):25-39.
40. Hirai M, Vernon LL, Clum GA, Skidmore ST. Psychometric properties and administration measurement invariance of social phobia symptom measures: Paper-pencil vs. Internet administrations. J Psychopathol Behav Assessment. 2011;33(4):470-9.
41. Lindner P, Martell C, Bergstrom J, Andersson G, Carlbring P. Clinical validation of a non-heteronormative version of the Social Interaction Anxiety Scale (SIAS). Health Qual Life Outcomes. 2013;11:209. doi: 10.1186/1477-7525-11-209.
42. Skolnick RB, Schare ML, Wyatt KP, Tillman MA. Aviophobia assessment: validating the Flight Anxiety Situations Questionnaire as a clinical identification measure. J Anxiety Disord. 2012;26(8):779-84. doi: 10.1016/j.janxdis.2012.07.006.
43. Coles ME, Cook LM, Blake TR. Assessing obsessive compulsive symptoms and cognitions on the internet: evidence for the comparability of paper and Internet administration. Behav Res Ther. 2007;45(9):2232-40. doi: 10.1016/j.brat.2006.12.009.
44. Storch EA, Khanna M, Merlo LJ, Loew BA, Franklin M, Reid JM, et al. Children's Florida Obsessive Compulsive Inventory: psychometric properties and feasibility of a self-report measure of obsessive-compulsive symptoms in youth. Child Psychiatry HumDev. 2009;40(3):467-83. doi: 10.1007/s10578-009-0138-9.
45. Williams MT, Turkheimer E, Schmidt KM, Oltmanns TF. Ethnic identification biases responses to the Padua Inventory for obsessive-compulsive disorder. Assessment. 2005;12(2):174-85. doi: 10.1177/1073191105275620.
46. Elhai JD, Miller ME, Ford JD, Biehn TL, Palmieri PA, Frueh BC. Posttraumatic stress disorder in DSM-5: estimates of prevalence and symptom structure in a nonclinical sample of college students. J Anxiety Disord. 2012;26(1):58-64. doi: 10.1016/j.janxdis.2011.08.013.
47. Miller MW, Wolf EJ, Kilpatrick D, Resnick H, Marx BP, Holowka DW et al. The Prevalence and Latent Structure of Proposed DSM-5 Posttraumatic Stress Disorder Symptoms in U.S. National and Veteran Samples. Psychological Trauma: Theory, Research, Practice, and Policy. 2012.
48. Read JP, Farrow SM, Jaanimagi U, Ouimette P. Assessing trauma and traumatic stress via the Internet: Measurement equivalence and participant reactions. Traumatology. 2009;15(1):94-102.
49. Callahan JL, Borja SE, Hynan MT. Modification of the Perinatal PTSD Questionnaire to enhance clinical utility. J Perinatol. 2006;26(9):533-9. doi: 10.1038/sj.jp.7211562.
50. Verkuil B, Brosschot JF. The online version of the Dutch Penn State Worry Questionnaire: factor structure, predictive validity and reliability. J Anxiety Disord. 2012;26(8):844-8. doi: 10.1016/j.janxdis.2012.08.002.
51. Herrero J, Meneses J. Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: A comparison to pencil and paper responses among Internet users. Comput Hum Behav. 2006;22(5):830-46.
52. Kortge R, Meade T, Tennant A. Interpersonal and intrapersonal functions of deliberate self-harm (DSH): A psychometric examination of the Inventory of Statements About Self-Injury (ISAS) scale. Behaviour Change. 2013;30(1):24-35.
53. Hemelrijk E, van Ballegooijen W, Donker T, van Straten A, Kerkhof A. Internet-based screening for suicidal ideation in common mental disorders. Crisis. 2012;33(4):215-21. doi: 10.1027/0227-5910/a000142.
54. Kline P. Handbook of psychological testing. London: Routledge; 1993.
55. van Spijker BA, Batterham PJ, Calear AL, Farrer L, Christensen H, Reynolds J, et al. The suicidal ideation attributes scale (SIDAS): Community-based validation study of a new scale for the measurement of suicidal ideation. Suicide Life Threat Behav. 2014;44(4):408-19. doi: 10.1111/sltb.12084.
56. Hirschfeld G, von Brachel R. Multiple-Group confirmatory factor analysis in R-A tutorial in measurement invariance with continuous and ordinal indicators. Pract Assessment, Res Eval. 2014;19(7):2.
57. Groves RM, Fowler Jr FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology (Vol. 561). Hoboken, NJ: John Wiley & Sons; 2011.
58. Palmier-Claus JE, Ainsworth J, Machin M, Barrowclough C, Dunn G, Barkus E, et al. he feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry. 2012;12:172. doi: 10.1186/1471-244X-12-172.