Sample size requirements for the internal validation of psychiatric scales

International Journal of Methods in Psychiatric Research

Volumn 20, Issue 4, 2011, Pages 235-249

  Rouquette, Alexandra ^a,b   Falissard, Bruno ^a,c,d  

^a Universite Paris Sud   (France)

^b ANGERS UNIVERSITY HOSPITAL   (France)

^c UNIVERSITÉ PARIS DESCARTES   (France)

^d HÔPITAL PAUL BROUSSE   (France)

Author keywords

Factor analysis; Psychiatry; Questionnaires; Sample size; Validation studies

Indexed keywords

ARTICLE; FACTORIAL ANALYSIS; FACTUAL DATABASE; FEMALE; HUMAN; MALE; MENTAL DISEASE; PSYCHOLOGICAL RATING SCALE; REPRODUCIBILITY; RETROSPECTIVE STUDY; SAMPLE SIZE; STANDARD; STATISTICS;

DATABASES, FACTUAL; FACTOR ANALYSIS, STATISTICAL; FEMALE; HUMANS; MALE; MENTAL DISORDERS; PSYCHIATRIC STATUS RATING SCALES; REPRODUCIBILITY OF RESULTS; RETROSPECTIVE STUDIES; SAMPLE SIZE;

EID: 82155175665     PISSN: 10498931     EISSN: 15570657     Source Type: Journal    
DOI: 10.1002/mpr.352     Document Type: Article

References (88)

1. Adachi N., Onuma T., Nishiwaki S., Murauchi S., Akanuma N., Ishida S., Takei N. (2000) Inter-ictal and post-ictal psychoses in frontal lobe epilepsy: A retrospective comparison with psychoses in temporal lobe epilepsy. Seizure, 9(5), 328-335, DOI: 10.1053/seiz.2000.0413S1059-1311(00)90413-8 [pii]
2. Aleamoni L.M. (1973) Effects of size of sample on eigenvalues, observed communalities, and factor loadings. Journal of Applied Psychology, 58(2), 266-269, DOI: 10.1037/h0035429
3. Basker M., Moses P.D., Russell S., Russell P.S. (2007) The psychometric properties of Beck Depression Inventory for adolescent depression in a primary-care paediatric setting in India. Child and Adolescent Psychiatry and Mental Health, 1(1), 8-14, DOI: 1753-2000-1-8 [pii]10.1186/1753-2000-1-8
4. Beck A.T. (1991) Relationship between the Beck Anxiety Inventory and the Hamilton Anxiety Rating Scale with anxious outpatients. Journal of Anxiety Disorders, 5(3), 213-223, DOI: 10.1016/0887-6185(91)90002-B
5. Beck A.T., Ward C.H., Mendelson M., Mock J., Erbaugh J. (1961) An inventory for measuring depression. Archives of General Psychiatry, 4(6), 561-571.
6. Bell M.D., Lysaker P.H., Beam-Goulet J.L., Milstein R.M., Lindenmayer J.P. (1994) Five-component model of schizophrenia: Assessing the factorial invariance of the positive and negative syndrome scale. Psychiatry Research, 52(3), 295-303, DOI: 0165-1781(94)90075-2 [pii]
7. Bonicatto S., Dew A.M., Soria J.J. (1998) Analysis of the psychometric properties of the Spanish version of the Beck Depression Inventory in Argentina. Psychiatry Research, 79(3), 277-285, DOI: S0165-1781(98)00047-X [pii]
8. Bonilla J., Bernal G., Santos A., Santos D. (2004) A revised Spanish version of the Beck Depression Inventory: Psychometric properties with a Puerto Rican sample of college students. Journal of Clinical Psychology, 60(1), 119-130, DOI: 10.1002/jclp.10195
9. Brown T.A. (2006) Confirmatory Factor Analysis for Applied Research, New York, The Guilford Press.
10. Castro-Costa E., Uchoa E., Firmo J.O., Lima-Costa M.F., Prince M. (2008) Association of cognitive impairment, activity limitation with latent traits in the GHQ-12 in the older elderly. The Bambui Health and Aging Study (BHAS). Aging Clinical and Experimental Research, 20(6), 562-568, DOI: 5323 [pii]
11. Cattell R.B. (1978) The Scientific Use of Factor Analysis in Behavioral and Life Sciences, New York, Plenum Press.
12. Chapman L.K., Williams S.R., Mast B.T., Woodruff-Borden J. (2009) A confirmatory factor analysis of the Beck Anxiety Inventory in African American and European American young adults. Journal of Anxiety Disorders, 23(3), 387-392, DOI: S0887-6185(08)00218-1 [pii]10.1016/j.janxdis.2008.12.003
13. Comrey A.L. (1978) Common methodological problems in factor analytic studies. Journal of Consulting and Clinical Psychology, 46(4), 648-659, DOI: 10.1037/0022-006X.46.4.648
14. Comrey A.L., Lee H.B. (1992) A First Course in Factor Analysis, Hillsdale, NJ, Lawrence Erlbaum Associates.
15. Costello A.B., Osborn J.W. (2005) Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1-9. [13th October 2011].
16. Dagnan D., Jahoda A., McDowell K., Masson J., Banks P., Hare D. (2008) The psychometric properties of the Hospital Anxiety and Depressions Scale adapted for use with people with intellectual disabilities. Journal of Intellectual Disability Research, 52(11), 942-949, DOI: JIR1053 [pii]10.1111/j.1365-2788.2008.01053.x
17. Dawkins N., Cloherty M.E., Gracey F., Evans J.J. (2006) The factor structure of the Hospital Anxiety and Depression Scale in acquired brain injury. Brain Injury, 20(12), 1235-1239, DOI: V041611313646066 [pii]10.1080/02699050601076414
18. Everitt B.S. (1975) Multivariate analysis: The need for data, and other problems. The British Journal of Psychiatry, 126(3), 237-240, DOI:10.1192/bjp.126.3.237
19. Fabrigar L.R., Wegener D.T., MacCallum R.C., Strahan E.J. (1999) Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299, DOI: 10.1037/1082-989X.4.3.272
20. Fan X., Thompson B. (2001) Confidence intervals about score reliability coefficients, please: An EPM guideline editorial. Educational and Psychological Measurement, 61(4), 517-531, DOI: 10.1177/0013164401614001
21. Farrell G.A. (1998) The mental health of hospital nurses in Tasmania as measured by the 12-item General Health Questionnaire. Journal of Advanced Nursing, 28(4), 707-712, DOI: 10.1046/j.1365-2648.1998.00735
22. Fedorowicz V.J., Falissard B., Foulon C., Dardennes R., Divac S.M., Guelfi J.D., Rouillon F. (2007) Factors associated with suicidal behaviors in a large French sample of inpatients with eating disorders. International Journal of Eating Disorders, 40(7), 589-595, DOI: 10.1002/eat.20415
23. Feldt L.S. (1965) The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30(3), 357-570, DOI: 10.1007/BF02289499
24. Floyd F.J., Widaman K.F. (1995) Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286-299, DOI: 10.1037/1040-3590.7.3.286
25. Ford J.K., MacCallum R.C., Tait M. (1986) The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology, 39(2), 291-314, DOI: 10.1111/j.1744-6570.1986.tb00583.x
26. Fresan A., De la Fuente-Sandoval C., Loyzaga C., Garcia-Anaya M., Meyenberg N., Nicolini H., Apiquian R. (2005) A forced five-dimensional factor analysis and concurrent validity of the Positive and Negative Syndrome Scale in Mexican schizophrenic patients. Schizophrenia Research, 72(2-3), 123-129, DOI: 10.1016/j.schres.2004.03.021
27. Friedman S., Samuelian J.C., Lancrenon S., Even C., Chiarelli P. (2001) Three-dimensional structure of the Hospital Anxiety and Depression Scale in a large French primary care population suffering from major depression. Psychiatry Research, 104(3), 247-257, DOI: S0165-1781(01)00309-2 [pii]
28. Gabryelewicz T., Styczynska M., Pfeffer A., Wasiak B., Barczak A., Luczywek E., Androsiuk W., Barcikowska M. (2004) Prevalence of major and minor depression in elderly persons with mild cognitive impairment - MADRS factor analysis. International Journal of Geriatric Psychiatry, 19(12), 1168-1172, DOI: 10.1002/gps.1235
29. Galinowski A., Lehert P. (1995) Structural validity of MADRS during antidepressant treatment. International Clinical Psychopharmacology, 10(3), 157-161, DOI: 10.1097/00004850-199510030-00004
30. Gorenstein C., Andrade L., Vieira Filho A.H., Tung T.C., Artes R. (1999) Psychometric properties of the Portuguese version of the Beck Depression Inventory on Brazilian college students. Journal of Clinical Psychology, 55(5), 553-562, DOI: 10.1002/(SICI)1097-4679(199905)55:5<553::AID-JCLP3>3.0.CO;2-D [pii]
31. Gorsuch R.L. (1983) Factor Analysis, London, Lawrence Erlbaum Associates.
32. Green S.A., Yang Y. (2009) Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155-167, DOI: 10.1007/S11336-008-9099-3
33. Grunebaum M.F., Keilp J., Li S., Ellis S.P., Burke A.K., Oquendo M.A., Mann J.J. (2005) Symptom components of standard depression scales and past suicidal behavior. Journal of Affective Disorders, 87(1), 73-82, DOI: S0165-0327(05)00073-X [pii] 10.1016/j.jad.2005.03.002
34. Guadagnoli E., Velicer W.F. (1988) Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265-275, DOI: 10.1037/0033-2909.103.2.265
35. Hamilton M. (1960) A rating scale for depression. Journal of Neurology, Neurosurgery & Psychiatry, 23(1), 56-62.
36. Hankins M. (2008) The factor structure of the twelve item General Health Questionnaire (GHQ-12): the result of negative phrasing? Clinical Practice and Epidemiology in Mental Health, 4(4), 10, DOI: 1745-0179-4-10 [pii]10.1186/1745-0179-4-10
37. Harvey P.D., Davidson M., White L., Keefe R.S., Hirschowitz J., Mohs R.C., Davis K.L. (1996) Empirical evaluation of the factorial structure of clinical symptoms in schizophrenia: Effects of typical neuroleptics on the brief psychiatric rating scale. Biological Psychiatry, 40(8), 755-760, DOI: 0006-3223(95)00486-6 [pii]10.1016/0006-3223(95)00486-6
38. Helm H.W. Jr., Boward M.D. (2003) Factor structure of the Beck Depression Inventory in a university sample. Psychological Reports, 92(1), 53-61.
39. Hogarty K.Y., Hines C.V., Kromrey J.D., Ferron J.M., Mumford K.R. (2005) The quality of factor solutions in exploratory factor analysis: The influence of sample size, communality, and overdetermination. Educational and Psychological Measurement, 65(2), 202-206, DOI: 10.1177/0013164404267287
40. Honey G.D., Sharma T., Suckling J., Giampietro V., Soni W., Williams S.C., Bullmore E.T. (2003) The functional neuroanatomy of schizophrenic subsyndromes. Psychological Medicine, 33(6), 1007-1018, DOI: 10.1017/S0033291703007864
41. Hu Y., Stewart-Brown S., Twigg L., Weich S. (2007) Can the 12-item General Health Questionnaire be used to measure positive mental health? Psychological Medicine, 37(7), 1005-1013, DOI: 10.1017/S0033291707009993
42. Iwata N., Mishima N., Okabe K., Kobayashi N., Hashiguchi E., Egashira K. (2000) Psychometric properties of the State-Trait Anxiety Inventory among Japanese clinical outpatients. Journal of Clinical Psychology, 56(6), 793-806, DOI: 10.1002/(SICI)1097-4679(200006)56:6<793::AID-JCLP8>3.0.CO;2-4 [pii]
43. Iwata N., Mishima N., Shimizu T., Mizoue T., Fukuhara M., Hidano T., Spielberger C.D. (1998) Positive and negative affect in the factor structure of the State-Trait Anxiety Inventory for Japanese workers. Psychological Reports, 82(2), 651-656, DOI: 10.2466/PR0.82.2.651-656
44. Jackson J.E. (1991) A User's Guide to Principal Components, New York, John Wiley & Sons.
45. Jo S.A., Park M.H., Jo I., Ryu S.H., Han C. (2007) Usefulness of Beck Depression Inventory (BDI) in the Korean elderly population. International Journal of Geriatric Psychiatry, 22(3), 218-223, DOI: 10.1002/gps.1664
46. Kabacoff R.I., Segal D.L., Hersen M., Van Hasselt V.B. (1997) Psychometric properties and diagnostic utility of the Beck Anxiety Inventory and the State-Trait Anxiety Inventory with older adult psychiatric outpatients. Journal of Anxiety Disorders, 11(1), 33-47, DOI: S0887618596000333 [pii]
47. Kay S.R., Sevy S. (1990) Pyramidical model of schizophrenia. Schizophrenia Bulletin, 16(3), 537-545, DOI: 10.1016/0165-1781(94)90075-2
48. Kilic C., Rezaki M., Rezaki B., Kaplan I., Ozgen G., Sagduyu A., Ozturk M.O. (1997) General Health Questionnaire (GHQ12 &GHQ28): Psychometric properties and factor structure of the scales in a Turkish primary care sample. Social Psychiatry and Psychiatric Epidemiology, 32(6), 327-331, DOI: 10.1007/BF00805437
49. Killgore W.D. (1999) Empirically derived factor indices for the Beck Depression Inventory. Psychological Reports, 84(3 Pt 1), 1005-1013.
50. Lachar D., Bailley S.E., Rhoades H.M., Espadas A., Aponte M., Cowan K.A., Gummattira P., Kopecky C.R., Wassef A. (2001) New subscales for an anchored version of the Brief Psychiatric Rating Scale: Construction, reliability, and validity in acute psychiatric admissions. Psychological Assessment, 13(3), 384-395, DOI: 10.1037/1040-3590.13.3.384
51. Lancon C., Reine G., Llorca P.M., Auquier P. (1999) Validity and reliability of the French-language version of the Positive and Negative Syndrome Scale (PANSS). Acta Psychiatrica Scandinavica, 100(3), 237-243, DOI: 10.1111/j.1600-0447.1999.tb10851.x
52. Lee K.H., Harris A.W., Loughland C.M., Williams L.M. (2003) The five symptom dimensions and depression in schizophrenia. Psychopathology, 36(5), 226-233, DOI: 10.1159/000073447 PSP2003036005226 [pii]
53. Lindenmayer J.P., Czobor P., Volavka J., Lieberman J.A., Citrome L., Sheitman B., McEvoy J.P., Cooper T.B., Chakos M. (2004) Effects of atypical antipsychotics on the syndromal profile in treatment-resistant schizophrenia. The Journal of Clinical Psychiatry, 65(4), 551-556.
54. Loo R. (1983) Caveat on sample sizes in factor analysis. Perceptual and Motor Skills, 56(2), 371-374.
55. Lopez-Castedo A., Fernandez L. (2005) Psychometric properties of the Spanish version of the 12-item General Health Questionnaire in adolescents. Perceptual and Motor Skills, 100(3 Pt 1), 676-80.
56. Loza B., Kucharska-Pietura K., Kopacz G., Debowska G. (2003) Factor structure of paranoid schizophrenia: A prospective study. Psychopathology, 36(3), 132-141, DOI: 10.1159/000071258 PSP2003036003132 [pii]
57. Lykouras L., Oulis P., Psarros K., Daskalopoulou E., Botsis A., Christodoulou G.N., Stefanis C. (2000) Five-factor model of schizophrenic psychopathology: How valid is it? European Archives of Psychiatry and Clinical Neuroscience, 250(2), 93-100, DOI: 10.1007/s004060070041
58. MacCallum R.C., Widaman K.F., Zhang S., Hong S. (1999) Sample size in factor analysis. Psychological Methods, 4(1), 84-99, DOI: 10.1037/1082-989X.4.1.84
59. Mundfrom D.J., Shaw D.G., Ke T.L. (2005) Minimum sample size recommendations for conducting factor analyses. International Journal of Testing, 5(2), 159-168, DOI: 10.1207/s15327574ijt0502_4
60. Munoz D.J., Chen E., Fischer S., Roehrig M., Sanchez-Johnson L., Alverdy J., Dymek-Valentine M., le Grange D. (2007) Considerations for the use of the Beck Depression Inventory in the assessment of weight-loss surgery seeking patients. Obesity Surgery, 17(8), 1097-1101, DOI: 10.1007/s11695-007-9185-0
61. Nunnaly J.C. (1978) Psychometric Theory, New York, McGraw-Hill.
62. Olden M., Rosenfeld B., Pessin H., Breitbart W. (2009) Measuring depression at the end of life: Is the Hamilton Depression Rating Scale a valid instrument? Assessment, 16(1), 43-54, DOI: 1073191108320415 [pii] 10.1177/1073191108320415
63. Pallant J.F., Bailey C.M. (2005) Assessment of the structure of the Hospital Anxiety and Depression Scale in musculoskeletal patients. Health and Quality of Life Outcomes, 3(12), 82, DOI: 1477-7525-3-82 [pii] 10.1186/1477-7525-3-82
64. Parker R.D., Flint E.P., Bosworth H.B., Pieper C.F., Steffens D.C. (2003) A three-factor analytic model of the MADRS in geriatric depression. International Journal of Geriatric Psychology, 18(1), 73-77, DOI: 10.1002/gps.776
65. Peterson R.A. (1994) A meta-analysis of Cronbach's coefficient alpha. Journal of Consumer Research, 21(2), 381-391, DOI: 10.1086/209405
66. Powell R. (2003) Psychometric properties of the Beck Depression Inventory and the Zung Self Rating Depression Scale in adults with mental retardation. Mental Retardation, 41(2), 88-95, DOI: 10.1352/0047-6765(2003)041<0088: PPOTBD>2.0.CO;2
67. R Development Core Team. (2008) R: A Language and Environment for Statistical Computing, Vienna, R Foundation for Statistical Computing.
68. Revelle W. (2008) R Documentation: Procedures for Personality, Psychometric, and Psychological Research, help pages for package 'psych' version 1.0-58: Principal Axis Factor Analysis (function: fa): CRAN. [13th October 2011].
69. Salamero M., Marcos T., Gutierrez F., Rebull E. (1994) Factorial study of the BDI in pregnant women. Psychological Medicine, 24(4), 1031-1035.
70. Salokangas R.K., Honkonen T., Stengard E., Koivisto A.M. (2002) Symptom dimensions and their association with outcome and treatment setting in long-term schizophrenia. Results of the DSP project. Nordic Journal of Psychiatry, 56(5), 319-327, DOI: 10.1080/080394802760322079
71. Sanchez-Lopez M.P., Dresch V. (2008) The 12-Item General Health Questionnaire (GHQ-12): Reliability, external validity and factor structure in the Spanish population. Psicothema, 20(4), 839-843.
72. Serretti A., Jori M.C., Casadei G., Ravizza L., Smeraldi E., Akiskal H. (1999) Delineating psychopathologic clusters within dysthymia: A study of 512 out-patients without major depression. Journal of Affective Disorders, 56(1), 17-25, DOI: S0165-0327(99)00056-7 [pii]
73. Shek D.T. (1990) Reliability and factorial structure of the Chinese version of the Beck Depression Inventory. Journal of Clinical Psychology, 46(1), 35-43, DOI: 10.1002/1097-4679(199001)46:1<35: AID-JCLP2270460106>3.0.CO; 2-W
74. Sijtsma K. (2009a) On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1), 107-120, DOI: 10.1007/S11336-008-9101-0
75. Sijtsma K. (2009b) Reliability beyond theory and into practice. Psychometrika, 74(1), 169-173, DOI: 10.1007/S11336-008-9103-Y
76. Smith A.B., Selby P.J., Velikova G., Stark D., Wright E.P., Gould A., Cull A. (2002) Factor analysis of the Hospital Anxiety and Depression Scale from a large cancer population. Psychology and Psychotherapy, 75(Pt 2), 165-176.
77. Steer R.A., Beck A.T., Brown G. (1989) Sex differences on the revised Beck Depression Inventory for outpatients with affective disorders. Journal of Personal Assessment, 53(4), 693-702, DOI: 10.1207/s15327752jpa5304_6
78. Steer R.A., Kumar G., Ranieri W.F., Beck A.T. (1995) Use of the Beck Anxiety Inventory with adolescent psychiatric outpatients. Psychological Reports, 76(2), 459-65.
79. Steer R.A., Rissmiller D.J., Ranieri W.F., Beck A.T. (1993) Structure of the computer-assisted Beck Anxiety Inventory with psychiatric inpatients. Journal of Personal Assessment, 60(3), 532-542, DOI: 10.1207/s15327752jpa6003_10
80. Uslu R.I., Kapci E.G., Oncu B., Ugurlu M., Turkcapar H. (2008) Psychometric properties and cut-off scores of the Beck Depression Inventory-II in Turkish adolescents. Journal of Clinical Psychological Medicine S15(3), 225-233, DOI: 10.1007/s10880-008-9122-y
81. Velicer W.F., Fava J.L. (1998) Effects of variable and subject sampling on factor pattern recovery. Psychological Methods, 3(2), 231-251, DOI: 10.1037/1082-989X.3.2.231
82. Velicer W.F., Peacock A.C., Jackson D.N. (1982) A comparison of component and factor patterns: A Monte Carlo approach. Multivariate Behavioral Research, 17(3), 371-388, DOI: 10.1207/s15327906mbr1703_5
83. Ventura J., Nuechterlein K.H., Subotnik K.L., Gutkind D., Gilbert E.A. (2000) Symptom dimensions in recent-onset schizophrenia and mania: A principal components analysis of the 24-item Brief Psychiatric Rating Scale. Psychiatry Research, 97(2-3), 129-135, DOI: S0165178100002286 [pii]
84. Villalta-Gil V., Vilaplana M., Ochoa S., Dolz M., Usall J., Haro J.M., Almenara J., Gonzalez J.L., Lagares C. (2006) Four symptom dimensions in outpatients with schizophrenia. Comprehensive Psychiatry, 47(5), 384-388, DOI: S0010-440X(06)00019-8 [pii] 10.1016/j.comppsych.2006.01.005
85. Wang Y.P., Andrade L.H., Gorenstein C. (2005) Validation of the Beck Depression Inventory for a Portuguese-speaking Chinese community in Brazil. Brazilian Journal of Medical and Biological Research, 38(3), 399-408, DOI: S0100-879X2005000300011 [pii] /S0100-879X2005000300011
86. Werneke U., Goldberg D.P., Yalcin I., Ustun B.T. (2000) The stability of the factor structure of the General Health Questionnaire. Psychological Medicine, 30(4), 823-829, DOI: 10.1017/S0033291799002287
87. Widaman K.F. (1993) Common factor analysis versus principal component analysis: Differential bias in representing model parameters? Multivariate Behavioral Research, 28(3), 263-311, DOI: 10.1207/s15327906mbr2803_1
88. Woolrich R.A., Kennedy P., Tasiemski T. (2006) A preliminary psychometric evaluation of the Hospital Anxiety and Depression Scale (HADS) in 963 people living with a spinal cord injury. Psychology, Health & Medicine, 11(1), 80-90, DOI: J07M627G23552610 [pii] 10.1080/13548500500294211