Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS®): Depression, anxiety, and anger

Assessment

Volumn 18, Issue 3, 2011, Pages 263-283

  Pilkonis, Paul A ^a   Choi, Seung W ^b   Reise, Steven P ^c   Stover, Angela M ^a   Riley, William T ^d   Cella, David ^b  

^a UNIVERSITY OF PITTSBURGH   (United States)

^b NORTHWESTERN UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d NATIONAL HEART LUNG AND BLOOD INSTITUTE   (United States)

Author keywords

anger; anxiety; depression; item response theory; measurement

Indexed keywords

ADOLESCENT; ADULT; AGED; ANGER; ANXIETY; ARTICLE; DEPRESSION; EMOTION; FACTORIAL ANALYSIS; FEMALE; HUMAN; INFORMATION PROCESSING; INFORMATION SYSTEM; INSTRUMENTATION; MALE; MENTAL STRESS; METHODOLOGY; MIDDLE AGED; ORGANIZATION AND MANAGEMENT; PSYCHOLOGIC TEST; PSYCHOLOGICAL ASPECT; PSYCHOLOGICAL MODEL; PSYCHOLOGICAL THEORY; PSYCHOMETRY; QUALITATIVE RESEARCH; QUESTIONNAIRE; STATISTICS;

ADOLESCENT; ADULT; AGED; ANGER; ANXIETY; DEPRESSION; EMOTIONS; FACTOR ANALYSIS, STATISTICAL; FEMALE; FOCUS GROUPS; HUMANS; INFORMATION SYSTEMS; INTERVIEW, PSYCHOLOGICAL; MALE; MIDDLE AGED; MODELS, PSYCHOLOGICAL; PSYCHOLOGICAL THEORY; PSYCHOMETRICS; QUALITATIVE RESEARCH; QUESTIONNAIRES; STATISTICS AS TOPIC; STRESS, PSYCHOLOGICAL; YOUNG ADULT;

EID: 79961190293     PISSN: 10731911     EISSN: 15523489     Source Type: Journal    
DOI: 10.1177/1073191111411667     Document Type: Article

References (110)

1. American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: Author.
2. Beck, A.T., Epstein, N., Brown, G., & Steer, R.A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893-897.
3. Bejar, I.I. (1977). An application of the continuous response level model to personality measurement. Applied Psychological Measurement, 1, 509-521.
4. Bernstein, I.H., Rush, A.J., Thomas, C.J., Woo, A., & Trivedi, M.H. (2006). Item response analysis of the Inventory of Depressive Symptomatology. Neuropsychiatric Disease and Treatment, 2, 557-564.
5. Berzon, R., Patrick, D., Guyatt, G., & Conley, J.M. (1994). Intellectual property considerations in the development and use of HRQL measures for clinical trial research. Quality of Life Research, 3, 273-277.
6. Bjorner, J.B., Chang, C.H., Thissen, D., & Reeve, B.B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16, 95-108.
7. Bjorner, J.B., Smith, K.J., Orlando, M., Stone, C.A., Thissen, D., & Xiaowa, S. (2006). IRTFIT: A macro for item fit and local dependence tests under IRT models [Computer software]. Chapel Hill, NC: L. L. Thurstone Psychometric Laboratory.
8. Bock, R.D. (1997). A brief history of item response theory. Educational Measurement: Issues and Practice, 16, 21-33.
9. Bode, R.K., Lai, J.S., Cella, D., & Heinemann, A.W. (2003). Issues in the development of an item bank. Archives of Physical Medicine and Rehabilitation, 84(Suppl.2), S52-S60.
10. Brod, M., Tesler, L.E., & Christensen, T.L. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research, 18, 1263-1278.
11. Brown, T.A., Chorpita, B.F., & Barlow, D.H. (1998). Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology, 107, 179-192.
12. Buss, A.H., & Perry, M. (1992). The aggression questionnaire. Journal of Personality and Social Psychology, 59, 73-81.
13. Buysse, D.J., Moul, D.E., Germain, A., Yu, L., Stover, A.M., Dodds, N.E., Pilkonis, P. A. (2010). Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep, 33, 781-792.
14. Carmody, T.J., Rush, A.J., Bernstein, I., Warden, D., Brannan, S., Burnham, D., Trivedi, M.H. (2006). The Montgomery Asberg and the Hamilton ratings of depression: A comparison of measures. European Neuropsychopharmacology, 16, 601-611.
15. Cella, D., Gershon, R., Lai, J.-S., & Choi, S. (2007). The future of outcomes measurement: Item banking, tailored short forms, and computerized adaptive assessment. Quality of Life Research, 16, 133-144.
16. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Hays, R.D. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63, 1179-1194.
17. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., Matthias, R. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(Suppl.1), S3-S11.
18. Chang, C.-H., Cella, D., Clarke, S., Heinemann, A., Von Roenn, J.H., & Harvey, R. (2003). Should symptoms be scaled for intensity, frequency, or both? Palliative & Supportive Care, 1,51-60.
19. Chang, C.-H., & Reeve, B.B. (2005). Item response theory and its applications to patient-reported outcomes measurement. Evaluation & the Health Professions, 28, 264-282.
20. Choi, S.W., Reise, S.P., Pilkonis, P.A., Hays, R.D., & Cella, D. (2010). Efficiency of static and computer adaptive short forms compared to full length measures of depressive symptoms. Quality of Life Research, 19, 125-136.
21. Choi, S.W., & Swartz, J.R. (2009). Comparison of CAT item selection criteria for polytomous items. Applied Psychological Measurement, 33, 419-440.
22. Chorpita, B.F., Albano, A.M., & Barlow, D.H. (1998). The structure of negative emotions in a clinical sample of children and adolescents. Journal of Abnormal Psychology, 107, 74-85.
23. Clark, D.C., vonAmmon Cavanaugh, S., & Gibbons, R.D. (1983). The core symptoms of depression in medical and psychiatric patients. Journal of Nervous and Mental Disease, 171, 705-713.
24. Clark, L.A., & Watson, D. (1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100, 316-336.
25. Cox, D.E., & Harrison, D.W. (2008). Models of anger: Contributions from psychophysiology, neuropsychology, and the cognitive behavioral perspective. Brain Structure & Function, 212, 371-385.
26. de Jong-Gierveld, J., & Kamphuis, F. (1985). The development of a Rasch-type loneliness scale. Applied Psychological Measurement, 9, 289-299.
27. Derogatis, L.R. (1983). SCL-90-R: Administration, scoring and procedures manual II. Towson, MD: Clinical Psychometric Research.
28. DeWalt, D.A., Rothrock, N., Yount, S., & Stone, A.A. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(Suppl. 1), S12-S21.
29. Fliege, H., Becker, J., Walter, O.B., Bjorner, J.B., Klapp, B.F., & Rose, M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14, 2277-2291.
30. Food and Drug Administration. (2009). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. Retrieved fromhttp://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf.
31. Forbey, J.D., & Ben-Porath, Y.S. (2007). Computerized adaptive personality testing: A review and illustration with the MMPI-2 computerized adaptive version. Psychological Assessment, 19, 14-24.
32. Forgays, D.K., Spielberger, C.D., Ottaway, S.A., & Forgays, D.G. (1998). Factor structure of the State-Trait Anger Expression Inventory for middle-aged men and women. Assessment, 5, 141-155.
33. Fries, J.F., Cella, D., Rose, M., Krishnan, E., & Bruce, B. (2009). Progress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing. Journal of Rheumatology, 36, 2061-2066.
34. Gardner, W., Kelleher, K.J. & Pajer, K.A. (2002). Multidimensional adaptive testing for mental health problems in primary care. Medical Care, 40, 812-823.
35. Gardner, W., Shear, K., Kelleher, K.J., Pajer, K.A., Mammen, O., Buysse, D., & Frank, E. (2004). Computerized adaptive measurement of depression: A simulation study. BMC Psychiatry, 4, 13.
36. Gershon, R.C., Rothrock, N.E., Hanrahan, R., Jansky, L.J., Harniss, M., & Riley, W. (2010). Development of a clinical outcomes research survey application: Assessment Center. Quality of Life Research, 19, 677-685.
37. Gibbons, R.D., Bock, D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, R., Stover, A. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, 4-19.
38. Gibbons, R.D., Clark, D.C., vonAmmon Cavanaugh, S., & Davis, J.M. (1985). Application of modern psychometric theory in psychiatric research. Journal of Psychiatric Research, 19, 43-55.
39. Gibbons, R.D., & Hedeker, D. (1992). Full-information item bi-factor analysis. Psychometrika, 57, 423-436.
40. Gibbons, R.D., Weiss, D.J., Kupfer, D.J., Frank, E., Fagiolini, A., Grochocinski, V.J., Immekus, J.C. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, 361-368.
41. Gollwitzer, M., Eid, M., & Jurgensen, R. (2005). Response styles in the assessment of anger expression. Psychological Assessment, 17, 56-69.
42. Gray, J.A. (1987). The psychology of fear and stress (2nd ed.). Cambridge, England: Cambridge University Press.
43. Hawthorne, G., Mouthaan, J., Forbes, D., & Novaco, R.W. (2006). Response categories and anger measurement: Do fewer categories result in poorer measurement? Development of the DAR5. Social Psychiatry & Psychiatric Epidemiology, 41, 164-172.
44. Hays, R.D., Bjorner, J., Revicki, D.A., Spritzer, K., & Cella, D. (2009). Development of physical and mental health summary scores from the Patient-Reported Outcomes Measurement Information System (PROMIS) global items. Quality of Life Research, 18, 873-880.
45. Kelly, M.A.R., Morse, J.Q., Stover, A., Hofkens, T., Huisman, E., Shulman, S., Pilkonis, P.A. (2011). Describing depression: Congruence between patient experiences and clinical assessments. British Journal of Clinical Psychology, 50, 46-66.
46. Kendel, F., Wirtz, M., Dunkel, A., Lehmkuhl, E., Hetzer, R., & Regitz-Zagrosek, V. (2010). Screening for depression: Rasch analysis of the dimensional structure of the PHQ-9 and the HADS-D. Journal of Affective Disorders, 122, 241-246.
47. Kim, Y., Pilkonis, P.A., Frank, E., Thase, M.E., & Reynolds, C.F., III. (2002). Differential functioning of the Beck Depression Inventory in late-life patients: Use of item response theory. Psychology and Aging, 17, 379-391.
48. Kingsbury, G.G., & Weiss, D.J. (1980). An alternate-forms reliability and concurrent validity comparison of Bayesian adaptive and conventional ability tests (Research Report 80-5). Minneapolis, MN: University of Minnesota.
49. Kingsbury, G.G., & Weiss, D.J. (1983). A comparison of IRTbased adaptive mastery testing and a sequential mastery testing procedure. In D. J. Weiss (Ed.), New horizons in testing: Latent trait theory and computerized adaptive testing (pp. 257-283). New York, NY: Academic Press.
50. Klem, M.L., Saghafi, E., Abromitis, R., Stover, A., Dew, M.A., & Pilkonis, P.A. (2009). Building PROMIS item banks: Librarians as co-investigators. Quality of Life Research, 18, 881-888.
51. Krueger, R.F. (1999). The structure of common mental disorders. Archives of General Psychiatry, 56, 921-926.
52. Krueger, R.F., & Finger, M.S. (2001). Using item response theory to understand comorbidity among anxiety and unipolar mood disorders. Psychological Assessment, 13, 140-151.
53. Krueger, R.F., McGue, M., & Iacono, W.G. (2001). The higher-order structure of common DSM mental disorders: Internalization, externalization, and their connections to personality. Personality and Individual Differences, 30, 1245-1259.
54. Lara, D.G., Pinto, O., Akiskal, K., & Akiskal, H.S. (2006). Toward an integrative model of the spectrum of mood, behavioral, and personality disorders based on fear and anger traits: 1. Clinical implications. Journal of Affective Disorders, 94, 67-87.
55. Liu, H., Cella, D., Gershon, R., Shen, J., Morales, L.S., Riley, W., & Hays, R.D. (2010). Representativeness of the Patient-Reported Outcomes Measurement Information System Inter-net panel. Journal of Clinical Epidemiology, 63, 1169-1178.
56. Loevinger, J. (1948). The technique of homogenous tests compared with some aspects of scale analysis and factor analysis. Psychological Bulletin, 45, 507-530.
57. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
58. Martin, R., Wan, C., David, J.P., Wegner, E., Olson, B.D., & Watson, D. (2000). Style of anger expression: Relation to expressivity, personality, and health. Personality and Social Psychology Bulletin, 25, 1196-1207.
59. Martin, R., Watson, D., & Wan, C.K. (2000). A three-factor model of trait anger: Dimensions of affect, behavior, and cognition. Journal of Personality, 68, 869-897.
60. McBride, J.R., & Martin, J.R. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.), New horizons in testing: Latent trait theory and computerized adaptive testing (pp. 223-236). New York, NY: Academic Press.
61. McHugh, R.K., & Behar, E. (2009). Readability of self-report measures of depression and anxiety. Journal of Consulting and Clinical Psychology, 77, 1100-1112.
62. Meijer, R.R., & Baneke, J.J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9, 354-368.
63. MetaMetrics, Inc. (2008). What does the Lexile measure mean? Durham, NC: Author.
64. Mokkink, L.B., Terwee, C.B., Stratford, P.W., Alonso, J., Patrick, D.L., Riphagen, I., de Vet, H.C.W. (2009). Evaluation of the methodological quality of systematic reviews of health status measurement instruments. Quality of Life Research, 18, 313-333.
65. Molenaar, I.W., & Sijtsma, K. (2000). User's manual for MSP5 for Windows. Groningen, Netherlands: IEC ProGAMMA.
66. Muhr, T., & Freise, S. (2004). User's manual for ATLAS.ti 5.0 (2nd ed.). Berlin, Germany: Scientific Software Development.
67. Mui, A.C., Burnette, D., & Chen, L.M. (2001). Cross-cultural assessment of geriatric depression: A review of the CES-D and GDS. Journal of Mental Health and Aging, 7, 137-164.
68. Muthén, L.K., & Muthén, B.O. (2006). Mplus user's guide. Los Angeles, CA: Author.
69. Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289-298.
70. Pallant, J.F., & Tennant, A. (2007). An introduction to the Rasch measurement model: An example using the Hospital Anxiety and Depression Scale (HADS). British Journal of Clinical Psychology, 46, 1-18.
71. Pedersen, R.D., Pallay, A.G., & Rudolph, R.L. (2002). Can improvement in well-being and functioning be distinguished from depression improvement in antidepressant clinical trials? Quality of Life Research, 11, 9-17.
72. Polimetrix, Inc. (2006). Scientific sampling for online research. Palo Alto, CA: Author.
73. Quilty, L.C., Zhang, K.A., & Bagby, M.R. (2010). The latent symptom structure of the Beck Depression Inventory-II in outpatients with major depression. Psychological Assessment, 22, 603-608.
74. Radloff, L.S. (1977). A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385-401.
75. Ramsay, J.O. (2000). TestGraph: A program for the graphical analysis of multiple choice test and questionnaire data [Computer software and manual]. Montreal, Quebec, Canada: McGill University.
76. Reeve, B.B., Hays, R.D., Bjorner, J.B., Cook, K.F., Crane, P.K., Teresi, J.A., Cella, D. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(Suppl.1), S22-S31.
77. Reise, S.P., Moore, M.T., & Haviland, M.G. (IN PRESS). Applying item response theory models to psychological data. In K. F. Geisinger (Ed.), Handbook of testing and assessment in psychology. Washington, DC: American Psychological Association.
78. Reise, S.P., Morizot, J., & Hays, R.D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19-31.
79. Reise, S.P., & Waller, N.G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 25-46.
80. Revicki, D.A., Chen, W.-H., Harnam, N., Cook, K., Amtmann, D., Callahan, L.F., Keefe, F.J. (2009). Development and psychometric analysis of the PROMIS pain behavior item bank Pain, 146, 158-169.
81. Revicki, D.A., & Schwartz, C.E. (2009). Intellectual property rights and good research practice. Quality of Life Research, 18, 1279-1280.
82. Revicki, D.A., & Sloan, J. (2007). Practical and philosophical issues surrounding a national item bank: If we build it will they come? Quality of Life Research, 16, 167-174.
83. Riley, W.T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E.A., & Cella, D. (2010). Patient-Reported Outcomes Measurement Information System (PROMIS) domain names and definitions revisions: Further assessment of content validity in IRT-derived item banks. Quality of Life Research, 19, 1311-1321.
84. Roberson-Nay, R., Strong, D.R., Nay, W.T., Beidel, D.C., & Turner, S.M. (2007). Development of an abbreviated Social Phobia and Anxiety Inventory (SPAI) using item response theory: The SPAI-23. Psychological Assessment, 19, 133-145.
85. Roth, W.M., & Roychoudhury, A. (1991). Nonmetric multidimensional item analysis in the construction of an anxiety attitude survey. Educational and Psychological Measurement, 51, 931-942.
86. Rothrock, N.E., Hays, R.D., Spritzer, K., Yount, S.E., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 63, 1195-1204.
87. Santor, D.A., Gregus, M., & Welch, A. (2006). Eight decades of measurement in depression. Measurement, 4, 135-155.
88. Shafer, A.B. (2006). Meta-analysis of the factor structures of four depression questionnaires: Beck, CES-D, Hamilton, and Zung. Journal of Clinical Psychology, 62, 123-146.
89. Sijtsma, K., Meijer, R.R., & van der Ark, L.A. (2011). Mokken Scale Analysis as time goes by: An update for scaling practitioners. Personality and Individual Differences, 50, 31-37.
90. Simms, L.J., Gros, D.F., Watson, D., & O'Hara, M.W. (2008). Parsing the general and specific components of depression and anxiety with bifactor modeling. Depression and Anxiety, 25, E34-E46.
91. Steinberg, L., & Thissen, D. (1996). Use of item response theory and the testlet concept in the measurement of psychopathology. Psychological Methods, 1, 81-97.
92. Stone, C.A. (1992). Recovery of marginal maximum likelihood estimates in the two-parameter logistic response model: An evaluation of MULTILOG. Applied Psychological Measurement, 16, 1-16.
93. Teresi, J.A., Ocepek-Welikson, K., Kleinman, M., Eimicke, J.P., Crane, P.K., Jones, R.N., Cella, D. (2009). Analysis of differential item functioning in the depression item bank from the Patient Reported Outcome Measurement Information System (PROMIS): An item response theory approach. Psychology Science Quarterly, 51, 148-180.
94. Thissen, D., Chen, W.-H., & Bock, R.D. (2003). MULTILOG (version 7) [Computer software]. Lincolnwood, IL: Scientific Software International.
95. Thissen, D., & Steinberg, L. (1983). An item response theory for personality and attitude scales: Item analysis using restricted factor analysis. Applied Psychological Measurement, 7, 211-226.
96. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Lawrence Erlbaum.
97. Vancleef, L.M.G., Vlaeyen, J.W.S., & Peters, M.L. (2009). Dimensional and componential structure of a hierarchical organization of pain-related anxiety constructs. Psychological Assessment, 21, 340-351.
98. Waller, N.G., & Reise, S.P. (1989). Computerized adaptive personality assessment: An illustration with the Absorption scale. Journal of Personality and Social Psychology, 57, 1051-8.
99. Walter, O.B., Becker, J., Bjorner, J.B., Fliege, H., Klapp, B.F., & Rose, M. (2007). Development and evaluation of a computer adaptive test for anxiety (Anxiety-CAT). Quality of Life Research, 16, 143-155.
100. Ware, J.E. (2003). Conceptualization and measurement of health-related quality of life: Comments on an evolving field. Archives of Physical Medicine and Rehabilitation, 84 (Suppl. 2), S43-S51.
101. Watson, D., Clark, L.A., Weber, K., Assenheimer, J.A., Strauss, M.E., & McCormick, R.A. (1995a). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptoms. Journal of Abnormal Psychology, 104, 3-14.
102. Watson, D., Clark, L.A., Weber, K., Assenheimer, J.S., Strauss, M.E., & McCormick, R.A. (1995b). Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. Journal of Abnormal Psychology, 104, 15-25.
103. Weiss, D.J. (1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53, 774-789.
104. Weiss, D.J. (2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37, 70-84.
105. Willis, G.B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: SAGE.
106. Winkielman, P., Knäuper, B., & Schwarz, N. (1998). Looking back at anger: Reference periods change the interpretation of emotion frequency questions. Journal of Personality and Social Psychology, 75, 719-728.
107. Woods, C.M. (2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253-270.
108. World Health Organization. (2007). Constitution of the World Health Organization: Basic documents (46th ed.). Geneva, Switzerland: Author.
109. Zinbarg, R.E., & Barlow, D.H. (1996). Structure of anxiety and the anxiety disorders: A hierarchical model. Journal of Abnormal Psychology, 105, 181-193.
110. Zumbo, B.D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.