Using effect sizes for research reporting: Examples using item response theory to analyze differential item functioning

Psychological Methods

Volumn 11, Issue 4, 2006, Pages 402-415

  Steinberg, Lynne ^a   Thissen, David ^b  

^a UNIVERSITY OF HOUSTON   (United States)

^b University of North Carolina at Chapel Hill   (United States)

Author keywords

DIF; Differential item functioning; Effect size; IRT; Item response theory

Indexed keywords

ARTICLE; HUMAN; METHODOLOGY; MONTE CARLO METHOD; PSYCHOLOGICAL MODEL; PSYCHOLOGY; SAMPLE SIZE; STATISTICS;

HUMANS; MODELS, PSYCHOLOGICAL; MONTE CARLO METHOD; PSYCHOLOGY; SAMPLE SIZE;

EID: 33845613579     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/1082-989X.11.4.402     Document Type: Article

References (51)

1. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
2. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Reading, MA: Addison-Wesley.
3. Bock, R. D., & Jones, L. V. (1968). The measurement and prediction of judgment and choice. New York: Holden-Day.
4. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
5. Collins, W. C., Raju, N. S., & Edwards, J. E. (2000). Assessing differential functioning in a satisfaction scale. Journal of Applied Psychology, 85, 451-461.
6. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Mahwah, NJ: Erlbaum.
7. Ellis, B. B. (1989). Differential item functioning: Implications for test translations. Journal of Applied Psychology, 74, 912-921.
8. Graham, J. R. (2000). MMPI-2: Assessing personality and psychopathology. New York: Oxford University Press.
9. Green, B. F. (1954). Attitude measurement. In G. Lindzey (Ed.), Handbook of social psychology (Vol. 1, pp. 335-369). Cambridge, MA: Addison-Wesley.
10. Hamilton, J. C., & Shuminsky, T. R. (1990). Self-awareness mediates the relationship between serial position and item reliability. Journal of Personality and Social Psychology, 59, 1301-1307.
11. Hancock, T. D. (1999). Differential trait and symptom functioning of MMPI-2 items in substance abusers and the restandardization sample: An item response theory approach. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.
12. Hedges, L. V. (1992). Meta-analysis. Journal of Educational Statistics, 17, 279-296.
13. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
14. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145), Hillsdale, NJ: Erlbaum.
15. Holland, P. W., & Wainer, H. (1993). Differential item functioning. Mahwah, NJ: Erlbaum.
16. Jones, L. V., & Tukey, J. H. (2000). A sensible formulation of the significance test. Psychological Methods, 5, 411-414.
17. Jorgensen, R., & Steinberg, L. (2006). Development and validation of anger experience and expression scales using the methods of item response theory. Unpublished manuscript.
18. Khoo, S. T. (2001). Assessing program effects in the presence of treatment-baseline interactions: A latent curve approach. Psychological Methods, 6, 234-257.
19. Knowles, E. S. (1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 57, 351-357.
20. Lazarsfeld, P. F. (1950). The logical and mathematical foundation of latent structure analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfeld, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction (pp. 362-412). New York: Wiley.
21. Li, H., & Stout, W. (1996). A new procedure for detecting crossing DIF. Psychometrika, 61, 647-677.
22. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
23. Monahan, P. O., McHorney, C. A., Stump, T. E., & Perkins, A. J. (in press). Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. Journal of Educational and Behavioral Statistics.
24. Nagin, D. S., & Tremblay, R. E. (2001). Analyzing developmental trajectories of distinct but related behaviors: A group-based method. Psychological Methods, 6, 18-34.
25. Orlando, M., & Marshall, G. N. (2002). Differential item functioning in a Spanish translation of the PTSD checklist: Detection and evaluation of impact. Psychological Assessment, 14, 50-59.
26. Orlando, M., Thissen, D., Teresi, J., Kleinman, M., & Ocepek-Welikson, K. (in press). Identification of differential item functioning using item response theory and the likelihood-model comparison approach: Application to the Mini-Mental Status Examination. Medical Care.
27. Penny, J., & Johnson, R. L. (1999). How group differences in matching criterion distribution and IRT item difficulty can influence the magnitude of the Mantel-Haenszel chi-square DIF index. Journal of Experimental Education, 67, 343-366.
28. Raju, N. S., van der Linden, W. J., & Fleer, P. F. (1995). IRT-based internal measures of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
29. Reeve, B. B. (2000). Item- and scale-level analysis of clinical and nonclinical sample responses to the MMPI-2 depression scales employing item response theory. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.
30. Roussos, L., Schnipke, D., & Pashley, P. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Behavioral and Educational Statistics, 24, 293-322.
31. Roussos, L., & Stout, W. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
32. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, 34(17, Pt. 2).
33. Samejima, F. (1997). Graded response model. In W. J. van der Linden & Ronald K. Hambleton (Eds.), Handbook of item response theory (pp. 85-100). New York: Springer.
34. Santor, D. A., Ramsay, J. O., & Zuroff, D. C. (1994). Nonparametric item analyses of the Beck Depression Inventory: Evaluating gender item bias and response option weights. Psychological Assessment, 6, 255-270.
35. Schaeffer, N. C. (1988). An application of item response theory to the measurement of depression. In C. C. Clogg (Ed.), Sociological methodology, (Vol. 18, pp. 271-307). Washington, DC: American Sociological Association.
36. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
37. Spielberger, C. D., Gorsuch, R. L., & Lushene, R. E. (1979). Manual for the STAI. Palo Alto, CA: Consulting Psychologists Press.
38. Steinberg, L. (1994). Context and serial-order effects in personality measurement: Limits on the generality of measuring changes the measure. Journal of Personality and Social Psychology, 66, 341-349.
39. Steinberg, L. (2001). The consequences of pairing questions: Context effects in personality measurement. Journal of Personality and Social Psychology, 81, 332-342.
40. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
41. Thissen, D. (2001). IRTLRDIF V. 2.0: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning [Computer software]. Retrieved February 1, 2001, from http://www.unc.edu/~dthissen/dl.html
42. Thissen, D., & Steinberg, L. (1988). Data analysis using item response theory. Psychological Bulletin, 104, 385-395.
43. Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Erlbaum.
44. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Mahwah, NJ: Erlbaum.
45. Thissen, D., & Wainer, H. (1990). Confidence envelopes for item response theory. Journal of Educational Statistics, 15, 113-128.
46. Tukey, J. W. (1991). The philosophy of multiple comparisons. Statistical Science, 6, 100-116.
47. Wainer, H. (1993). Model-based standardized measurement of an item's differential impact. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 123-135). Mahwah, NJ: Erlbaum.
48. Wainer, H., Sireci, S. G., & Thissen, D. (1991). Differential testlet functioning: Definitions and detection. Journal of Educational Measurement, 28, 197-219.
49. Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 595-604.
50. Williams, V. S. L. (1993). An evaluation of item response theory for the investigation of differential item functioning. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.
51. Williams, V. S. L. (1997). The "unbiased" anchor: Bridging the gap between DIF and item bias. Applied Measurement in Education, 10, 353-367.