Measuring Constructs in Family Science: How Can Item Response Theory Improve Precision and Validity?

Journal of Marriage and Family

Volumn 77, Issue 1, 2015, Pages 147-176

  Gordon, Rachel A ^a  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

Author keywords

measurement; Methods; Theory

Indexed keywords

EID: 84920936900     PISSN: 00222445     EISSN: 17413737     Source Type: Journal    
DOI: 10.1111/jomf.12157     Document Type: Article

References (113)

1. Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and Profile. Burlington: University of Vermont, Department of Psychiatry.
2. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington: University of Vermont, Research Center for Children, Youth, & Families.
3. Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral training in statistics, measurement, and methodology in psychology. American Psychologist, 63, 32-50. doi:10.1037/0003-066X.63.1.32
4. Aiken, L. S., West, S. G., Sechrest, L. B., & Reno, R. R. (1990). Graduate training in statistics, methodology, and measurement in psychology. American Psychologist, 45, 721-734. doi:10.1037/0003-066X.45.6.721
5. Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42, I7-I16. doi:10.1097/01.mlr.0000103528.48582.7c
6. Asparhouhov, T. (2005). Sampling weights in latent variable modeling. Structural Equation Modeling, 12, 411-434. doi:10.1207/s15328007sem1203_4
7. Barnes, G. M., Hoffman, J. H., Welte, J. W., Farrell, M. P., & Dintcheff, B. A. (2006). Effects of parental monitoring and peer deviance on substance use and delinquency. Journal of Marriage and Family, 68, 1084-1104. doi:10.1111/j.1741-3737.2006.00315.x
8. Bauer, D. J., & Hussong, A. M. (2009). Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods, 14, 101-125. doi:10.1037/a0015583
9. Bingenheimer, J. B., Raudenbush, S. W., Leventhal, T., & Brooks-Gunn, J. (2005). Measurement equivalence and differential item functioning in family psychology. Journal of Family Psychology, 19, 441-455. doi:10.1037/0893-3200.19.3.441
10. Black, R. A., & Butler, S. F. (2012). Using the GLIMMIX procedure in SAS 9.3 to fit a standard dichotomous Rasch and hierarchical 1-PL IRT model. Applied Psychological Measurement, 36, 237-248. doi:10.1177/0146621612441857
11. Blinkhorn, S. F. (1997). Past imperfect, future conditional: Fifty years of test theory. British Journal of Mathematical and Statistical Psychology, 50, 175-185. doi:10.1111/j.2044-8317.1997.tb01139.x
12. Bock, R. D., Gibbons, R., & Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261-280. doi:10.1177/014662168801200305
13. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Erlbaum.
14. Bridges, M., Cohen, S. R., McGuire, L. W., Yamada, H., Fuller, B., Mireles, L., & Scott, L. (2012). "Bien educado": Measuring the social behaviors of Mexican American children. Early Childhood Research Quarterly, 27, 555-567. doi:10.1016/j.ecresq.2012.01.005
15. Brown, T. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.
16. Browning, C. R. (2002). The span of collective efficacy: Extending social disorganization theory to partner violence. Journal of Marriage and Family, 64, 833-850. doi:10.1111/j.1741-3737.2002.00833.x
17. Browning, C. R., & Burrington, L. A. (2006). Racial differences in sexual and fertility attitudes in an urban setting. Journal of Marriage and Family, 68, 236-351. doi:10.1111/j.1741-3737.2006.00244.x
18. Buehler, C. (2006). Parents and peers in relation to early adolescent problem behavior. Journal of Marriage and Family, 68, 109-124. doi:10.1111/j.1741-3737.2006.00237.x
19. Cai, L. (2013). flexMIRT Version 2: Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.
20. Cai, L., Du Toit, S. H. C., & Thissen, D. (2011). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Chicago: Scientific Software International.
21. Chalmers, R. P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. Retrieved from www.jstatsoft.org/v48/i06/paper
22. Chen, W., & Thissen, D. (1997). Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265-289. doi:10.2307/1165285
23. Cho, Y. I., Martin, M. J., Conger, R. D., & Widaman, K. F. (2010). Differential item functioning on antisocial behavior scale items for adolescents and young adults from single-parent and two-parent families. Journal of Psychopathology and Behavioral Assessment, 32, 157-168. doi:10.1007/s10862-009-9145-1
24. Chorpita, B. F., Reise, S., Weisz, J. R., Grubbs, K., Becker, K. D., Krull, J. L., & The Research Network on Youth Mental Health. (2010). Evaluation of the Brief Problem Checklist: Child and caregiver interviews to measure clinical progress. Journal of Consulting and Clinical Psychology, 78, 526-536. doi:10.1037/a0019602
25. Cole, J. C., Rabin, A. S., Smith, T. L., & Kaufman, A. S. (2004). Development and validation of a Rasch-derived CES-D short form. Psychological Assessment, 16, 360-372. doi:10.1037/1040-3590.16.4.360
26. Coley, R. L., Ribar, D., & Votruba-Drzal, E. (2011). Do children's behavior problems limit poor women's labor market success? Journal of Marriage and Family, 73, 33-45. doi:10.1111/j.1741-3737.2010.00787.x
27. Conrad, K. J., Conrad, K. M., Mazza, J., Riley, B. B., Funk, R., Stein, M. A., & Dennis, M. L. (2012). Dimensionality, hierarchical structure, age generalizability, and criterion validity of the GAIN's behavioral complexity scale. Psychological Assessment, 24, 913-924. doi:10.1037/a0028196
28. Conrad, K. J., Riley, B. B., Conrad, K. M., Chan, Y., & Dennis, M. L. (2010). Validation of the Crime and Violence Scale (CVS) against the Rasch measurement model including differences by gender, race, and age. Evaluation Review, 34, 83-115. doi:10.1177/0193841X10362162
29. Culpepper, S. A. (2013). Reliability and precision of total scores and IRT estimates as a function of polytomous IRT parameters and latent trait distributions. Applied Psychological Measurement, 37, 201-225. doi:10.1177/0146621612470210
30. Curran, P. J., Hussong, A. M., Cai, L., Huang, W., Chassin, L., Sher, K. J., & Zucker, R. A. (2008). Pooling data from multiple longitudinal studies: The role of item response theory in integrative data analysis. Developmental Psychology, 44, 365-380. doi:10.1037/0012-1649.44.2.365
31. de Ayala, R. J. (2009). The theory and practice of item response theory. New York: Guilford Press.
32. DeWalt, D. A., Thissen, D., Stucky, B. D., Langer, M. M., DeWitt, E. M., Irwin, D. E., Varni, J. W. (2013). PROMIS Pediatric Peer Relationships Scale: Development of a peer relationships item bank as part of social health measurement. Health Psychology, 32, 1093-1103. doi:10.1037/a0032670
33. Dodd, B. G., & Koch, W. R. (1987). Effects of variations in item step values on item and test information in the partial credit model. Applied Psychological Measurement, 11, 371-384. doi:10.1177/014662168701100403
34. Draxler, C. (2010). Sample size determination for Rasch model tests. Psychometrika, 75, 708-724. doi:10.1007/s11336-010-9182-4
35. Elliott, D. S., Huizinga, D., & Ageton, S. S. (1985). Explaining delinquency and drug use. Beverly Hills, CA: Sage.
36. Embretson, S. E. (1986). Item response theory models and spurious interaction effects in factorial ANOVA designs. Applied Psychological Measurement, 20, 201-212. doi:10.1177/014662169602000302
37. Embretson, S. E., & Reise, S. R. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
38. Fincham, F. D., & Rogge, R. (2010). Understanding relationship quality: Theoretical challenges and new tools for assessment. Journal of Family Theory and Review, 2, 227-242. doi:10.1111/j.1756-2589.2010.00059.x
39. Fomby, P., & Bosick, S. J. (2013). Family instability and the transition to adulthood. Journal of Marriage and Family, 75, 1266-1287. doi:10.1111/jomf.12063
40. Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365. doi:10.1037/0022-3514.78.2.350
41. Funk, J. L., & Rogge, R. D. (2007). Testing the ruler with item response theory: Increasing precision of measurement for relationship satisfaction with the Couples Satisfaction Index. Journal of Family Psychology, 21, 572-583. doi:10.1037/0893-3200.21.4.572
42. Gibbons, R. D., Bock, R. D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., Stover, A. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, 4-19. doi:10.1177/0146621606289485
43. Gibson, C. L., Ward, J. T., Wright, J. P., Beaver, K. M., & Delisi, M. (2010). Where does gender fit in the measurement of self-control? Criminal Justice and Behavior, 37, 883-903. doi:10.1177/0093854810369082
44. Gordon, R. A. (2012). Applied statistics for the social and health sciences. New York: Routledge.
45. Gordon, R. A., Colwell, N., Fujimoto, K., Abner, K., Kaestner, R., Wakschlag, L., & Korenman, S. (2014). Rating young children's behaviors: Similarities and differences among parents, family child care providers and teachers. Chicago: Institute of Government and Public Affairs.
46. Gordon, R. A., Fujimoto, K., Kaestner, R., Korenman, S., & Abner, K. (2013). An assessment of the validity of the ECERS-R with implications for measure of child care quality and relations to child development. Developmental Psychology, 49, 146-160. doi:10.1037/a0027899
47. Guilleux, A., Blanchin, M., Hardouin, J., & Sebille, V. (2014). Power and sample size determination in the Rasch model. PLOS One, 9, e83652. doi:10.1371/journal.pone.0083652
48. Hitchcock, J. H., Nastasi, B. K., Dai, D. Y., Newman, J., Jayasena, A., Bernstein-Moore, R., Varjas, K. (2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of School Psychology, 43, 259-278. doi:10.1016/j.jsp.2005.04.007
49. Ho, A. (2014, March). Item response theory. Workshop conducted at the meeting of the Society for Research in Educational Effectiveness, Washington, DC.
50. Houts, C. R., & Edwards, M. C. (2013). The performance of local dependence measures with psychological data. Applied Psychological Measurement, 37, 541-562. doi:10.1177/0146621613491456
51. Hui, C. H., & Triandis, H. C. (1985). Measurement in cross-cultural psychology: A review and comparison of strategies. Journal of Cross-Cultural Psychology, 16, 131-152. doi:10.1177/0022002185016002001
52. IBM. (2014). Item response theory/Rasch models in SPSS statistics. Armonk, NY: Author. Retrieved from www-01.ibm.com/support/docview.wss?uid=swg21488442
53. Ip, E. H. (2001). Testing for local dependency in dichotomous and polytomous item response models. Psychometrika, 66, 109-132. doi:10.1007/BF02295736
54. Jiao, H., Kamata, A., Wang, S., & Jin, Y. (2012). A multilevel testlet model for dual local dependence. Journal of Educational Measurement, 49, 82-100. doi:10.1111/j.1745-3984.2011.00161.x
55. Johnson, S., Li, J., Kendall, G., Strazdins, L., & Jacoby, P. (2013). Mothers' and fathers' work hours, child gender, and behavior in middle childhood. Journal of Marriage and Family, 75, 56-74. doi:10.1111/j.1741-3737.2012.01030.x
56. Joint Committee on Standards for Educational and Psychological Testing of the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
57. Joint Committee on Standards for Educational and Psychological Testing of the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
58. Jöreskog, K. G., & Moustaki, I. (2001). Factor analysis of ordinal variables: A comparison of three approaches. Multivariate Behavioral Research, 36, 347-387.
59. Kang, S., & Waller, N. (2005). Moderated multiple regression, spurious interaction effects, and IRT. Applied Psychological Measurement, 29, 87-105. doi:10.1177/0146621604272737
60. Krishnakumar, A., Buehler, C., & Barber, B. K. (2004). Cross-ethnic equivalence of socialization measures in European American and African American youth. Journal of Marriage and Family, 66, 809-820. doi:10.1111/j.0022-2445.2004.00054.x
61. Lambert, M. C., Schmitt, N., Samms-Vaughan, M. E., An, J. S., Fairclough, M., & Nutter, C. A. (2003). Is it prudent to administer all items for each child behavior checklist cross-informant syndrome? Evaluating the psychometric properties of the Youth Self-Reported Dimensions with confirmatory factor analysis and item response theory. Psychological Assessment, 15, 550-568. doi:10.1037/1040-3590.15.4.550
62. Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7, 328.
63. Linacre, J. M. (2012). Winsteps® Rasch measurement computer program user's guide. Beaverton, OR: Winsteps.
64. Long, J. S. (1997). Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.
65. McArdle, J. J., Grimm, K., Hamagami, F., Bowles, R., & Meredith, W. (2009). Modeling life-space growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement. Psychological Methods, 14, 126-149. doi:10.1037/a0015857
66. McLoyd, V. C., & Smith, J. (2002). Physical discipline and behavior problems in African American, European American, and Hispanic children: Emotional support as a moderator. Journal of Marriage and Family, 64, 40-53. doi:10.1111/j.1741-3737.2002.00040.x
67. Mislevy, R. J. (1996). Test theory reconceived. Journal of Educational Measurement, 33, 379-416. doi:10.1111/j.1745-3984.1996.tb00498.x
68. Muraki, E., & Carlson, J. E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90. doi:10.1177/014662169501900109
69. Muthén, B. (1983). Latent variable structural equation modeling with categorical data. Journal of Econometrics, 22, 43-65. doi:10.1016/0304-4076(83)90093-3
70. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620. doi:10.1207/S15328007SEM0904_8
71. Muthén, L. K., & Muthén, B. O. (2012). Mplus user's guide (7th ed.) Los Angeles: Author.
72. Nering, M. L., & Ostini, R. (2010). Handbook of polytomous item response theory models. New York: Routledge.
73. Oberski, D. L. (2014). lavaan.survey: An R package for complex survey analysis of structural equation models. Journal of Statistical Software, 57, 1-27.
74. Osborne, C., & McLanahan, S. (2007). Partnership instability and child well-being. Journal of Marriage and Family, 69, 1065-1083. doi:10.1111/j.1741-3737.2007.00431.x
75. Osgood, D. W., McMorris, B. J., & Potenza, M. T. (2002). Analyzing multiple-item measures of crime and deviance: I. Item response theory scaling. Journal of Quantitative Criminology, 18, 267-296. doi:10.1023/A:1016008004010
76. Osterlind, S. J., & Everson, H. T. (2009). Differential item functioning (2nd ed.). Thousand Oaks: Sage. doi:10.4135/9781412993913
77. Paek, I., & Han, K. T. (2012). IRTRO 2.1 for Windows. Applied Psychological Measurement, 37, 242-252. doi:10.1177/0146621612468223
78. Peterson, J. L., & Zill, N. (1986). Marital disruption, parent-child relationships, and behavior problems in children. Journal of Marriage and the Family, 48, 295-307. doi:10.2307/352397
79. Peterson, R. A. (2000). A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis. Marketing Letters, 11, 261-275. doi:10.1023/A:1008191211004
80. Piquero, A. R., Macintosh, R., & Hickman, M. (2002). The validity of a self-reported delinquency scale: Comparisons across gender, age, race and place of residence. Sociological Methods and Research, 30, 492-529. doi:10.1177/0049124102030004002
81. Raîche, G. (2005). Critical eigenvalue sizes (variances) in standardized residual principal components analysis. Rasch Measurement Transactions, 19, 1011-1012.
82. Rapport, M. D., LaFond, S. V., & Sivo, S. A. (2009). Unidimensionality and developmental trajectory of aggressive behavior in clinically-referred boys: A Rasch analysis. Journal of Psychopathology and Behavioral Assessment, 31, 309-319. doi:10.1007/s10862-008-9125-x
83. Raudenbush, S. W., Johnson, C., & Sampson, R. J. (2003). A multivariate, multilevel Rasch model with application to self-reported criminal behavior. Sociological Methodology, 33, 169-211. doi:10.1111/j.0081-1750.2003.t01-1-00130.x
84. Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications and promise in psychological research. Current Directions in Psychological Science, 14, 95-101. doi:10.1111/j.0963-7214.2005.00342.x
85. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566. doi:10.1037/0033-2909.114.3.552
86. Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17, 1-25.
87. Rocque, M., Posick, C., & Zimmerman, G. M. (2013). Measuring up: Assessing the measurement properties of two self-control scales. Deviant Behavior, 34, 534-556. doi:10.1080/01639625.2012.748619
88. SAS Institute. (2013). Proc IRT (Experimental). In SAS/STAT 13.1 user's guide. Cary, NC: Author.
89. Schwab, D. P. (1980). Construct validity in organizational behavior. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (pp. 2-43). Greenwich: JAI Press.
90. Shafer, A. B. (2006). Meta-analyses of the factor structures of four depression questionnaires: Beck, CES-D, Hamilton, and Zung. Journal of Clinical Psychology, 62, 123-146. doi:10.1002/jclp.20213
91. Sheu, C., Chen, C., Su, Y., & Wang, W. (2005). Using SAS PROC NLMIXED to fit item response theory models. Behavior Research Methods, 37, 202-218. doi:10.3758/BF03192688
92. Skrondal, A., & Rabe-Hesketh, S. (2007). Latent variable modeling: A survey. Scandinavian Journal of Statistics, 34, 712-745. doi:10.1111/j.1467-9469.2007.00573.x
93. Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G., & Sharpe, M. (2008). Rasch fit statistics and sample size consideration for polytomous data. BMC Medical Research Methodology, 8, 33. doi:10.1186/1471-2288-8-33
94. Smith, R. M. (1996). A comparison of methods for determining dimensionality in Rasch measurement. Structural Equation Modeling, 3, 25-40. doi:10.1080/10705519609540027
95. Smith, R. M. (2004). Fit analysis in latent trait measurement models. In E. V. Smith & R. M. Smith (Eds.), Introduction to Rasch measurement: Theory, models and applications (pp. 73-92). Maple Grove, MN: JAM Press.
96. Stapleton, L. M. (2006). An assessment of practical solutions for structural equation modeling with complex sample data. Structural Equation Modeling, 13, 28-58. doi:10.1207/s15328007sem1301_2
97. StataCorp. (2013). Stata: Release 13 [Computer software]. College Station, TX: Author.
98. Steinberg, L., & Thissen, D. (1996). Uses of item response theory and the testlet concept in the measurement of psychopathology. Psychological Methods, 1, 81-97. doi:10.1037/1082-989X.1.1.81
99. Studts, C. R., & van Zyl, M. A. (2013). Identification of developmentally appropriate screening items for disruptive behavior problems in preschoolers. Journal of Abnormal Child Psychology, 41, 851-863. doi:10.1007/s10802-013-9738-8
100. Sweeten, G. (2012). Scaling criminal offending. Journal of Quantitative Criminology, 28, 533-557. doi:10.1007/s10940-011-9160-8
101. Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408. doi:10.1007/BF02294363
102. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Erlbaum.
103. Thoemmes, F., MacKinnon, D. P., & Reiser, M. R. (2010). Power analysis for complex mediational designs using Monte Carlo methods. Structural Equation Modeling, 17, 510-534.
104. Thornberry, T. P., & Krohn, M. D. (2000). The self-report method for measuring delinquency and crime. In Crime and justice 2000: Vol. 4. Measurement and analysis of crime and justice (pp. 33-83). Washington, DC: U.S. Department of Justice.
105. Turney, K. (2011). Chronic and proximate depression among mothers: Implications for child well-being. Journal of Marriage and Family, 73, 149-163. doi:10.1111/j.1741-3737.2010.00795.x
106. Wang, W., Cheng, Y., & Wilson, M. (2005). Local item dependence for items across tests connected by common stimuli. Educational and Psychological Measurement, 65, 5-27. doi:10.1177/0013164404268676
107. White, J. M., & Klein, D. M. (2008). Family theories. Thousand Oaks: Sage.
108. Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Erlbaum.
109. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58-79. doi:10.1037/1082-989X.12.1.58
110. Wolfe, F. (2001). raschcvt [User-written Stata program]. Retrieved from www.winsteps.com/winman/datafromstatafiles.htm
111. Wright, B. D., & Stone, M. H. (1979). Best test design: Rasch measurement. Chicago: MESA Press.
112. Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACER Conquest, Version 2: Generalised items response modeling software [Computer software]. Victoria, Australia: ACER Press.
113. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213. doi:10.1111/j.1745-3984.1993.tb00423.x