What sources contribute to variance in observer ratings? Using generalizability theory to assess construct validity of psychological measures

Infant and Child Development

Volumn 17, Issue 3, 2008, Pages 269-284

  Lakes, Kimberley D ^a   Hoyt, William T ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Caregiver child relationships; Construct validity; Measurement; Observer ratings; Parent child relationships

Indexed keywords

EID: 51449085129     PISSN: 15227227     EISSN: 15227219     Source Type: Journal    
DOI: 10.1002/icd.551     Document Type: Article

References (24)

1. Aksan, N., Kochanska, G., & Ortmann, M. R. (2006). Mutually responsive orientation between parents and their young children: Toward methodological advances in the science of relationships. Developmental Psychology, 42, 833-848. DOI: 10.1037/0012-1649.42.5.833
2. Borkenau, P., Mauer, N., Riemann, R., Spinath, F. M., & Angleitner, A. (2004). Thin slices of behavior as cues of personality and intelligence. Journal of Personality and Social Psychology, 86, 599-614. DOI: 10.1037/0022-3514.86.4.599
3. Brennan, R. L. (2001). Generalizability theory. New York: Springer.
4. Cronbach, L. J. (1995). Giving method variance its due. In P. E. Shrout & T. S. Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 145-157). Hillsdale, NJ: Lawrence Erlbaum.
5. Cronbach, L. J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
6. Cronbach, L. J., Gleser, G. C., Nanda, A. N., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: John Wiley.
7. Crick, J. E., & Brennan, R. L. (1982). GENOVA: A generalized analysis of variance system (FORTRAN IV computer program and manual). Dorchester: Computer Facilities, Boston: University of Massachusetts, Free download: http://www.uiowa.edu/∼itp/pages/SWGENOVA.SHTML
8. Dennis, T. (2006). Emotional self-regulation in preschoolers: The interplay of child approach reactivity, parenting, and control capacities. Developmental Psychology, 42, 84-97. DOI: 10.1037/0012-1649.42.1.84
9. Eyberg, S. M., Robinson, E. A., (1981). Dyadic parent-child interaction coding system: A manual. Washington: The Parenting Clinic Department of Family and Child Nursing, School of Nursing, University of Washington.
10. Hoyt, W. T. (2000). Rater bias in psychological research: When is it a problem and what can we do about it? Psychological Methods, 5, 64-86.
11. Hoyt, W. T., Fincham, F., McCullough, M. E., Maio, G., & Davila, J. (2005). Responses to interpersonal transgressions in families: Forgivingness, forgivability, and relationship-specific effects. Journal of Personality and Social Psychology, 89, 375-394. DOI: 10.1037/0022-3514.89.3.375
12. Hoyt, W. T., & Kerns, M. D. (1999). Magnitude and moderators of bias in observer ratings: A meta-analysis. Psychological Methods, 4, 403-424. DOI: 10.1037/1082-989X.4.4.403
13. Hoyt, W. T., & McCullough, M. E. (2005). Issues in the multimodal measurement of forgiveness. In E.L. Worthington, Jr. (Ed.), Handbook of Forgiveness (pp. 109-123). New York: Brunner-Routledge.
14. Hoyt, W. T., & Melby, J. N. (1999). Dependability of measurement in counseling psychology: An introduction to generalizability theory. The Counseling Psychologist, 27, 325-352.
15. Hoyt, W. T., Warbasse, R. E., & Chu, E. Y. (2006). Construct validity in counseling psychology research. The Counseling Psychologist, 769-805.
16. Kenny, D. A. (1994). Interpersonal perception: A social relations analysis. New York: Guilford.
17. Kenny, D. A., Mohr, C., & Levesque, M. (2001). A social relations partitioning variance of dyadic behavior. Psychological Bulletin, 127, 128-141.
18. Lakes, K. D., & Hoyt, W. T. (in press). Applications of generalizability theory to clinical child psychology research. Journal of Clinical Child and Adolescent Psychology.
19. McCullough, M. E., Hoyt, W. T., & Rachal, K. C. (2000). What we know (and need to know) about assessing forgiveness constructs. In M. E. McCullough, K. Pargament, & C. Thoresen (Eds.), Forgiveness: Theory, research, and practice (pp. 65-88). New York: Guilford.
20. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.) (pp. 13-103). New York: Macmillan.
21. Ross, L., & Nisbett, R.E. (1991). The person and the situation: Perspectives of social psychology. New York: McGraw-Hill.
22. Schmidt, F. L., & Hunter, J. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1(2), 199-223.
23. Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychological Methods, 8(2), 206-224.
24. Shavelson, R. J., & Webb, N. M. (1991). Generalizability Theory: A primer. Newbury Park, CA: Sage.