The use of generalizability theory to estimate data reliability in single-subject observational research

Psychology in the Schools

Volumn 44, Issue 5, 2007, Pages 433-439

  Lei, Pui Wa ^a   Smith, Maria ^a   Suen, Hoi K ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34250819895     PISSN: 00333085     EISSN: 15206807     Source Type: Journal    
DOI: 10.1002/pits.20235     Document Type: Article

References (21)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
2. Brennan, R.L. (2001). An essay on the history and future of reliability from the perspective of replications. Journal of Educational Measurement, 38, 295-317.
3. Cohen, R.J., & Swerdlik, M.E. (1999). Psychological testing and assessment: An introduction to tests and measurement. Mountain View, CA: Mayfield.
4. Cone, J.D. (1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8, 411-426.
5. Crick, J.E., & Brennan, R.L. (1982). GENOVA: A generalized analysis of variance system (FORTRAN IV computer program and manual). Dorchester, MA: Computer Facilities, University of Massachusetts at Boston.
6. Cronbach, L.J., Gleser, G.C., Nanda, H., & Ratjaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
7. de Weerth, C., van Geert, P., & Hoijtink, H. (1999). Intraindividual variability in infant behavior. Developmental Psychology, 35, 1102-1112.
8. Gresham, F.M. (2003). Establishing the technical adequacy of functional behavioral assessment: Conceptual and measurement challenges. Behavioral Disorders, 28, 282-298.
9. Hartmann, D.P. (1977). Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10, 103-116.
10. Hartmann, D.P. (1982). Assessing the dependability of observational data. In D.P. Hartmann (Ed.), New directions for the methodology of the behavioral sciences: Using observers to study behavior (p. 51-66). San Francisco: Jossey-Bass.
11. Hintze, J.M., & Matthews, W.J. (2004). The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral observation. School Psychology Review, 33, 158-170.
12. Hughes, M.A., & Garrett, D.E. (1990). Intercoder reliability estimation approaches in marketing: A generalizability theory framework for quantitative data. Journal of Marketing Research, 27, 185-195.
13. Kozaki, Y. (2004). Using GENOVA and FACETS to set multiple standards on performance assessment for certification in medical translation from Japanese into English. Language Testing, 21, 1-27.
14. Kratochwill, T.R. (1978). Single subject research: Strategies for evaluating change, (p. 22) New York: Academic Press. Ragan, B.G., & Kang, M. (2005). Reliability: Current issues and concerns. Athletic Therapy Today, 10, 30-34.
15. Reckase, M.D. (1995). The reliability of ratings versus the reliability of scores. Educational Measurement: Issues and Practice, 14, 31.
16. Rogosa, D., & Ghandour, G. (1991). Statistical models for behavioral observations. Journal of Educational Statistics, 16, 157-252.
17. Rowley, G. (1978). The relationship of reliability in classroom research to the amount of observation: An extension of the Spearman-Brown formula. Journal of Educational Measurement, 15, 165-180.
18. Rowley, G.L. (1989). Assessing error in behavioral data: Problems of sequencing. Journal of Educational Measurement, 26, 273-284.
19. Suen, H.K., & Ary, D. (1989). Analyzing quantitative behavioral observation data. Hillsdale, NJ: Erlbaum.
20. Suen, H.K. & Rzasa, S.E. (2004). Psychometric foundations of behavioral assessment. In S.N. Haynes & E.M. Heiby (Eds.), Comprehensive handbook of psychological assessment. Vol. 3: Behavioral assessment (pp. 37-56). Hoboken, NJ: Wiley.
21. Wilson, M.S., & Reschly, D.J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 9-23.