The influence of observation length on the dependability of data

School Psychology Quarterly

Volumn 27, Issue 4, 2012, Pages 187-197

  Ferguson, Tyler David ^a   Briesch, Amy M ^a   Volpe, Robert J ^a   Daniels, Brian ^a  

^a NORTHEASTERN UNIVERSITY   (United States)

Author keywords

Behavioral assessment; Generalizability theory; Seventh grade students; Student engagement; Systematic direct observation

Indexed keywords

ADOLESCENT; ADOLESCENT BEHAVIOR; ARTICLE; FEMALE; HUMAN; MALE; METHODOLOGY; OBSERVER VARIATION; PSYCHOLOGICAL ASPECT; PSYCHOLOGY; PSYCHOMETRY; REPRODUCIBILITY; STATISTICS; STUDENT; TIME; UNITED STATES; URBAN POPULATION;

ADOLESCENT; ADOLESCENT BEHAVIOR; FEMALE; HUMANS; MALE; NEW ENGLAND; OBSERVER VARIATION; PSYCHOLOGY, EDUCATIONAL; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; STUDENTS; TIME FACTORS; URBAN POPULATION;

EID: 84874560474     PISSN: 10453830     EISSN: None     Source Type: Journal    
DOI: 10.1037/spq0000005     Document Type: Article

References (30)

1. Brennan, R., Gao, X., & Colton, D. (1995). Generalizability analyses of Work Key Listening and Writing tests. Educational and Psychological Measurement, 55, 157-176.
2. Briesch, A. M., Chafouleas, S. M., & Riley-Tillman, T. C. (2010). Generalizability and dependability of behavior assessment methods to estimate academic engagement: A comparison of systematic direct observation and Direct Behavior Rating. School Psychology Review, 39, 408-421.
3. Briesch, A. M., & Volpe, R. J. (2007). Important considerations in the selection of progress monitoring measures for classroom behaviors. School Psychology Forum, 1, 59-74.
4. Chafouleas, S. M., Riley-Tillman, T. C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York, NY: The Guilford Press.
5. Chiu, C., & Wolfe, E. (2002). A method for analyzing sparse data matrices in the generalizability theory framework. Applied Psychological Measurement, 26, 321-338.
6. Cone, J. D. (1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8, 411-426.
7. Gardner, F. (2000). Methodological issues in the direct observation of parent-child interaction: Do observational findings reflect the natural behavior or participants? Clinical Child and Family Psychology Review, 3, 185-198.
8. Heyman, R. E., Chaudry, B. R., Treboux, D., Crowell, J., Lord, C., Vivian, D., & Waters, E. B. (2001). How much observational data is enough? An empirical test using marital interaction coding. Behavior Therapy, 32, 107-122.
9. Hintze, J. M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507-519.
10. Hintze, J. M., & Matthews, W. J. (2004). The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral observation. School Psychology Review, 33, 258-270.
11. Hintze, J. M., Volpe, R., & Shapiro, E. (2002). Best practices in systematic direct observation of student behavior. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 993- 1006). Bethesda, MD: National Association of School Psychologists.
12. Kent, R., O'Leary, D., Dietz, A., & Diament, C. (1979). Comparison of observational recordings in vivo, via mirror, and via television. Journal of Applied Behavior Analysis, 12, 517-522.
13. Leff, S. S., & Lakin, R. (2005). Playground-based observational systems: A review and implications for practitioners and researchers. School Psychology Review, 34, 475-489
14. McConaughy, S., & Achenbach, T. M. (2009). Manual for the ASEBA Direct Observation Form. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.
15. McConaughy, S., & Ritter, D. (2008). Best practices in multimethod assessment of emotional and behavioral disorders. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 697-715). Bethesda, MD: National Association of School Psychologists.
16. McWilliam, R. A., & Ware, W. B. (1994). The reliability of observations of young children's engagement: An application of generalizability theory. Journal of Early Intervention, 18, 34-47.
17. Salvia, J., Ysselydke, J. E., & Bolt, S. (2010). Assessment in special and inclusive education (11th ed.). Boston, MA: Houghton Mifflin.
18. Sattler, J. M. (2002). Assessment of children: Behavioral and clinical applications (4th ed.). San Diego, CA: Jerome M. Sattler, Publisher, Inc.
19. Shapiro, E. S. (2004). Academic skills problems workbook. New York, NY: Guilford Press.
20. Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred for social/behavioral/emotional problems. Psychology in the Schools, 41, 551-561.
21. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory. Newbury Park, CA: Sage.
22. Smith, P. L. (1981). Gaining accuracy in generalizability theory: Using multiple designs. Journal of Educational Measurement, 18, 147-154.
23. Steege, M., & Watson, T. (2008). Best practices in functional behavioral assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 337-348). Bethesda, MD: National Association of School Psychologists.
24. Thorndike, R. M. (2005). Measurement and evaluation in psychology and education (7th ed.). Upper Saddle River, NJ: Pearson.
25. Volpe, R., DiPerna, J., Hintze, J., & Shapiro, E. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454-474.
26. Volpe, R. J., McConaughy, S. H., & Hintze, J. M. (2009). Generalizability of classroom behavior problem and on-task scores from the Direct Observation Form. School Psychology Review, 38, 382-401.
27. Waters, E. (1978). The reliability and stability of individual differences in infant-mother attachment. Child Development, 49, 483-494.
28. Watkins, M. W., & Pacheco, M. (2000). Interobserver agreement in behavioral research: Importance and calculation. Journal of Behavioral Education, 10, 205-212.
29. Webb, N., Rowley, G., & Shavelson, R. (1988). Using generalizability theory in counseling and development. Measurement and Evaluation in Counseling and Development, 21, 81-90.
30. Wilson, M. S., & Reschly, D. J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 9-23