An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students

Journal of School Psychology

Volumn 48, Issue 3, 2010, Pages 219-246

  Chafouleas, Sandra M ^a   Briesch, Amy M ^b   Riley Tillman, T Chris ^c   Christ, Theodore J ^d   Black, Anne C ^e   Kilgus, Stephen P ^a  

^a UNIVERSITY OF CONNECTICUT   (United States)

^b NORTHEASTERN UNIVERSITY   (United States)

^c EAST CAROLINA UNIVERSITY   (United States)

^d UNIVERSITY OF MINNESOTA   (United States)

^e YALE UNIVERSITY   (United States)

Author keywords

Behavior assessment; DBR; Dependability; Direct Behavior Rating; Generalizability

Indexed keywords

ANALYSIS OF VARIANCE; ARTICLE; BEHAVIOR DISORDER; CHILD; CHILD BEHAVIOR; EDUCATION; FEMALE; HEALTH CARE QUALITY; HUMAN; MALE; METHODOLOGY; SCHOOL; STATISTICAL MODEL; STUDENT;

ANALYSIS OF VARIANCE; CHILD; CHILD BEHAVIOR; CHILD BEHAVIOR DISORDERS; EDUCATIONAL MEASUREMENT; FEMALE; HUMANS; MALE; MODELS, STATISTICAL; PROGRAM EVALUATION; SCHOOLS; STUDENTS;

EID: 77951111104     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2010.02.001     Document Type: Article

References (56)

1. Brennan R.L. Generalizability theory 2001, Springer-Verlag, New York.
2. Chafouleas S.M., McDougal J.L., Riley-Tillman T.C., Panahon C.J., Hilt A.M. What do daily behavior report cards (DBRCs) measure? An initial comparison of DBRCs with direct observation for off-task behavior. Psychology in the Schools 2005, 42:669-676.
3. Chafouleas S.M., Christ T.J., Riley-Tillman T.C., Briesch A.M., Chanese J.A.M. Generalizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School Psychology Review 2007, 36:63-79.
4. Chafouleas S.M., Riley-Tillman T.C., Sugai G. School-based behavioral assessment: Informing intervention and instruction 2007, The Guilford Press, New York.
5. Chafouleas S.M., Christ T.J., Riley-Tillman T.C. Generalizability and dependability of scaling gradients on Direct Behavior Ratings (DBRs). Educational and Psychological Measurement 2009, 69:157-173.
6. Chafouleas S.M., Kilgus S.P., Hernandez P. Using Direct Behavior Rating (DBR) to screen for school social risk: A preliminary comparison of methods in a kindergarten sample. Assessment for Effective Intervention 2009, 34:214-223.
7. Chafouleas S.M., Riley-Tillman T.C., Christ T.J. Direct Behavior Rating (DBR): An emerging method for assessing social behavior within a tiered intervention system. Assessment for Effective Intervention 2009, 34:195-200.
8. Chafouleas, S. M., Riley-Tillman, T. C., Kilgus, S. P., Amon, J., Jaffery, R., & Brooks, S. (2009d). Critical components of DBR training to enhance rater accuracy: An investigation of training content and exposure. Unpublished manuscript.
9. Christ T.J., Riley-Tillman T.C., Chafouleas S.M. Foundation for the development and use of Direct Behavior Rating (DBR) to assess and evaluate student behavior. Assessment for Effective Intervention 2009, 34:201-213.
10. Christ, T. J., Riley-Tillman, T. C., Chafouleas, S. M., & Boice, C. H. (2009b). Direct Behavior Rating (DBR): Generalizability and dependability across raters and observations by facet and universe of generalization. Unpublished manuscript.
11. Cone J.D. The behavioral assessment grid (BAG): A conceptual framework and a taxonomy. Behavior Therapy 1978, 9:882-888.
12. Cone J.D. Psychometric considerations. Handbook of behavioral assessment 1981, 38-68. Pergamon Press, Elmsford, NY. 2nd Ed. M. Hersen, A.S. Bellack (Eds.).
13. Applied behavior analysis 2006, Prentice Hall, Upper Saddle River, NJ. 2nd Ed. J.O. Cooper, T.E. Heron, W.L. Heward (Eds.).
14. Cronbach L. Processes affecting scores on understanding of others and assumed similarity. Psychological Bulletin 1955, 52:177-193.
15. Erchul W.P., Martens B.K. School consultation: Conceptual and empirical bases of practice 2002, Plenum Press, New York. 2nd ed.
16. Feeley T.H. Comment on halo effects in rating and evaluation research. Human Communication Research 2002, 28:578-586.
17. Foster S.L., Bell-Dolan D.J., Burge D.A. Behavioral observation. Behavioral assessment: A practical handbook 1988, Pergamon, NY. 3rd edition. A.S. Bellack, M. Hersen (Eds.).
18. Greenwood C.R., Terry B., Marquis J., Walker D. Confirming a performance-based instructional model. School Psychology Review 1994, 23:652-668.
19. Gresham F.M. Current status and future directions of school-based behavioral interventions. School Psychology Review 2004, 33:326-343.
20. Gresham F.M., Elliott S.N. Social skills rating system manual 1990, American Guidance Service, Circle Pines.
21. Hintze J.M., Matthews W.J. The generalizability of systematic direct observation across time and setting: A preliminary investigation of the psychometrics of behavioral observation. School Psychology Review 2004, 33:258-270.
22. Hintze J.M., Volpe R.J., Shapiro E.S. Best practices in the systematic direct observation of student behavior. Best practices in school psychology IV 2002, 993-1006. National Association of School Psychologists, Washington, DC, US. A. Thomas, J. Grimes (Eds.).
23. Irvin L.K., Tobin T.J., Sprague J.R., Sugai G., Vincent C.G. Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions 2004, 6:131-147.
24. Kelly T.L. Interpretations of educational measures 1927, World Book, Yonkers, NY.
25. Kline P. Handbook of psychological testing 2000, Routledge, New York.
26. Kratochwill T.R., Sheridan S.M., Carlson J., Lasecki K.L. Advances in behavioral assessment. The handbook of school psychology 1999, 350-382. Wiley, New York. 3rd ed. C.R. Reynolds, T.R. Gutkin (Eds.).
27. LeBel T.J., Kilgus S.P., Briesch A.M., Chafouleas S.M. The impact of training on the accuracy of teacher-completed Direct Behavior Ratings (DBRs). Journal of Positive Behavior Interventions 2010, 12:55-63.
28. Marcoulides G.A. Generalizability theory: Picking up where the Rasch IRT leaves off?. The new rules of measurement 1999, 129-152. Lawrence Erlbaum, Mahwah, NJ. S.E. Embertson, S.L. Hershberger (Eds.).
29. Mash E.J., Foster S.L. Exporting analogue behavioral observation from research to clinical practice: Useful or cost-defective?. Psychological Assessment 2001, 13:86-96.
30. McIntosh K., Campbell A.L., Carter D.R., Zumbo B.D. Concurrent validity of office discipline referrals and cut points used in schoolwide positive behavior support. Behavioral Disorders 2009, 34:100-113.
31. Merrell K.W. Behavioral, social, and emotional assessment of children and adolescents 2008, Erlbaum, Mahwah, NJ. 3rd ed.
32. Meyer G.J., Finn S.E., Eyde L.D., Kay G.G., Moreland K.L., Dies R.R., et al. Psychological testing and psychological assessment: A review of the evidence and issues. American Psychologist 2001, 56:128-165.
33. Pelham W.E., Fabiano G.A., Massetti G.M. Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology 2005, 34:449-476.
34. Reschly D.J., Bergstrom M.K. Response to intervention. The handbook of school psychology 2009, 434-459. Wiley, Hoboken, NJ. 4th ed. T. Gutkin, C. Reynolds (Eds.).
35. Riley-Tillman T.C., Chafouleas S.M., Briesch A.M. A school practitioner's guide to using Daily Behavior Report Cards to monitor student behavior. Psychology in the Schools 2007, 44:77-90.
36. Riley-Tillman T.C., Chafouleas S.M., Sassu K.A., Chanese J.M., Glazer A.D. Examining agreement between Direct Behavior Ratings (DBRs) and systematic direct observation data for on-task and disruptive behavior. Journal of Positive Behavior Interventions 2008, 10:136-143.
37. Riley-Tillman T.C., Chafouleas S.M., Briesch A.M., Eckert T.L. Daily behavior report cards and systematic direct observation: An investigation of the acceptability, reported training and use, and decision reliability among school psychologists. Journal of Behavioral Education 2008, 17:313-327.
38. Riley-Tillman T.C., Chafouleas S.M., Christ T.J., Briesch A.M., LeBel T.J. The impact of wording and behavioral specificity on the accuracy of Direct Behavior Ratings (DBRs). School Psychology Quarterly 2009, 24:1-12.
39. Riley-Tillman T.C., Methe S.A., Weegar K. Examining the use of Direct Behavior Rating methodology on classwide formative assessment: A case study. Assessment for Effective Intervention 2009, 34:224-230.
40. Riley-Tillman, T. C., Christ, T. J., Chafouleas, S. M., Boice, C. H., & Briesch, A. M. (2009c). DBR observation duration and scaling anchoring: Effects on accuracy of ratings. Unpublished manuscript.
41. Riley-Tillman, T. C., Harrison, S., Amon, J., & Brooks, S. (2009d). An investigation of rating accuracy on DBR following training involving practice and feedback. Unpublished manuscript.
42. Rubin D.B. Multiple imputation for nonresponse in surveys 1987, J. Wiley, New York.
43. Salvia J., Ysselydke J.E., Bolt S. Assessment in special and inclusive education 2010, Houghton Mifflin, Boston. 11th ed.
44. Schafer J.L. Analysis of incomplete multivariate data 1997, Chapman & Hall, New York.
45. Schafer J.L. Multiple imputation: A primer. Statistical Methods for Medical Research 1999, 8:3-15.
46. Schafer J.L., Olsen M.K. Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research 1998, 33:545-571.
47. Schlientz M.D., Riley-Tillman T.C., Briesch A.M., Walcott C.M., Chafouleas S.M. The impact of training on the accuracy of Direct Behavior Ratings (DBRs). School Psychology Quarterly 2009, 24:73-83.
48. Shapiro E.S. Academic skills problems workbook 2004, Guilford Press, New York.
49. Shapiro E.S., Kratochwill T.R. Behavioral assessment in schools: Theory, research and clinical foundations 2000, The Guilford Press, New York. 2nd ed.
50. Shavelson R.J., Webb N.M. Generalizability theory: A primer 1991, Sage, Newbury Park, CA.
51. Stewart R.M., Benner G.J., Matella R.C., Marchand-Martella N.E. Three-tier models of reading and behavior: A research review. Journal of Positive Behavior Interventions 2007, 9:239-253.
52. Sugai G., Sprague J.R., Horner R.H., Walker H.M. Preventing school violence: The use of office discipline referrals to assess and monitor school-wide discipline interventions. Journal of Emotional and Behavioral Disorders 2000, 8:94-101.
53. Tilly W.D. The evolution of school psychology to science-based practice: Problem solving and the three-tiered model. Best practices in school psychology-V 2008, 17-36. National Association of School Psychologists, Bethesda, MD. A. Thomas, J. Grimes (Eds.).
54. Volpe, R. J. & Chafouleas, S. M. (in press). Assessment of externalizing behavioral deficits. In M. A. Bray & T. J. Kehle. (Eds.), Oxford handbook of school psychology. New York, NY: Oxford University Press.
55. Walker H.M., Ramsey E., Gresham F.M. Heading off disruptive behavior: How early intervention can reduce defiant behavior and win back teaching time. American Educator 2003, 26:6-45.
56. Wright J.A., Dusek J.B. Compiling school base rates for disruptive behaviors from student disciplinary referral data. School Psychology Review 1998, 27:138-147.