The Impact of Item Wording and Behavioral Specificity on the Accuracy of Direct Behavior Ratings (DBRs)

School Psychology Quarterly

Volumn 24, Issue 1, 2009, Pages 1-12

  Riley Tillman, T Chris ^a   Chafouleas, Sandra M ^b   Christ, Theodore ^c   Briesch, Amy M ^b   LeBel, Teresa J ^b  

^a EAST CAROLINA UNIVERSITY   (United States)

^b UNIVERSITY OF CONNECTICUT   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

Author keywords

behavioral assessment; direct behavior ratings (DBRs); formative assessment

Indexed keywords

EID: 67749114397     PISSN: 10453830     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0015248     Document Type: Article

References (35)

1. Ahlawat K.S. On the negative valence items in self-report measures. Journal of General Psychology 112 (1988) 89-99
2. Alberto P.A., and Troutman A.C. Applied behavior analysis for teachers (2003), Merrill Prentice Hall, Upper Saddle River, NJ
3. Borman W. Consistency of rating accuracy and rating errors in the judgment of human performance. Organizational Behavior and Human Performance 20 (1977) 238-252
4. Borman W. Exploring upper limits of reliability and validity in job performance ratings. Journal of Applied Psychology 63 (1978) 135-144
5. Chafouleas, S. M., Christ, T. H., & Riley-Tillman, T. C. (2007a). Generalizability and dependability of scaling gradients on direct behavior ratings (DBRs). Manuscript submitted for publication.
6. Chafouleas S.M., Christ T.H., Riley-Tillman T.C., Briesch A.M., and Chanese J.A.M. Generalizability and dependability of direct behavior ratings (DBRs) to assess social behavior of preschoolers. School Psychology Review 36 (2007) 63-79
7. Chafouleas S.M., McDougal J.L., Riley-Tillman T.C., Panahon C.J., and Hilt A.M. What do daily behavior report cards (DBRCs) measure? An initial comparison of DBRCs with direct observation for off-task behavior. Psychology in the Schools 42 (2005) 669-676
8. Chafouleas S.M., Riley-Tillman T.C., and Sassu K.A. Acceptability and reported use of daily behavior report cards among teachers. Journal of Positive Behavior Interventions 8 (2006) 174-182
9. Chafouleas S.M., Riley-Tillman T.C., and Sugai G. School-based behavioral assessment: Informing intervention and instruction (2007), Guilford Press, New York
10. Chang L. Connotatively inconsistent test items. Applied Measurement in Education 3 (1995) 199-209
11. Clonan S.M., Chafouleas S.M., McDougal J.L., and Riley-Tillman T.C. Positive psychology goes to school: Are we there yet?. Psychology in the Schools 41 (2004) 101-111
12. Cronbach L.J. Processes affecting scores on "understanding of others" and "assumed similarity.". Psychological Bulletin 52 (1955) 177-193
13. Crone D.A., Horner R.H., and Hawken L.S. Responding to problem behavior in schools: The Behavior Education Program (2004), Guilford Press, New York
14. Edelbrock C. Informant reports. In: Shapiro E.S., and Kratochwill T.R. (Eds). Behavioral assessment in schools: Conceptual foundations and practical applications (1988), Guilford Press, New York 351-383
15. Gettinger M., and Callan Stoiber K. Excellence in teaching: Review of instructional and environmental variables. In: Reynolds C., and Gutkin T. (Eds). The handbook of school psychology 3 (1999), Wiley, New York 383-409
16. LeBel, T. J., Chafouleas, S. M., & Sugai, G. (2006). Project VIABLE: Rationale for selection of behaviors to be investigated. Unpublished manuscript, University of Connecticut, Storrs.
17. Lewis T., and Sugai G. Descriptive and experimental analysis of teacher and peer attention and the use of assessment-based intervention to improve pro-social behavior. Journal of Behavioral Education 6 (1996) 7-24
18. Merrell K.W. Informant reports: Theory and research in using child behavior rating scales in school settings. In: Shapiro E., and Kratochwill T. (Eds). Behavior assessment in schools: Theory, research, and clinical foundations. 2nd ed. (2000), Guilford Press, New York 223-256
19. Merrell K.W. Behavioral, social, and emotional assessment of children and adolescents. 2nd ed. (2003), Erlbaum, Hillsdale, NJ. http://www.swis.org School Wide Information System (SWIS) Web site. Retrieved July 6, 2006, from http://www.swis.org
20. Ory J.C. Item placement and wording effects on overall ratings. Educational and Psychological Measurement 42 (1982) 767-775
21. Peterson L.D., Young K.R., and Salzberg C.L. Using self-management procedures to improve classroom social skills in multiple general education settings. Education and Treatment of Children 29 (2006) 1-21
22. Pett M.A., Lackey N.R., and Sullivan J.J. Making sense of factor analysis: The use of factor analysis for instrument development in health care research (2003), Sage, Thousand Oaks, CA
23. Riley-Tillman T.C., Chafouleas S.M., and Briesch A.M. A school practitioner's guide to using daily behavior report cards to monitor interventions. Psychology in the Schools 44 (2007) 77-89
24. Riley-Tillman T.C., Chafouleas S.M., Sassu K.A., Chanese J.A.M., and Glazer A.D. Examining the agreement of direct behavior ratings and systematic direct observation for on-task and disruptive behavior. Journal of Positive Behavior Interventions 10 (2008) 136-143
25. Riley-Tillman, T. C., Christ, T. J., Chafouleas, S. M., & Boice, C. H. (2008). The impact of observation duration on the generalizability and dependability of direct behavior ratings. Manuscript in preparation.
26. Saudargas R.A. State Event Classroom Observation System (SECOS) observation manual (1997), University of Tennessee, Knoxville
27. Shapiro E.S. Academic skills problems: Direct assessment and intervention. 3rd ed. (2004), Guilford Press, New York
28. Sugai G., and Horner R. A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review 35 (2006) 245-259
29. Sulsky L.M., and Balzer W.K. Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. Journal of Applied Psychology 73 (1988) 497-506
30. Tabachnick B.G., and Fidell L.S. Using multivariate statistics (2001), Allyn & Bacon, Boston
31. Tapp J. MOOSES (Multi-Option Observation System for Experimental Studies) (2004). http://www.kc.vanderbilt.edu/mooses/mooses.html Retrieved November 15, 2007, from http://www.kc.vanderbilt.edu/mooses/mooses.html
32. Todd A., Horner R., and Sugai G. Self-monitoring and self-recruited praise. Journal of Positive Behavior Interventions 1 (1999) 66-76
33. Walker H.M. Teacher social behavior standards and expectations as determinants of classroom ecology, teacher behavior and child outcomes: Final report (1985), CEPM Publications, Eugene, OR
34. Walker H.M., Ramsey E., and Gresham F.M. Heading off disruptive behavior: How early intervention can reduce defiant behavior-and win back teaching time. American Educator, Winter (2003). http://www.aft.org/pubs-reports/american_educator/winter03-04/early_inte rvention.html Available from http://www.aft.org/pubs-reports/american_educator/winter03-04/early_inte rvention.html
35. Walker H.M., and Rankin R. Assessing the behavioral expectations and demands of less restrictive settings. School Psychology Review 12 (1983) 274-284