Agreement among student, parent and teacher ratings of school inclusion: A multitrait-multimethod analysis

Journal of School Psychology

Volumn 82, Issue , 2020, Pages 1-16

  Schwab, Susanne ^a,b   Zurbriggen, Carmen L A ^c   Venetz, Martin ^d  

^a UNIVERSITY OF VIENNA   (Austria)

^b NORTH WEST UNIVERSITY   (South Africa)

^c BIELEFELD UNIVERSITY   (Germany)

^d University of Applied Sciences of Special Needs Education Zurich   (Switzerland)

Author keywords

Academic self concept; Emotional inclusion; MTMM; Multi informant research; Social inclusion

Indexed keywords

ADULT; ARTICLE; CASE REPORT; CLINICAL ARTICLE; FATHER; FEMALE; GENDER; HUMAN; HUMAN EXPERIMENT; MALE; MULTITRAIT MULTIMETHOD; PERCEPTION; QUESTIONNAIRE; SELF REPORT; TEACHER; WORLD HEALTH ORGANIZATION;

EID: 85088365082     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2020.07.003     Document Type: Article

References (70)

1. Achenbach, T.M., Multi-informant and multicultural advances in evidence-based assessment of students' behavioral/emotional/social difficulties. European Journal of Psychological Assessment 34 (2018), 127–140, 10.1027/1015-5759/a000448.
2. Achenbach, T.M., McConaughy, S.H., Howell, C.T., Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin 101 (1987), 213–232.
3. Adams, D., Harris, A., Jones, M.S., Exploring teachers' and parents' perceptions on social inclusion practices in Malaysia. Pertanika Journal of Social Science and Humanities 25:4 (2017), 1721–1738.
4. Armstrong, D., Armstrong, A.C., Spandagou, I., Inclusion: By choice or by chance?. International Journal of Inclusive Education 15:1 (2011), 29–39, 10.1080/13603116.2010.496192.
5. Bear, G.G., Minke, K.M., Manning, M.A., Self-concept of students with learning disabilities: A meta-analysis. School Psychology Review 31:3 (2002), 405–427.
6. Bossaert, G., Colpin, H., Pijl, S.J., Petry, K., Truly included? A literature study focusing on the social dimension of inclusion in education. International Journal of Inclusive Education 17:1 (2013), 60–79, 10.1080/13603116.2011.580464.
7. Brown, T.A., Confirmatory factor analysis for applied research. 2nd ed., 2015, The Guilford Press, New York, NY.
8. Cambra, C., Silvestre, N., Students with special educational needs in the inclusive classroom: Social integration and self-concept. European Journal of Special Needs Education 18:2 (2003), 197–208, 10.1080/0885625032000078989.
9. Chapman, J.W., Learning disabled children's self-concepts. Review of Educational Research 58:3 (1988), 347–371.
10. Crayen, C., Geiser, C., Scheithauer, H., Eid, M., Evaluating interventions with multimethod data: A structural equation modeling approach. Structural Equation Modeling: A Multidisciplinary Journal 18:4 (2011), 497–524, 10.1080/10705511.2011.607068.
11. De Los Reyes, A., Introduction to the special section: More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child & Adolescent Psychology 40:1 (2011), 1–9, 10.1080/15374416.2011.533405.
12. De Los Reyes, A., Augenstein, T.M., Wang, M., Thomas, S.A., Drabick, D., Burgers, D.E., Rabinowitz, J., The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin 141 (2015), 858–900.
13. De Monchy, M., Pijl, S.J., Zandberg, T.J., Discrepancies in judging social inclusion and bullying of pupils with behavior problems. European Journal of Special Needs Education 19:3 (2004), 317–330, 10.1080/0885625042000262488.
14. DeVries, J.M., Voß, S., Gebhardt, M., Do learners with special education needs really feel included? Evidence from the perception inclusion questionnaire and strengths and difficulties questionnaire. Research in Developmental Disabilities 83 (2018), 28–36, 10.1016/j.ridd.2018.07.007.
15. DiStefano, C., Liu, J., Jiang, N., Shi, D., Examination of the weighted root mean square residual: Evidence for trustworthiness?. Structural Equation Modeling: A Multidisciplinary Journal 25:3 (2018), 453–466, 10.1080/10705511.2017.1390394.
16. Dunn, T.J., Baguley, T., Brunsden, V., From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology 105 (2014), 399–412, 10.1111/bjop.12046.
17. Eid, M., A multitrait–multimethod model with minimal assumptions. Psychometrika 65:2 (2000), 241–261.
18. Eid, M., Lischetzke, T., Nussbeck, F.W., Trierweiler, L.I., Separating trait from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M–1) model. Psychological Methods 8:1 (2003), 38–60, 10.1037/1082-989X.8.1.38.
19. Emerson, E., Hatton, C., The mental health of children and adolescents with learning disabilities in Britain. 2007, Lancaster University, Lancaster, Institute for Health Research.
20. Funder, D.C., On the accuracy of personality judgment: A realistic approach. Psychological Review 102:4 (1995), 652–670, 10.1037/0033-295X.102.4.652.
21. Funder, D.C., Accurate personality judgment. Current Directions in Psychological Science 21 (2012), 177–182, 10.1177/0963721412445309.
22. Geiser, C., Mandelmann, S.D., Tan, M., Grigorenko, E.L., Multitrait-multimethod assessment of giftedness: An application of the correlated traits-correlated (methods - 1) model. Structural Equation Modeling: A Multidisciplinary Journal 23:1 (2016), 76–90, 10.1080/10705511.2014.937792.
23. Gilman, R., Huebner, E.S., Furlong, M.J., Handbook of positive psychology in schools. 2nd ed., 2014, Routledge/Taylor & Francis.
24. Givvin, K.B., Stipek, D.J., Salmon, J.M., MacGyvers, V.L., In the eyes of the beholder: Students' and teachers' judgments of students' motivation. Teaching and Teacher Education 17:3 (2001), 321–331.
25. Gomez, R., Correlated trait-correlated method minus one analysis of the convergent and discriminant validities of the strengths and difficulties questionnaire. Assessment 21:3 (2014), 372–382, 10.1177/1073191112457588.
26. Goodman, R., Psychometric properties of the strengths and difficulties questionnaire (SDQ). Journal of the American Academy of Child & Adolescent Psychiatry 40:11 (2001), 1337–1345.
27. Gresham, F.M., Elliott, S., Metallo, S., Byrd, S., Wilson, E., Cassidy, K., Cross-informant agreement of children's social-emotional skills: An investigation of ratings by teachers, parents, and students from a nationally representative sample. Psychology in the Schools 55:2 (2018), 208–223.
28. Hu, L., Bentler, P.M., Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6:1 (1999), 1–55.
29. Huang, C., Self-concept and academic achievement: A meta-analysis of longitudinal relations. Journal of School Psychology 49:5 (2011), 505–528, 10.1016/j.jsp.2011.07.001.
30. IDEA, Individuals with disability education act amendments of 1997 [IDEA]. Retrieved from https://www.congress.gov/105/plaws/publ17/PLAW-105publ17.pdf, 1997. (Accessed 14 December 2019)
31. Jöreskog, K.G., Goldberger, A.S., Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association 70:351 (1975), 631–639.
32. Kane, M., Validating the interpretations and use of test scores. Journal of Educational Measurement 50 (2013), 1–73.
33. Kane, M., Validity studies commentary. Educational Assessment 25 (2020), 83–89.
34. Kline, R.B., Principles and practice of structural equation modeling. 4th ed., 2015, Guilford Press, New York, NY.
35. Knickenberg, M., Zurbriggen, C.L.A., Venetz, M., Gebhardt, M., Schwab, S., Assessing dimensions of inclusion from students' perspective – Measurement invariance across students with learning disabilities in different educational settings. European Journal of Special Needs Education, 2019, 10.1080/08856257.2019.1646958 Published online 25 July 2019.
36. Koch, T., Kelava, A., Eid, M., Analyzing different types of moderated method effects in confirmatory factor models for structurally different methods. Structural Equation Modeling 25:2 (2018), 179–200, 10.1080/10705511.2017.1373595.
37. Koster, M., Pijl, S.J., van Houten, E., Being part of the peer group: A literature study focusing on the social dimension of inclusion in education. International Journal of Inclusive Education 13:2 (2009), 117–140.
38. Koster, M., Pijl, S.J., van Houten, E., Nakken, H., The social position and development of pupils with SEN in mainstream Dutch schools. European Journal of Special Needs Education 22:1 (2007), 31–46, 10.1080/08856250601082265.
39. Kwon, K., Kim, E.M., Sheridan, S.M., A contextual approach to social skills assessment in the peer group: Who is the best judge?. School Psychology Quarterly 27:3 (2012), 121–133.
40. Maas, C.J.M., Lensvelt-Mulders, G.J.L.M., Hox, J.J., A multilevel multitrait-multimethod analysis. Methodology 5:3 (2009), 72–77, 10.1027/1614-2241.5.3.72.
41. Marsh, H.W., Hau, K.-T., Wen, Z., In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu und Bentler's (1999) findings. Structural Equation Modeling: A Multidisciplinary Journal 11:3 (2004), 320–341, 10.1207/s15328007sem1103_2.
42. Marsh, H.W., Hocevar, D., A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology 73:1 (1988), 107–117, 10.1037/0021-9010.73.1.107.
43. Marsh, H.W., Morin, A.J.S., Parker, P.D., Kaur, G., Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology 10 (2014), 85–119, 10.1146/annurev-clinpsy-032813-153700.
44. McCoy, S., Banks, J., Simply academic? Why children with special educational needs don't like school. European Journal of Special Needs Education 27:1 (2012), 81–97.
45. Miller, F.G., Johnson, A.H., Yu, H., Chafouleas, S.M., McCoach, D.B., Riley-Tillman, T., Welsh, M.E., Methods matter: A multi-trait multi-method analysis of student behavior. Journal of School Psychology 68 (2018), 53–72, 10.1016/j.jsp.2018.01.002.
46. Muthén, L.K., Muthén, B., Mplus user's guide. Eigthth ed., 1998–2017, Muthén & Muthén, Los Angeles, CA.
47. Nussbeck, F.W., Eid, M., Lischetzke, T., Analysing multitrait-multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results?. British Journal of Mathematical and Statistical Psychology 59:1 (2006), 195–213, 10.1348/000711005X67490.
48. Oh-Young, C., Filler, J., A meta-analysis of the effects of placement on academic and social skill outcome measures of students with disabilities. Research in Developmental Disabilities 47 (2015), 80–92.
49. Olino, T.M., Klein, D.N., Psychometric comparison of self- and informant-reports of personality. Assessment 22:6 (2015), 655–664.
50. Praetorius, A.-K., Berner, V.-D., Zeinz, H., Scheunpflug, A., Dresel, M., Judgment confidence and judgment accuracy of teachers in judging self-concepts of students. Journal of Educational Research 106 (2013), 64–76.
51. Rhemtulla, M., Brosseau-Liard, P.E., Savalei, V., When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods 17:3 (2012), 354–373, 10.1037/a0029315.
52. Roos, S., Lohbeck, A., Petermann, F., Petermann, U., Schultheiss, O.C., Nitkowski, D., Petersen, R., Fremd- und Selbsturteile von Lehrern und Schülern im Rahmen psychologischer Diagnostik [External and self-ratings of teachers and students in psychological diagnostic]. Zeitschrift für Psychiatrie, Psychologie und Psychotherapie 64:3 (2016), 199–207, 10.1024/1661-4747/a000279.
53. Ruijs, N.M., Peetsma, T.D., Effects of inclusion on students with and without special educational needs reviewed. Educational Research Review 4:2 (2009), 67–79.
54. Schwab, S., Peer-relations of students with special educational needs in inclusive education. Polenghi, S., Fiorucci, M., Agostinetto, L., (eds.) Diritti Cittadinanza Inclusione, 2018, Pensa MultiMedia, 15–24.
55. Schwab, S., Inclusive and special education in Europe. Oxford research encyclopedia of education, 2020, Oxford University Press, 10.1093/acrefore/9780190264093.013.1230.
56. Schwab, S., Gebhardt, M., Hessels, M.G.P., Nusser, L., Predicting a high rate of self-assessed and parent-assessed peer problems – Is it typical for students with disabilities?. Research in Developmental Disabilities 49/50 (2016), 196–204, 10.1016/j.ridd.2015.11.026.
57. Schwab, S., Kopp-Sixt, S., Bernat, E., Kritische Einblicke in die Gutachtenerstellung zum sonderpädagogischen Förderbedarf in Österreich [Critical insights into expert reports on special educational needs in Austria]. Heilpädagogische Forschung 41:3 (2015), 136–141.
58. Skrzypiec, G., Askell-Williams, H., Slee, P., Rudzinski, A., Students with self-identified special educational needs and disabilities (si-SEND): Flourishing or languishing!. International Journal of Disability, Development and Education 63:1 (2016), 7–26.
59. Suldo, S.M., Riley, K.N., Shaffer, E.J., Academic correlates of children and adolescents' life satisfaction. School Psychology International 27 (2006), 567–582.
60. United Nations, Convention on the rights of persons with disabilities and optional protocol. 2007, UN, New York, NY.
61. Urhahne, D., Zhu, M., Accuracy of teachers' judgments of students' subjective well-being. Learning and Individual Differences 43 (2015), 226–232.
62. van der Ende, J., Verhulst, F.C., Tiemeier, H., Agreement of informants on emotional and behavioral problems from childhood to adulthood. Psychological Assessment 24:2 (2012), 293–300.
63. Venetz, M., Zurbriggen, C., Eckhart, M., Entwicklung und erste Validierung einer Kurzversion des “Fragebogens zur Erfassung von Dimensionen der Integration von Schülern (FDI 4-6)” von Haeberlin, Moser, Bless und Klaghofer [Development and preliminary validation of a short questionnaire based on the FDI 4-6 by Haeberlin, Moser, Bless, and Klaghofer measuring the educational integration of pupils]. Empirische Sonderpädagogik 6:2 (2014), 99–113.
64. Venetz, M., Zurbriggen, C. L. A., Eckhart, M., Schwab, S., & Hessels, M. G. P. (2015). The perceptions of inclusion questionnaire (PIQ). German version. Retrieved from http://www.piqinfo.ch Accessed 14 December 2019.
65. Venetz, M., Zurbriggen, C.L.A., Schwab, S., What do teachers think about their students' inclusion? Consistency of students' self-reports and teacher ratings. Frontiers in Psychology, 10(1637), 2019, 10.3389/fpsyg.2019.01637.
66. Zeleke, S., Self-concepts of students with learning disabilities and their normally achieving peers: A critical review. European Journal of Special Needs Education 19:2 (2004), 145–170.
67. Zhang, X., Noor, R., Savalei, V., Examining the effect of reverse worded items on the factor structure of the need for cognition scale. PLoS One, 11(6), 2016, e0157795, 10.1371/journal.pone.0157795.
68. Zhu, M., Urhahne, D., Assessing teachers' judgements of students' academic motivation and emotions across two rating methods. Educational Research and Evaluation 20:5 (2014), 411–427.
69. Zurbriggen, C., Venetz, M., Schwab, S., Hessels, M.G.P., A psychometric analysis of the student version of the perceptions of inclusion questionnaire (PIQ). European Journal of Psychological Assessment 35:5 (2019), 641–649, 10.1027/1015-5759/a00044.
70. Zurbriggen, C.L.A., Venetz, M., Hinni, C., The quality of experience of students with and without special educational needs in everyday life and when relating to peers. European Journal of Special Needs Education 33:2 (2018), 205–220, 10.1080/08856257.2018.1424777.