Disentangling Shape from Level Effects in Person-Centered Analyses: An Illustration Based on University Teachers’ Multidimensional Profiles of Effectiveness

Structural Equation Modeling

Volumn 22, Issue 1, 2015, Pages 39-59

  Morin, Alexandre J S ^a   Marsh, Herbert W ^a,b,c  

^a AUSTRALIAN CATHOLIC UNIVERSITY   (Australia)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c KING SAUD UNIVERSITY   (Saudi Arabia)

Author keywords

factor mixture analyses; latent profile analyses; level; performance evaluation; shape; students evaluations; teacher profiles; university teaching

Indexed keywords

MIXTURES; QUALITY CONTROL; STUDENTS; TEACHING;

LEVEL; MIXTURE ANALYSIS; PERFORMANCE EVALUATION; PROFILE ANALYSIS; SHAPE; TEACHER PROFILES; UNIVERSITY TEACHING;

FACTOR ANALYSIS;

EID: 84918554423     PISSN: 10705511     EISSN: 15328007     Source Type: Journal    
DOI: 10.1080/10705511.2014.919825     Document Type: Article

References (107)

1. Allport, G. (1937). Personality: A psychological interpretation. New York, NY: Holt, Rinehart & Winston.
2. Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 261–271.
3. Bailey, K. D. (1994). Typologies and taxonomies: An introduction to classification techniques. Thousand Oaks, CA: Sage.
4. Bauer, D. J. (2007). Observations on the use of growth mixture models in psychological research. Multivariate Behavioral Research, 42, 757–786.
5. Bauer, D. J., & Curran, P. J. (2004). The integration of continuous and discrete latent variable models: Potential problems and promising opportunities. Psychological Methods, 9, 3–29.
6. Bauer, D. J., & Shanahan, M. J. (2007). Modeling complex interactions: Person-centered and variable-centered approaches. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling ecological and contextual effects in longitudinal studies of human development (pp. 255–283). Mahwah, NJ: Erlbaum.
7. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analyses of covariance structures. Psychological Bulletin, 88, 588–606.
8. Bergman, L. R., Magnusson, D., & El-Khouri, B. M. (2003). Studying individual development in an interindividual context: A person-oriented approach. New York, NY: Psychology Press.
9. Bergman, L. R., & Trost, K. (2006). The person-oriented vs. the variable-oriented approach: Are they complementary, opposites, or different worlds? Merrill-Palmer Quarterly, 52, 601–632.
10. Black, P., McCormick, R., James, M., & Pedder, D. (2006). Assessment for learning and learning how to learn: A theoretical enquiry. Research Papers in Education, 21, 119–132.
11. Brekelmans, M., Levy, J., & Rodriguez, R. (1993). A typology of teacher communication style. In T. Wubbels & J. Levy (Eds.), Do you know what you look like? (pp. 46–55). London, UK: Falmer.
12. Brewer, M. B. (1991). The social self: On being the same and different at the same time. Personality & Social Psychology Bulletin, 17, 475–482.
13. Brok, P. D., Fisher, D., Brekelmans, M., Wubbels, T., & Rickards, T. (2006). Secondary teachers’ interpersonal behavior in Singapore, Brunei and Australia: A cross-national comparison. Asia Pacific Journal of Education, 26, 79–95.
14. Canrinus, E. T., Helms-Lorenz, M., Beijaard, D., Buitink, J., & Hoffman, A. (2011). Profiling teachers’ sense of professional identity. Educational Studies, 37, 593–608.
15. Cattell, R. B. (1952). The three basic factor-analytic designs: Their interrelations and derivatives. Psychological Bulletin, 49, 499–520.
16. Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Structural Equation Modeling, 41, 189–225.
17. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
18. Cross, S. E., & Madson, L. (1997). Models of the self: Self-construals and gender. Psychological Bulletin, 122, 5–37.
19. Cudeck, R., & Henly, S. J. (2003). A realistic perspective on pattern representation in growth data: Comment on Bauer and Curran (2003). Psychological Methods, 8, 378–383.
20. De Boek, P., Wilson, M., & Acton, G. S. (2005). A conceptual framework for distinguishing categories and dimensions. Psychological Review, 112, 129–158.
21. Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. C., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait–multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253.
22. Feldman, K. A. (1997). Identifying exemplary teachers and teaching: Evidence from student ratings. In R. P. Perry & J. C. Smart (Eds.), Effective teaching in higher education: Research and Practice (pp. 368–395). New York, NY: Agathon.
23. Fletcher, C. (2001). Performance appraisal and management: The developing research agenda. Journal of Occupational and Organizational Psychology, 74, 473–487.
24. Hativa, N. (1996). University instructors’ ratings profiles: Stability over time, and disciplinary differences. Research in Higher Education, 37, 341–365.
25. Helgeson, V. S. (1994). Relation of agency and communion to well-being: Evidence and potential explanations. Psychological Bulletin, 116, 412–428.
26. Henson, J. M., Reise, S. P., & Kim, K. H. (2007). Detecting mixtures from structural model differences using latent variable mixture modeling: A comparison of relative model fit statistics. Structural Equation Modeling, 14, 202–226.
27. Hershberger, S. L. (2006). The problem of equivalent structural models. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (pp. 13–41). Greenwich, CT: Information Age.
28. Hipp, J. R., & Bauer, D. J. (2006). Local solutions in the estimation of growth mixture models. Psychological Methods, 11, 36–53.
29. Hobson, C. J., & Gibson, F. W. (1983). Policy capturing as an approach to understanding and improving performance appraisal: A review of the literature. Academy of Management Review, 8, 640–649.
30. Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41–54.
31. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453.
32. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
33. James, M., & Pedder, D. (2006). Beyond method: Assessment and learning practices and values. The Curriculum Journal, 17, 109–138.
34. Jöreskog, K., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Lincolnwood, IL: Scientific Software International.
35. Krueger, R. F., Markon, K. E., Patrick, C. J., & Iacono, W. G. (2005). Externalizing psychopathology in adulthood: A dimensional-spectrum conceptualization and its implication for DSM–IV. Journal of Abnormal Psychology, 114, 537–550.
36. Kuo, P.-H., Aggen, S. H., Prescott, C. A., Kendler, K. S., & Neale, M. C. (2008). Using factor mixture modeling approach in alcohol dependence in a general population sample. Drug and Alcohol Dependence, 98, 105–114.
37. Latham, G. P., & Mann, S. (2006). Advances in the science of performance appraisal: Implications for practice. In G. P. Hodgkinson & J. K. Ford (Eds.), International review of industrial and organizational psychology (Vol. 26, pp. 295–337). Hoboken, NJ: Wiley.
38. Lubke, G. H., & Muthén, B. O. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21–39.
39. Lubke, G. H., & Muthén, B. O. (2007). Performance of factor mixture models as a function of model size, covariate effects, and class-specific parameters. Structural Equation Modeling, 14, 26–47.
40. Lubke, G., & Neale, M. (2006). Distinguishing between latent classes and continuous factors: Resolution by maximum likelihood? Multivariate Behavioral Research, 41, 499–532.
41. Lubke, G., & Neale, M. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models? Multivariate Behavioral Research, 43, 592–620.
42. Lubke, G., & Tueller, S. (2010). Latent class detection and class assignment: A comparison of the MAXEIG taxometric procedure and factor mixture modeling approaches. Structural Equation Modeling, 17, 605–628.
43. MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–40.
44. Magidson, J., & Vermunt, J. K. (2002). Latent class models for clustering: A comparison with K-means. Canadian Journal of Marketing Research, 20, 37–44.
45. Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching. British Journal of Educational Psychology, 52, 77–95.
46. Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253–388.
47. Marsh, H. W. (2007a). Self-concept theory, measurement and research into practice: The role of self-concept in educational psychology. Leicester, UK: British Psychological Society.
48. Marsh, H. W. (2007b). Students’ evaluations of university teaching: Dimensionality, reliability, validity, biases, and usefulness. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: Evidence-based perspective (pp. 319–383). Dordrecht, The Netherlands: Springer.
49. Marsh, H. W., & Bailey, M. (1993). Multidimensionality of students’ evaluations of teaching effectiveness: A profile analysis. Journal of Higher Education, 64, 1–18.
50. Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391–410.
51. Marsh, H. W., & Hau, K.-T. (2007). Latent-variable models in educational psychology: The need for methodological–substantive synergies. Contemporary Educational Psychology, 32, 151–171.
52. Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit evaluation in structural equation modeling. In A. Maydeu-Olivares & J. McArdle (Eds.), Psychometrics: A festschrift to Roderick P. McDonald (pp. 275–340). Hillsdale, NJ: Erlbaum.
53. Marsh, H. W., & Hocevar, D. (1991). The multidimensionality of students’ evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level. Teaching and Teacher Education, 7, 9–18.
54. Marsh, H. W., Lüdtke, O., Trautwein, U., & Morin, A. J. S. (2009). Latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to the internal/external frame of reference model. Structural Equation Modeling, 16, 1–35.
55. Marsh, H. W., Muthén, B. O., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439–476.
56. Marsh, H. W., & Roche, L. A. (1993). The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal, 30, 217–251.
57. Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low workloads on students’ evaluations of teaching: Popular myth, bias, validity or innocent bystanders? Journal of Educational Psychology, 92, 202–228.
58. Masyn, K. E., Henderson, C. E., & Greenbaum, P. E. (2010). Exploring the latent structures of psychological constructs in social development using the dimensional-categorical spectrum. Social Development, 19, 473–493.
59. McClelland, D. C. (1987). Human motivation. New York, NY: Cambridge University Press.
60. McLachlan, G., & Peel, D. (2000). Finite mixture models. New York, NY: Wiley.
61. Meece, J. L., Anderman, E. M., & Anderman, L. H. (2005). Classroom goal structure, student motivation and academic achievement. Annual Review of Psychology, 57, 487–503.
62. Midgley, C. (2002). Goals, goal structures, and patterns of adaptive learning. Mahwah, NJ: Erlbaum.
63. Morin, A. J. S., Maïano, C., Nagengast, B., Marsh, H. W., Morizot, J., & Janosz, M. (2011). General growth mixture analysis of adolescents’ developmental trajectories of anxiety: The impact of untested invariance assumptions on interpretations. Structural Equation Modeling, 18, 613–648.
64. Morin, A. J. S., Marsh, H. W., & Nagengast, B. (2013). Exploratory structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed., pp. 395–436). Greenwich, CT: Information Age.
65. Morin, A. J. S., Morizot, J., Boudrias, J.-S., & Madore, I. (2011). A multifoci person-centered perspective on workplace affective commitment: A latent profile/factor mixture analysis. Organizational Research Methods, 14, 58–90.
66. Morin, A. J. S., Tran, A., & Caci, H. (2013). Factorial validity of the ADHD Adult Symptom Rating Scale in a French community sample: Results from the ChiP-ARD study. Journal of Attention Disorders. Advance online publication. doi:10.1177/1087054713488825
67. Morin, A. J. S., Rodriguez, D., Fallu, J.-S., Maïano, C., & Janosz, M. (2012). Academic achievement and adolescent smoking: A general growth mixture model. Addiction, 107, 819–828.
68. Muthén, B. O. (2001). Latent variable mixture modeling. In G. A. Marcoulides & R. E. Schumaker (Eds.), New developments in structural equation modeling (pp. 1–33). Mahwah, NJ: Erlbaum.
69. Muthén, B. O. (2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29, 81–117.
70. Muthén, B. O. (2003). Statistical and substantive checking in growth mixture modeling: Comment on Bauer and Curran (2003). Psychological Methods, 8, 369–377.
71. Muthén, B. O. (2006). Should substance use disorders be considered as categorical of dimensional? Addiction, 101(Suppl. 1), 6–16.
72. Muthén, B. O., & Asparouhov, T. (2006). Item response mixture modeling: Application to tobacco dependence criteria. Addictive Behaviors, 31, 1050–1066.
73. Muthén, B. O., & Asparouhov, T. (2009). Growth mixture modeling: Analysis with non-Gaussian random effects. In G. Fitzmaurice, M. Davidian, G. Verbeke, & G. Molenberghs (Eds.), Longitudinal data analysis (pp. 143–165). Boca Raton, FL: Chapman & Hall/CRC.
74. Muthén, B. O., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics, 55, 463–469.
75. Muthén, L. K., & Muthén, B. O. 1998–2010. Mplus user’s guide. Los Angeles CA: Muthén & Muthén.
76. Myers, I. B., & McCaulley, M. H. (1985). Manual: A guide to the development and use of the Myers–Briggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press.
77. Nagin, D. S. (2010). Group-based trajectory modeling: An overview. In A. R. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 53–67). New York, NY: Springer.
78. Nylund, K. L., Asparouhov, T., & Muthén, B. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14, 535–569.
79. Petras, H., & Masyn, K. (2010). General growth mixture analysis with antecedents and consequences of change. In A. R. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 69–100). New York, NY: Springer.
80. Oscarson, D. J., & Finch, C. R. (1979). Adoption-proneness among trade and industrial teachers as measured by cluster analysis. Paper presented at the 73rd annual convention of the American Vocational Association, Anaheim, CA.
81. Pedder, D. (2007). Profiling teachers’ professional learning practices and values: Differences between and within schools. The Curriculum Journal, 18, 231–252.
82. Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science and recommendations on how to control it. Annual Review of Psychology, 63, 539–569.
83. Prawat, R. S. (1985). Affective versus cognitive goal orientations in elementary teachers. American Educational Research Journal, 22, 587–604.
84. Reb, J., & Greguras, G. J. (2010). Understanding performance ratings: Dynamic performance, attributions, and rating purpose. Journal of Applied Psychology, 95, 213–220.
85. Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667–696.
86. Renaud, R. D., & Murray, H. G. (2005). Factorial validity of student ratings of instruction. Research in Higher Education, 46, 929–953.
87. Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment and Evaluation in Higher Education, 30, 387–415.
88. Rickards, T., Brok, P. D., & Fisher, D. (2005). The Australian science teacher: A typology of teacher–student interpersonal behaviour in Australia. Learning Environments Research, 8, 267–287.
89. Roseth, C. J., Johnson, D. W., & Johnson, R. T. (2008). Promoting early adolescents’ achievement and peer relationships: The effects of cooperative, competitive, and individualistic goal structures. Psychological Bulletin, 134, 223–246.
90. Sang, G., Valcke, M., Braak, J. V., & Tondeur, J. (2009). Investigating teachers’ educational beliefs in Chinese primary schools: Socioeconomic and geographical perspectives. Asia-Pacific Journal of Teacher Education, 37, 363–377.
91. Shavelson, R. J., & Stern, P. (1981). Research on teachers’ pedagogical thoughts, judgments, decisions and behaviors. Review of Educational Research, 51, 455–498.
92. Steinley, D., & Brusco, M. J. (2011). Evaluating mixture modeling for clustering: Recommendations and cautions. Psychological Methods, 16, 63–79.
93. Steinley, D., & McDonald, R. P. (2007). Examining factor scores distributions to determine the nature of latent spaces. Multivariate Behavioral Research, 42, 133–156.
94. Sterba, S. K., & Pek, J. (2012). Individual influence on model selection. Psychological Methods, 17, 582–599.
95. Swank, P. R., Taylor, R. D., Brady, M. P., & Freiberg, H. J. (1989). Sensitivity of classroom observation systems: Measuring teacher effectiveness. Journal of Experimental Education, 57, 171–186.
96. Tay, L., Newman, D. A., & Vermunt, J. K. (2011). Using mixed-measurement item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence. Organizational Research Methods, 14, 147–176.
97. Tofighi, D., & Enders, C. (2008). Identifying the correct number of classes in growth mixture models. In G. R. Hancock & K. M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 317–341). Charlotte, NC: Information Age.
98. Tolvanen, A. (2007). Latent growth mixture modeling: A simulation study (Unpublished doctoral dissertation). University of Jyvaskyla, Jyvaskyla, Finland.
99. Uebersax, J. S. (1999). Probit latent class analysis with dichotomous or ordered category measures: Conditional independence/dependence. Applied Psychological Measurement, 23, 283–297.
100. Vermunt, J. K. (2011). K-means may perform as well as mixture model clustering but may also be much worse: Comment on Steinley and Brusco (2011). Psychological Methods, 16, 82–88.
101. Vermunt, J. K., & Magidson, J. (2000). Latent GOLD 1.0 user’s manual. Boston, MA: Statistical Innovations.
102. Vermunt, J. K., & Magidson, J. (2002). Latent class cluster analysis. In J. Hagenaars & A. McCutcheon (Eds.), Applied latent class models (pp. 89–106). New York, NY: Cambridge University Press.
103. Waller, N. G., & Meehl, P. E. (1998). Multivariate taxometric procedures: Distinguishing types from continua. Thousand Oaks, CA: Sage.
104. Wang, C. K. J., & Liu, W. C. (2008). Teachers’ motivation to teach national education in Singapore: A self-determination theory approach. Asia Pacific Journal of Education, 28, 395–410.
105. Winterbottom, M., Taber, K., Brindley, S., Fisher, L., Finney, J., & Riga, F. (2008). Understanding differences in teachers’ values and practices in assessment. Teacher Development, 12, 15–35.
106. Woolley, S. L., Benjamin, W. J. J., & Woolley, A. W. (2004). Construct validity of a self-report measure of teacher beliefs related to constructivist and traditional approaches to teaching and learning. Educational and Psychological Measurement, 64, 319–331.
107. Yang, C. (2006). Evaluating latent class analyses in qualitative phenotype identification. Computational Statistics & Data Analysis, 50, 1090–1104