Separating Cognitive and Content Domains in Mathematical Competence

Educational Assessment

Volumn 19, Issue 4, 2014, Pages 243-266

  Harks, Birgit ^a   Klieme, Eckhard ^a   Hartig, Johannes ^a   Leiss, Dominik ^b  

^a GERMAN INSTITUTE FOR INTERNATIONAL EDUCATIONAL RESEARCH   (Germany)

^b LEUPHANA UNIVERSITY OF LÜNEBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84911161784     PISSN: 10627197     EISSN: None     Source Type: Journal    
DOI: 10.1080/10627197.2014.964114     Document Type: Article

References (74)

1. Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23. doi:10.1177/0146621697211001
2. Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York, NY: Longman.
3. Asparouhov, T., & Muthén, B. (2005). Multivariate statistical modeling with survey data. Proceedings of the Federal Committee on Statistical Methodology (FCSM) Research Conference. Retrieved from http://www.fcsm.gov/05papers/Asparouhov_Muthen_IIA.pdf
4. Babcock, B. (2011). Estimating a noncompensatory IRT model using metropolis within Gibbs sampling. Applied Psychological Measurement, 35, 317–329. doi:10.1177/0146621610392366
5. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7–74. doi:10.1080/0969595980050102
6. Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York, NY: David McKay.
7. Blum, W., & Leiss, D. (2005). Modellieren im Unterricht mit der “Tanken”-Aufgabe [Modeling in instruction using the “refueling”-task]. Mathematik lehren, 128, 18–21.
8. Blum, W., & Leiss, D. (2007). Investigating quality mathematics teaching—the DISUM project. In C. Bergsten & B. Grevholm (Eds.), Developing and researching quality in mathematics teaching and learning (pp. 3–16). Linköping, Sweden: SMDF.
9. Blum, W., Neubrand, M., Ehmke, T., Senkbeil, M., Jordan, A., Ulfig, F., & Carstensen, C. (2004). Mathematische Kompetenz [Mathematical competence]. In PISA-Konsortium Deutschland (Ed.), PISA 2003. Der Bildungsstand der Jugendlichen in Deutschland–Ergebnisse des zweiten internationalen Vergleichs (pp. 47–92). Münster, Germany: Waxmann.
10. Borg, I. (1986). Facettentheorie: Prinzipien und Beispiele [Facet theory: Principles and examples]. Psychologische Rundschau, 37, 121–137.
11. Brunner, M. (2006). Mathematische Schülerleistung: Struktur, Schulformunterschiede und Validität [Student achievement in mathematics: Structure, school type differences and validity] (Unpublished doctoral dissertation). Humboldt-Universität zu Berlin, Germany. Retrieved from http://library.mpib-berlin.mpg.de/diss/Brunner_Dissertation.pdf
12. Bürgermeister, A., Klimczak, M., Klieme, E., Rakoczy, K., Blum, W., Leiß, D., … Besser, M. (2011). Leistungsbeurteilung im Mathematikunterricht–Eine Darstellung des Projekts “Nutzung und Auswirkungen der Kompetenzmessung in mathematischen Lehr-Lernprozessen” [Assessment in mathematics instruction–A description of the project “Conditions and Consequences of Classroom Assessment”]. Schulpädagogik-heute, 2(3), 1–18.
13. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. doi:10.1037/0021-9010.78.1.98
14. Csapó, B. (2010). Goals of learning and the organization of knowledge. Zeitschrift für Pädagogik, 56, 12–27. Retrieved from http://www.beltz.de/de/nc/paedagogik/zeitschriften/zeitschrift-fuer-paedagogik.html
15. DiBello, L., Roussos, L. A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. V. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 979–1027). Amsterdam, the Netherlands: Elsevier.
16. Else-Quest, N. M., Hyde, J. S., & Linn, M. C. (2010). Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin, 136, 103–127. doi:10.1037/a0018053
17. Frey, E., Hartig, J., & Rupp, A. A. (2009). An NCME instructional module on booklet designs in large-scale assessments of student achievement: Theory and practice. Educational Measurement: Issues and Practice, 28(3), 39–53. doi:10.1111/j.1745-3992.2009.00154.x
18. Gustafsson, J. E. (1994). Hierarchical models of intelligence and educational achievement. In A. Demetriou & A. Efklides (Eds.), Intelligence, mind, and reasoning: Structure and development (pp. 45–73). Amsterdam, the Netherlands: Elsevier.
19. Haberman, S. J., & Sinharay, S. (2010). Reporting of subscores using multidimensional item response theory. Psychometrika, 75, 209–227. doi:10.1007/s11336-010-9158-4
20. Harks, B. (2013). Kompetenzdiagnostik und Rückmeldung–zwei Komponenten formativen Assessments [Competence diagnostics and feedback–Two components of formative assessment] (Unpublished doctoral dissertation). Goethe-Universität, Frankfurt am Main, Germany.
21. Harks, B., Rakoczy, K., Hattie, J., Besser, M., & Klieme, E. (2014). The effects of feedback on achievement, interest and self-evaluation: The role of feedback's perceived usefulness. Educational Psychology, 34, 269–290. doi:10.1080/01443410.2013.785384
22. Harlen, W. (2008). Editior's introduction. In W. Harlen (Ed.), Student assessment and testing (pp. xix-xlvi). London, UK: Sage.
23. Hartig, J., & Höhler, J. (2008). Representation of competencies in multidimensional IRT models with within-item and between-item multidimensionality. Zeitschrift für Psychologie/Journal of Psychology, 216, 89–101. doi:10.1027/0044-3409.216.2.89
24. Hartig, J., & Höhler, J. (2009). Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation, 35, 57–63. doi:10.1016/j.stueduc.2009.10.002
25. Hattie, J. (2003). Formative and summative interpretations of assessment information. Retrieved from https://cdn.auckland.ac.nz/assets/education/hattie/docs/formative-and-summative-assessment-%282003%29.pdf
26. Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107, 139–155. doi:10.1037/0033-2909.107.2.139
27. Kaiser, G., & Steisel, T. (2000). Results of an analysis of the TIMS study from a gender perspektive. Zentralblatt für Didaktik der Mathematik, 32, 18–24. doi:10.1007/BF02652735
28. Klieme, E. (2000). Fachleistungen im voruniversitären Mathematik- und Physikunterricht: Theoretische Grundlagen, Kompetenzstufen und Unterrichtsschwerpunkte [Subject specific achievement in preuniversity mathematics and physics instruction: Theoretical basis, competence levels and instructional focuses]. In J. Baumert, W. Bos, & R. Lehmann (Eds.), TIMSS/III. Dritte Internationale Mathematik- und Naturwissenschaftsstudie–Mathematische und naturwissenschaftliche Bildung am Ende der Schullaufbahn: Bd. 2. Mathematische und physikalische Kompetenzen am Ende der gymnasialen Oberstufe (pp. 57–128). Opladen, Germany: Leske & Budrich.
29. Klieme, E., Hartig, J., & Rauch, D. (2008). The concept of competence in educational context. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 3–22). Göttingen, Germany: Hogrefe.
30. Klieme, E., Neubrand, M., & Lüdtke, O. (2001). Mathematische Grundbildung: Testkonzeption und Ergebnisse. [Mathematical basic education: Test conception and results]. In J. Baumert, E. Klieme, M. Neubrand, M. Prenzel, U. Schiefele, W. Schneider, … M. Weiß (Eds.), PISA 2000. Basiskompetenzen von Schülerinnen und Schülern im internationalen Vergleich (pp. 139–190). Opladen, Germany: Leske & Budrich.
31. Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras study—Investigating effects of teaching and learning in Swiss and German mathematics classrooms. In T. Janik & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 137–160). Münster, Germany: Waxmann.
32. Koeppen, K., Hartig, J., Klieme, E., & Leutner, D. (2008). Current issues in competence modeling and assessment. Zeitschrift für Psychologie/Journal of Psychology, 216, 61–73. doi:10.1027/0044-3409.216.2.61
33. Kunina-Habenicht, O. (2010). Theoretical and practical considerations for implementing diagnostic classification models: Insights from simulation-based and applied research (Unpublished doctoral dissertation). Humboldt-Universität zu Berlin, Germany. Retrieved from http://edoc.hu-berlin.de/dissertationen/kunina-habenicht-olga-2010-06-03/PDF/kunina-habenicht.pdf
34. Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2009). A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models. Studies in Educational Evaluation, 35, 64–70. doi:10.1016/j.stueduc.2009.10.003
35. Kupermintz, H., & Snow, R. E. (1997). Enhancing the validity and usefulness of large-scale educational assessments: III. NELS: 88 mathematics achievement to 12th grade. American Educational Research Journal, 34, 124–150. doi:10.3102/00028312034001124
36. Lane, S., Stone, C. A., Ankenmann, R. D., & Liu, M. (1995). Examination of the assumptions and properties of the graded item response model: An example using a mathematics performance assessment. Applied Measurement in Education, 8, 313–340. doi:10.1207/s15324818ame0804_3
37. Leiss, D., & Blum, W. (2006). Beschreibung zentraler mathematischer Kompetenzen [Description of central mathematical competencies]. In W. Blum, C. Drüke-Noe, R. Hartung, & O. Köller (Eds.), Bildungsstandards Mathematik: Konkret (pp. 33–50). Berlin, Germany: Cornelsen Scriptor.
38. Lindberg, S. M., Hyde, J. S., Petersen, J. L., & Linn, M. C. (2010). New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin, 136, 1123–1135. doi:10.1037/a0021276
39. Liu, O. L., Wilson, M., & Paek, I. (2008). A multidimensional Rasch analysis of gender differences in PISA mathematics. Journal of Applied Measurement, 9, 18–35.
40. McClelland, D. C. (1973). Testing for competence rather than for “intelligence.” American Psychologist, 28, 1–14. doi:10.1037/h0034092
41. Mullis, I. V. S., Martin, M. O., & Foy, P. (2008). TIMSS 2007 international mathematics report: Findings from IEA's trends in international mathematics and science study at the fourth and eighth grades. Chestnut Hill, MA: Boston College.
42. Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., & Chrostowski, S. J. (2004). TIMSS 2003 international mathematics report findings from IEA's trends in international mathematics and science study at the fourth and eighth grades. Chestnut Hill, MA: Boston College.
43. Mullis, I. V. S., Martin, M. O., Ruddock, G. J., O'Sullivan, C. Y., & Preuschoff, C. (2009). TIMSS 2011 assessment frameworks. Chestnut Hill, MA: Boston College.
44. Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus (Version 7.0) [Computer software]. Los Angeles, CA: Muthén & Muthén.
45. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.
46. Niss, M. (2003). Mathematical competencies and the learning of mathematics: The Danish KOM project. In A. Gagatsis & S. Papastavridis (Eds.), Mediterranean Conference on Mathematical Education (pp. 115–124). Athens, Greece: Hellenic Mathematical Society and Cyprus Mathematical Society.
47. OECD. (2002). PISA 2000 technical report. Paris, France: Author.
48. OECD. (2003). The PISA 2003 assessment framework—Mathematics, reading, science, and problem solving knowledge and skills. Paris, France: Author.
49. OECD. (2004). Learning for tomorrow's world. First results from PISA 2003. Paris, France: Author.
50. OECD. (2009a). PISA 2006 technical report. Paris, France: Author.
51. OECD. (2009b). PISA 2009 assessment framework. Key competencies in reading, mathematics and science. Paris, France: Author.
52. OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris, France: Author.
53. Rakoczy, K., Harks, B., Klieme, E., Blum, W., & Hochweber, J. (2013). Written feedback in mathematics: Mediated by students' perception, moderated by goal orientation. Learning and Instruction, 27, 63–73. doi:10.1016/j.learninstruc.2013.03.002
54. Reckase, M. D. (2009). Multidimensional item response theory (statistics for social and behavioral sciences). New York, NY: Springer.
55. Resnick, L. B., Cauzinille-Marmeche, E., & Mathieu, J. (1987). Understanding algebra. In J. A. Szoboda, & D. Rogers (Eds.), Cognitive processes in mathematics (pp. 169–203). Oxford, England: Clarendon.
56. Rittle-Johnson, B., Matthews, P. G., Taylor, R. S., & McEldoon, K. L. (2011). Assessing knowledge of mathematical equivalence: A construct-modeling approach. Journal of Educational Psychology, 103, 85–104. doi:10.1037/a0021334
57. Roberts, M. J. (2007). Integrating the mind: Domain general versus domain specific processes in higher cognition. Hove, England: Psychology Press.
58. Robitzsch, A. (2009). Methodische Herausforderungen bei der Kalibrierung von Leistungstests [Methodological challenges in calibrating performance tests]. In A. Bremerich-Vos, D. Granzer, & O. Köller (Eds.), Bildungsstandards Deutsch und Mathematik (pp. 42–106). Weinheim, Germany: Beltz Pädagogik.
59. Rost, J. (2004). Lehrbuch Testtheorie–Testkonstruktion [Textbook test theory–test construction]. Bern, Switzerland: Huber.
60. Russel, M. (2010). Technology-aided formative assessment and learning: New developments and applications. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of formative assessment (pp. 125–138). New York, NY: Routledge.
61. Ryan, K. E., & Chiu, S. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14, 73–90. doi:10.1207/S15324818AME1401_06
62. Sinharay, S. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47, 150–174. doi:10.1111/j.1745-3984.2010.00106.x
63. Stobart, G. (2006). The validity of formative assessment. In J. Gardner (Ed.), Assessment and learning (pp. 133–146). London, UK: Sage.
64. Stone, C. A., Ye, F., Zu, X., & Lane, S. (2010). Providing subscale scores for diagnostic information: A case study when the test is essentially unidimensional. Applied Measurement in Education, 23, 63–86. doi:10.1080/08957340903423651
65. Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313–324. doi:10.1111/j.1745-3984.2007.00041.x
66. Vasilyeva, M., Ludlow, L. H., Casey, B. M., & St. Onge, C. (2008). Examination of the psychometric properties of the measurement skills assessment. Educational and Psychological Measurement, 69, 106–130. doi:10.1177/0013164408318774
67. Walker, C. M., & Beretvas, S. N. (2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40, 255–275. doi:10.1111/j.1745-3984.2003.tb01107.x
68. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen & L. H. Salganik (Eds.), Defining and selecting key competencies (pp. 45–65). Bern, Switzerland: Hogrefe & Huber.
69. Wiliam, D. (2006). Formative assessment: Getting the focus right. Educational Assessment, 11, 283–289. doi:10.1080/10627197.2006.9652993
70. Winkelmann, H. (2009). Validierung der länderübergreifenden Bildungsstandards für mathematische Kompetenzen im Primarbereich [Validation of cross-country education standards for mathematics competence in primary education] (Unpublished doctoral dissertation). Humboldt-Universität zu Berlin, Germany.
71. Winkelmann, H., & Robitzsch, A. (2009). Modelle mathematischer Kompetenzen: Empirische Befunde zur Dimensionalität [Models of mathematical competence: Empirical findings on dimensionality]. In A. Bremerich-Vos, D. Granzer, & O. Köller (Eds.), Bildungsstandards Deutsch und Mathematik (pp. 169–196). Weinheim, Germany: Beltz Pädagogik.
72. Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18, 93–113. doi:10.1007/BF03217438
73. Young, W. J., Cho, Y., Ling, G., Cline, F., Steinberg, J., & Stone, E. (2008). Validity and fairness of state standards-based assessments for English language learners. Educational Assessment, 13, 170–192. doi:10.1080/10627190802394388
74. Zhang, J. (2004). Comparison of unidimensional and multidimensional approaches to IRT parameter estimation (ETS Research Rep. No. 04-44). Princeton, NJ: Educational Testing Service.