The science and design of assessment in engineering education

Cambridge Handbook of Engineering Education Research

Volumn , Issue , 2015, Pages 571-598

  Pellegrino, James W ^a   Di Bello, Louis V ^a   Brophy, Sean P ^b  

^a UNIVERSITY OF ILLINOIS AT CHICAGO   (United States)

^b Purdue University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DESIGN; EDUCATION COMPUTING; ENGINEERING EDUCATION; ENGINEERING RESEARCH; STATISTICAL METHODS; STEM (SCIENCE, TECHNOLOGY, ENGINEERING AND MATHEMATICS); STUDENTS;

BACK-GROUND KNOWLEDGE; CONCEPTUAL FRAMEWORKS; EDUCATIONAL ASSESSMENT; ENGINEERING EDUCATION RESEARCH; NATIONAL RESEARCH COUNCIL; SCIENCE , TECHNOLOGY , ENGINEERING , AND MATHEMATICS EDUCATIONS; SCIENTIFIC FOUNDATIONS; STATISTICAL INFERENCE;

EDUCATION;

EID: 84908140266     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9781139013451.036     Document Type: Chapter

References (58)

1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education (AERA/APA/NCME). (1999). Standards for educational and psychological testing. Washington, DC: Author.
2. Atman, C. J., Kilgore, D., Mc Kenna, A. (2008). Characterizing design learning: A mixed-methods study of engineering designers’ use of language. Journal of Engineering Education, 97(3), 309-326.
3. Baker, F. (2001). The basics of item response theory. ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD. Retrieved from http://echo.edres.org:8080/irt/baker/http://edres.org/irt/baker/
4. Barr, R. W., Cone, J., Roselli, R. J., Brophy, S. P. (2003). Initial experiences using an interactive classroom participation system (CPS) for presenting the iron cross biomechanics module. In Proceedings of the 2003 ASEE Gulf-Southwest Annual Conference, the University of Texas at Arlington. Lubbock, TX.
5. Bayes, T. P. (1763). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions ofthe Royal Society of London, 53, 370-418
6. Beatty, I. D., Gerace, W. J. (2009). Technology-enhanced formative assessment: A research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18(2), 1146-1162.
7. Bransford, J. D., Stein, B. S. (1993). The IDEAL problem solver (2nd ed.). New York, NY: W. H. Freeman.
8. Brophy, S., Klein, S., Portsmore, M., Rogers, C. (2008). Advancing engineering in P-12 classrooms. Journal of Engineering Education, 97(3), 369-387.
9. Brophy, S., Li, S. (2011). Problem definition in design by first year engineering students. In Proceedings of the American Society of Engineering Education Annual Conference, Vancouver, BC, Canada.
10. Chi, M. T. H., Glaser, R., Rees, E. (1982). Expertise in problem solving. In R. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1, p. 7-76). Hillsdale, NJ: Lawrence Erlbaum.
11. Cordray, D. S., Harris, T. R., Klein, S. (2009). A research synthesis of the effectiveness, replicability, and generality of the VaNTH challenge-based instructional models in bioengineering. Journal of Engineering Education, 98, 335-348.
12. Crocker, L., Algina, J. (2006). Introduction to classicaland modern test theory. New York, NY: Wadsworth.
13. Crouch, C. H., Mazur, E. (2001). Peer instruction: Ten years of experience andresults. American Journal of Physics, 69, 970-977.
14. DiBello, L. V., Roussos, L. A., Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics, Vol. 26: Psychometrics (p. 979-1030). Amsterdam: Elsevier.
15. DiBello, L., Stout, W. F. (2003). Student profile scoring methods for informative assessment. In H. Yanai, A. Okada, K. Shigemasu, J. J. Meulmann (Eds.), New developments in psychometrics (p. 81-92). Tokyo: Springer.
16. DiBello, L. V., Stout, W. F. (Eds.). (2007) Special issue on IRT-based cognitive diagnostic models and related methods, Journal of Educational Measurement, 44(4), 285-292.
17. Dym, C. L., Agogino, A. M., Eris, V., Frey, D., Leifer, L. J. (2005). Engineering design, thinking, teaching and learning. Journal ofEngineer-ing Education, 94(1), 103-120.
18. Haertel, E. H., Lorie, W. A. (2004). Validating standards-based test score interpretations. Measurement: Interdisciplinary Research and Perspectives, 2(2), 61-103.
19. Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74.
20. Hambleton, R. K., Swaminathan, H., Rogers, H. J. (1991). Fundamentals of item response theory. Measurement Methods for the Social Science. Newbury Park, CA: SAGE.
21. Hatano, G., Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azuma, K. Hakuta (Eds.), Child development and education in Japan. New York, NY: W. H. Freeman.
22. Hestenes, D., Wells, M., Swackhamer, G. (1992, March). Force concept inventory. The Physics Teacher, 30, 159-166.
23. Hickey, D., Pellegrino, J. W. (2005). Theory, level, and function: Three dimensions for understanding transfer and student assessment. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (p. 251-293). Greenwich, CT: Information Age.
24. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., p. 17-64). Westport, CT: Praeger.
25. Lord, R. M., Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
26. Marion, S., Pellegrino, J. W. (2006, Winter). A validity framework for evaluating the technical quality of alternate assessments. Educational Measurement: Issues and Practice, 4757.
27. Martin, T., Petrosino, A. J., Rivale, S., Diller, K. R. (2006). The development of adaptive expertise in biotransport. New Directions for Teaching and Learning, 108, 35-47.
28. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 1323.
29. Mislevy, R. J., Haertel, G. (2006). Implications of evidence-centered design for educational assessment. Educational Measurement: Issues and Practice, 25, 6-20.
30. Mislevy, R. J., Riconscente, M. M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of test development (p. 61-90). Mahwah, NJ: Lawrence Erlbaum.
31. Mislevy, R. J., Steinberg, L., Almond, R. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-67.
32. National Research Council. (2003). Assessment in support of learning and instruction: Bridging the gap between large-scale and classroom assessment. Washington, DC: The National Academies Press.
33. Nersessian, N. J. (2009). How do engineering scientists think? Model-based simulation in biomedical engineering laboratories. Topics in Cognitive Science, 1, 730-757.
34. Pellegrino, J. W., Chudowsky, N., Glaser, R. (Eds.). (2001). Knowing what students know: The science and design ofeducational assessment. Washington, DC: The National Academies Press.
35. Pellegrino, J. W., Hickey, D. (2006). Educational assessment: Towards better alignment between theory and practice. In L. Verschaf-fel, F. Dochy, M. Boekaerts, S. Vosniadou (Eds.), Instructional psychology: Past, present and future trends. Sixteen essays in honour of Erik De Corte (p. 169-189). Oxford: Elsevier.
36. Polya, G. (1945). How to solve it. Princeton, NJ: Princeton University Press.
37. Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223-231.
38. Prince, M. J., Felder, R. M. (2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95, 123-138.
39. Roselli, R. J., Brophy, S. P. (2003). Redesigning a biomechanics course using challenge-based instruction. IEEE Engineering in Medicine and Biology Magazine, 22(4), 66-70.
40. Roselli, R. J., Brophy, S. P. (2006). Experiences with formative assessment in engineering classrooms. Journal of Engineering Education, 95(4), 325-333.
41. Roselli, R. J., Howard, L., Brophy, S. P. (2006). A computer-based free body diagram assistant. Computer Applications in Engineering Education, 14(4), 281-290.
42. Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L., Klein, S. (2002). On the evaluation of systemic science education reform: Searching for instructional sensitivity. Journal of Research in Science Teaching, 39, 369-393.
43. Santiago-Roman, A. I. (2009). Fitting cognitive diagnostic assessment to the concept assessment tool for statics (CATS) (Doctoral dissertation). Purdue University, Lafayette, IN.
44. Santiago-Roman, A. I., Streveler, R. A., DiBello, L. (2010). The development of estimated cognitive attribute profiles for the Concept Assessment Tool for Statics. Paper presented at the 40th ASEE/IEEE Frontiers in Education Conference, Washington, DC.
45. Santiago-Roman, A. I., Streveler, R. A., Steif, P., DiBello, L. V. (2010). The development of a Q-matrix for the Concept Assessment Tool for Statics. Paper presented at the ERM Division of the ASEE Annual Conference and Exposition, Louisville, KY.
46. Schum, D. (1987). Evidence and inference for the intelligence analyst. Lantham, MD: University Press of America.
47. Schwartz, D. L., Arena, D. (2009). Choice-based assessments for the digital age. Stanford University, School of Education. Retrieved from http://courses.ischool.berkeley.edu/1290-pm4e/f10/sites/default/files/assessment_ schwartz.pdf
48. Schwartz, D. L., Bransford, J. D., Sears, D. (2005). Efficiency and innovation in transfer. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (p. 1-51). Greenwich, CT: Information Age.
49. Smith, K. A., Johnson, D. W., Johnson, R. W., Sheppard, S. D. (2005). Pedagogies of engagement: Classroom-based practices. Journal of Engineering Education, 94(1), 87101.
50. Steif, P. S., Dantzler, J. A. (2004). A statics concept inventory: Development and psychometric analysis. Journal of Engineering Education, 94 (4), 363-371.
51. Steif, P. S., Hansen, M. A. (2006). Comparisons between performances in a statics concept inventory and course examinations, International Journal of Engineering Education, 22, 1070-1076.
52. Steif, P. S., Hansen, M. A. (2007). New practices for administering and analyzing the results of concept inventories. Journal of Engineering Education, 96, 205-212.
53. Stigler, S. M. (1982). Thomas Bayes’ Bayesian inference. Journal of the Royal Statistical Society, Series A, 145, 250-258.
54. Toulmin, S. E. (2003). The uses of argument. Cambridge: Cambridge University Press.
55. van der Linden, W. J., Hambleton, R. K. (Eds.) (1996). Handbook of modern item response theory. New York, NY: Springer.
56. Walker, J. T., Cordray, D. S., King, P. H., Brophy, S. P. (2006). Design scenarios as an assessment of adaptive expertise. International Journal of Engineering Education, 22(1), 1-7.
57. Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass.
58. Woods, D. R. (1997). Developing problemsolving skills: The Mc Master problem solving program. Journal of Engineering Education, 86(2) 75-92.