Validity: Meaning and relevancy in assessment for engineering education research

Journal of Engineering Education

Volumn 104, Issue 2, 2015, Pages 108-118

  Douglas, Kerrie Anna ^a   Purzer, Şenay ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84928186965     PISSN: 10694730     EISSN: 21689830     Source Type: Journal    
DOI: 10.1002/jee.20070     Document Type: Editorial

References (42)

1. Ahn, B., Cox, M. F., London, J., Cekic, O., & Zhu, J. (2014). Creating an instrument to measure leadership, change, and synthesis in engineering undergraduates. Journal of Engineering Education, 103(1), 115-136. doi:10.1002/jee.20036
2. American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (1999). Standards for educational and psychological testing (1999 ed.). Washington, DC: American Educational Research Association.
3. American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (2014). Standards for educational and psychological testing (2014 ed.). Washington, DC: American Educational Research Association.
4. Bodner, G. M., & Guay, R. B. (1997). The Purdue Visualization of Rotations Test. Chemical Educator, 2(4), 1-18. doi:10.1333/s00897970138a
5. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.
6. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. doi:10.1037/h0040957
7. Douglas, K. A., & Strobel, J. (2014). Hopes and Goals Survey for use in STEM elementary education. International Journal of Technology and Design Education, 1-15. doi:10.1007/s10798-014-9277-9
8. Douglas, K. A., Yale, M. S., Bennett, D. E., Haugan, M. P., & Bryan, L. A. (2014) Evaluation of Colorado Learning Attitudes about Science Survey. Physics Review Special Topics: Physics Education Research, 10, 020128-1-10. doi:10.1103/PhysRevSTPER.10.020128
9. Fortenberry, N. L. (2006). An extensive agenda for engineering education research. Journal of Engineering Education, 95(1), 3-5. doi:10.1002/j.2168-9830.2006.tb00872.x
10. Gabriele, G. A. (2005). Advancing engineering education in a flattened world. Journal of Engineering Education, 94(3), 285-286. doi:10.1002/j.2168-9830.2005.tb00852.x
11. Guay, R. B. (1976). Purdue Spatial Visualization Test. West Lafayette, IN: Purdue Research Foundation.
12. Haghighi, K. (2005). Quiet no longer: Birth of a new discipline. Journal of Engineering Education, 94(4), 351-353. doi:10.1002/j.2168-9830.2005.tb00862.x
13. Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527-535. doi:10.1037/0033-2909.112.3.527
14. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Washington, DC: National Council on Measurement in Education and the American Council on Education.
15. Kerns, S. E. (2005). Keeping us on the same page. Journal of Engineering Education, 94(2), 205. doi:10.1002/j.2168-9830.2005.tb00840.x
16. Kubiszyn, T., & Borich, G. (2007). Educational testing and measurement (8th. ed.). Hoboken, NJ: John Wiley & Sons.
17. Landy, F. J. (1986). Stamp collecting versus science: Validation as hypothesis testing. American Psychologist, 41(11), 1183.
18. Lohmann, J. R. (2005). Building a community of scholars: The role of the Journal of Engineering Education as a research journal. Journal of Engineering Education, 94(1), 1-6. doi:10.1002/j.2168-9830.2005.tb00823.x
19. Maeda, Y., & Yoon, S. Y. (2013). A meta-analysis on gender differences in mental rotation ability measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R). Educational Psychology Review, 25(1), 69-94. doi:10.1007/s10648-012-9215-x
20. Maeda, Y., Yoon, S. Y., Kim-Kang, K., & Imbrie, P. K. (2013). Psychometric properties of the Revised PSVT:R for measuring first year engineering students' spatial ability. Human Computer Interaction in Engineering Education [Special issue]. International Journal of Engineering Education, 29(3), 763-776.
21. Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational researcher, 18(2), 5-11.
22. Messick, S. (1990). Validity of test interpretation and use. Retrieved from http://files.eric.ed.gov/fulltext/ED395031.pdf
23. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23. doi:10.3102/0013189X023002013
24. Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741-749.
25. Mislevy, R. J. (1994). Evidence and inference in educational assessment. Psychometrika, 59(4), 439-483. doi:10.1007/BF02294388
26. Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-62.
27. National Engineering Education Research Colloquies, Steering Committee. (2006). The research agenda for the new discipline of engineering education. Journal of Engineering Education, 95(4), 259-261.
28. National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
29. Olds, B. M., Moskal, B. M., & Miller, R. L. (2005). Assessment in engineering education: Evolution, approaches and future collaborations. Journal of Engineering Education, 94(1), 13-25. doi:10.1002/j.2168-9830.2005.tb00826.x
30. Pellegrino, J. W. (2012). Assessment of science learning: Living in interesting times. Journal of Research in Science Teaching, 49(6), 831-841. doi:10.1002/tea.21032
31. Radcliffe, D. F. (2006). Shaping the discipline of engineering education. Journal of Engineering Education, 95(4), 263-264. doi:10.1002/j.2168-9830.2006.tb00901.x
32. Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350.
33. Songer, N. B., & Ruiz-Primo, M. A. (2012). Assessment and science education: Our essential new priority? Journal of Research in Science Teaching, 49(6), 683-690. doi:10.1002/tea.21033
34. Streveler, R. A., Miller, R. L., Santiago-Román, A. I., Nelson, M. A., Geist, M. R., & Olds, B. M. (2011). Rigorous methodology for concept inventory development: Using the 'assessment triangle' to develop and test the Thermal and Transport Science Concept Inventory (TTCI). International Journal of Engineering Education, 27(5), 968-984.
35. Streveler, R. A., & Smith, K. A. (2006). Conducting rigorous research in engineering education. Journal of Engineering Education, 95(2), 103-105. doi:10.1002/j.2168-9830.2006.tb00882.x
36. Validity. (n.d.). In Merriam-Webster's online dictionary (11th ed.). Retrieved from http://m-w.com/dictionary/validity
37. Validation. (n.d.). Website of the National Council of Measurement in Education. Retrieved from http://ncme.org/resource-center/glossa
38. Wankat, P. C., Felder, R. M., Smith, K. A., & Oreovicz, F. S. (2002). The scholarship of teaching and learning in engineering. In M. T. Huber & S. P. Morrealle (Eds.), Disciplinary styles in the scholarship of teaching and learning: Exploring common ground (pp. 217-237). Sterling, VA: Stylus Publishing.
39. Weng, L. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64(6), 956-972. doi:10.1177/0013164404268674
40. Wise, L. (2014, September 12). Major Changes in the 2014 Standards. Presentation at the Standards Public Briefing on Standards for Educational and Psychological Testing: Essential Guidance and Key Developments in a New Era of Testing, Washington, DC. Retrieved from http://mp125118.cdn.mediaplatform.com/125118/wc/mp/4000/5592/5599/39268/Archive/default.htm
41. Wormley, D. N. (2006). A year of dialogue focused on engineering education research. Journal of Engineering Education, 95(3), 179. doi:10.1002/j.2168-9830.2006.tb00890.x
42. Yoon, S. Y., Evans, M. G., & Strobel, J. (2014). Validation of the teaching engineering self-efficacy scale for K-12 teachers: A structural equation modeling approach. Journal of Engineering Education, 103(3), 463-485. doi:10.1002/jee.20049