Performance-based assessment of graduate student research skills: timing, trajectory, and potential thresholds

Studies in Higher Education

Volumn 38, Issue 5, 2013, Pages 693-710

  Timmerman, Briana Crotwell ^a   Feldon, David ^b   Maher, Michelle ^a   Strickland, Denise ^a,b   Gilmore, Joanna ^c  

^a UNIVERSITY OF SOUTH CAROLINA   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

^c UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

graduate education; higher degree research; performance assessment; threshold concepts

Indexed keywords

EID: 84878639051     PISSN: 03075079     EISSN: 1470174X     Source Type: Journal    
DOI: 10.1080/03075079.2011.590971     Document Type: Article

References (39)

1. Caicedo, J. M., Pierce, C. E., Flora, J., Timmerman, B., Nichols, A. P., Graf, W. and Ray, T. Forthcoming. " Instructional environment to stimulate critical thought of freshmen civil engineering students ". In Advances in Engineering Education
2. Carnegie Initiative on the Doctorate. 2001. Overview of doctoral educational studies and reports: 1990-present, Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
3. Davis, G. and Fiske, P. 2001. The 2000 national doctoral program survey University of Missouri-Columbia: National Association of Graduate-Professional Students
4. Dehaan, R. L. 2005. The impending revolution in undergraduate science education. Journal of Science Education and Technology, 14 (2): 253 - 70. (doi:10.1007/s10956-005-4425-3)
5. Dunbar, K. 2000. How scientists think in the real world: Implications for science education. Journal of Applied Developmental Psychology, 21 (1): 49 - 58. (doi:10.1016/S0193-3973(99)00050-7)
6. Ericsson, K. A. and Charness, N. 1994. Expert performance: Its structure and acquisition. American Psychologist, 49 (8): 725 - 47. (doi:10.1037/0003-066X.49.8.725)
7. Feldon, D. 2007. The implications of research on expertise for curriculum and pedagogy. Educational Psychology Review, 19 (2): 91 - 110. (doi:10.1007/s10648-006-9009-0)
8. Feldon, D. 2010. Do psychology researchers tell it like it is? A microgenetic analysis of research strategies and self-report accuracy along a continuum of expertise. Instructional Science, 38 (4): 395 - 415. (doi:10.1007/s11251-008-9085-2)
9. Feldon, D. F., Maher, M. M. and Timmerman, B. E. 2010. Performance-based data in the study of STEM PhD education. Science, 329 (5989): 282 - 3. (doi:10.1126/science.1191269)
10. Feldon, D. F., Timmerman, B. E., Showman, R. and Stowe, K. A. 2010. Translating expertise into effective instruction: The impacts of cognitive task analysis (CTA) on laboratory report quality and student retention in the biological sciences. Journal of Research in Science Teaching, 47 (10): 1165 - 85. (doi:10.1002/tea.20382)
11. Golde, C. M. 2001. Findings of the survey of doctoral education and career preparation: A report to the preparing future faculty program, Madison: University of Wisconsin.
12. Hackett, E. and Rhoten, D. 2009. The snowbird charrette: Integrative interdisciplinary collaboration in environmental research design. Minerva, 47 (4): 407 - 40. (doi:10.1007/s11024-009-9136-0)
13. Halonen, J. S., Bosack, T., Clay, S., McCarthy, M., Dunn, D. S., Hill, G. W., McEntarffer, R., Mehrotra, C., Nesmith, R., Weaver, K. A. and Whitlock, K. 2003. A rubric for learning, teaching, and assessing scientific inquiry in psychology. Teaching of Psychology, 30 (3): 196 - 208. (doi:10.1207/S15328023TOP3003_01)
14. Hinds, P. J. 1999. The curse of expertise: The effects of expertise and debiasing methods on predictions of novice performance. Journal of Experimental Psychology: Applied, 5 (2): 205 - 21. (doi:10.1037/1076-898X.5.2.205)
15. Hofstein, A. and Lunetta, V. N. 2004. The laboratory in science education: Foundations for the twenty-first century. Science Education, 88 (1): 28 - 54. (doi:10.1002/sce.10106)
16. Johnson, R. L., Penny, J. and Gordon, B. 2000. The relation between score resolution methods and interrater reliability: An empirical study of an analytic scoring rubric. Applied Measurement in Education, 13 (2): 121 - 38. (doi:10.1207/S15324818AME1302_1)
17. Johnson, R. L., Penny, J., Gordon, B., Shumate, S. R. and Fisher, S. P. 2005. Resolving score differences in the rating of writing samples: Does discussion improve the accuracy of scores?. Language Assessment Quarterly: An International Journal, 2 (2): 117 - 46. (doi:10.1207/s15434311laq0202_2)
18. Keys, C. W. 1994. The development of scientific reasoning skills in conjunction with collaborative writing assignments: An interpretive study of six ninth-grade students. Journal of Research in Science Teaching, 31 (9): 1003 - 22. (doi:10.1002/tea.3660310912)
19. Kiley, M. 2008. Quality in postgraduate education, Canberra: Australian National University. Centre for Educational Development and Academic Methods 2008. http://www.qpr.edu.au/overview.html
20. Kiley, M. 2009. Identifying threshold concepts and proposing strategies to support doctoral candidates. Innovations in Education and Teaching International, 46 (3): 293 - 304. (doi:10.1080/14703290903069001)
21. Kiley, M. and Wisker, G. 2009. Threshold concepts in research education and evidence of threshold crossing. Higher Education Research & Development, 28 (4): 431 - 41. (doi:10.1080/07294360903067930)
22. Land, R., Meyer, J. H.F. and Smith, J. 2008. Threshold concepts within the disciplines, Edited by: Land, R., Meyer, J. H.F. and Smith, J. Rotterdam: Sense.
23. Lawson, A. E. 2008. Biology: An inquiry approach, 2, Dubuque, IA: Kendall/Hunt Publishing Company.
24. Marx, R. W., Blumenfeld, P. C., Krajcik, J. S., Fishman, B., Soloway, E., Geier, R. and Tal, R. T. 2004. Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform. Journal of Research in Science Teaching, 41 (10): 1063 - 80. (doi:10.1002/tea.20039)
25. Mervis, J. 2000. Graduate educators struggle to grade themselves. Science, 287 (5453): 568 - 70. (doi:10.1126/science.287.5453.568)
26. Meyer, J. H.F. and Land, R. 2006. Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge, Edited by: Meyer, J. H.F. and Land, R. Abingdon, UK: Routledge.
27. National Research Council. 1996. National science education standards, Washington, DC: National Academy Press.
28. Onwuegbuzie, A. J. 2003. Modeling statistics achievement among graduate students. Educational and Psychological Measurement, 63 (6): 1020 - 38. (doi:10.1177/0013164402250989)
29. Ostriker, J. P. and Kuh, C. V. 2003. Assessing research-doctorate programs: A methodology study, Edited by: Ostriker, J. P. and Kuh, C. V. Washington, DC: National Academies Press.
30. Popper, K. 1959. The logic of scientific discovery, New York: Routledge.
31. Roberts, G. 2002. Set for success: The supply of people with science, technology, engineering and mathematical skills HM Treasury, April 2002. http://www.employment-studies.co.uk/pubs/report.php?id=1440robert
32. Sandoval, W. A. and Reiser, B. J. 2004. Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88 (3): 345 - 72. (doi:10.1002/sce.10130)
33. Schroeder, C. M., Scott, T. P., Tolson, H., Huang, T.-Y. and Lee, Y.-H. 2007. A meta-analysis of national research: Effects of teaching strategies on student achievement in science in the United States. Journal of Research in Science Teaching, 44 (10): 1436 - 60. (doi:10.1002/tea.20212)
34. Schunn, C. D. and Anderson, J. R. 1999. The generality/specificity of expertise in scientific reasoning. Cogitive Science, 23 (3): 337 - 70. (doi:10.1207/s15516709cog2303_3)
35. Sevian, H. and Gonsalves, L. 2008. Analysing how scientists explain their research: A rubric for measuring the effectiveness of scientific explanations. International Journal of Science Education, 30 (11): 1441 - 67. (doi:10.1080/09500690802267579)
36. Seymour, E. 2001. Tracking the processes of change in US undergraduate education in science, mathematics, engineering and technology. Science Education, 86 (1): 79 - 105. (doi:10.1002/sce.1044)
37. Timmerman, B. E., Strickland, D. C., Johnson, R. L. and Payne, J. 2011. Development of a 'universal' rubric for assessing undergraduates' science inquiry and reasoning skills using scientific writing across multiple courses. Assessment & Evaluation in Higher Education, 36 (5): 509 - 47. (doi:10.1080/02602930903540991)
38. Willison, J. and O'Regan, K. 2007. Commonly known, commonly not known, totally unknown: A framework for students becoming researchers. Higher Education Research & Development, 26 (4): 393 - 409. (doi:10.1080/07294360701658609)
39. Zimmerman, C. 2000. The development of scientific reasoning skills. Developmental Review, 20 (1): 99 - 149. (doi:10.1006/drev.1999.0497)