Development and validation of a rubric for diagnosing students' experimental design knowledge and difficulties

CBE Life Sciences Education

Volumn 13, Issue 2, 2014, Pages 265-284

  Dasgupta, Annwesa P ^a   Anderson, Trevor R ^b   Pelaez, Nancy ^a  

^a PURDUE UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EDUCATION; HUMAN; KNOWLEDGE; LEARNING; PROCEDURES; PSYCHOLOGY; REPRODUCIBILITY; STANDARDS; STUDENT; VALIDATION STUDY;

EDUCATIONAL MEASUREMENT; HUMANS; KNOWLEDGE; LEARNING; REPRODUCIBILITY OF RESULTS; STUDENTS;

EID: 84901820567     PISSN: None     EISSN: 19317913     Source Type: Journal    
DOI: 10.1187/cbe.13-09-0192     Document Type: Article

References (74)

1. American Association for the Advancement of Science (2010). Vision and Change: A Call to Action, Washington, DC.
2. Association of American Colleges and Universities (2013). Scientific Thinking and Integrative Reasoning Skills, Washington, DC. Association of American Medical Colleges and the Howard Hughes
3. Medical Institute (2009). Report of Scientific Foundations for Future Physicians Committee, Washington, DC.
4. Barnett SM, Ceci SJ (2002). When and where do we apply what we learn? A taxonomy for far transfer. Psychol Bull 128, 612.
5. Beck CW, Blumer LW (2012). Inquiry-based ecology laboratory courses improve student confidence and scientific reasoning skills. Ecosphere 3, 112.
6. Box GE, Hunter JS, Hunter WG (2005). Statistics for Experimenters: Design, Innovation, and Discovery, 2nd ed., Hoboken, NJ: Wiley.
7. Bullock M, ZieglerA (1999). Scientific reasoning: developmental and individual differences. In: Individual Development from 3 to 12, ed. FEWeinert andWSchneider, Cambridge, UK: CambridgeUniversity Press, 38-54.
8. Burns JC, Okey JR, Wise KC (1985). Development of an integrated process skill test: TIPS II. J Res Sci Teach 22, 169-177.
9. Chen Z, Klahr D (1999). All other things being equal: acquisition and transfer of the control of variables strategy. Child Dev 70, 1098-1120.
10. ChiMT, GlaserRE, FarrMJ (1988). The Nature of Expertise, Hillsdale, NJ: Erlbaum.
11. Clase KL, Gundlach E, Pelaez NJ (2010). Calibrated peer review for computer-assisted learning of biological research competencies. Biochem Mol Biol Educ 38, 290-295.
12. Cohen JA (1960). Coefficient of agreement for nominal scales. Educ Psychol Meas 20, 37-46.
13. College Board (2006). AP Statistics Free-Response Questions. http://apcentral.collegeboard.com/apc/public/repository/_ap06 _frq_statistics_51653.pdf (accessed 20 December 2013).
14. College Board (2009). AP Statistics Free-Response Question Form B. http://apcentral.collegeboard.com/apc/public/repository/ap09 _frq_statistics_formb.pdf (accessed 20 December 2013).
15. Colon-Berlingeri M, Burrowes PA (2011). Teaching biology through statistics: application of statistical methods in genetics and zoology courses. CBE Life Sci Educ 10, 259-267.
16. Cox DR, Reid R (2000). The Theory of the Design of Experiments, Boca Raton, FL: Chapman and Hall/CRC.
17. D'Costa AR, Schlueter MA (2013). Scaffolded instruction improves student understanding of the scientific method and experimental design. Am Biol Teach 75, 18-28.
18. Dolan E, Grady J (2010). Recognizing students' scientific reasoning: a tool for categorizing complexity of reasoning during teaching by inquiry. J Sci Teach Educ 21, 31-55.
19. Duschl RA, Schweingruber HA, Shouse AW (2007). Taking Science to School: Learning and Teaching Science inGrades K-8, Washington, DC: National Academies Press.
20. Feldon DF, Timmerman BC, Stowe KA, Showman R (2010). Translating expertise into effective instruction: the impacts of cognitive task analysis (CTA) on lab report quality and student retention in the biological sciences. J Res Sci Teach 47, 1165-1185.
21. Fuller RG (ed.) (2002). A Love of Discovery: Science Education-The Second Career of Robert Karplus, New York: Kluwer.
22. Gill J, Walsh J (2010). Control group. In: Encyclopedia of Research Design, ed. N Salkind, Thousand Oaks, CA: Sage.
23. Gormally C, Brickman P, Lutz M (2012). Developing a Test of Scientific Literacy Skills (TOSLS): measuring undergraduates' evaluation of scientific information and arguments. CBE Life Sci Educ 11, 364-377.
24. Grayson DJ, Anderson TR, Crossley LG (2001). A four-level framework for identifying and classifying student conceptual and reasoning difficulties. Int J Sci Educ 23, 611-622.
25. Griffith AB (2007). Semester-long engagement in science inquiry improves students' understanding of experimental design. Teach Issues Exp Ecol 5, Research #2. www.ecoed.net/tiee/vol/v5/ research/griffith/pdf/Griffith_2007.pdf (assessed 3 May 2014).
26. Grunwald S, Hartman A (2010). A case-based approach improves science students' experimental variable identification skills. J Coll Sci Teach 39, 28-33.
27. Gutwill-Wise J (2001). The impact of active and context-based learning in introductory chemistry courses: an early evaluation of the modular approach. J Chem Ed 78, 684-690.
28. Harker AR (2009). Full application of the scientific method in an undergraduate teaching laboratory: a reality-based approach to experiential student-directed instruction. J Coll Sci Teach 29, 97-100.
29. Hiebert SM (2007). Teaching simple experimental design to undergraduates: do your students understand the basics. AdvPhysiol Educ 31, 82-92.
30. Holmes D, Moody P, Dine D (2011). Research Methods for the Biosciences, Oxford, UK: Oxford University Press.
31. Kanari Z, Millar R (2004). Reasoning from data: how students collect and interpret data in science investigations. J Res Sci Teach 41, 748-769.
32. Kardash CM (2000). Evaluation of an undergraduate research experience: perceptions of undergraduate interns and their faculty mentors. J Educ Psychol 92, 191-201.
33. Kish L (1965). Survey Sampling, New York:Wiley.
34. Klahr D, Fay AL, Dunbar K (1993). Heuristics for scientific experimentation: a developmental study. Cogn Psychol 25, 111-146.
35. Kloser M, Brownell S, Shavelson R, Fukami T (2013). Effects of a research-based ecology lab course: a study of nonvolunteer achievement, self-confidence, and perception of lab course purpose. J Coll Sci Teach 42, 72-81.
36. Koehler DJ (1994). Hypothesis generation and confidence in judgment. J Exp Psychol Learn 20, 461-469.
37. Kuhn D, Dean D (2005). Is developing scientific thinking all about learning to control variables. Psychol Sci 16, 866-870.
38. Kuhn D, Pearsall S (2000). Developmental origins of scientific thinking. J Cogn Dev 1, 113-129.
39. Kuhn D, Schauble L, Garcia-Mila M (1992). Cross-domain development of scientific reasoning. Cogn Instr 9, 285-327.
40. Landis JR, Koch GG (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159-174.
41. Laursen S, Hunter AB, Seymour E, Thiry H, Melton G (2010). Undergraduate Research in the Sciences: Engaging Students in Real Science, San Francisco: Jossey-Bass.
42. Lawson AE, Drake N, Johnson J, Kwon YJ, Scarpone C (2000). How good are students at testing alternative explanations of unseen entities? Am Biol Teach 62, 249-255.
43. Lawson AE, Snitgen DA (1982). Teaching formal reasoning in a college biology course for preservice teachers. J Res Sci Teach 19, 233-248.
44. Libarkin J, Ording G (2012). The utility of writing assignments in undergraduate bioscience. CBE Life Sci Educ 11, 39-46.
45. Lopatto D (2003). The essential features of undergraduate research. Counc Undergrad Res Q 24, 139-142.
46. Lopatto D (2004). Survey of Undergraduate Research Experiences (SURE): first findings. Cell Biol Educ 3, 270-277.
47. Lopatto D (2007). Undergraduate research experiences support science career decisions and active learning. CBE Life Sci Educ 6, 297-306.
48. Lopatto D (2008). Exploring the benefits of undergraduate research: the SURE survey. In: Creating EffectiveUndergraduate Research Programs in Science, ed. R Taraban and RL Blanton, New York: Teacher's College Press, 112-132.
49. Metz AM (2008). Teaching statistics in biology: using inquiry-based learning to strengthen understanding of statistical analysis in biology laboratory courses. CBE Life Sci Educ 7, 317-326.
50. National Institute of Standards and Technology/SEMATECH (2003). Engineering Statistics Handbook, Section 3.1.3.6. www.itl.nist.gov/div898/handbook/ppc/section1/ppc136.htm (accessed 20 December 2013).
51. National Research Council (2007). Rising above the gathering storm: energizing and employing America for a brighter economic future, Washington, DC: National Academies Press.
52. Park J, Pak S (1997). Students' responses to experimental evidence based on perceptions of causality and availability of evidence. J Res Sci Teach, 34, pp. 57-67.
53. Picone C, Rhode J, Hyatt L, Parshall T (2007). Assessing gains in undergraduate students' abilities to analyze graphical data. Teach Issues Exp Ecol 5, Research #1. www.esa.org/tiee/vol/v5/research/ picone/article.html (accessed 20 December 2013).
54. Purdue University (2010). Second Purdue Symposium on Psychological Sciences-Psychology of Science: Implicit and Explicit Processes. http://www.purdue.edu/hhs/psy/research/symposium/ 2ndPSPS2010.php (accessed 20 December 2013).
55. Quinn GGP, KeoughMJ (2002). ExperimentalDesign and Data Analysis for Biologists, Cambridge, UK: Cambridge University Press.
56. Ramsey FL, Schafer DW (2012). The Statistical Sleuth: A Course in Methods of Data Analysis, Boston: Brooks/Cole.
57. Roth WM, McGinn MK, Bowen GM (1998). How prepared are preservice teachers to teach scientific inquiry? Levels of performance in scientific representation practices. J Sci Teach Educ 9, 25-48.
58. Rubin DB (1973). The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics 29, 185-203.
59. Ruxton GD, Colegrave N (2006). Experimental Design for the Life Sciences, Oxford, UK: Oxford University Press.
60. Salangam J (2007). The impact of a prelaboratory discussion on nonbiology majors abilities to plan scientific inquiry. Master's Thesis, Fullerton: California State University.
61. Schauble L (1990). Belief revision in children: the role of prior knowledge and strategies for generating evidence. J Exp Child Psychol 49, 31-57.
62. Schauble L (1996). The development of scientific reasoning in knowledge-rich contexts. Dev Psychol 32, 102-119.
63. Schauble L, Glaser R (1990). Scientific thinking in children and adults. Contrib Hum Dev 21, 9-27.
64. Shi J, Power JM, Klymkowsky MW (2011). Revealing student thinking about experimental design and the roles of control experiments. Int J Sch Teach Learn 5, 1-16.
65. Singer S, Hilton M, Schweingruber H (2006). America's Lab Report: Investigations in High School Science, Washington, DC: National Academies Press.
66. Sirum K, Humburg J (2011). The Experimental Design Ability Test (EDAT). Bioscene 37, 8-16.
67. SRI International (2003). Diffusion and Testing a New Drug 1. http://pals.sri.com/tasks/9-12/Testdrug/ (accessed 20 December 2013).
68. Sternberg RJ (2003). What is an "expert student?" Educ Res 32, 5-9.
69. Thiry H, Weston TJ, Laursen SL, Hunter AB (2012). The benefits of multi-year research experiences: differences in novice and experienced students' reported gains from undergraduate research. CBE Life Sci Educ 11, 260-272.
70. Timmerman B, Strickland DC, Johnson RL, Payne JR (2011). Development of a "universal" rubric for assessing undergraduates' scientific reasoning skills using scientific writing. Assess Eval High Educ 36, 509-547.
71. Tobin KG, Capie W (1981). The development and validation of a pencil and paper test of logical thinking. Educ Psychol Measurement 41, 413-424.
72. Tobin KG, CapieW (1982). Relationships between classroom process variables andmiddle-school science achievement. J Educ Psychol 74, 441.
73. Wei CA, Woodin T (2011). Undergraduate research experiences in biology: alternatives to the apprenticeship model. CBE Life Sci Educ 10, 123-131.
74. Wuensch KL (2001). When Does Correlation Imply Causation? http://core.ecu.edu/psyc/wuenschk/stathelp/Correlation-Causation.htm (accessed 20 December 2013).