The testing effect: An intervention on behalf of low-skilled comprehenders in general chemistry

Journal of Chemical Education

Volumn 91, Issue 12, 2014, Pages 2045-2057

  Pyburn, Daniel T ^a   Pazicni, Samuel ^a   Benassi, Victor A ^b   Tappin, Elizabeth M ^b  

^a Univ of New Hampshire   (United States)

^b UNIVERSITY OF NEW HAMPSHIRE   (United States)

Author keywords

Chemical education research; First Year undergraduate general; Testing Assessment

Indexed keywords

EID: 84916632058     PISSN: 00219584     EISSN: 19381328     Source Type: Journal    
DOI: 10.1021/ed4009045     Document Type: Article

References (102)

1. Gosser, D. K.; Roth, V. The workshop chemistry project: Peer-led team-learning. J. Chem. Educ. 1998, 75 (2), 185-187.
2. Gosser, D. K.; Strozak, V. S.; Cracolice, M. S. Peer-Led Team Learning: Workshops for General Chemistry; Prentice Hall: Upper Saddle River, NJ, 2001.
3. Tien, L. T.; Roth, V.; Kampmeier, J. A. Implementation of a peer-led team learning instructional approach in an undergraduate organic chemistry course. J. Res. Sci. Teach. 2002, 39 (7), 606-632.
4. Báez-Galib, R.; Colón-Cruz, H.; Resto, W.; Rubi, M. R. Chem-2-Chem: A one-to-one supportive learning environment for chemistry. J. Chem. Educ. 2005, 82 (12), 1859-1863.
5. Lewis, S. E.; Lewis, J. E. Departing from lectures: An evaluation of a peer-led guided inquiry alternative. J. Chem. Educ. 2005, 82 (1), 135-139.
6. Wamser, C. C. Peer-led team learning in organic chemistry: Effects on student performance, success, and persistence in college. J. Chem. Educ. 2006, 83 (10), 1562-1566.
7. Hockings, S. C.; DeAngeles, K. J.; Frey, R. F. Peer-led team learning in general chemistry: Implementation and evaluation. J. Chem. Educ. 2008, 85 (7), 990-996.
8. Lewis, S. E.; Lewis, J. E. Seeking effectiveness and equity in a large college chemistry course: An HLM investigation of peer-led guided inquiry. J. Res. Sci. Teach. 2008, 45 (7), 794-811.
9. Lewis, S. E. Retention and reform: An evaluation of peer-led team learning. J. Chem. Educ. 2011, 88 (6), 703-707.
10. Mitchell, Y. D.; Ippolito, J.; Lewis, S. E. Evaluating peer-led team learning across the two semester general chemistry sequence. Chem. Educ. Res. Pract. 2012, 13 (3), 378-383.
11. Farrell, J. J.; Moog, R. S.; Spencer, J. N. A guided-inquiry general chemistry course. J. Chem. Educ. 1999, 76 (4), 570-574.
12. Minderhout, V.; Loertscher, J. Lecture-free biochemistry. Biochem. Mol. Biol. Educ. 2007, 35 (3), 172-180.
13. Daubenmire, P. L.; Bunce, D. M. In Process Oriented Guided Inquiry Learning (POGIL); Moog, R. S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2008; pp 87-99.
14. Ruder, S. M.; Hunnicutt, S. S. In Process Oriented Guided Inquiry Learning (POGIL); Moog, R. S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2008; pp 133-147.
15. Straumanis, A.; Simons, E. A. In Process Oriented Guided Inquiry Learning (POGIL); Moog, R. S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2008; pp 226-239.
16. Rajan, N.; Marcus, L. Student attitudes and learning outcomes from process oriented guided-inquiry learning (POGIL) strategy in an introductory chemistry course for non-science majors: An action research study. Chem. Educ. 2009, 14 (2), 85-93.
17. Brown, P. J. P. Process-oriented guided-inquiry learning in an introductory anatomy and physiology course with a diverse student population. Adv. Physiol. Educ. 2010, 34 (3), 150-155.
18. Brown, S. D. A process-oriented guided inquiry approach to teaching medicinal chemistry. Am. J. Pharm. Educ. 2010, 74 (7), 1-6.
19. Chase, A.; Pakhira, D.; Stains, M. Implementing process-oriented, guided-inquiry learning for the first time: Adaptations and short-term impacts on students' attitude and performance. J. Chem. Educ. 2013, 90 (4), 409-416.
20. Lynch, S. Equity and Science Education Reform; Lawrence Erlbaum Associates: Mahwah, NJ, 2000.
21. Graesser, A. C.; Millis, K. K.; Zwaan, R. A. Discourse comprehension. Annu. Rev. Psychol. 1997, 48 (1), 163-189.
22. Graesser, A. C.; McNamara, D. S. Computational analyses of multilevel discourse comprehension. Top. Cognit. Sci. 2011, 3 (2), 371-398.
23. McNamara, D. S.; Graesser, A. C.; McCarthy, P. A.; Cai, Z. Automated Evaluation of Text and Discourse with Coh-Metrix; Cambridge University Press: New York, NY, 2014; pp 14-17.
24. McNamara, D. S.; Magliano, J. In The Psychology of Learning and Motivation; Ross, B., Ed.; Academic Press: New York, NY, 2009; pp 298-384.
25. Gernsbacher, M. A. Language Comprehension As Structure Building; Lawrence Erlbaum Associates: Hillsdale, NJ, 1990.
26. Gernsbacher, M. A.; Varner, K. R.; Faust, M. E. Investigating differences in general comprehension skill. J. Exp. Psychol. Learn. Mem. Cogn. 1990, 16 (3), 430-445.
27. Graesser, A. C.; McNamara, D. S.; Van Lehn, K. Scaffolding deep comprehension strategies through Point&Query, AutoTutor, and iSTART. Educ. Psychol. 2005, 40 (4), 225-234.
28. McNamara, D. S. The importance of teaching reading strategies. Perspect. Language Literacy 2009, 35 (2), 34-40.
29. McNamara, D. S. Strategies to read and learn: Overcoming learning by consumption. Med. Educ. 2010, 44 (44), 340-346.
30. Azevedo, R. In Recent Innovations in Educational Technology That Facilitate Student Learning ; Robinson, D., Schraw, G., Eds.; Information Age Publishing: Charlotte, NC, 2008; pp 127-156.
31. Pyburn, D. T.; Pazicni, S. Applying the multilevel framework of discourse comprehension to evaluate the text characteristics of general chemistry texts. J. Chem. Educ. 2014, 91 (6), 778-783.
32. Pyburn, D. T.; Pazicni, S.; Benassi, V. A.; Tappin, E. E. Assessing the relation between language comprehension and performance in general chemistry. Chem. Educ. Res. Pract. 2013, 14 (4), 524-541.
33. Mayer, R. E. Multimedia Learning; Cambridge University Press: New York, NY, 2001; pp 41-62.
34. Mayer, R. E.; Wittrock, M. C. Problem Solving. In Handbook of Educational Psychology, 2nd ed.; Alexander, P. A., Winne, P. H., Eds.; Routledge: New York, NY, 2009; pp 287-303.
35. Johnstone, A. H. Chemistry teaching - Science or alchemy? J. Chem. Educ. 1997, 74 (3), 262-268.
36. Tsaparlis, G. Atomic and molecular structure in chemical education: A critical analysis from various perspectives of science education. J. Chem. Educ. 1997, 74 (8), 922-925.
37. Gabel, D. Improving teaching and learning through chemistry education research: A look to the future. J. Chem. Educ. 1999, 76 (4), 548-554.
38. Johnstone, A. H. You can't get there from here. J. Chem. Educ. 2010, 87 (1), 22-29.
39. Callender, A. A.; McDaniel, M. A. The benefits of embedded questions for low and high structure builders. J. Educ. Psychol. 2007, 99 (2), 339-348.
40. McDaniel, M. A.; Anderson, J. L.; Derbish, M. H.; Morrisette, N. Testing the testing effect in the classroom. Eur. J. Cogn. Psychol. 2007, 19 (4/5), 494-513.
41. Callender, A. A.; McDaniel, M. A. The limited benefits of rereading educational texts. Contemp. Educ. Psychol. 2009, 34 (1), 30-41.
42. Roediger, H. L.; Karpicke, J. D. The power of testing memory: Basic research and implications for educational practice. Perspect. Psychol. Sci. 2006, 1 (3), 181-210.
43. Hanawalt, N. G.; Tarr, A. G. The effect of recall on recognition. J. Exp. Psychol. 1961, 62 (4), 361-367.
44. Darley, C. F.; Murdock, B. B. Effects of prior free recall testing on final recall and recognition. J. Exp. Psychol. 1971, 91 (1), 66-73.
45. Hogan, R. M.; Kintsch, W. Differential effects of study and test trials on long-term recognition and recall. J. Verbal Learn. Verbal Behav. 1971, 10 (5), 562-567.
46. Bartlett, J. C. Effects of immediate testing on delayed retrieval: Search and recovery operations with four types of cue. J. Exp. Psychol.: Hum. Learn. Mem. 1977, 3 (6), 719-732.
47. Whitten, W. B.; Bjork, R. A. Learning from tests: Effects of spacing. J. Verbal Learn. Verb. Behav. 1977, 16 (4), 465-478.
48. Masson, M. E. J.; McDaniel, M. A. The role of organization processes in long-term retention. J. Exp. Psychol.: Hum. Learn. Mem. 1981, 7 (2), 100-110.
49. McDaniel, M. A.; Masson, M. E. J. Altering memory representations through retrieval. J. Exp. Psychol. Learn. Mem. Cogn. 1985, 11 (2), 371-385.
50. McDaniel, M. A.; Kowitz, M. D.; Dunay, P. K. Altering memory through recall: The effects of cue-guided retrieval processing. Mem. Cognit. 1989, 17 (4), 423-434.
51. Rohrer, D.; Taylor, K.; Sholar, B. Tests enhance the transfer of learning. J. Exp. Psychol.: Learn. Mem. Cogn. 2010, 36 (1), 233-239.
52. Roediger, H. L.; Agarwal, P. K.; McDaniel, M. A.; McDermott, K. B. Test-enhanced learning in the classroom: Long-term improvements from quizzing. J. Exp. Psychol.: Appl. 2011, 17 (4), 382-395.
53. Pressley, M.; McDaniel, M. A.; Turnure, J. E.; Wood, E.; Ahmed, M. Generation and precision of elaboration: Effects on intentional and incidental learning. J. Exp. Psychol.: Learn. Mem. Cogn. 1987, 13 (2), 291-300.
54. McDaniel, M. A.; Donnelly, C. M. Learning with analogy and elaborative interrogation. J. Educ. Psychol. 1996, 88 (3), 508-519.
55. Boudreau, R. L.; Wood, E.; Willoughby, T.; Specht, J. Evaluating the efficacy of elaborative strategies for remembering expository text. Alberta J. Educ. Res. 1999, 45 (2), 170-183.
56. Ozgungor, S.; Guthrie, J. T. Interactions among elaborative interrogation, knowledge, and interest in the process of constructing knowledge from text. J. Educ. Psychol. 2004, 96 (3), 437-443.
57. Ramsay, C. M.; Sperling, R. A.; Dornisch, M. M. A comparison of the effects of students' expository text comprehension strategies. Instr. Sci. 2010, 38 (6), 551-570.
58. Smith, B. L.; Holliday, W. G.; Austin, H. W. Students' comprehension of science textbooks using a question-based reading strategy. J. Res. Sci. Teach. 2010, 47 (4), 363-379.
59. Wood, E.; Willoughby, T.; McDermott, C.; Motz, M.; Kaspar, V.; Ducharme, M. Developmental differences in study behavior. J. Educ. Psychol. 1999, 91 (3), 527-536.
60. McDaniel, M. A.; Agarwal, P. K.; Huelser, B. J.; McDermott, K. B.; Roediger, H. L. Test-enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. J. Educ. Psychol. 2011, 103 (2), 399-414.
61. Little, J. L.; Bjork, E. L.; Bjork, R. A.; Angello, G. Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychol. Sci. 2012, 23 (11), 1337-1344.
62. Bereiter, C.; Bird, M. Use of thinking aloud in identification and teaching of reading comprehension strategies. Cogn. Instr. 1985, 2 (2), 131-156.
63. King, A.; Rosenshine, B. Effects of guided cooperative-questioning on children's knowledge construction. J. Exp. Educ. 1993, 61 (2), 127-148.
64. Reading Comprehension Strategies: Theory, Interventions, And Technologies; McNamara, D. S., Ed.; Lawrence Erlbaum Associates: Mahwah, NJ, 2007.
65. Palinscar, A. S.; Brown, A. L. Reciprocal teaching of comprehension-fostering and comprehension monitoring activities. Cogn. Inst. 1984, 1 (2), 117-175.
66. Krajcik, J.; McNeill, K. L.; Reiser, B. J. Learning-goals-driven design model: Developing curriculum materials that align with national standards and incorporate project-based pedagogy. Sci. Educ. 2008, 92 (1), 1-32.
67. Pyburn, D. T. Investigations of language comprehension and chemistry education. Ph.D. Thesis, University of New Hampshire, Durham, NH, 2014.
68. Fleiss, J. L. Measuring nominal scale agreement among many raters. Psychol. Bull. 1971, 76 (5), 378-382.
69. Landis, J. R.; Koch, G. G. The measurement of observer agreement for categorical data. Biometrics 1977, 33 (1), 159-174.
70. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook 1: Cognitive Domain; Bloom, B. S., Ed.; David McKay Company, Inc.: New York, 1956.
71. Educational Testing Service. Critical Reading Section. http://professionals.collegeboard.com/testing/sat-reasoning/about/sections/critical-reading (accessed Aug 2014).
72. Ewing, M.; Huff, K.; Andrews, M.; King, K. Assessing the Reliability of Skills Measured by the SAT; The College Board Office of Research and Analysis: New York, NY, 2005.
73. Hovey, N. W.; Krohn, A. An evaluation of the Toledo chemistry placement examination. J. Chem. Educ. 1963, 40 (7), 370.
74. American Chemical Society Division of Chemical Education Exams Institute. Composite Norms for the Toledo Examination 2009. http://chemexams.chem.iastate.edu/national-norms/tp09.html (accessed Aug 2014).
75. Cohen, J.; Cohen, P.; West, S. G.; Aiken, L. S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, 3rd ed: Lawrence Erlbaum Associates: Mahwah, NJ, 2003; pp 41-42.
76. Raudenbush, S. W.; Bryk, A. S. Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed.; Sage Publications: Thousand Oaks, CA, 2002.
77. Pazicni, S.; Bauer, C. F. Characterizing illusions of competence in introductory general chemistry students. Chem. Educ. Res. Pract. 2014, 15 (1), 24-34.
78. Villafañe, S. M.; Garcia, C. A.; Lewis, J. E. Exploring diverse students' trends in chemistry self-efficacy throughout a semester of college-level preparatory chemistry. Chem. Educ. Res. Pract. 2014, 15 (2), 114-127.
79. We note here one important difference in the results obtained using ML vs REML. Using REML to calculate intraclass correlations resulted in a substantially larger intraclass correlation (ρ = 0.18) for the third (between-classrooms) level of our nested data set. Unlike the results obtained using ML presented above, this intraclass correlation suggested that a three-level HLM was indeed appropriate. However, when the data were analyzed with a three-level HLM using the REML algorithm (Supporting Information, Section S7), results were similar to those obtained with a two-level HLM using the ML algorithm. Thus, for the sake of parsimony, we present here the results obtained with a two-level HLM using the ML algorithm.
80. Tabachnick, B. G.; Fidell, L. S. Using Multivariate Statistics, 6th ed.; Pearson: Boston, 2013.
81. Pazicni, S.; Pyburn, D. T. Intervening on behalf of low-skilled comprehenders in a university general chemistry course. In Applying Science of Learning in Education: Infusing Psychological Science into the Curriculum [Online];
82. Benassi, V. A.; Overson, C. E.; Hakala, C. M., Eds.; Society for the Teaching of Psychology, 2014; pp 279-292. http://teachpsych.org/ebooks/asle2014/index.php (accessed Aug 2014).
83. Kang, S. H. K.; McDermott, K. B.; Roediger, H. L. Test format and corrective feedback modify the effect of testing on long-term retention. Eur. J. Cogn. Psychol. 2007, 19 (4/5), 528-558.
84. Anderson, R. C.; Biddle, W. B. In The Psychology of Learning and Motivation; Bower, G. H., Ed.; Academic Press: New York, NY, 1975; pp 89-132.
85. Duchastel, P. C. Retention of prose following testing with different types of test. Contemp. Educ. Psychol. 1981, 6 (3), 217-226.
86. Hamaker, C. The effects of adjunct question on prose learning. Rev. Educ. Res. 1986, 56 (2), 212-242.
87. Foos, P. W.; Fisher, R. P. Using tests as learning opportunities. J. Educ. Psychol. 1988, 80 (2), 179-183.
88. McDaniel, M. A.; Wildman, K. M.; Anderson, J. L. Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. J. Appl. Res. Mem. Cogn. 2012, 1 (1), 18-26.
89. Agarwal, P. K.; Karpicke, J. D.; Kang, S. H. K.; Roediger, H. L.; McDermott, K. B. Examining the testing effect with open- and closed-book tests. Appl. Cogn. Psychol. 2008, 22 (7), 861-876.
90. Pintrich, P. R.; DeGroot, E. V. Motivational and self-regulated learning components of classroom academic performance. J. Educ. Psychol. 1990, 82 (1), 33-40.
91. Bauer, C. F. Attitude towards chemistry: A semantic differential instrument for assessing curriculum impacts. J. Chem. Educ. 2008, 85 (10), 1440-1445.
92. Treagust, D. F. Development and use of diagnostic tests to evaluate students' misconceptions in science. Int. J. Sci. Educ 1988, 10 (2), 159-169.
93. Peterson, R. F.; Treagust, D. F. Grade-12 students' misconceptions of covalent bonding and structure. J. Chem. Educ. 1989, 66 (6), 459-460.
94. National Research Council. Knowing What Students Know: The Science and Design of Educational Assessment; Pellegrino, J. W., Chudowsky, N., Glasser, R., Eds.; The National Academies Press: Washington, DC, 2001; pp 1-14.
95. Bretz, S. L.; Linenberger, K. J. Development of the enzyme-substrate interactions concept inventory. Biochem. Mol. Biol. Educ. 2012, 40 (4), 229-233.
96. Luxford, C. J.; Bretz, S. L. Development of bonding representations inventory to identify student misconceptions about covalent and ionic bonding representations. J. Chem. Educ. 2014, 91 (3), 312-320.
97. Glass, A. L.; Brill, G.; Ingate, M. Combined online and in-class pretesting improves exam performance in general psychology. Educ. Psychol. 2008, 28 (5), 483-503.
98. Bunce, D. M.; VandenPlas, J. R.; Havanki, K. L. Comparing the effectiveness on student achievement of a student response system versus online WebCT quizzes. J. Chem. Educ. 2006, 83 (3), 488-493.
99. Mayer, R. E.; Stull, A.; DeLeeuw, K.; Almeroth, K.; Bimber, B.; Chun, D.; Bulger, M.; Campbell, J.; Knight, A.; Zhang, H. Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemp. Educ. Psychol. 2009, 34 (1), 51-57.
100. MacArthur, J. R.; Jones, L. L. A review of literature reports of clickers applicable to college chemistry classrooms. Chem. Educ. Res. Pract. 2008, 9 (3), 187-195.
101. Smith, M. K.; Wood, W. B.; Adams, W. K.; Wieman, C.; Knight, J. K.; Guild, N.; Su, T. T. Why peer discussion improves student performance on in-class concept questions. Science 2009, 323 (5910), 122-124.
102. Lewis, S. E.; Lewis, J. E. Predicting at-risk students in general chemistry: Comparing formal thought to a general achievement measure. Chem. Educ. Res. Pract. 2007, 8 (1), 32-51.