Measuring learning gains in chemical education: A comparison of two methods

Journal of Chemical Education

Volumn 90, Issue 7, 2013, Pages 839-845

  Pentecost, Thomas C ^a   Barbera, Jack ^b  

^a GRAND VALLEY STATE UNIVERSITY   (United States)

^b UNIVERSITY OF NORTHERN COLORADO   (United States)

Author keywords

Chemical Education Research; Graduate Education Research; Testing Assessment

Indexed keywords

EID: 84880000436     PISSN: 00219584     EISSN: 19381328     Source Type: Journal    
DOI: 10.1021/ed400018v     Document Type: Article

References (53)

1. Field, A.; Hole, G. How To Design and Report Experiments; Sage Publications, LTD.: Los Angeles, CA, 2003.
2. Lewis, S. E.; Lewis, J. E. The Same or Not the Same: Equivalence as an Issue in Educational Research J. Chem. Educ. 2005, 82 (9) 1408-1412
3. Bereiter, C. Some Persisting Dilemmas in the Measurement of Change. In Problems in Measuring Change; Harris, C., Ed.; The University of Wisconsin Press: Madison, WI, 1963; pp 3-20.
4. Lord, F. M. The Measurement of Growth Educ. Psychol. Meas. 1956, 16 (4) 421-437
5. Willoughby, S. D.; Metz, A. Exploring Gender Differences with Different Gain Calculations in Astronomy and Biology Am. J. Phys. 2009, 77 (7) 651-657
6. Hake, R. R. Interactive-Engagement versus Traditional Methods: A Six-Thousand-Student Survey of Mechanics Test Data for Introductory Physics Courses Am. J. Phys. 1998, 66 (1) 64-74
7. Lord, F. M. Elementary Models for Measuring Change. In Problems in Measuring Change; Harris, C., Ed.; The University of Wisconsin Press: Madison, WI, 1963; pp 21-38.
8. Linn, R. L.; Slinde, J. A. The Determination of the Significance of Change between Pre- and Posttesting Periods Rev. Educ. Res. 1977, 47 (1) 121-150
9. Cronbach, L. J.; Furby, L. How Should We Measure "Change"-Or Should We? Psychol. Bull. 1970, 74 (1) 68-80
10. Goldstein, H. Measuring Changes in Educational Attainment over Time: Problems and Possibilities J. Educ. Meas. 1983, 20 (4) 369-377
11. Willett, J. B. Questions and Answers in the Measurement of Change J. Educ. Meas. 1988-1989, 15, 345-422
12. Rodosa, D. R.; Willett, J. B. Demonstrating the Reliability of the Difference Score in the Measurement of Change J. Educ. Meas. 1983, 20 (4) 335-343
13. Hake, R. R. Possible Palliatives for the Paralyzing Pre/Post Paranoia That Plagues Some PEP's J. Multidiscip. Eval. 2006, 6 (November) 59-71
14. Tornqvist, L.; Vartia, P.; Vartia, Y. How Should Relative Change Be Measured? Am. Statistician 1985, 39 (1) 43-46
15. University of Colorado Science Education Initiative. http://www.colorado. edu/sei/index.html (accessed May 2013).
16. Marx, J. D.; Cummings, K. Normalized Change Am. J. Phys. 2007, 75 (1) 87-91
17. Hestenes, D.; Wells, M.; Swackhamer, G. Force Concept Inventory Phys. Teach. 1992, 30 (3) 141-158
18. Hake, R. R. Lessons from the Physics Education Reform Effort. Conserv. Ecol. 2002, 5 (2), 28; http://www.consecol.org/vol5/iss2/art28/ (accessed May 2013).
19. Hake, R. R. Six Lessons from the Physics Education Reform Effort. http://www.physics.indiana.edu/∼hake/SixLessonsD.pdf (accessed May 2013).
20. Brogt, E.; Sabers, D.; Prather, E. E.; Deming, G. L.; Hufnagel, B.; Slater, T. F. Analysis of the Astronomy Diagnostic Test Astron. Educ. Rev. 2007, 6 (1) 25-42
21. Bunce, D. M.; Hutchinson, K. D. The Use of the GALT (Group Assessment of Logical Thinking) as a Predictor of Academic Success in College Chemistry J. Chem. Educ. 1993, 70 (3) 183-187
22. Barbera, J.; Adams, W. K.; Weiman, C. E.; Perkins, K. K. Modifying and Validating the Colorado Learning Attitudes about Science Survey for Use in Chemistry J. Chem. Educ. 2008, 85 (10) 1435-1439
23. Wright, B. D.; Linacre, J. M. Observations Are Always Ordinal; Measurements, However, Must Be Interval Arch. Phys. Med. Rehabil. 1989, 70 (12) 857-860
24. Embretson, S. E. The New Rules of Measurement Psychol. Assess. 1996, 8 (4) 341-349
25. Wright, B. Fundamental Measurement for Psychology. In The New Rules of Measurement: What Every Psychologists and Educator Should Know; Embretson, S. E.; Hershberger, S. L., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, 1999; pp 65-104.
26. Bond, T.; Fox, C. M. Applying the Rasch Model: Fundamental Measurement in the Human Sciences; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, 2007.
27. Wright, B. D.; Stone, M. H. Best Test Design; MESA Press: Chicago, IL, 1979.
28. Masters, G. N. Educational Measurement; Australian Council for Educational Research: Melbourne, 1996.
29. Liu, X.; Boone, W. J. Applications of Rasch Measurement in Science Education; JAM Press: Maple Grove, MN, 2006.
30. Claesgens, J.; Scalise, K.; Wilson, M.; Stacy, A. M. Mapping Student Understanding in Chemistry: The Perspectives of Chemists Sci. Educ. 2009, 93 (1) 56-85
31. Barbera, J. A Psychometric Analysis of the Chemical Concepts Inventory J. Chem. Educ. 2013, 90 (5) 546-553
32. Wei, S.; Liu, X.; Wang, Z.; Wang, X. Using Rasch Measurement To Develop a Computer Modeling-Based Instrument To Assess Students' Conceptual Understanding of Matter J. Chem. Educ. 2012, 89 (3) 335-345
33. Liu, X. Elementary to High School Students' Growth over an Academic Year in Understanding Concepts of Matter J. Chem. Educ. 2007, 84 (11) 1853-1856
34. Herrmann-Abell, C. F.; DeBoer, G. E. Using Distractor-Driven Standards-Based Multiple-Choice Assessments and Rasch Modeling To Investigate Hierarchies of Chemistry Misconceptions and Detect Structural Problems with Individual Items Chem. Educ. Res. Pract. 2011, 12 (2) 184-192
35. Scantlebury, K.; Boone, W. J. Designing Tests and Surveys for Chemical Education Research. In Nuts and Bolts of Chemical Education Research; Bunce, D. M.; Cole, R. S., Eds.; American Chemical Society: Washington DC, 2008; pp 149-169.
36. Thorndike, E. L. An Introduction to the Theory of Mental and Social Measurement; Teachers College, Columbia University: New York, 1904.
37. Wallace, C. S.; Bailey, J. Do Concept Inventories Actually Measure Anything? Astron. Educ. Rev. 2010, 9
38. Mulford, D. R.; Robinson, W. R. An Inventory for Alternate Conceptions among First-Semester General Chemistry Students J. Chem. Educ. 2002, 79 (6) 739-744
39. Cacciatore, K. L.; Sevian, H. Incrementally Approaching an Inquiry Lab Curriculum: Can Changing a Single Laboratory Experiment Improve Student Performance in General Chemistry? J. Chem. Educ. 2009, 86 (4) 498-505
40. Mayer, K. Addressing Students' Misconceptions about Gases, Mass, and Composition J. Chem. Educ. 2011, 88 (1) 111-115
41. Regan, A.; Childs, P.; Hayes, S. The Use of an Interventions Programme To Improve Undergraduate Students' Chemical Knowledge and Address Their Misconceptions Chem. Educ. Res. Pract. 2011, 12 (2) 219-227
42. Kruse, R. A.; Roehrig, G. H. A Comparison Study: Assessing Teacher's Conceptions with the Chemical Concepts Inventory J. Chem. Educ. 2005, 82 (8) 1246-1250
43. Chandrasegaran, A. L.; Treagust, D. F.; Mocerino, M. The Development of a Two-Tier Multiple-Choice Diagnostic Instrument for Evaluating Secondary School Students' Ability To Describe and Explain Chemical Reactions Using Multiple Levels of Representation Chem. Educ. Res. Pract. 2007, 8 (3) 293-307
44. Linacre, J. M. Winsteps: Rasch Measurement Computer Program; Winsteps.com: Beaverton, OR, 2012.
45. Bao, L. Theoretical Comparisons of Average Normalized Gain Calculations Am. J. Phys. 2006, 74 (10) 917-922
46. Wright, B. D. Time 1 to Time 2 (Pre-Test to Post-Test) Comparison and Equating: Racking and Stacking Rasch Meas. Trans. 1996, 10 (1) 478
47. Wilson, M. Cognitive Diagnosis Using Item Response Models Z. Psychol. 2008, 216 (2) 74-88
48. Wilson, M.; Scalise, K. Assessment To Improve Learning in Higher Education: The BEAR Assessment System Higher Educ. 2006, 52, 635-663
49. Wilson, M.; Sloane, K. From Principles to Practice: An Embedded Assessment System Appl. Meas. Educ. 2000, 12 (2) 181-208
50. Mislevy, R. On Approaches to Assessing Change; Clarification. http://www.education.umd.edu/EDMS/mislevy/papers/Gain/ (accessed May 2013).
51. Embretson, S. E. A Multidimensional Latent Trait Model for Measuring Learning and Change Psychometrika 1991, 56 (3) 495-515
52. Adams, R. J.; Wilson, M.; Wang, W.-C. The Multidimensional Random Coefficients Multinomial Logit Model Appl. Psychol. Meas. 1997, 21 (1) 1-23
53. Briggs, D. C.; Wilson, M. An Introduction to Multidimensional Measurement Using Rasch Models J. Appl. Meas. 2003, 4 (1) 87-100