Gathering evidence for validity during the design, development, and qualitative evaluation of Thermochemistry Concept Inventory items

Journal of Chemical Education

Volumn 90, Issue 12, 2013, Pages 1590-1601

  Wren, David ^a   Barbera, Jack ^b  

^a WAKE FOREST UNIVERSITY   (United States)

^b UNIVERSITY OF NORTHERN COLORADO   (United States)

Author keywords

Chemical Education Research; First Year Undergraduate General; Misconceptions Discrepant Events; Testing Assessment

Indexed keywords

EID: 84890499724     PISSN: 00219584     EISSN: 19381328     Source Type: Journal    
DOI: 10.1021/ed400384g     Document Type: Article

References (58)

1. National Research Council. Knowing What Students Know: The Science and Design of Educational Assessment; Committee on the Foundations of Assessment, James, P., Chudowsky, N., and Glaser, R., Eds.; National Academy Press: Washington, DC, 2001.
2. Barbera, J. and VandenPlas, J., All Assessment Material Are Not Created Equal: The Myths about Instrument Development, Validity, and Reliability. In Investigating Classroom Myths through Research on Teaching and Learning; Bunce, D., Ed.; American Chemical Society: Washington, DC, 2011.
3. Villafañe, S.; Loertscher, J.; Minderhout, V.; Lewis, J. Uncovering Students' Incorrect Ideas about Foundational Concepts for Biochemistry Chem. Educ. Res. Pract. 2011, 12, 210-218
4. Bunce, D. Survey Development J. Chem. Educ. 2008, 85 (10) 1439
5. Bretz, S. Qualitative Research Designs in Chemistry Education Research. In Nuts and Bolts of Chemical Education Research; Bunce, D.; Cole, R., Eds.; American Chemical Society: Washington, DC, 2008.
6. National Research Council. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering; Committee on the Status and Future Directions of Discipline-Based Education Research, Singer, S.; Nielsen, N.; Schweingruber, H., Eds.; National Academy Press: Washington, DC, 2012.
7. Arjoon, J.; Xu, X.; Lewis, J. Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence J. Chem. Educ. 2013, 90 (5) 536-545
8. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for Educational and Psychological Testing; American Educational Research Association: Washington, DC, 1999.
9. Towns, M.; Kraft, A. Review and Synthesis of Research in Chemical Education from 2000-2010; The National Academies National Research Council Board of Science Education: West Lafayette, IN, 2011.
10. Treagust, D. Development and Use of Diagnostic Tests To Evaluate Students' Misconceptions in Science. Int. J. Sci. Educ. 1988, 10 (2), 159-169.
11. Ausubel, D. Educational Psychology: A Cognitive View; Holt, Rinehart and Winston, Inc.: New York, 1968.
12. National Research Council. How People Learn: Brain, Mind, Experience, and School (expanded ed.); Committee on Developments in the Science of Learning, John, B.; Brown, A.; Cocking, R., Eds.; National Academy Press: Washington, DC, 2000.
13. Posner, G.; Strike, K.; Hewson, P.; Gertzog, W. A. Accommodation of a Scientific Conception: Toward a Theory of Conceptual Change Sci. Educ. 1982, 66 (2) 211-227
14. Chi, M. Three Types of Conceptual Change: Belief Revision, Mental Model Transformation, and Categorical Shift. In International Handbook of Research on Conceptual Change; Vonsiadou, S., Ed.; Routledge: New York, 2008; pp 61-82.
15. diSessa, A. A History of Conceptual Change Research: Threads and Fault Lines. In The Cambridge Handbook of the Learning Sciences; Sawyer, K., Ed.; Cambridge University Press: New York, 2006; pp 265-281.
16. Vonsiadou, S. Conceptual Change Research: An Introduction. In International Handbook of Research on Conceptual Change; Routledge: New York, 2008.
17. Chinn, C.; Brewer, W. An Empirical Test of a Taxonomy of Responses to Anomalous Data in Science J. Res. Sci. Teach. 1998, 35 (6) 623-654
18. Beall, H. Probing Student Misconceptions in Thermodynamics with In-Class Writing J. Chem. Educ. 1994, 71 (12) 1056-1057
19. Boo, H. K. Students' Understanding of Chemical Bonds and the Energetics of Chemical Reactions J. Res. Sci. Teach. 1998, 35 (5) 569-581
20. Brook, A.; Briggs, H.; Bell, B.; Driver, R. Aspects of Secondary Students' Understanding of Heat: Full Report; The University of Leeds, Centre for Studies in Science Education and Mathematics Education: Leeds, U.K., 1988.
21. Clark, D. Logitudinal Conceptual Change in Students' Understanding of Thermal Equilibrium: An Examination of the Process of Conceptual Restructuring Cognit. Instr. 2006, 24 (4) 467-563
22. Erickson, G.; Tiberghien, A. Heat and Temperature. In Children's Ideas in Science; Driver, R.; Guesne, E.; Tiberghien, A., Eds.; Open University Press: Milton Keynes, U.K., 1985.
23. Harrison, A.; Grayson, D.; Treagust, D. Investigating a Grade 11 Student's Evolving Conceptions of Heat and Temperature J. Res. Sci. Teach. 1999, 36 (1) 55-87
24. Leite, L. Heat and Temperature: An Analysis of How These Concepts Are Dealt With in Textbooks Eur. J. Teach. Educ. 1999, 22 (1) 75-88
25. Meltzer, D. Investigation of Students' Reasoning Regarding Heat, Work, and the First Law of Thermodynamics in an Introductory Calculus-Based General Physics Course Am. J. Phys. 2004, 72 (11) 1432-1446
26. Sözbilir, M.; Bennett, J. Turkish Prospective Chemistry Teachers' Misunderstandings of Enthalpy and Spontaneity Chem. Educ. 2006, 11, 355-363
27. Streveler, R. A.; Miller, R. L.; Santiago-Roman, A. I.; Nelson, M. A.; Geist, M. R.; Olds, B. M. Rigorous Methodology for Concept Inventory Development: Using the "Assessment Triangle" To Develop and Test the Thermal and Transport Science Concept Inventory (TTCI) Int. J. Eng. Educ. 2011, 27 (5) 968-984
28. Taber, K. Chemical Misconceptions-Prevention, Diagnosis, and Cure, Vol. 1: Theoretical Background; Royal Society of Chemistry: London, 2002.
29. Thomaz, M.; Malaquias, I.; Valente, M.; Antunes, M. J. An Attempt To Overcome Alternative Conceptions Related to Heat and Temperature Phys. Educ. 1995, 30 (19) 19-26
30. Carson, E. M.; Watson, J. R. Undergraduate Students' Understanding of Enthalpy Change Univ. Chem. Educ. 1999, 3 (2) 46-51
31. Hadfield, L.; Wieman, C. Students' Interpretations of Equations Related to the First Law of Thermodynamics J. Chem. Educ. 2010, 87 (7) 750-755
32. Lewis, E.; Lin, M. Heat Energy and Temperature Concepts of Adolescents, Adults, and Experts: Implications for Curricular Improvements J. Res. Sci. Teach. 2003, 40, S155-S175
33. Taber, K. Finding the Optimum Level of Simplification: The Case of Teaching about Heat and Temperature Phys. Educ. 2000, 35 (5) 320-325
34. Thomas, P.; Schwenz, R. College Physical Chemistry Students' Conceptions of Equilibrium and Fundamental Thermodynamics J. Res. Sci. Teach. 1998, 35 (10) 1151-1160
35. Yeo, S.; Zadnik, M. Introductory Thermal Concept Evaluation: Assessing Students' Understanding Phys. Teach. 2001, 39, 496-504
36. Chu, H.-E.; Treagust, D.; Yeo, S.; Zadnik, M. Evaluation of Students' Understanding of Thermal Concepts in Everyday Contexts Int. J. Sci. Educ. 2012, 34 (10) 1509-1534
37. Sreenivasulu, B.; Subramaniam, R. University Students' Understanding of Chemical Thermodynamics Int. J. Sci. Educ. 2012, 35 (4) 601-635
38. Prince, M.; Vigeant, M.; Nottis, K. Development of the Heat and Energy Concept Inventory: Preliminary Results on the Prevalence and Persistence of Engineering Students' Misconceptions J. Eng. Educ. 2012, 101 (3) 412-438
39. Furr, R. M.; Bacharach, V. Psychometrics: An Introduction; SAGE Publication Inc.: Thousand Oaks, CA, 2008.
40. Messick, S. Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performance as Scientific Inquiry into Score Meaning Am. Psychol. 1995, 50 (9) 741-749
41. Cronbach, L.; Meehl, P. Construct Validity in Psychological Test Psychol. Bulletin 1955, 52 (4) 281-302
42. Loevinger, J. Objective Tests As Instruments of Psychological Theory Psychol. Rep. 1957, 3 (Monogr. Suppl. 9) 635-694
43. Messick, S. Meaning and Values in Test Validation: The Science of Ethics and Assessment Educ. Res. 1989, 18 (5) 5-11
44. Brown, T.; LeMay, E.; Bursten, B.; Murphy, C. J.; Woodward, P. M. Chemistry: The Central Science, 12 th ed.; Prentice Hall: Upper Saddle River, NJ, 2012.
45. McMurry, J.; Fay, R. Chemistry, 5 th ed.; Pearson: Upper Saddle River, NJ, 2010.
46. Silberberg, M. Chemistry: The Molecular Nature of Matter and Change, 6 ed.; McGraw-Hill: New York, 2012.
47. Tro, N. Chemistry: A Molecular Approach, 1 st ed.; Pearson: Upper Saddle River, NJ, 2010.
48. Zumdahl, S.; Zumdahl, S. Chemistry, 8 th ed.; Cengage Learning: Independence, KY, 2010.
49. Bowen, C. Think-Aloud Methods in Chemistry Education J. Chem. Educ. 1994, 71 (3) 184-190
50. Cheung, D. Using Think-Aloud Protocols To Investigate Secondary School Chemistry Teachers' Misconceptions about Chemical Equilibrium. Chem. Educ. Res. Pract. 2009, 10, 97-108.
51. Beatty, P.; Willis, G. Research Synthesis: The Practice of Cognitive Interviewing Public Opin. Q. 2007, 71 (2) 287-311
52. van Someren, M.; Barnard, Y.; Sandberg, J. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes; Academic Press: London, 1994.
53. NVivo Qualitative Data Analysis Software; QSR International Pty Ltd.: 2008; Version 8.
54. Fleiss, J. Statistical Methods for Rates and Proportions: The Measurement of Interrater Agreement; John Wiley & Sons: New York, 1981.
55. Willis, G. Cognitive Interviewing: A "How To" Guide. Reducing Survey Error through Research on the Cognitive and Decision Processes in Surveys. Short course presented at the 1999 Meeting of the American Statistical Association, August 8-12, 1999, Baltimore, Maryland.
56. Qualtrics Labs Inc., Provo, UT, 2005.
57. Kesidou, S.; Duit, R. Students' Conceptions of the Second Law of Thermodynamics-An Interpretive Study J. Res. Sci. Teach. 1993, 30 (1) 85-106
58. Wilson, M. Constructing Measures: An Item Response Modeling Approach; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, 2005.