The Psychometric Properties of Classroom Response System Data: A Case Study

Journal of Science Education and Technology

Volumn 25, Issue 4, 2016, Pages 561-574

  Kortemeyer, Gerd ^a  

^a MICHIGAN STATE UNIVERSITY   (United States)

Author keywords

Classical Test Theory; Clickers; CRS; IRT; Item Response Theory; Physics; Psychometrics

Indexed keywords

EID: 84960118579     PISSN: 10590145     EISSN: 15731839     Source Type: Journal    
DOI: 10.1007/s10956-016-9613-9     Document Type: Article

References (40)

1. Barth-Cohen LA, Smith MK, Capps DK, Lewin JD, Shemwell JT, Stetzer MR (2016) What are middle school students talking about during clicker questions? Characterizing small-group conversations mediated by classroom response systems. J Sci Educ Technol 25(1):50–61
2. Beatty I, Gerace W (2009) Technology-enhanced formative assessment: a research-based pedagogy for teaching science with classroom response technology. J Sci Educ Technol 18(2):146–162. doi:10.1007/s10956-008-9140-4
3. Birnbaum A (1968) in Lord and Novick (1968), chap. Some latent trait models and their use in inferring an examinees ability, pp 374–472
4. Caldwell JE (2007) Clickers in the large classroom: current research and best-practice tips. CBE Life Sci Educ 6(1):9–20
5. Cardamone CN, Abbott JE, Rayyan S, Seaton DT, Pawl A, Pritchard DE (2011) Item response theory analysis of the mechanics baseline test. In: Physics education research conference 2011 AIP conference proceedings, pp 135–138
6. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16(3):297–334
7. Crouch CH, Mazur E (2001) Peer instruction: ten years of experience and results. Am J Phys 69(9):970–977
8. Dancy M, Henderson C (2010) Pedagogical practices and instructional change of physics faculty. Am J Phys 78(10):1056–1063
9. Ding L, Reay NW, Lee A, Bao L (2009) Are we asking the right questions? Validating clicker question sequences by student interviews. Am J Phys 77(7):643–650
10. Ding L, Beichner R (2009) Approaches to data analysis of multiple-choice questions. Phys Rev ST Phys Educ Res 5:020103. doi:10.1103/PhysRevSTPER.5.020103
11. Fagen AP, Crouch CH, Mazur E (2002) Peer instruction: results from a range of classrooms. Phys Teach 40(4):206–209
12. Henderson C, Dancy M, Niewiadomska-Bugaj M (2012) Use of research-based instructional strategies in introductory physics: where do faculty leave the innovation-decision process? Phys Rev Spec Top Phys Educ Res 8(2):020104
13. iClicker Classroom Response System (2003) https://www.iclicker.com. Accessed Mar 2016
14. James MC (2006) The effect of grading incentive on student discourse in Peer instruction. Am J Phys 74(8):689–691
15. James MC, Willoughby S (2011) Listening to student conversations during clicker questions: what you have not heard might surprise you!. Am J Phys 79(1):123–132
16. Kay RH, LeSage A (2009) Examining the benefits and challenges of using audience response systems: a review of the literature. Comput Educ 53(3):819–827
17. Keller C, Finkelstein N, Perkins K, Pollock S, Turpen C, Dubson M (2007) Research-based practices for effective clicker use. In: 2007 physics education research conference, vol 951, pp 128–131
18. King DB, Joshi S (2008) Gender differences in the use and effectiveness of personal response devices. J Sci Educ Technol 17(6):544–552
19. Kortemeyer G (2015) Scalable continual quality control of formative assessment items in an educational digital library: an empirical study. Int J Digit Libr 1–13. doi:10.1007/s00799-015-0145-3
20. Kortemeyer G, Kashy E, Benenson W, Bauer W (2008) Experiences using the open-source learning content management and assessment system LON-CAPA in introductory physics courses. Am J Phys 76:438–444
21. Kortemeyer G (2014) Extending item response theory to online homework. Phys Rev ST Phys Educ Res 10:010118. doi:10.1103/PhysRevSTPER.10.010118
22. Lantz M (2010) The use of ‘clickers’ in the classroom: teaching innovation or merely an amusing novelty? Comput Hum Behav 26(4):556
23. Lasry N (2008) Clickers or flashcards: is there really a difference? Phys Teach 46(4):242–244
24. Lasry N, Mazur E, Watkins J (2008) Peer instruction: from Harvard to the two-year college. Am J Phys 76(11):1066–1069
25. Lee YJ, Palazzo DJ, Warnakulasooriya R, Pritchard DE (2008) Patterns, correlates, and reduction of homework copying. Phys Rev ST Phys Educ Res 4:010102. doi:10.1103/PhysRevSTPER.4.010102
26. Lord FM, Novick MR (eds) (1968) Statistical theories of mental test scores. Addison-Wesley, Reading
27. Mazur E (1997) Peer instruction. Prentice Hall, Upper Saddle River
28. Morris GA, Branum-Martin L, Harshman N, Baker SD, Mazur E, Dutta S, Mzoughi T, McCauley V (2006) Testing the test: item response curves and test quality. Am J Phys 74(5):449–453
29. Nunnally J, Bernstein I (1994) Psychometric theory. McGraw-Hill, New York
30. Poole D (2012) The impact of anonymous and assigned use of student response systems on student achievement. J Interact Learn Res 23(2):101–112
31. R Development Core Team (2008) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. http://www.R-project.org. ISBN 3-900051-07-0
32. Rasch G (1993) Probabilistic models for some intelligence and attainment tests. MESA Press, Chicago
33. Reckase MD (1997) The past and future of multidimensional item response theory. Appl Psychol Meas 21(1):25–36
34. Richardson AM, Dunn PK, McDonald C, Oprescu F (2014) CRiSP: an instrument for assessing student perceptions of classroom response systems. J Sci Educ Technol 24(4):432–447
35. Richardson CT, O’Shea BW (2013) Assessing gender differences in response system questions for an introductory physics course. Am J Phys 81(3):231–236
36. Rizopoulos D (2006) Ltm: an R package for Latent Variable Modelling and Item Response Theory Analyses. J Stat Softw 17(5):1–25. http://www.jstatsoft.org/v17/i05/
37. Setzer JC, Wise SL, van den Heuvel JR, Ling G (2013) An investigation of examinee test-taking effort on a large-scale assessment. Appl Meas Educ 26(1):34–49
38. Turpen C, Finkelstein ND (2009) Not all interactive engagement is the same: variations in physics professors implementation of Peer Instruction. Phys Rev Spec Top Phys Educ Res 5(2):020101
39. White P, Syncox D, Alters B (2011) Clicking for grades? Really? Investigating the use of clickers for awarding grade-points in post-secondary education. J Interact Learn Environ 19(5):551–561
40. Willse JT (2014) CTT: Classical Test Theory Functions. https://cran.r-project.org/web/packages/CTT/. Accessed Mar 2016