Empirical recovery of argumentation learning progressions in scenario-based assessments of English language arts

Psicologia Educativa

Volumn 20, Issue 2, 2014, Pages 109-115

  Van Rijn, Peter W ^a   Graf, E Aurora ^b   Deane, Paul ^b  

^a Educational Testing Service Global   (Netherlands)

^b EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Item response theory; Learning progressions; Parallel forms; Writing assessment

Indexed keywords

EID: 84924525195     PISSN: 1135755X     EISSN: 21740526     Source Type: Journal    
DOI: 10.1016/j.pse.2014.11.004     Document Type: Article

References (38)

1. R. Adams, M. Wilson, and W.-C. Wang The multdimensional random coefficients multinomial logit model Applied Psychological Measurement 21 1997 1 23
2. Alonzo, A.; & Gotwals, A. (Eds.) (2012). Learning progressions in science: Current challenges and future directions. Rotterdam, The Netherlands: Sense.
3. D.G. Altman Practical statistics for medical research 1991 Chapman and Hall London, England
4. Arieli-Attali, M.; Wylie, E.C.; & Bauer, M.I. (2012, April). The use of three learning progressions in supporting formative assessment in middle school mathematics. Paper presented at the annual meeting of the American Educational Research Association (AERA), Vancouver, CA.
5. M.E. Avalon, L.S. Meyers, B.W. Davis, and N. Smits Distractor similarity and item-stem structure: Effects on item difficulty Applied Measurement in Education 20 2007 153 170
6. R. Bennett Cognitively based assessment of, for, and as learning (CBAL): A preliminary theory of action for summative and formative assessment Measurement 8 2010 70 91
7. P. Black, M. Wilson, and S.-Y. Yao Road maps for learning: A guide to navigation of learning progressions Measurement: Interdisciplinary Research and Perspectives 9 2011 71 123
8. M. Carr, and N. Alexeev Fluency, accuracy, and gender predict developmental trajectories of arithmetic strategies, Journal of Educational Psychology 103 2011 617 631
9. G.J. Cizek, and M.B. Bunch Standard setting: A guide to establishing and evaluating performance standards on tests 2007 Sage Thousand Oaks, CA
10. D.H. Clements, and J. Sarama Learning trajectories in mathematics education Mathematical Thinking and Learning 6 2004 81 89
11. Common Core State Standards Initiative (2010). Common core state standards for Englishlanguage arts and literacy in history/social studies, science, and technical subjects. Retrieved from http://www.corestandards.org/ELA-Literacy/.
12. Deane, P.; Fowles, M.; Baldwin, D.; & Persky, H. (2011). The CBAL summative writing assessment: A draft eighth-grade design (Research Memorandum 11-01). Princeton, NJ: Educational Testing Service.
13. P. Deane, and Y. Song A case study in principled assessment design: Designing assessments to measure and support the development of argumentative reading and writing skills Psicología Educativa 20 2014 99 108
14. A.L. Dudycha, and J.B. Carpenter Effects of item format on item discrimination and difficulty, Journal of Applied Psychology 58 1973 116 121
15. R. Duschl, S. Maeng, and A. Sezen Learning progressions and teaching sequences: A review and analysis Studies in Science Education 47 2011 123 182
16. S.E. Embretson, and J. Gorin Improving construct validity with cognitive psychology principles Journal of Educational Measurement 38 2001 343 368
17. J.L. Fleiss, B. Levin, and M. Paik Statistical methods for rates and proportions 3rd ed 2003 Wiley New York
18. Haberman, S. (2013). A general program for item-response analysis that employs the stabilized Newton-Raphson algorithm (ETS Research Report 13-32). Princeton, NJ: Educational Testing Service.
19. Mislevy, R.J.; Almond, R.G.; & Lukas, J.F. (2003). A brief introduction to evidence centered design (Research Report 03-16). Princeton, NJ: Educational Testing Service.
20. R.J. Mislevy, and G.D. Haertel Implications of evidence-centered design for educational testing Educational Measurement: Issues and Practice 25 4 2006 6 20
21. E. Muraki A generalized partial credit model: Application of an EM algorithm Applied Psychological Measurement 16 1992 159 176
22. S.E. Newstead, P. Bradon, S.J. Handley, I. Dennis, and J.S.B.T. Evans Predicting the difficulty of complex logical reasoning problems Thinking & Reasoning 12 2006 62 90
23. OECD (1999). Measuring student knowledge and skill. a new framework for assessment. Retrieved from http://www.oecd.org/edu/school/programmeforinternationalstu-dentassessmentpisa/33693997.pdf.
24. OECD (2013a, March). PISA 2015: Draft reading literacy framework. Retrieved from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Reading%20Fra-mework%20.pdf.
25. OECD (2013b, March). PISA 2015: Draft science framework. Retrieved from http://www. oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Science%20Framework%20.pdf.
26. M. Reckase Multidimensional item response theory 2009 Springer New York
27. A.A. Rupp, T. Ferne, and H. Choi How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective Language Testing 23 2006 441 474
28. C. Smith, M. Wiser, C. Anderson, and J. Krajcik Implications of research on children's learning for standards and assessment: A proposed learning progression for matter and the atomic-molecular theory Measurement: Interdisciplinary Research and Perspectives 4 2006 1 98
29. Song, Y.; Deane, P.; Graf, E.A.; & van Rijn, P.W. (2013). Using argumentation learning progressions to support teaching and assessments of English language arts (R & D Connections No. 22). Princeton, NJ: Educational Testing Service.
30. J. Steedle, and R. Shavelson Supporting valid interpretations of learning progression level diagnoses Journal of Research in Science Teaching 46 2009 669 715
31. Van der Schoot, F.C. J. A. (2001, April). The application of an IRT-based method for standard setting in a three-stage procedure. Paper presented at the annual meeting of the National Council of Measurement in Education (NCME), New Orleans, LA.
32. H. Wainer, and D. Thissen How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement: Issues and Practice 15 1 1996 22 29
33. W. Way, T.N. Ansley, and R.A. Forsyth The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimates Applied Psychological Measurement 12 1988 239 252
34. West, P.; Rutstein, D.; Mislevy, R.; Liu, J.; Levy, R.; DiCerbo, K.;... Behrens, J. (2012). A Bayesian network approach to modeling learning progressions. In A. Alonzo & A. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (p. 257-292). Rotterdam, The Netherlands: Sense.
35. D. Wilmot, A. Schoenfeld, M. Wilson, D. Champney, and W. Zahner Validating a learning progression in mathematical functions for college readiness Mathematical Thinking and Learning 1 2011 259 291
36. M. Wilson Measuring progressions: Assessment structures underlying a learning progression Journal of Research in Science Teaching 46 2009 716 730
37. W. Yen Scaling performance assessments: Strategies for managing local item dependence Journal of Educational Measurement 30 1993 187 213
38. R. Zwick, D. Senturk, J. Wang, and S.C. Loomis An investigation of alternative methods for item mapping in the National Assessment of Educational Progress Educational Measurement: Issues and Practice 20 2 2001 15 25