Tuned models of peer assessment in MOOCs

Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013

Volumn , Issue , 2013, Pages

  Piech, Chris ^a   Huang, Jonathan ^a   Chen, Zhenghao ^b   Do, Chuong ^b   Ng, Andrew ^b   Koller, Daphne ^b  

^a STANFORD UNIVERSITY   (United States)

^b Coursera ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GRADING; STUDENTS;

HUMAN EXPERT; ONLINE COURSE; PEER ASSESSMENT;

DATA MINING;

EID: 85084016189     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. K. Ashley and I. Goldin. Toward ai-enhanced computer-supported peer review in legal education. In 24th International Conference on Legal Knowledge and Information Systems (JURIX), volume 235, 2011.
2. Y. Bachrach, T. Minka, J. Guiver, and T. Graepel. How To Grade a Test Without Knowing the Answers - A Bayesian Graphical Model for Adaptive Crowdsourcing and Aptitude Testing. In The 29th Annual International Conference on Machine Learning, ICML’12, 2012.
3. F. Baker. The basics of item response theory. ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, 2001.
4. W. H. Batchelder and A. K. Romney. Test theory without an answer key. Psychometrika, 53:71–92, 1988.
5. L. Charlin, R. Zemel, and C. Boutilier. A framework for optimizing paper matching. In Proceedings of Uncertainty in Artificial Intelligence (UAI’11), 2011.
6. A. Dasgupta and A. Ghosh. Crowdsourced judgement elicitation with endogenous proficiency. In Proceedings of the 22th international conference on World wide web. ACM, 2013., 2013.
7. S. Geman and D. Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE PAMI, (6):721–741, 1984.
8. I. Goldin. Accounting for peer reviewer bias with bayesian models. In Proceedings of the Workshop on Intelligent Support for Learning Groups at the 11th International Conference on Intelligent Tutoring Systems, 2012.
9. I. M. Goldin and K. D. Ashley. Peering inside peer review with bayesian models. In Proceedings of the 15th international conference on Artificial intelligence in education, AIED’11, pages 90–97, Berlin, Heidelberg, 2011. Springer-Verlag.
10. V. E. Johnson. On bayesian analysis of multi-rater ordinal data: An application to automated essay grading. Journal of the American Statistical Association, 91:42–51, 1996.
11. E. Kamar, S. Hacker, and E. Horvitz. Combining human and machine intelligence in large-scale crowdsourcing. In In AAMAS, 2012.
12. G. Karabatsos and W. Batchelder. Markov chain estimation for test theory without an answer key. Psychometrika, 68(3):373–389, 2003.
13. C. Kulkarni, K. Pang-Wei, H. Le, D. Chia, K. Papadopoulos, D. Koller, and S. R. Klemmer. Scaling self and peer assessment to the global design classroom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013.
14. R. J. Mislevy, R. G. Almond, D. Yan, and L. S. Steinberg. Bayes nets in educational assessment: Where the numbers come from. In Proceedings of the fifteenth conference on uncertainty in artificial intelligence, pages 437–446. Morgan Kaufmann Publishers Inc., 1999.
15. F. Å. Nielsen. A new anew: Evaluation of a word list for sentiment analysis in microblogs. arXiv preprint arXiv:1103.2903, 2011.
16. S. Rogers, M. Girolami, and T. Polajnar. Semi-parametric analysis of multi-rater data. Statistics and Computing, 20(3):317–334, July 2010.
17. A. A. Russell. Calibrated peer review - a writing and critical-thinking instructional tool. Teaching Tips: Innovations in Undergraduate Science Instruction, page 54, 2004.
18. P. M. Sadler and E. Good. The impact of self-and peer-grading on student learning. Educational assessment, 11(1):1–31, 2006.
19. J. Whitehill, P. Ruvolo, T. fan Wu, J. Bergsma, and J. Movellan. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems 22, pages 2035–2043. MIT Press, 2009.