A user study on the automated assessment of reviews

CEUR Workshop Proceedings

Volumn 1009, Issue , 2013, Pages

  Ramachandran, Lakshmi ^a   Gehringer, Edward F ^a  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Metareview metrics; Review quality assessment; User experience survey

Indexed keywords

ARTIFICIAL INTELLIGENCE; INTELLECTUAL PROPERTY;

ACADEMIC REVIEWS; AUTOMATED ASSESSMENT; AUTOMATED METAREVIEWING; METAREVIEW METRICS; PURCHASE DECISION; REVIEW QUALITY; TIMELY FEEDBACK; USER EXPERIENCE;

AUTOMATION;

EID: 84925014543     PISSN: 16130073     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. K. Cho and C. D. Schunn, "Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system," Computer Education, vol. 48, pp. 409-426, April 2007.
2. E. F. Gehringer, L. M. Ehresman, and W. P. Conger, S. Gu, "Reusable learning objects through peer review: The expertiza approach," in Innovate: Journal of Online Education, 2007.
3. E. F. Gehringer, "Expertiza: Managing feedback in collaborative learning." in Monitoring and Assessment in Online Collaborative Environments: Emergent Computational Technologies for E-Learning Support, 2010, pp. 75-96.
4. R. Rada, A. Michailidis, and W. Wang, "Collaborative hypermedia in a classroom setting," J. Educ. Multimedia Hypermedia, vol. 3, pp. 21-36, January 1994.
5. C. KAijhne, K. Bauhm, and J. Z. Yue, "Reviewing the reviewers: A study of author perception on peer reviews in computer science." in CollaborateCom'10, 2010, pp. 1-8.
6. P. Wessa and A. De Rycker, "Reviewing peer reviews: a rule-based approach," in International Conference on E-Learning (ICEL), 2010, pp. 408-418.
7. J. Burstein, D. Marcu, and K. Knight, "Finding the write stuff: Automatic identification of discourse structure in student essays," IEEE Intelligent Systems, vol. 18, pp. 32-39, January 2003.
8. P. W. Foltz, S. Gilliam, and S. A. Kendall, "Supporting content-based feedback in online writing evaluation with LSA," Interactive Learning Environments, vol. 8, pp. 111-129, 2000.
9. S. van Rooyen, N. Black, and F. Godlee, "Development of the review quality instrument (rqi) for assessing peer reviews of manuscripts," Journal of Clinical Epidemiology, vol. 52, no. 7, pp. 625-629, 1999.
10. M. M. Nelson and C. D. Schunn, "The nature of feedback: How different types of peer feedback affect writing performance," in Instructional Science, vol. 27, 2009, pp. 375-401.
11. W. Xiong, D. J. Litman, and C. D. Schunn, "Assessing reviewer's performance based on mining problem localization in peer-review data." in EDM, 2010, pp. 211-220.
12. K. Cho, "Machine classification of peer comments in physics," in Educational Data Mining, 2008, pp. 192-196.
13. R. Zhang and T. Tran, "Review recommendation with graphical model and em algorithm," in Proceedings of the 19th international conference on World wide web, ser. WWW '10, 2010, pp. 1219-1220.
14. S. Moghaddam, M. Jamali, and M. Ester, "Review recommendation: personalized prediction of the quality of online reviews," in Proceedings of the 20th ACM international conference on Information and knowledge management, ser. CIKM '11, 2011, pp. 2249-2252.
15. P. D. Turney, "Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews," in In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 2002.
16. N. Dalvi, R. Kumar, B. Pang, and A. Tomkins, "Matching reviews to objects using a language model," in Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2 - Volume 2, ser. EMNLP '09, 2009, pp. 609-618.
17. I. Titov and R. McDonald, "Modeling online reviews with multi-grain topic models," in Proceedings of the 17th international conference on World Wide Web, ser. WWW '08, 2008, pp. 111-120.
18. E.-P. Lim, V.-A. Nguyen, N. Jindal, B. Liu, and H. W. Lauw, "Detecting product review spammers using rating behaviors," in Proceedings of the 19th ACM international conference on Information and knowledge management, ser. CIKM '10, 2010, pp. 939-948.
19. L. Ramachandran and E. F. Gehringer, "A word-order based graph representation for relevance identification [poster]," CIKM 2012, 21st ACM Conference on Information and Knowledge Management, October 2012.
20. B. Liu, M. Hu, and J. Cheng, "Opinion observer: analyzing and comparing opinions on the web," in Proceedings of the 14th International Conference on World Wide Web, 2005, pp. 342-351.
21. M. Kuniavsky, Smart Things: Ubiquitous Computing User Experience Design: Ubiquitous Computing User Experience Design. Morgan Kaufmann, 2010.
22. E. L.-C. Law, V. Roto, M. Hassenzahl, A. P. Vermeeren, and J. Kort, "Understanding, scoping and defining user experience: a survey approach," in Proceedings of the 27th international conference on Human factors in computing systems. ACM, 2009, pp. 719-728.
23. F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics bulletin, vol. 1, no. 6, pp. 80-83, 1945.
24. E. Tognini-Bonelli, "Corpus linguistics at work," Computational Linguistics, vol. 28, no. 4, pp. 583-583, 2002.