Assessing the ease of use in the environment and markers' acceptance of on screen marking: A Rasch measurement perspective

Educational Research and Evaluation

Volumn 19, Issue 5, 2013, Pages 461-483

  Yan, Zi ^a   Coniam, David ^a  

^a THE EDUCATION UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

acceptance of OSM; differential item functioning; ease of use in the OSM environment; on screen marking; Rasch measurement

Indexed keywords

EID: 84879615039     PISSN: 13803611     EISSN: 17444187     Source Type: Journal    
DOI: 10.1080/13803611.2013.793604     Document Type: Article

References (38)

1. Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(Supplement 1), 1-7.
2. Anshel, M. H., Weatherby, N. L., Kang, M., & Watson, T. (2009). Rasch calibration of a unidimensional perfectionism inventory for sport. Psychology of Sport and Exercise, 10, 210-216. doi: 10.1016/j.psychsport.2008.07.006
3. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Erlbaum.
4. Bos, W., Goy, M., Howie, S. J., Kupari, P., & Wendt, H. (2011). Editorial: Rasch measurement in educational contexts Special issue 2: Applications of Rasch measurement in large-scale assessments. Educational Research and Evaluation, 17, 413-417. doi: 10.1080/13803611.2011.634580
5. Chan, A. H. S., So, J. C. Y., & Tsang, S. N. H. (2011). Developing optimum interface design for on-screen Chinese proofreading tasks. In Proceedings of the 1st international conference on Human interface and the management of information: Interacting with information - Volume Part II (pp. 3-10). Berlin, Germany: Springer-Verlag.
6. Chen, C.-H., & Chien, Y.-H. (2005). Effect of dynamic display and speed of display movement on reading Chinese text presented on a small screen. Perceptual and Motor Skills, 100, 865-873. doi: 10.2466/PMS.100.3.865-873
7. Coniam, D. (2009a). A comparison of onscreen and paper-based marking in the Hong Kong public examination system. Educational Research and Evaluation, 15, 243-263. doi: 10.1080/13803610902972940
8. Coniam, D. (2009b). Validating onscreen marking in Hong Kong. Asia Pacific Education Review, 11, 423-431. http://www.springerlink.com/content/l6663m43jn501317/ doi: 10.1007/s12564-009-9068-2
9. Coniam, D. (2011). A qualitative examination of the attitudes of Liberal Studies markers towards onscreen marking. British Journal of Educational Technology, 42, 1042-1054. doi: 10.1111/j.1467-8535.2010.01136.x
10. 2
11. Coniam, D., & Yeung, S.-C. A. (2010). Markers' perceptions regarding the onscreen marking of Liberal Studies in the Hong Kong public examination system. Asia Pacific Journal of Education, 30, 249-271. doi: 10.1080/02188791.2010.495836
12. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319-340. doi: 10.2307/249008
13. Fowles, D., & Adams, C. (2005, September). How does assessment differ when e-marking replaces paper-based marking? Paper presented at the 31st International Association for Educational Assessment Conference, Abuja, Nigeria.
14. Hart, D. L., & Wright, B. D. (2002). Development of an index of physical functional health status in rehabilitation. Archives of Physical Medicine and Rehabilitation, 83, 655-665. doi: 10.1053/apmr.2002.31178
15. Johnson, M., Nádas, R., & Bell, J. F. (2009). Marking essays on screen: An investigation into the reliability of marking extended subjective texts. British Journal of Educational Technology, 41, 814-826. doi: 10.1111/j.1467-8535.2009.00979.x
16. Legislative Council Panel on Education. (2005). Grant to support the modernization and development of the Hong Kong Examinations and Assessment Authority's examination systems (LC Paper No. CB(2)323/05-06(01)). Retrieved from http://www.legco.gov.hk/yr05-06/english/panels/ed/papers/ed1114cb2-323-1e.pdf
17. Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85-106.
18. Linacre, J. M. (2006). A user's guide to WINSTEPS/MINISTEP: Rasch-model computer programs. Chicago, IL: Winsteps.com.
19. Linacre, J. M. (2011). WINSTEPS: Rasch measurement computer program. Chicago, IL: Winsteps.com.
20. Muís, K. R., Winne, P. H., & Edwards, O. V. (2009). Modern psychometrics for assessing achievement goal orientation: A Rasch analysis. British Journal of Educational Psychology, 79, 547-576. doi: 10.1348/000709908X383472
21. Powers, D., Kubota, M., Bentley, J., Farnum, M., Swartz, R., & Willard, A. (1997). A pilot test of on-line essay scoring (ETS Report RM-97-07). Princeton, NJ: Educational Testing Service.
22. Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research), expanded edition (1980) with foreword and afterword by B. D. Wright. Chicago, IL: University of Chicago Press.
23. Schulz, W., & Fraillon, J. (2011). The analysis of measurement equivalence in international studies using the Rasch model. Educational Research and Evaluation, 17, 447-464. doi: 10.1080/13803611.2011.630559
24. Shaw, S. (2008). Essay marking on-screen: Implications for assessment validity. E-Learning, 5(3), 256-274. doi: 10.2304/elea.2008.5.3.256
25. Shaw, S., & Imam, H. (2008, September). On-screen essay marking reliability: Towards an understanding of marker assessment behaviour. Paper presented at the International Association for Educational Assessment Conference, Cambridge, UK.
26. Törmäkangas, K. (2011). Advantages of the Rasch measurement model in analysing educational tests: An applicator's reflection. Educational Research and Evaluation, 17, 307-320. doi: 10.1080/13803611.2011.630562
27. Twing, J., Nichols, P., & Harrison, I. (2003, June). The comparability of paper-based and image-based marking of a high-stakes, large-scale writing assessment. Paper presented at the International Association for Educational Assessment Conference, Manchester, UK.
28. Verhelst, N. D., & Glas, C. A. (1995). The one parameter logistic model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 215-237). New York, NY: Springer.
29. Wang, W. C. (2000). Modeling effects of differential item functioning in polytomous items. Journal of Applied Measurement, 1, 63-82.
30. Weigle, S. C. (2002). Assessing writing. Cambridge, UK: Cambridge University Press.
31. Wendt, H., Bos, W., & Goy, M. (2011). On applications of Rasch models in international comparative large-scale assessments: A historical review. Educational Research and Evaluation, 17, 419-446. doi: 10.1080/13803611.2011.634582
32. Wright, B. D. (1992). IRT in the 1990s: Which models work best? Rasch Measurement Transactions, 6, 196-200.
33. Wright, B. D. (1997). A history of social science measurement. Educational Measurement: Issues and Practice, 16(4), 33-45. doi: 10.1111/j.1745-3992.1997.tb00606.x
34. Yan, Z., & Bond, T. G. (2011). Developing a Rasch measurement physical fitness scale for Hong Kong primary school-aged students. Measurement in Physical Education and Exercise Science, 15, 182-203. doi: 10.1080/1091367X.2011.590772
35. Yen, N.-S., Tsai, J.-L., Chen, P.-L., Lin, H.-Y., & Chen, A. L. P. (2011). Effects of typographic variables on eye-movement measures in reading Chinese from a screen. Behaviour & Information Technology, 30, 797-808. doi: 10.1080/0144929X.2010.523900
36. Zhang, Y., Powers, D., Wright, W., & Morgan, R. (2003). Applying the online scoring network (OSN) to advanced program placement program (AP) Tests (ETS Research Report RR-03-12). Princeton, NJ: Educational Testing Service.
37. Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
38. Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36, 1-28. doi: 10.1111/j.1745-3984.1999.tb00543.x