A Blueprint for Computer-Assisted Assessment

A Blueprint for Computer-Assisted Assessment

Volumn , Issue , 2003, Pages 1-202

  Bull, Joanna ^a   McKenna, Colleen ^b  

^a University College London ^*

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85138923097     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203464687     Document Type: Book

References (178)

1. American Physical Society (1996) Fighting the Gender Gap: standardised tests are a poor indicators of ability in physics, American Physical Society, News On-line, http://www.aps.org/apsnews/0796/11538.html (accessed January 2003).
2. Angoff, W.H. and Schrader, W.B. (1981) A study of alternative methods for equating right scores to formula scores. Research Report, New Jersey: Educational Testing Service.
3. Antonelli, M.A. (1997) Accuracy of second-year medical students self-assessment of clinical skills, Academic Medicine 72: 563–5.
4. Askew, S. and Lodge, C. (2000) Gifts, ping-pong and loops – linking feedback and learning, in S. Askew (ed) Feedback for Learning, London: Routledge.
5. Atkins, M. (1993) Evaluating interactive technologies for learning, Journal of Curriculum Studies 25(4): 333–42.
6. Baggott, G. and Rayne, R. (2001) Learning support for mature, part-time, evening students: providing feedback via frequent, computer-based assessments, in M. Danson (ed.) Proceedings of the 5th International Computer assisted Assessment Conference, 2–3 July 2001, Loughborough: Loughborough University, pp. 9–20, http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
7. Baldwin, B.A. and Howard, T.P. (1983) Intertopical sequencing of examination questions: an empirical evaluation, Journal of Accounting Education 1(2): 89–96.
8. Barnett, R. (1995) Improving Higher Education: total quality care, Buckingham: Society for Research in Higher Education and Open University Press.
9. Beevers, C.E., Cherry, B.G., Foster, M.G. and McGuire, G.R. (1991) Software Tools for Computer Aided Learning in Mathematics, Aldershot, UK: Ashgate Publishing Ltd.
10. Beevers, C.E., Foster, M.G., McGuire, G.R. and Renshaw, J.H. (1992) Some
11. Problems of Mathematical CAL, Journal of Computing Education 18: 119–25.
12. Benford, S., Burke, E. and Foxley, E. (1992) Automatic assessment of computer programs in the Ceilidh system, IEEE-SC International Software Metrics Symposium.
13. Bennett, R.E. (1998) Reinventing Assessment: speculations on the future of large-scale educational testing, Research Report. New Jersey: Educational Testing Service. Available from http://www.ets.org/research/newpubs.html (accessed January 2003).
14. Biometric Consortium (2002) http://www.biometrics.org/ (accessed January 2003).
15. Bloom, B.S. (ed.), Englehart, M.D., Furst, E.J., Hill, W.H. and Krathwohl, D.R. (1956) Taxonomy of Educational Objectives: handbook 1: cognitive domain, New York: Longmans.
16. Borga, J. (2002) Finger scanning technology monitors school employees, Education Week on the Web, October 23, 2002, http://www.edweek.org/ew/ewstory.cfm?slug=08biometric.h22 (accessed January 2003).
17. Boud, D. (1995) Enhancing Learning Through Self-Assessment, London: Kogan Page.
18. Boyle, S. (1984) The effect of variations in answer-sheet format on aptitude test performance, Journal of Occupational Psychology 57: 323–6.
19. Brew, A. (1999) Towards autonomous assessment: using self-assessment and peer-assessment, in S. Brown and A. Glasner (1999) Assessment Matters in Higher Education, Buckingham: Society for Research in Higher Education.
20. Brewster, S., Masters M.M. and Glendye, A. (2000) Haptic virtual reality for training veterinary students, paper presented at ED-MEDIA99, Seattle, USA. Available from http://www.dcs.gla.ac.uk/~michelle (accessed January 2003).
21. Britain, S. and Liber, O. (2000) A Framework for Pedagogical Evaluation of Virtual Learning Environments, JISC Technology Applications Programme.
22. Bristol: Joint Information Systems Committee report, http://www.jtap.ac.uk/reports/htm/jtap-041.html (accessed January 2003).
23. British Educational Communications and Technology Agency (2001) Computer games in education (CGE) project. http://forum.ngfl.gov.uk/Images/vtc/Games_and_education/GamesReportfinal.rtf (accessed January 2003).
24. British Standards Institute (2002) Code of practice for the use of information technology in the delivery of assessments, http://www.bsi-global.com (accessed January 2003).
25. Brosnan, M. (1999) Computer anxiety in students: should computer-based assessment be used at all? in S. Brown, J. Bull and P. Race (eds) (1999) Computer-assisted Assessment in Higher Education, London: Kogan Page.
26. Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education, London: Routledge.
27. Brown, S. (1999) Assessing practice, in S. Brown and A. Glasner (eds) (1999) Assessment Matters in Higher Education: Choosing and Using Diverse Approaches, Buckingham: Society for Research in Higher Education and Open University Press, pp. 95–105.
28. Bull, J. (1993) Using Technology to Assess Student Learning, Sheffield: The Universities’ Staff Development Unit and the Universities of Kent and Leeds, TLTP Project ALTER.
29. Bull, J. and McKenna, C. (2000)CAA centre update, in H. Cooper and S. Clowes (eds) Proceedings of the 4th International Computer-assisted Assessment Conference, 21–22 June 2000, Loughborough: Loughborough University, pp. 11–19 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
30. Bull, J. and Dalziel, J. (2003) Assessing question banks, in A. Littlejohn, Reusing Resources for Networked Learning, London: Kogan Page.
31. Bull, J., Conole, G., Danson, M., Davis, H., Sclater, N. and White, S. (2002) Rethinking assessment through learning technologies, in Proceedings of 18th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, 8–11 November 2002, UNITEC, Auckland, New Zealand.
32. Burbules, N. (1998) Rhetorics of the web: hyperreading and critical literacy, in I. Snyder (1998) Page to Screen: Taking Literacy into the Electronic Era, London: Routledge, pp. 102–22.
33. Burstein, J., Kaplan, R., Wolff, S. and Chi Lu (1996) Using lexical semantic techniques to classify free-response, in Proceedings of SIGLEX 1996 Workshop, Annual Meeting of the Association of Computational Linguistics, University of California, Santa Cruz.
34. Burstein, J., Kukich, K., Wolff, S., Lu, C. and Chodorow, M. (1998) Enriching automated scoring using discourse marking, in Proceedings of the Workshop on Discourse Relations and Discourse Marking, Annual Meeting of the Association of Computational Linguistics, August 1998, Montreal, Canada.
35. Burstein, J. and Chodorow, M. (1999) Automated essay scoring for non-native english speakers, Joint Symposium of the Association of Computational Linguistics and the International Association of Language Learning Technologies, Workshop on Computer-Mediated Language Assessment and Evaluation of Natural Language Processing, June 1999, College Park, Maryland.
36. Burstein, J., Leacock, C. and Swartz, R. (2001) Automated evaluation of essays and short answers, in M. Danson (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference, 2–3 July 2001, Loughborough: Loughborough University, pp. 41–53 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
37. Burstein, J. and Macru, D. (1999) The Benefits of Modularity in an Automated Essay Scoring System, Educational Testing Services, available from http://www.ets.org/research/dload/colinga4.pdf (accessed January 2003).
38. Burton, R. (2001) Quantifying the effects of chance in multiple choice and true/false tests: question selection and guessing of answers, in Assessment and Evaluation in Higher Education 26(1): 41–50.
39. Choppin, B (1979) Testing the questions – the Rasch model and item banking, MESA Research Memoranda 49, MESA Psychometric Laboratory, University of Chicago, http://www.rasch.org/memos.htm (accessed January 2003).
40. Chung, G.K.W.K. and O’Neill, H.F., Jr. (1997) Methodological approaches to on-line scoring of essays. (Delivered to OERI, Award No. R305B60002.) Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
41. Clarke, A. (1999) Evaluation Research: An introduction to principles, methods and practice, London: Sage.
42. Cleave-Hogg, D., Morgan, P. and Guest, C. (2000) Evaluation of medical students performance in anaesthesia using a CAE Med-link simulator system, in H. Cooper and S. Clowes (eds) Proceedings of the 4th International Computer-assisted Assessment Conference, 21–22 June 2001, Loughborough: Loughborough University, pp. 119–28 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
43. Cook, J. (2002) Evaluating learning technology resources guide, Learning and Teaching Support Network, Generic Centre and the Association for Learning Technology, http://www.ltsn.ac.uk/genericcentre/index.asp?id=17149 (accessed January 2003).
44. Creme, P. (1995) Assessing ‘seminar work’: students as teachers, in P. Knight (ed) Assessment for Learning in Higher Education, London: Kogan Page.
45. Creme, P. (2000) The personal in university writing: uses of reflective learning journals in M. Lea and B. Stierer (eds) Student Writing in Higher Education: New Contexts, Buckingham: Society for Research in Higher Education and Open University Press.
46. Cronbach, L. (1982) Issues in planning evaluations, in L. Cronbach (ed.), Designing Evaluations of Educational and Social Programs, San Francisco: Jossey-Bass.
47. Cronbach, L.J. (1949) Essentials of Psychological Testing, New York: Harper and Row.
48. Dacre, J. and Haq, I. (2002) Evaluation of a clinical skills website and its effect on student performance. Presentation at Teaching and Learning at University College London http://www.ucl.ac.uk/Library/TL2002 (accessed January 2003).
49. Daly, C. and Waldron, J. (2002) Introductory programming, problem solving and computer assisted assessment, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 95–105 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
50. Dalziel, J. (2001) Enhancing web-based learning with computer assisted assessment: pedagogical and technical considerations, in M. Danson (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp. 99–107 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
51. Dalziel, J. and Gazzard, S. (1999) Next generation computer assisted assessment software: the design and implementation of WebMCQ in M. Danson and R. Sherratt (eds) Proceedings of the 3rd International Computer-assisted Assessment Conference, 16–17 June 1999, Loughborough: Loughborough University, pp. 61–71 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
52. Davies, P. (2002) There’s no confidence in multiple-choice testing…, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 119–31 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
53. Dearing, R. (1997) Higher Education in the Learning Society–Summary Report, The National Committee of Inquiry into Higher Education, London: HMSO.
54. Denton, P. (2001) Generating and e-mailing feedback to students using MSOffice, in Danson M. (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp.157–74 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
55. DTI (2001) The UK Games Industry and Higher Education, DTIreport, April 2001, http://www.dti.gov.uk/cii/services/contentindustry/games_skills.pdf (accessed January 2003).
56. Draper, S., Brown, M., Edgerton, E., Henderson, F., McAteer, E., Smith, E., Watt, H. (1994) Observing and Measuring the Performance of Educational Technology, TILT Report No. 1, Glasgow: University of Glasgow.
57. Draper, S., Brown, M., Henderson, F., McAteer, E. (1997) Integrative evaluation: an emerging role for classroom studies of CAL, Computers in Education 26(1–3): 17–35.
58. Drasgow, F. and Oslon-Buchanan, J.B. (1999) Innovations in Computerised Assessment, Mahwah, New Jersey: Lawrence Erlbaum Associates.
59. Ehrmann, S.C. (1998) Studying teaching, learning and technology: a toolkit from the Flashlight program, Active Learning 9: 36–8.
60. Entwistle, N.J., McCune, V. and Hounsell, J. (2002) Approaches to studying and perceptions of university teaching-learning environments: concepts, measures and preliminary findings, Occasional Report No. 1, September, 2002, ETL Project, Universities of Edinburgh, Coventry and Durham. Available from URL: http://www.ed.ac.uk/etl (accessed January 2003).
61. Evans, A.W., McKenna, C. and Oliver, M. (2002) Self-assessment in medical practice, Journal of the Royal Society of Medicine 95: 511–13.
62. Falchikov, N. (1995) Improving feedback to and from students in P. Knight (ed.) Assessment for Learning in Higher Education, London: Kogan Page.
63. Farthing, D. and McPhee, D. (1999) Multiple choice for honours-level students? A statistical evaluation, in M. Danson and R. Sherratt (eds) Proceedings of the 3rd International Computer-assisted Assessment Conference, 16–17 June 1999, Loughborough: Loughborough University, pp. 105–116 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
64. Fiddes, D.J., Korabinski, A.A., McGuire, G.R., Youngson, M.A. and McMillan, D. (2002) Are mathematics exam results affected by the mode of delivery, ALT-J 10: 61–9.
65. Foltz, P.W. (1996) Latent semantic analysis for text-based research, in Behavior Research Methods, Instruments and Computers 28(2): 197–202.
66. Foxely, E., Higgins, C., Tsintsifas, A. and Symoniedes, P. (1999) The Ceildh Coursemaster System: an introduction, 4th Java in Computing Conference, LTSN Centre for Information and Computer Sciences http://www.ics.ltsn.ac.uk/pub/Jicc4/foxley.pdf (accessed January 2003).
67. Foxley, E., Higgins, C., Hegazy, T., Symeonidis, P. and Tsintsifas, A. (2001) The CourseMaster CBA System: improvements over Ceilidh, in M. Danson (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp. 189–201 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
68. Gardner-Medwin, A.R. (1995) Confidence assessment in the teaching of basic science, in ALT-J 3(1).
69. Gibbs, G. (1999) Using assessment strategically to change the way students learn, in S. Brown and A. Glasner (eds) (1999) Assessment Matters in Higher Education: choosing and using diverse approaches, Buckingham: Society for Research in Higher Education and Open University Press.
70. Glasner, A. (1999) Innovations in student assessment: a system-wide perspective, in S. Brown and A. Glasner (eds) (1999) Assessment Matters in Higher Education: choosing and using diverse approaches, Buckingham: Society for Research in Higher Education and Open University Press.
71. Green, B.F. (1983) Adaptive testing by computer, in R.B. Ekstrom (ed.) Measurement, Technology, and Individuality in Education: new directions for testing and measurement, No 17, San Francisco, California: Jossey-Bass.
72. Gronlund, N.E. (1988) How to Construct Achievement Tests, Englewood Cliffs, New Jersey: Prentice Hall.
73. Grove-Stephenson, I., Sclater, N. and Heath, A. (2002) Building a compliance audit for BS7988 ‘Code of practice for the use of information technology for the delivery of assessments’, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 133–36 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
74. Gruber, R.A., (1987) Sequencing exam questions relative to topic presentation, Journal of Accounting Education, pp. 77–86.
75. Haladyna, T.M. (1997) Writing Test Items to Evaluate Higher Order Thinking, Boston: Allyn and Bacon.
76. Halstead, P. (1994) Computer aided learning and assessment for programming skills, Monitor 4: 76–8 CTI Centre for Computing.
77. Hambleton, R.K, Swaminathan, H. and Rogers, H.J. (1991) Fundamentals of Item Response Theory, California: Sage Publications.
78. Hammond, M. (1994) Measuring the impact of IT on learning, Journal of Computer Assisted Learning 10: 251–60.
79. Heard, S., Nicol, J. and Heath, S. (1997a) Protocol for the Implementation of Summative Computer-assisted Assessment Examinations, Aberdeen: Mertal Publications, University of Aberdeen.
80. Heard, S., Nicol, J. and Heath, S. (1997b) Setting Effective Objective Tests, Aberdeen: Mertal Publications, University of Aberdeen.
81. Henkel, M. (2000) Academic Identities and Policy Change in Higher Education, London: Jessica Kingsley.
82. Higgins, R., Hartley, P. and Skelton, A. (2001) Getting the message across: the problem of communicating assessment feedback. Points for debate, in Teaching in Higher Education 6(2): 269–74.
83. Holley, D. and Oliver, M. (2000) Pedagogy and new power relationships, in The International Journal of Management Education 1(1): 11–21.
84. Irving, A., Read, M., Hunt, A. and Knight, S. (2000) Uses of information technology in exam revision, in H. Cooper and S. Clowes (eds) Proceedings of the 4th International Computer-assisted Assessment Conference, 21–22 June 2001, Loughborough: Loughborough University, pp. 99–107 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
85. Isaacs, G. (1994) About multiple choice questions, Multiple Choice Testing: Green Guide, No.16, Cambelltown, New South Wales: HERDSA, pp. 4–22.
86. Jacobs, D. (2002) Personnal communication with the authors. Royal Veterinary College.
87. Kafai, Y.B. (2001) The Educational Potential of Electronic Games: from gamesto-teach to games-to-learn, playing by the rules, Chicago: Cultural Policy Center, University of Chicago.
88. King, T. (1997) Recommendations for managing the implementation of computer aided assessment, Active Learning, 6: 23–6.
89. King, T. and White, I. (1999) A University wide protocol for the conduct of computer assisted assessment, Technical Report No. 3, Integrating People and Technology, Fund for the Development of Teaching and Learning project, Portsmouth: University of Portsmouth.
90. King, T., Billinge, D., Callear D., Wilson, S., Wilson, A. and Briggs, J. (1998) Developing and evaluation a CAA protocol for university students, in W. Wade and M. Danson (eds), Proceedings of the 2nd Annual Computer-assisted Assessment Conference, 17–18 June 1998, Loughborough: University of Loughborough, pp.17–23 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
91. Kirriemuir, J. (2002a) The Relevance of Gaming and Gaming Consoles to the Higher and Further Education Learning Experience, Joint Information Systems Committee, Technology Watch Report, February/March 2002 http://www.jisc.ac.uk/techwatch/reports/tsw_02–01.rtf (accessed January 2003).
92. Kirriemuir, J. (2002b) Video gaming, education and digital learning technologies, D-Lib Magazine, February, 8, 2, http://www.dlib.org/dlib/february02/kirriemuir/02kirriemuir.html (accessed January 2003).
93. Kneale, P.E. (1997) The rise of the strategic student: how can we adapt to cope? in S. Armstrong, G. Thompson and S. Brown (eds) Facing up to Radical Changes in Universities and Colleges, London: Staff and Educational Development Agency, Kogan Page, pp. 119–39.
94. Knight, P. (2002a) Being a Teacher in Higher Education, Buckingham: Society for Research in Higher Education and Open University Press.
95. Knight, P. (2002b) Summative assessment in higher education: practices in disarray, Studies in Higher Education 27(2): 275–86.
96. Knight, P. and Trowler, P. (2001) Departmental Leadership in Higher Education, Buckingham: Society for Research in Higher Education and Open University Press.
97. Kniveton, B.H. (1996) A correlation analysis of multiple-choice and essay assessment measures, Research in Education 56: 73–84.
98. Kress, G. (1998) Visual and verbal modes of representation in electronically mediated communication: the potentials of new forms of text, in I. Snyder (1998) Page to Screen: Taking Literacy into the Electronic Era, London: Routledge, pp. 53–79.
99. Kuminek, P.A. and Pilkington, R.M. (2001) Helping the tutor facilitate debate to improve literacy using computer mediated communication, IEEE International Conference on Advanced Learning Technologies: Issues, Achievements and Challenges, Madison, Wisconsin, 6–8 August, IEEE Computer Society, ISBN 0–7695–1013–2.
100. Kvale, S. (1996) Interviews as knowledge construction, Qualitative Research: Space for Critique and Creativity Symposium, XXVI, International Congress of Psychology, Montreal, pp. 1–18.
101. Land, R. and Bayne, S. (2002) Screen or monitor? Issues of surveillance and disciplinary power in online learning environments, in C. Rust (ed.) Proceedings of Improving Student Learning Conference, Oxford: Oxford Centre for Staff and Learning Development, Oxford Brookes University.
102. Laurillard, D., Stratfold, M., Luckin, R., Plowman, L. and Taylor, J.(2000) Affordances for learning in a non-linear narrative medium, in Journal of Interactive Media in Education, 2, http://www-jime.open.ac.uk/00/2/ (accessed January 2003).
103. Laurillard, D. (1993) Rethinking University Teaching: a framework for the effective use of educational technology, London: Routledge.
104. Lea, M. (2000) Computer conferencing: new possibilities for writing and learning in Higher Education, in M. Lea and B. Stierer (2000) Student Writing in Higher Education, Buckingham: SRHE and Open University Press, pp. 69–85.
105. Lea, M. (2001) Computer conferencing and assessment, in Studies in Higher Education 26(2): 163–79.
106. Lillis, T. (2001) Student Writing: Access, regulation, desire, London: Routledge.
107. Lillis, T. and Turner, J. (2001) Student writing in higher education: contemporary confusion, traditional concerns, in Teaching in Higher Education 6(1): 57–68.
108. Lord, F. (1970) Some test theory for tailored testing, in Holtzman, W. H. (ed.) Computer-assisted instruction, testing, and guidance, pp. 139–83. New York: Harper Row.
109. MacDonald, C. (2002) Virtual Private Networks: an overview, in Intranet Journal, http://www.intranetjournal.com/foundation/vpn–1.shtml (accessed January 2003).
110. Mackenzie, D. (1999) Recent development in the Tripartite Interactive Assessment Delivery System (TRIADS) in M. Danson and R. Sherratt (eds) Proceedings of the 3rd International Computer-assisted Assessment Conference, 16–17 June 1999, Loughborough: Loughborough University, pp. 235–50 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
111. Mackenzie, D. and O’Hare, D. (2002) Empirical prediction of the measurement scale and base level ‘guess factor’ for advanced computer-based assessments, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: University of Loughborough, pp. 187- 201.
112. Malone, T. (1981) Toward a theory of intrinsically motivating instruction, in Cognitive Science 4: 333–69.
113. Mason, R. (1992) Methodologies for evaluating applications of computer conferencing, PLUM Report No. 31, Open University, Milton Keynes.
114. Massey, A.J. (1995) Evaluation and Analysis of Examination Data: Some guidelines for reporting and interpretation, Cambridge: University of Cambridge Local Examination Syndicate, Internal Report.
115. Matthews, J. (1981) The Use of Objective Tests, Lancaster: University of Lancaster Press (revised edn.)
116. Maughan, S., Peet, D. and Willmott, A. (2001) On-line formative assessment item banking and learning support, in M. Danson (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp. 335–46 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
117. McAlpine, M. (2001) Using computer-assisted assessment to facilitate resource selection, in M. Danson (ed.) Proceedings of the 5th International Computer assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp. 285–91 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
118. McAlpine, M. (2002a) A summary of methods of item analysis, Bluepaper Number 2, Computer-assisted Assessment Centre, http://www.caacentre.ac.uk ISBN1–904020–02-x (accessed January 2003).
119. McAlpine, M. (2002b) Design requirements of a databank, Bluepaper Number 3, Computer-assisted Assessment Centre, http://www.caacentre.ac.uk ISBN 1–904020–04–6 (accessed January 2003).
120. McBeath, R.J. (ed.) (1992) Instructing and Evaluating Higher Education: a guidebook for planning learning outcomes, Englewood Cliffs, New Jersey: Educational Technology Publications.
121. McGuire, G.R., Youngson, M.A., Korabinski, A.A. (2002) Partial credit in mathematics exams – a comparison of traditional and CAA exams, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: University of Loughborough, pp. 223–30.
122. McKenna, C. (2000) Using computers in humanities assessment, in Computers and Texts 18/19: 6–7.
123. McKenna, C. (2001) Who’s in control? Considering issues of power and control associated with the use of CAA: a discussion session in M. Danson (ed.)Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp. 305–8 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
124. McKenna, C. (2002) What is electronic literacy and where is it best positioned within the academic curriculum?, in C. Rust (ed.) Proceedings of Improving Student Learning Conference, Oxford: Oxford Centre for Staff and Learning Development, Oxford Brookes University.
125. McKenna, C. and Bull, J. (2000) Quality assurance of computer-assisted assessment: practical and strategic issues, in Quality Assurance in Higher Education 8(1): 24–31.
126. McKenna, C., Loewenberger, P., Bull, J. and Collins, C. (in press) Results of a national survey of computer-assisted assessment, Bluepaper No. 4, Computer assisted Assessment Centre, http://www.caacentre.ac.uk (accessed January 2003).
127. Meredyth, D., Russell, N., Blackwood, L., Thomas, J. and Wise, P. (1999) Real time, computers, change and schooling, Department of Education Training and Youth Affairs, Commonwealth of Australia. http://www.detya.gov.au/archive/schools/publications/1999/realtime.pdf (accessed January 2003).
128. Messick, C. (1989) Validity, in R.L. Linn (ed.) (1989) Educational Measurement (3rd edn), New York: Macmillan, pp. 13–103.
129. Morgan, C. and O’Reilly, M. (1994) Assessing Open and Distance Learners, London: Kogan Page.
130. Moss, P.A. (1995) Themes and variations in validity theory, in Educational Measurement: issues and practice 14(2): 5–12.
131. Moss, P.A, (1994) Can there be validity without reliability? Educational Researcher 23(2): 5–12.
132. Mulligan, B. (1999) Pilot study on the impact of frequent computerized assessment on student work rates, in M. Danson and R. Sherratt (eds) Proceedings of the 3rd International Computer-assisted Assessment Conference, 16–17 June 1999, Loughborough: Loughborough University, pp. 137–47 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
133. Newstead, S. and Dennis, I. (1994) Examiners examined: the reliability of exam marking in psychology, The Psychologist 7: 216–19.
134. OCSLD (1994) Strategies for Diversifying Assessment in Higher Education, Oxford: Oxford Centre for Staff and Learning Development, Oxford Brookes University.
135. Oliver, M. and Conole, G. (1998) Evaluating communication and information technologies: a toolkit for practitioners, Active Learning 8: 3–8.
136. Orsmond P., Merry S. and Reiling K. (1997) A study in self-assessment: tutor and students’ perceptions of performance criteria, Assessment and Evaluation in Higher Education 24(4): 357–69.
137. Outtz, J.L. (1998) Testing medium, validity and test performance, in M.D. Hakel (1998) Beyond Multiple Choice: evaluating alternative to traditional testing for selection, New Jersey: Lawrence Erlbaum Associates.
138. Page, E.B. and Peterson, N. (1995) The computer moves into essay grading: updating the ancient test, Phi Delta Kappa, March: 561–5.
139. Parlett, M.R. and Hamilton, D. (1972/77/87) Evaluation as illumination: a new approach to the study of innovatory programmes, (1972) workshop at Cambridge, and unpublished report Occasional paper 9, Edinburgh: Centre for Research in the Educational Sciences, University of Edinburgh.
140. Patton, M.Q. (1990) Qualitative Evaluation and Research Methods, London: Sage.
141. Pawson, R. and Tilley, N. (1997) Realistic Evaluation, London: Sage.
142. Paxton, M. (2000) A linguistic perspective on multiple choice questioning, in Assessment and Evaluation in Higher Education 25(2): 109–19.
143. Pennington, M.C. (1996) Writing the natural way on computer, Computer Assisted Language Learning 9: 2–3, 125–42.
144. Perkin, M. (1999) Validating formative and summative assessment in S. Brown, J. Bull, and P. Race (eds) Computer-Assisted Assessment in Higher Education, London: Kogan Page, pp. 57–70.
145. Pincas, A. (2000) New literacies and future educational culture, in ALT-J 8(2):69–79.
146. Pritchett, N. and Zakrzewski, S. (1996) Interactive computer assessment of large groups: student responses, Innovations in Education and Training International 33(3): 242–7.
147. Proctor, A. and Donoghue, D. (1994) Computer based assessment: a case study in geography, in Active Learning 1 December: 29–34.
148. Race, P. (1999) Why assess innovatively? in S. Brown and A. Glasner (eds) (1999) Assessment Matters in Higher Education, Buckingham: Society for Research in Higher Education and Open University Press.
149. Ramsden, P. (1992) Learning to Teach in Higher Education, Routledge: London.
150. Rasch, G. (1980). Probabilistic Models for some Intelligence and Attainment Tests, Chicago: University of Chicago Press (original work published 1960).
151. Reynolds, M. and Trehan, K. (2000) Assessment: a critical perspective, Studies in Higher Education 25(3): 267–78.
152. Ricketts, C. and Wilks, S. (2002) What factors affect student opinions of computer assisted assessment? in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 307–16 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
153. Robinson, J. (1999) Computer-assisted peer review in S. Brown, J. Bull, and P. Race (eds), Computer-Assisted Assessment in Higher Education, London:Kogan Page.
154. Rowland, S. (2000) The Enquiring University Teacher, Buckingham: Society for Research in Higher Education and Open University Press.
155. Royal College of Physicians (2002) MRPC (UK) Part One and Part Two Written Examinations: introduction of new format, http://www.mrcpuk.org/plain/changes.html (accessed January 2003).
156. Rudner, L. (1998) An on-line, interactive, computer adaptive testing mini-tutorial, ERIC Clearinghouse on Assessment and Evaluation Digest, http://www.eriche.org/ (accessed January 2003).
157. Schmidt, N.G., Norman, G.R. and Boshuzen, H.P.A. (1990) A cognitive perspective on medical expertise: theory and implications, Academic Medicine 65: 611–21.
158. Sclater, N., Low, B. and Barr, N. (2002) Interoperability with CAA: does it work in practice? in M. Danson (ed.) Proceedings of the 6th International Computer assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 317–27 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
159. Seale, C. (1999) The Quality of Qualitative Research, London: Sage Publications.
160. Shetzer, H. and Warschauer, M. (2000) An electronic literacy approach to network-based language teaching, in M. Warschauer and R. Kern (eds) (2000) Network-based Language Teaching: concepts and practice, New York: Cambridge University Press.
161. Smith, B. (1998) Adopting a strategic approach to managing change in learning and teaching in M. Kaplan (ed.) To Improve the Academy, 18, Stillwater, OK: New Forums Press and the Professional and Organizational Development Network in Higher Education.
162. Snyder, I. (1998) Page to Screen: Taking Literacy into the Electronic Era, London: Routledge.
163. Stephens, D., Bull, J. and Wade, W. (1998) Computer-assisted assessment: suggested guidelines for an institutional strategy, Assessment and Evaluation in Higher Education 23(3): 283–94.
164. Stephens, D., Sargent, G. and Brew, I. (2001) Comparison of assessed work marking software: implications for the ideal Integrated Marking Tool (IMT) in M. Danson (ed.) Proceedings of the 5th International Computer-assisted Assessment Conference 2–3 July 2001, Loughborough: Loughborough University, pp.471–86 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
165. Stern E. (1998) The Evaluation of Policy and the Policies of Evaluation, Tavistock Institute of Human Relations Annual Review, London: Tavistock Institute.
166. Stout, D.E. and Wygal, D.E. (1994) Additional empirical evidence on the relationship between exam sequencing and accounting student performance, Advances in Accounting 8: 133–52.
167. Stuart, S. (2002) An electronically enhanced philosophy learning environment, paper presented at Computing and Philosophy Conference. 8–10 August 2002, Carnegie Mellon.
168. Tait, H. and Entwistle, N. (1996) Identifying students at risk through ineffective study strategies in Higher Education 31: 97–116.
169. Taylor, C., Jamieson, J., Eignor, D. and Kirsch, I. (1998) The relationship between computer familiarity and performance on computer-based TOEFL test tasks, Research Report 61, March, New Jersey: Educational Testing Service, ftp://ftp.ets.org/pub/toefl/275757.pdf (accessed January 2003).
170. Thissen, D., Wainer, H. and Wang, S.–B. (1994) Are tests comprising both multiple-choice and free-response items necessarily less unidimensional than multiple-choice tests? An analysis of two tests, in Journal of Educational Measurement, 31(2): 113–23.
171. Vermunt, J.D. (1998) The regulation of constructive learning processes in British Journal of Educational Psychology, 68: 149–71.
172. Vermunt, J.D. and Verloop, N. (1999) Congruence and friction between learning and teaching in Learning and Instruction, 9: 257–80.
173. Wainer, H. (2000) Computerised Adaptive Testing: a primer, New Jersey: Lawrence Erlbaum Associates.
174. Ward, W.C., Frederiksen, N. and Carlson, S.B. (1980) Construct validity of freeresponse and machine-scorable forms of a test, Journal of Educational Measurement 7(1):11–29.
175. Whittington, C.D. and Campbell, L.M. (1999) Task-oriented learning on the web, Innovations in Education and Training International 36:1: 26–33.
176. Wood, J. and Burrow, M. (2002) Formative assessment in engineering using ‘TRIADS’ software, in M. Danson (ed.) Proceedings of the 6th International Computer-assisted Assessment Conference, 9–10 July 2002, Loughborough: Loughborough University, pp. 369–380 http://www.lboro.ac.uk/service/ltd/flicaa/conferences.html (accessed January 2003).
177. Wright, B.D. and Bell, S.R. (1984) Item banks: what, why, how, Memoranda 43, MESA Psychometric Laboratory, Chicago: University of Chicago http://www.rasch.org/memo43.htm (accessed January 2003).
178. Zakrzewski, S. and Bull, J. (1998) The mass implementation and evaluation of computer-based assessments, Assessment and Evaluation in Higher Education 23:2: 141–52.