Toward an integrated model for designing assessment systems: An analysis of the current status of computer-based assessments in science

Computers and Education

Volumn 68, Issue , 2013, Pages 388-403

  Kuo, Che Yu ^a   Wu, Hsin Kai ^a  

^a NATIONAL TAIWAN NORMAL UNIVERSITY   (Taiwan)

Author keywords

Architectures for educational technology system; Post secondary education; Secondary education; Simulations

Indexed keywords

ARCHITECTURES FOR EDUCATIONAL TECHNOLOGY SYSTEM; COMPUTER-BASED ASSESSMENTS; FORMATIVE AND SUMMATIVE ASSESSMENTS; INTERACTIVE MULTIMEDIA; ITEM RESPONSE THEORY; POSTSECONDARY EDUCATION; PROBLEM SOLVING SKILLS; SIMULATIONS;

INTERACTIVE COMPUTER SYSTEMS; MULTIMEDIA SYSTEMS;

COMPUTER AIDED ANALYSIS;

EID: 84879960065     PISSN: 03601315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.compedu.2013.06.002     Document Type: Article

References (90)

1. B. Adrian, and D. Beverley Computer managed student assessment: a case study The Canadian Journal for the Study of Adult Education 13 1 1999 41 60
2. T. Ager Online placement testing in mathematics and chemistry Journal of Computer-based Instruction 20 2 1993 52 57
3. H.S. Ashton, C.E. Beevers, A.A. Korabinski, and M.A. Youngson Investigating the medium effect in computer-aided assessment of school chemistry and college computing national examinations British Journal of Educational Technology 36 5 2005 771 787 (Pubitemid 43906763)
4. G. Baggott, and R.C. Rayne The use of computer-based assessment in a field biology module Bioscience Education Journal 9 2007 Retrieval June 27, 2012, from http://www.bioscience.heacademy.ac.uk/journal/vol9/beej-9-5.pdf
5. S. Basu, C. Roberts, D.I. Newble, and M. Snaith Competence in the musculoskeletal system: assessing the progression of knowledge through an undergraduate medical course Medical Education 38 2004 1253 1260 (Pubitemid 39647376)
6. C. Bax, G. Baggott, E. Howey, C. Pellet-Many, E. Rayne, and M. Neonaki Evaluation of formative computer-based assessment by cell biology students with differing entry qualifications and ethnicity Bioscience Education Journal 8 2006 Retrieval June 27, 2012, from http://www.bioscience.heacademy.ac.uk/journal/ vol8/beej-8-5.pdf
7. R.E. Bennett, and I.I. Bejar Validity and automated scoring: it's not only the scoring Educational Measurement: Issues and Practices 1998 9 17
8. R.E. Bennett, M. Goodman, J. Hessinger, H. Kahn, J. Ligget, and F. Marshall Using multimedia in large-scale computer-based testing programs Computers in Human Behavior 15 1999 283 294
9. R.E. Bennett, H. Persky, A. Weiss, and F. Jenkins Measuring problem solving with technology: a demonstration study for NAEP Journal of Technology, Learning, and Assessment 8 8 2010 Retrieved January 7, 2012 from http://www.jtla.org
10. A.K. Bersky Effect of audiovisual enhancement on problem-solving and decision-making activities during a computerized clinical simulation test (CST™) of nursing competence Evaluation & the Health Professions 17 4 1994 446 464 (Pubitemid 24351464)
11. Black Testing: Friend or foe? Theory and practice of assessment and testing 1998 Falmer Press New York, NY
12. K.C. Bloom, and L.B. Trice The efficacy of individualized computerized testing in nursing education Computers in Nursing 15 2 1997 82 88 (Pubitemid 127758295)
13. C.W. Bowen Item design considerations for computer-based testing of student learning in chemistry Journal of Chemical Education 75 9 1998 1172 1175 (Pubitemid 128590206)
14. C.A. Brewer Near real-time assessment of student learning and understanding in biology course BioScience 54 11 2004 1034 1039 (Pubitemid 39471333)
15. B. Bridgeman Experiences from large-scale computer-based testing in the USA F. Scheuermann, J. Björnsson, The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing 2009 European Communities Luxembourg 39 44
16. J.E. Brown, S.R. Hinze, and J.W. Pellegrino Technology and formative assessment T.L. Good, 21st century education Technology vol. 2 2008 Sage Thousand Oak, CA 245 255
17. C. Bunderson, D. Inouye, and J. Olsen The four generations of computerized education measurement R.L. Linn, Educational measurement 3rd ed. 1989 American Council on Education and Macmillan Publishers New York, NY 367 407
18. P. Caudle, J. Bigness, J. Daniels, M. Gillmore-Kahn, and J. Knestrick Implementing computer-based testing in distance education for advanced practice nurses: lessons learned Nursing Education Perspectives 32 5 2011 328 332
19. K.E. Chang, Y.T. Sung, R.B. Chang, and S.C. Lin A new assessment for computer-based concept mapping Educational Technology & Society 8 3 2005 138 148
20. H. Chen, and C. Chuang The learning effectiveness of nursing students using online testing as an assistant tool: a cluster randomized controlled trial Nurse Education Today 32 3 2012 208 213
21. G. Chung, L.F. de Vries, A.M. Cheak, R.H. Stevens, and W.L. Bewley Cognitive process validation of an online problem solving assessment Computers in Human Behavior 18 6 2002 669 684
22. B.E. Clauser, M.J. Margolis, S.G. Clyman, and L.P. Ross Development of automated scoring algorithms for complex performance assessments Journal of Educational Measurement 34 1997 141 161
23. S.S. Cook, R. Kase, L. Middelton, and R.B. Monsen Portfolio evaluation for professional competence: credentialing in genetics for nurses Journal of Professional Nursing 19 2 2003 85 90 (Pubitemid 36656580)
24. K.J. Crippen, and B. Earl Considering the efficacy of web-based worked examples in introductory chemistry Journal of Computers in Mathematics and Science Teaching 23 2 2004 151 167
25. G. Dam, and S. Kaufmann Computer assessment of interview data using latent semantic analysis Behavior Research Methods 40 1 2008 8 20
26. T. Deutsch, K. Herrmann, T. Frese, and H. Sandolzer Implementing computer-based assessment - a web-based mock examination changes attitudes Computers & Education 58 2012 1068 1075
27. F. Drasgow, and K. Mattern New tests and new items: opportunities and issues D. Bartram, R. Hambleton, Computer-based testing and the internet: Issues and advances 2006 Wiley London, UK 59 76
28. K. Dunbar What scientific thinking reveals about the nature of cognition K. Crowley, C. Schunn, T. Okada, Designing for science: Implications from everyday, classroom, and professional settings 2001 Erlbaum Mahwah, NJ 21 49
29. G. Duque Web-based evaluation of medical clerkships: a new approach to immediacy and efficacy of feedback and assessment Medical Teacher 25 5 2003 510 514 (Pubitemid 37270569)
30. R.A. Duschl, H.A. Schweingruber, and A.W. Shouse Taking science to school: Learning and teaching science in grades K-8 2007 National Academy Press Washington, DC
31. J.T. Fitzgerald, F.M. Wolf, W.K. Davis, M.L. Barclay, M.E. Bozynski, and K.R. Chamberlain A preliminary study of the impact of case specificity on computer-based assessment of medical student clinical performance Evaluation & The Health Professions 17 3 1994 307 321 (Pubitemid 24265487)
32. D. Fry The database format question: an alternative to multiple choice and free format for computer based testing Computers & Education 14 5 1990 395 401
33. S. Ganguli, M. Camacho, C.S. Yam, and I. Pedrosa Preparing first-year radiology residents and assessing their readiness for on-call responsibilities: results over 5 years American Journal of Roentgenology 192 2 2009 534 539
34. M.J. Gilmer, J. Murley, and E. Kyzer Web-based testing procedure for nursing students Journal of Nursing Education 42 8 2003 377 380 (Pubitemid 40081583)
35. G.J. Hwang, C.L. Hsiao, and C.R. Tseng A computer-assisted approach to diagnosing student learning problem in science course Journal of Information Science and Engineering 19 2 2003 229 248
36. S. Jordan Online interactive assessment in teaching science: a view from the Open University Education in Science 231 2009 16 17
37. S. Jordan Using interactive computer-based assessment to support beginning distance learners of science Open Learning: The Journal of Open and Distance Learning 26 2 2011 147 164
38. D. Klahr Exploring science: The cognition and development of discovery processes 2000 MIT Press Cambridge, MA
39. R. Kozma Transforming education: assessing and teaching 21st century skills F. Scheuermann, J. Björnsson, The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing 2009 European Communities Luxembourg 13 23
40. S. Krasne, P.F. Wimmers, A. Relan, and T.A. Drake Differential effects of two types of formative assessment in predicting performance of first-year medical students Advances in Health Sciences Education 11 2006 155 171 (Pubitemid 43806865)
41. C.D. Kreiter, K. Ferguson, and L.D. Gruppen Evaluating the usefulness of computerized adaptive testing for medical in-course assessment Academic Medicine 74 10 1999 1125 1128 (Pubitemid 29493388)
42. R. Kuo, W. Lien, M. Chang, and J. Heh Analyzing problem's difficulty based on neural networks and knowledge map Educational Technology & Society 7 2 2004 42 50 (Pubitemid 38817112)
43. F. Lazarinis, S. Green, and E. Pearson Creating personalized assessments based on learner knowledge and objectives in a hypermedia web testing application Computers & Education 55 4 2010 1732 1743
44. G. Lee, and P. Weerakoon The role of computer-aided assessment in health professional education: a comparison of student performance in computer-based and paper-and-pen multiple choice tests Medical Teacher 23 2 2001 152 157 (Pubitemid 32288377)
45. S.A. Lieberman, A.W. Frye, S.D. Litwins, K.A. Rasumusson, and J.R. Boulet Introduction of patient video clips into computer-based testing: effects on item statistics and reliability estimates Academic Medicine 78 10 2003 48 51
46. E.C. Lim, R.C. Seet, V.M. Oh, B. Chia, M. Aw, and S. Quak Computer-based testing of the modified essay question: the Singapore experience Medical Teacher 28 2007 261 268
47. R.M. Luecht Multidimensional computerized adaptive testing in a certification or licensure context Applied Psychological Measurement 20 4 1996 389 404
48. R.M. Luecht, and R.J. Nungster Some practical examples of computer-adaptive sequential testing Journal of Educational Measurement 35 3 1998 229 249 (Pubitemid 128184242)
49. M.E. Lunz, and B.A. Bergstrom The effect of review on student ability and test efficiency for computerized adaptive tests Applied Psychological Measurement 16 1 1992 33 40
50. M.E. Lunz, and B.A. Bergstrom An empirical study of computerized adaptive test administration conditions Journal of Educational Measurement 31 3 1994 251 263
51. M.E. Lunz, and B. Bergstrom Computerized adaptive testing: tracking candidate response patterns Journal of Educational Computing Research 13 2 1995 151 162 (Pubitemid 126049316)
52. S. Messick The interplay of evidence and consequences in the validation of performance assessments Educational Researcher 23 2 1995 13 23
53. D. Micklewright, L. Pearsall, M. Sellens, and N. Billam Changes in approaches to learning among undergraduate sports science students following a programme of weekly online assessments Journal of Hospitality, Leisure, Sport and Tourism Education 9 2 2010 141 155
54. R.T. Millos, D.L. Gordon, S.B. Issenberg, P.S. Reynolds, S.L. Lewis, and W.C. McGahie Development of a reliable multimedia, computer-based measure of clinical skills in bedside neurology Academic Medicine 78 10 2003 52 54
55. E.E.S. Mina, C. Eifert, M. Ireland, C. Fine, V. Micevski, and R. Wojtiuk The development of an online instrument for prior learning assessment and recognition of internationally educated nurses: a pilot study International Review of Research in Open and Distance Learning 12 1 2011 100 110
56. National Research Council Knowing what students know: the science and design of educational assessment. Committee on the Foundations of Assessment J. Pellegrino, N. Chudowsky, R. Glaser, Board on testing and assessment, Center for education. Division of Behavioral and social sciences and education 2001 National Academy Press Washington, DC
57. G. Norman Research in clinical reasoning: past history and current trends Medical Education 39 4 2005 418 427
58. OECD PISA computer-based assessment of student skills in science 2010 OECD Paris, France
59. R.W. Ogilvie, C.T. Trusk, and A. Blue Students' attitudes towards computer testing in a basic science course Medical Education 33 1999 828 831 (Pubitemid 29520652)
60. C.G. Parshall, T. Davey, and P. Pashley Innovative item types for computerized testing W.J. van der Linden, C.A.W. Glas, Computerized adaptive testing: Theory and practice 2000 Kluwer Academic Publishers Netherlands 129 148
61. H. Pashler, P. Bain, B. Bottage, A. Graesser, K. Koedinger, and M. McDaiel Organizing instruction and study to improve student learning (No. NCER 2007-2004) 2007 National Center for Educational Research, Institute of Education Sciences, U.S. Department of Education Washington, DC
62. M. Peat, and S. Franklin Supporting student learning: the use of computer-based formative assessment modules British Journal of Educational Technology 33 5 2002 515 523
63. M.W. Peterson, J. Gordon, S. Elliott, and C. Kreiter Computer-based testing: initial report of extensive use in a medical school curriculum Teaching and Learning in Medicine 16 1 2004 51 59 (Pubitemid 38292932)
64. E.S. Quellmalz, M.J. Timms, M.D. Silberglitt, and B.C. Buckley Science assessments for all: integrating science simulations into balanced state science assessment systems Journal of Research in Science Teaching 49 3 2012 363 393
65. M. Richardson, J.A. Baird, J. Ridgway, M. Ripley, D. Shorrocks-Taylor, and M. Swan Challenging minds? Students' perceptions of computer-based world class tests of problem solving Computers in Human Behavior 18 6 2002 633 649
66. K. Scalise, D.J. Bernbaurn, M. Timms, S.V. Harrell, K. Burmester, and C.A. Kennedy Adaptive technology for e-learning: principles and case studies of an emerging field Journal of the American Society for Information Science and Technology 58 14 2007 2295 2309
67. K. Scalise, M. Timms, A. Moorjani, L. Clark, K. Holtermann, and P.S. Irvin Student learning in science simulations: design features that promote learning gains Journal of Research in Science Teaching 48 9 2011 1050 1078
68. J. Schacter, H.E. Herl, G.K.W.K. Chung, R.A. Dennis, and H.F. O'Neil Jr. Computer-based performance assessments: a solution to the narrow measurement and reporting of problem-solving Computers in Human Behavior 15 1999 403 418
69. L. Schuwirth, C. van der Vleuten, H. Stoffers, and A. Peperkamp Computerized long-menu questions as an alternative to open-ended questions in computerized assessment Medical Education 30 1996 50 55 (Pubitemid 26097097)
70. R.E. Shaw, J.A. Effken, B.R. Fajen, S.R. Garrett, and A. Morris An ecological approach to the on-line assessment of problem-solving paths: principles and applications Instructional Science 25 2 1997 151 166
71. E. Sheader, I. Gouldsborough, and R. Grady Staff and student perceptions of computer-assisted assessment for physiology practical classes Advances in Physiology Education 30 2006 174 180 (Pubitemid 44749376)
72. A.N. Siriwardena, H. Dixon, C. Blow, B. Irish, and P. Milne Performance and views of examiners in the applied knowledge test for the nMRCGP licensing examination British Journal of General Practice 59 2009 38 43
73. C. Smaill The implementation and evaluation of OASIS: a web-based learning and assessment tool for large classes IEEE Transactions on Education 48 4 2005 658 663 (Pubitemid 41723898)
74. R. Stevens, J. Ikeda, A. Casillas, J. Palacio-Cayetano, and S. Clyman Artificial neural network-based performance assessments Computers in Human Behavior 15 1999 295 313
75. E.M. Taricani, and R.B. Clariana A technique for automatically scoring open-ended concept maps Educational Technology Research and Development 54 1 2006 65 82 (Pubitemid 43587832)
76. A. Thin Using online microassessments to drive student learning Bioscience Education 7 2006 Retrieved January 7, 2012 from http://www. bioscience.heacademy.ac.uk/journal/vol7/beej-7
77. A. Thissen-Roe, E. Hunt, and J. Minstrell The DIAGNOSER project: combining assessment and learning Behavior Research Methods, Instruments, & Computers 36 2 2004 234 240 (Pubitemid 39196121)
78. N.A. Thompson, and D.J. Weiss Computerized and adaptive testing in educational assessment F. Scheuermann, J. Björnsson, The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing 2009 European Communities Luxembourg 127 133
79. R.L. Thorndike Measurement and evaluation in psychology and education 1997 Prentice Hall Upper Saddle River, NJ
80. D.L. Trumpower, and G.S. Sarwar Effectiveness of structural feedback provided by pathfinder networks Journal Educational Computing Research 43 1 2010 7 24
81. G. Velan, P. Jones, H. McNeil, and R. Kumar Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning BMC Medical Education 8 2008 Retrieved January 7, 2012 from http://www.biomedcentral.com/1472-6920/8/52
82. T. Vendlinski, and R. Stevens Assessing student problem-solving skills with complex computer-based tasks Journal of Technology, Learning, and Assessment 1 3 2002 Available from: http://www.jtla.org
83. T.H. Wang What strategies are effective for formative assessment in an e-learning environment? Journal of Computer Assisted Learning 23 2007 171 186
84. T.H. Wang Web-based quiz-game-like formative assessment: development and evaluation Computers & Education 51 2008 1247 1263
85. K.H. Wang, T.H. Wang, W.L. Wang, and S.C. Huang Learning styles and formative assessment strategy: enhancing student achievement in web-based learning Journal of Computer Assisted Learning 22 2006 207 217 (Pubitemid 43937512)
86. T.H. Wang, K.H. Wang, W.L. Wang, S.C. Huang, and S.Y. Chen Web-based Assessment and Test Analyses (WATA) system: development and evaluation Journal of Computer Assisted Learning 20 1 2004 59 71 (Pubitemid 38883638)
87. N.A. Williams, W. Bland, and G. Christie Improving student achievement and satisfaction by adopting a blended learning approach to inorganic chemistry Chemistry Education Research and Practice 9 2008 43 50
88. H.C. Wu, C.Y. Chang, C.L.D. Chen, T.K. Yeh, and C.C. Liu Comparison of earth science achievement between animation-based and graphic-based testing designs Research in Science Education 40 5 2010 639 673
89. H.-K. Wu, S.W.-Y. Lee, H.-Y. Chang, and J.-C. Liang Current status, opportunities and challenges of augmented reality in education Computers Education 62 2013 41 49
90. A.L. Zenisy, and S.G. Sireci Technological innovations in large scale assessment Applied Measurement in Education 15 4 2002 337 362