Testing reasoning ability with handheld computers, notebooks, and paper and pencil

European Journal of Psychological Assessment

Volumn 26, Issue 4, 2010, Pages 284-292

  Schroeders, Ulrich ^a   Wilhelm, Oliver ^a  

^a HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Computerized testing; Equivalence; Handheld computer; Reasoning ability; Test media

Indexed keywords

EID: 77956998434     PISSN: 10155759     EISSN: None     Source Type: Journal    
DOI: 10.1027/1015-5759/a000038     Document Type: Article

References (32)

1. American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least square estimation in confirmatory factor analysis. Structural Equation Modeling, 13, 186-203.
3. Borsboom, D., & Mellenbergh, G. J. (2002). True scores, latent variables, and constructs: A comment on Schmidt and Hunter. Intelligence, 30, 505-514.
4. Bridgeman, B., Lennon, M. L., & Jackenthal, A. (2003). Effects of screen size, screen resolution, and display rate on computer-based test performance. Applied Measurement in Education, 16, 191-205.
5. Carpenter, P. A., Just, M., & Shell, P. (1990). What one intelligence test measures: A theoretical account of processing in the Raven's progressive matrices test. Psychological Review, 97, 404-431.
6. Carroll, J. B. (1993). Human cognitive abilities: A survey of factoranalytic studies. Cambridge, MA: Cambridge University Press.
7. Drasgow, F., & Mattern, K. (2005). New tests and new items: Opportunities and issues. In D. Bartram & R. K. Hambleton (Eds.), Computer-based testing and the Internet: Issues and advances. New York: Wiley.
8. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241-261.
9. Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396.
10. Farcot, M., & Latour, T. (2009). Transitioning to computer-based assessments: A question of costs. In F. Scheuermann & J. Björnsson (Eds.), The transition to computer-based assessment (pp. 108-116). Office for Official Publications of the European Communities, Luxembourg. Retrieved from http://crell.jrc.it/RP/reporttransitionpdf
11. International Test Commission (ITC). (2005). International guidelines on computer-based and Internet-delivered testing. Retrieved from http://www.intestcom.org/guidelines
12. Irvine, S. H., & Kylonnen, P. C. (2002). The foundations of item generation for mass testing. In S. H. Irvine & P. C. Kylonnen (Eds.), Item generation for test development (pp. 3-34). Mahwah, NJ: Erlbaum.
13. Johnson-Laird, P. N. (1997). Rules and illusions: A critical study of Rip's The Psychology of Proof. Minds and Machines, 7, 387-407.
14. Kim, J.-P. (1999, October). Meta-analysis of equivalence of computerized and P&P tests on ability measures. Paper presented at the annual meeting of the Midwestern Educational Research Association, Chicago, IL.
15. Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity? Intelligence, 14, 389-433.
16. Leeson, H. V. (2006). The mode effect: A literature review of human and technological issues in computerized testing. International Journal of Testing, 6, 1-24.
17. Lustig, C., Hasher, L., & Tonev, S. T. (2006). Distraction as a determinant of processing speed. Psychonomic Bulletin and Review, 13, 619-625.
18. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
19. Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive-ability tests: A meta-analysis. Psychological Bulletin, 114, 449-458.
20. Neuman, G., & Baydoun, R. (1998). Computerization of paperand-pencil tests: When are they equivalent? Applied Psychological Measurement, 22, 71-83.
21. Overton, R. C., Taylor, L. R., Zickar, M. J., & Harms, H. J. (1996). The pen-based computer as an alternative platform for test administration. Personnel Psychology, 49, 455-464.
22. Primi, R. (2001). Complexity of geometric inductive reasoning tasks: Contribution to the understanding of fluid intelligence. Intelligence, 30, 41-70.
23. Schroeders, U. (2009). Testing for equivalence of test data across media. In F. Scheuermann & J. Björnsson (Eds.), The transition to computer-based assessment (pp. 164-170). Office for Official Publications of the European Communities, Luxembourg. Retrieved from http://crell.jrc.it/RP/ reporttransitionpdf
24. Schroeders, U., Wilhelm, O., & Schipolowski, S. (2010). Internet-based ability testing: Problems and opportunities. In S. D. Gosling & J. A. Johnson (Eds.), Advanced methods for behavioral research on the Internet (pp. 131-148). Washington, DC: American Psychological Association.
25. Segall, N., Doolen, T. L., & Porter, J. D. (2004). A usability comparison of PDA-based quizzes and paper-and-pencil quizzes. Computers and Education, 45, 417-432.
26. Spearman, C. (1904). "General intelligence", objectively determined and measured. American Journal of Psychology, 15, 201-293.
27. Taylor, C., Kirsch, I., Eignor, D., & Jamieson, J. (1999). Examining the relationship between computer familiarity and performance on computer-based language tasks. Language Learning, 49, 219-274.
28. Tseng, H.-M., Tiplady, B., Macleod, H. A., & Wright, P. (1998). Computer anxiety:A comparison of pen-based personal digital assistants, conventional computer, and paper assessment of mood and performance. British Journal of Psychology, 89, 599-610.
29. van Lent, G. (2008). Important considerations in e-assessment. In F. Scheuermann & A. Guimarães Pereira (Eds.), Toward a research agenda on computer-based assessment (pp. 97-103). Office for Official Publications of the European Communities, Luxembourg. Retrieved from http://crell.jrc.it/CBA/EU- Report-CBApdf
30. Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2007). A meta-analysis of testing mode effects in grade K-12 mathematics tests. Educational and Psychological Measurement, 67, 219-238.
31. Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2008). Comparability of computer-based and paper-and-pencil testing in K-12 reading assessments: A meta-analysis of testing mode effects. Educational and Psychological Measurement, 68, 5-24.
32. Wilhelm, O. (2005). Measuring reasoning ability. In O. Wilhelm & R. W. Engle (Eds.), Understanding and measuring intelligence (pp. 373-392). London: Sage.