Controlling response shift bias: The use of the retrospective pre-test design in the evaluation of a master's programme

Assessment and Evaluation in Higher Education

Volumn 33, Issue 6, 2008, Pages 699-709

  Drennan, Jonathan ^a   Hyde, Abbey ^a  

^a UNIVERSITY COLLEGE DUBLIN   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 75649103708     PISSN: 02602938     EISSN: 1469297X     Source Type: Journal    
DOI: 10.1080/02602930701773026     Document Type: Article

References (34)

1. Bray, J., S. Maxwell, and G. Howard. 1984. Methods of analysis with response-shift bias. Educational and Psychological Measurement 44: 781-804.
2. Cantrell, P. 2003. Traditional vs. retrospective pre-tests for measuring science teaching efficacy beliefs in preservice teachers. School Science and Mathematics 103: 177-185.
3. Cohen, J. 1988. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum.
4. Dillman, D. 2000. Mail and internet surveys: The tailored design approach. 2nd ed. New York: Wiley.
5. Drennan, J. 2003. Cognitive interviews: Verbal data in the development and pre-testing of questionnaires. Journal of Advanced Nursing 42: 57-63.
6. Farel, A., K. Umble, and B. Polhamus. 2001. Impact of an online analytic skills course. Evaluation and the Health Professions 24: 446-459.
7. Field, A. 2005. Discovering dtatistics Using SPSS. 2nd ed. London: Sage Publications.
8. Gal, I., and L. Ginsburg. 1994. The role of beliefs and attitudes in learning statistics: Towards an assessment framework. Journal of Statistics Education 2: 1-15.
9. Garfield, J., and A. Ahlgren. 1988. Difficulties in learning basic concepts in probability and statistics: Implications for research. Journal for Research in Mathematics Education 19: 44-63.
10. Goedhart, H., and J. Hoogstraten. 1992. The retrospective pre-test and the role of pre-test information in evaluative studies. Psychological Reports 70: 699-704.
11. Hoogstraten, J. 1982. The retrospective pre-test in an educational training context. Journal of Experimental Education 50: 200-204.
12. Howard, G. 1980. Response shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review 4: 93-106.
13. Howard, G., and P. Dailey. 1979. Response shift bias: A source of contamination of self-report measures. Journal of Applied Psychology 64: 144-150.
14. Howard, G., R. Schmeck, and J. Bray. 1979. Internal invalidity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement 16: 129-135.
15. Lam, T., and P. Bengo. 2002. A comparison of three retrospective self-reporting methods of measuring change in instructional practice. American Journal of Evaluation 24: 65-80.
16. Leech, N., K. Barrett, and G. Morgan. 2005. SPSS for intermediate statistics: Use and interpretation. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum.
17. Murtonen, M., and E. Lehtinen. 2003. Difficulties experienced by education and sociology students in quantitative methods courses. Studies in Higher Education 28: 171-185.
18. Pohl, N. 1982. Using retrospective pre-ratings to counteract response-shift confounding. Journal of Experimental Education 50: 211-214.
19. Pratt, C, W. McGuigan, and A. Katzev. 2000. Measuring program outcomes: Using retrospective pre-test methodology. American Journal of Evaluation 21: 341-349.
20. Rohs, F. 1999. Response shift bias: A problem in evaluating leadership development with self-report pretest-post-test measures. Journal of Agricultural Education 40: 28-37.
21. Rolls, F. 2002. Improving the evaluation of leadership programs: Control response shift. Journal of Leadership Education 1: 1-12.
22. Shadish, W., T. Cook, and D. Campbell. 2002. Experimental and quasi-experimental designs for generalised causal inference. Boston, MA: Houghton Mifflin.
23. Sprangers, M. 1988a. Response shift and the retrospective pretest: On the usefulness of retrospective pretest-posttest designs in detecting training related response shifts. Doctoral diss., Het instituut van Onderzoek van het Onderwijs S.V.O., 'S Gravenhage.
24. Sprangers, M. 1988b. A further note on the necessity of including retrospective pre-test in self-report pre-test-post-test designs to detect training effectiveness. Tijdschrift voor Onderwijs Research 13: 353-355.
25. Sprangers, M. 1989a. Response-shift bias in program evaluation. Impact Assessment Bulletin 7: 153-166.
26. Sprangers, M. 1989b. Subject bias and the retrospective pre-test in retrospect. Bulletin of the Psychonomic Society 27: 11-14.
27. Sprangers, M., and J. Hoogstraten. 1987. Response-style effects, response-shift bias and bogus-pipeline. Psychological Reports 61: 579-585.
28. Sprangers, M., and J. Hoogstraten. 1988a. On delay and reassessment of retrospective ratings. Journal of Experimental Education 56: 148-153.
29. Sprangers, M., and J. Hoogstraten. 1988b. Response-style effects, response-shift bias and bogus-pipeline: A replication. Psychological Reports 62: 11-16.
30. Sprangers, M., and J. Hoogstraten. 1989. Pretesting effects in retrospective pre-test-post-test designs. Journal of Applied Psychology 74: 265-272.
31. Sprangers, M., and J. Hoogstraten. 1991. Subject bias in three self-report measures of change. Methodika 5: 1-13.
32. Townsend, M., D. Moore, B. Tuck, and K. Wilton. 1998. Self-concept and anxiety in university students studying social science statistics within a co-operative learning structure. Educational Psychology 18: 41-54.
33. Townsend, M., and K. Wilton. 2003. Evaluating change towards mathematics using the 'then-now' procedure in a cooperative learning programme. British Journal of Educational Psychology 73: 473-487.
34. Umble, K., V. Upshaw, S. Orton, and M. Kelly. 2000. Using the post-then method to assess learner change. Presentation at the American Association of Higher Education Assessment Conference, North Carolina, June.