Gilding the outcome by tarnishing the past: Inflationary biases in retrospective pretests

American Journal of Evaluation

Volumn 30, Issue 1, 2009, Pages 31-43

  Taylor, Paul J ^a   Russ Eft, Darlene F ^b   Taylor, Hazel ^c  

^a UNIVERSITY OF WAIKATO   (New Zealand)

^b OREGON STATE UNIVERSITY   (United States)

^c UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Pretest bias; Pretests; Response shift bias; Retrospective pretests

Indexed keywords

EID: 61349151751     PISSN: 10982140     EISSN: 15570878     Source Type: Journal    
DOI: 10.1177/1098214008328517     Document Type: Article

References (40)

1. Birkenbach, X.C. (1986). Measuring alpha, beta, and gamma change. South African Journal of Psychology, 16, 1-7.
2. Bray, J.H., & Howard, G.S. (1980). Methodological considerations in the evaluation of a teacher-training program. Journal of Educational Psychology, 72, 62-70.
3. Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.
4. Conway, M., & Ross, M. (1984). Getting what you want by revising what you had. Journal of Personality and Social Psychology, 47, 738-748.
5. Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Boston, MA: Houghton Mifflin.
6. Cook, T.D., Campbell, D.T., & Peracchio, L. (1990). Quasi experimentation. In M. D, Dunnette & L. M, Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 491-576). Palo Alto, CA: Consulting Psychologists Press.
7. Decker, P.J., & Nathan, B.R. (1985). Behavior modeling training: Principles and applications. New York: Praeger.
8. Goldstein, A.P., & Sorcher, M.A. (1974). Changing supervisory behavior. New York: Pergoman Press.
9. Golembiewski, R.T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12, 133-157.
10. Hill, L.G., & Betz, D.L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26, 501-517.
11. Hoogstraten, J. (1982). The retrospective pretest in an educational training context. Journal of Experimental Education, 50, 200-204.
12. Hoogstraten, J. (1985). Influence of objective measures on self-reports in a retrospective pretest-posttest design. Journal of Experimental Education, 53, 207-210.
13. Howard, G.S., & Dailey, P.R. (1979). Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology, 64, 144-150.
14. Howard, G.S., Dailey, P.R., & Gulanick, N.A. (1979). The feasibility of informed pretests in attenuating response-shift bias. Applied Psychology Measurement, 3, 1-23.
15. Howard, G.S., Ralph, K.M., Gulanick, N.A., Maxwell, S.E., Nance, D.W., & Gerber, S.K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychological Measurement, 3, 1-23.
16. Howard, G.S., Schmeck, R.R., & Bray, J.H. (1979). Internal invalidity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement, 16, 129-135.
17. Krosnick, J.A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
18. Lam, T.C.M., & Bengo, P. (2003). A comparison of three retrospective self-reporting methods of measuring change in instructional practice. American Journal of Evaluation, 24, 65-80.
19. Menon, G. (1994). Judgments of behavioral frequencies: Memory search and retrieval strategies. In Schwarz. N, & Sudman. S (Eds.), Autobiographical memory and the validity of retrospective reports (pp. 161-172). New York: Springer-Verlag.
20. Paulhus, D.L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, & D. N. Jackson (Eds.), The role of constructs in psychological and educational measurement (pp. 49-69). Mahwah, NJ: Lawrence Erlbaum.
21. Pohl, N.F. (1982). Using retrospective pre-ratings to counteract response-shift confounding. Journal of Experimental Education, 51, 211-214.
22. Pratt, C.C., Mcguigan, W.M., & Katzev, A.R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341-350.
23. Rhodes, J.E., & Jason, L.A. (1987). The retrospective pretest: An alternative approach in evaluating drug intervention programs. Journal of Drug Education, 17, 345-356.
24. Rice, R.E., & Contractor, N.S. (1990). Conceptualizing effects of office information systems: A methodology and application for the study of alpha, beta, and gamma changes. Decision Sciences, 21, 301-317.
25. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341-357.
26. Schwarz, N. (2007). Cognitive aspects of survey methodology. Applied Cognitive Psychology, 21, 277-287.
27. Schwarz, N., & Oyserman, D. (2001). Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation, 22, 127-160.
28. N. Schwarz, & S. Sudman (Eds.), (1994). Autobiographical memory and the validity of retrospective reports. New York: Springer-Verlag.
29. Sibthorp, J., Paisley, K., Gookin, J., & Ward, P. (2007). Addressing response-shift bias: Retrospective pretests in recreation research and evaluation. Journal of Leisure Research, 39, 295-315.
30. Sprangers, M., & Hoogstraten, J. (1988). On delay and reassessment of retrospective preratings. Journal of Experimental Education, 56, 148-153.
31. Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest designs. Journal of Applied Psychology, 74, 265-272.
32. Sprangers, M.A.G., & Hoogstraten, J. (1991). Subject bias in three self-report measures of change. Methodika, 5, 1-13.
33. Stieglitz, R.-D. (1990). Studies on the validity of the retrospective pretest in therapy research. Zeitschrift fur Klinische Psychologie, 19, 144-150.
34. Taylor, P.J., Russ-Eft, D., & Taylor, H. (in press). The transfer of management training from alternative perspectives. Journal of Applied Psychology.
35. Terborg, J.R., & Davis, G.A. (1982). Evaluation of a new method for assessing change to planned job redesign and applied to Hackman and Olham's job characteristics model. Organizational Behavior and Human Performance, 29, 112-128.
36. Tourangeau, R. (2000). Remembering what happened: Memory errors and survey reports. In Stone. A. A, Turkkan. J. S, Bachrach. C. A, Jobe. J. B, Kurtzman. H. S, & Cain. V. S (Eds.), The science of self-report: Implications for research and practice. Mahwah, NJ: Lawrence Erlbaum.
37. Tourangeau, R., Rips, L.J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
38. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.
39. Wilson, A.E., & Ross, M. (2001). From chump to champ: People's appraisals of their earlier and present selves. Journal of Personality and Social Psychology, 80, 572-584.
40. Wilson, D.B., & Lipsey, M.W. (2001). The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Methods, 6, 413-429.