Measuring change in a short-term educational program using a retrospective pretest design

American Journal of Evaluation

Volumn 30, Issue 2, 2009, Pages 189-202

  Moore, Debra ^a   Tananis, Cynthia A ^a  

^a UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Gifted education; Perceived competence; Pre post test; Response shift bias; Retrospective pre post test

Indexed keywords

EID: 68949189677     PISSN: 10982140     EISSN: 15570878     Source Type: Journal    
DOI: 10.1177/1098214009334506     Document Type: Article

References (27)

1. Bamberger, M., Rugh, J., Church, M., & Fort, L. (2004). Shoestring evaluation: Designing impact evaluations under budget, time, and data constraints. American Journal of Evaluation, 25, 5-37.
2. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.
3. Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response shift bias. Educational and Psychological Measurement, 44, 781-804.
4. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Co.
5. Cantrell, P. (2003). Traditional vs. retrospective pretests for measuring science teaching efficacy beliefs in preservice teachers. School Science and Mathematics, 103, 177-185.
6. Choi, N. (2004). Sex role group differences in specifying academic and general self-efficacy. The Journal of Psychology, 138, 149-159.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8. Cooke, B. (1998). Beyond basic training. Evaluation Exchange, 4, 8-9.
9. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
10. D'Eon, M., Sadownik, L., Harrison, A., & Nation, J. (2008). Using self-assessments to detect workshop success: Do they work? American Journal of Evaluation, 29, 92-98.
11. Gutek, B. A., & Winter, S. J. (1992). Consistency of job satisfaction across situations: Fact or framing artifact? Journal of Vocational Behavior, 41, 61-78.
12. Hill, L. G. & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26, 501-517.
13. Hoogstraten, J. (1982). The retrospective pre-test in an educational training context. Journal of Experimental Education, 50, 200-204.
14. Howard, G. S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review, 4, 93-106.
15. Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychological Measurement, 3, 1-23.
16. Howard, G. S., Schmeck, R. R., & Bary, J. H. (1979). Internal invalidity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement, 16, 129-135.
17. Klatt, J., & Taylor-Powell, E. (2005a). Using the retrospective post-then-pre design, quick tips #27. Program Development and Evaluation. Madison, WI: University of Wisconsin-Extension.
18. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567.
19. Krosnick, J. A., & Fabrigar, L. R. (1998). Designing good questionnaires: Insights from psychology. New York: Oxford University Press.
20. Lent, R. W., Brown, S. D., & Gore, P. A. (1997). Discriminant and predictive validity of academic self-concept, academic self-efficacy and mathematics specific self-efficacy. Journal of Counseling Psychology, 44, 307-315.
21. Manthei, R. J. (1997). The response-shift bias in a counselor education programme. British Journal of Guidance and Counseling, 25, 229-237.
22. Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 49-69). Mahwah, NJ: Erlbaum.
23. Pratt, C. C., Mcguigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341-349.
24. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341-357.
25. Schwarz, N. (2007). Cognitive aspects of survey methodology. Applied Cognitive Psychology, 21, 277-287.
26. Taylor, P., Russ, Eft, D., & Taylor, H. (2009). Gilding the outcome by tarnishing the past: Inflationary biases in retrospective pretests. American Journal of Evaluation, 30, 31-43.
27. Wolfinger, R., & Chang, M. (2007). Comparing the SAS GLM and MIXED procedures for repeated measures. SAS Institute). Retrieved March 6, 2008, from http://support.sas.com\rnd\app\papers\mixedglm.pdf