Addressing response-shift bias: Retrospective pretests in recreation research and evaluation

Journal of Leisure Research

Volumn 39, Issue 2, 2007, Pages 295-315

  Sibthorp, Jim ^a   Paisley, Karen ^a   Gookin, John ^b   Ward, Peter ^c  

^a UNIVERSITY OF UTAH   (United States)

^b National Outdoor Leadership School   (United States)

^c BRIGHAM YOUNG UNIVERSITY   (United States)

Author keywords

Outcome measurement; Program evaluation; Response shift bias

Indexed keywords

ASSESSMENT METHOD; MEASUREMENT METHOD; RECREATIONAL ACTIVITY; RESEARCH; THEORETICAL STUDY;

EID: 34250757581     PISSN: 00222216     EISSN: None     Source Type: Journal    
DOI: 10.1080/00222216.2007.11950109     Document Type: Article

References (40)

1. Bialeschki, D., Sibthorp, J., & Ellis, G. (2006) Intentionality and targeted youth development outcomes: Four years at Morry's Camp. 2006 NRPA Leisure Research Symposium, Seattle, WA, 59.
2. Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response-shift bias. Educational & Psychological Measurement, 44, 781-804.
3. Breetvelt, I. S., & Van Dam, F. S. (1991). Underreporting by cancer patients: The case of response-shift. Social Science Medicine, 32, 981-987.
4. Caldwell, L. L., & Baldwin, C. K. (2004). Preliminary effects of a leisure education program to promote healthy use of free time among middle school adolescents. Journal of Leisure Research, 36, 310-335.
5. Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research Chicago: Rand McNally.
6. Cantrell, P. (2003). Traditional vs. retrospective pretests for measuring science teaching efficacy beliefs in preservice teachers. School Science and Mathematics, 103(4), 177-185.
7. Christensen, J. (1995). Statistical and methodological issues in leisure research. In L. Barnett (Ed.), Research about leisure: Past, Present, and future (2nd ed.) (pp. 231-251) Champaign, IL: Sagamore.
8. Cronbach, L. J., & Furby, L. (1970). How we should measure "change" - or should we? Psychological Bulletin, 74, 68-80.
9. Goedhart, H., & Hoogstraten, J. (1992). The retrospective pretest and the role of pretest information in valuative studies. Psychological Reports, 70, 699-705.
10. Green, G., Kleiber, D., & Tarrant, M. (2000). The effect of an adventure-based recreation program on the development of resiliency in low income minority youth. Journal of Park and Recreation Administration, 18(3). 76-97.
11. Hoogstraten J. (1985). Influence of objective measures on self-reports in a retrospective pretest-posttest design. Journal of Experiment Education, 53(4), 207-210.
12. Hopkins, K. D. (1986). Affective dependent measures: The use of a response integrity scale to enhance the validity of experimental and quasi-experimental research. Journal of Special Education, 20, 43-47.
13. Howard, G. S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review, 4(1), 93-106.
14. Howard, G. S., & Dailey, P. R. (1979). Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology, 66(2), 144-151.
15. Howard, G. S., Dailey, P. R., & Gulanick, N. A. (1979). The feasibility of informed pretests in attenuating response-shift bias. Applied Psychological Measurement, 3, 481-494.
16. Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W. & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluation and a re-evaluation retrospective pretests. Applied Psychological Measurement, 3(1), 1-23.
17. Howard, G. S., Schmeck, R. R., & Bray, J. H. (1979). Internal invalidity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement, 16, 129-135.
18. Hurtes, K. P., Allen, L. R., Stevens, B. W., & Lee, C. (2000). Benefits-Based Programming: Making an impact on youth at-risk. Journal of Park and Recreation Administration, 18(1), 34-49.
19. Koele, P. & Hoogstraten, J. (1988). A method for analyzing retrospective pretest/posttest designs: I. Theory. Bulletin of the Psychonomic Society, 26, 51-54.
20. Lam, T. & Bengo, P. (2003). A comparison of three retrospective self-reporting methods of measuring change in instructional practice. American Journal of Evaluation, 24(1), 65-80
21. Lee, T. M., Paterson, J. G., & Chan, C. C. (1994). The effect of occupational therapy education on students' perceived attitudes towards persons with disabilities. American Journal of Occupational Therapy, 48, 633-638.
22. Mann, S. (1997). Implications of the response-shift bias for management. Journal of Management Development, 16 328-337.
23. Manthei, R. J. (1997). The response-shift bias in a counsellors education programme. British Journal of Guidance & Counselling, 25(2), 229-238.
24. Mezoff, B. (1981). Pre- then posttesting: a tool to improve the accuracy of management training program evaluation. Performance and Instruction. 20(8), 10-11, 16.
25. Pearson, R. W., Ross, M., & Dawes, R. M. (1991). Personal recall and the limits of retrospective questions in surveys. In J. Tanur (Ed.), Questions about questions (pp. 65-100). New York: Russell Sage Foundation.
26. Pohl, N. F. (1982). Using retrospective pre-ratings to counteract response-shift confounding. Journal of Experimental Education, 50, 211-214.
27. Pratt, C. C., Mcgulgan, W. M., & Katzer, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341-350.
28. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
29. Rhodes, J. & Jason, L. (1987). The retrospective pretest: An alternative approach in evaluating drug prevention programs. Journal of Drug Education, 17, 345-356.
30. Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726-748.
31. Rohs, F. R. (1999). Response-shift bias: A problem in evaluating leadership development with self-report pretest-posttest measures. Journal of Agricultural Education, 40(4), 28-37.
32. Rohs, F. R, & Longone, C. L. (1997). Increased accuracy in measuring leadership impacts. The Journal of Leadership Studies, 4(1), 150-158.
33. Searle, M. S., Mahon, M. J., Iso-Ahola, S. E., Sodrolias, H. A., & Van Dyck, J. (1995). Enhancing a sense of independence and psychological well-being among the elderly: A field experiment. Journal of Leisure Research, 27, 107-124.
34. Sibthorp, J., Paisley, K., Gookin, J., & Ward, P. (2005). Participant Development through Adventure-Based Recreation Programming: A Model from The National Outdoor Leadership School [Abstract]. Proceedings of the 2005 NRPA Leisure Research Symposium, San Antonio, TX, 11.
35. Sprangers, M., & Hoogstraten, J. (1988a). On delay and reassessments of retrospective ratings. Journal of Experimental Education, 56(3), 148-153.
36. Sprangers, M., & Hoogstraten, J. (1988b). Response-style effects, response-shift bias, and a bogus pipeline: A replication. Psychological Reports, 62(1), 11-16.
37. Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest designs. Journal of Applied Psychology, 74(2), 265-273.
38. Townsend, M. & Wilton, K. (2003). Evaluating change in attitude towards mathematics using the 'then-now' procedure in a cooperative learning programme. British Journal of Educational Psychology, 73, 473-488.
39. Toupence, R. & Townsend, C. (2000). Leadership development and youth camping: Determining a relationship. In A. Stringer et. al. (Eds.). Coalition for Education in the Outdoors Fifth Research Symposium Proceedings (pp. 82-88). Cortland, NY: CEO.
40. Witt, P. (2000). If leisure research is to matter II. Journal of Leisure Research, 32, 186-189.