The inclusion of 'then-test' questions in post-test questionnaires alters post-test responses: A randomized study of bias in health program evaluation

Quality of Life Research

Volumn 21, Issue 3, 2012, Pages 487-494

  Nolte, Sandra ^a   Elsworth, Gerald R ^a   Sinclair, Andrew J ^a   Osborne, Richard H ^a  

^a DEAKIN UNIVERSITY   (Australia)

Author keywords

Bias; Chronic disease; Patient education; Program evaluation; Response shift; Retrospective pre test; Self management; Then test

Indexed keywords

ADAPTIVE BEHAVIOR; ADULT; AGED; ARTICLE; ATTITUDE TO HEALTH; AUSTRALIA; CHRONIC DISEASE; CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; FEMALE; HEALTH CARE QUALITY; HEALTH SURVEY; HUMAN; MALE; METHODOLOGY; MIDDLE AGED; PATIENT EDUCATION; PSYCHOLOGICAL ASPECT; PSYCHOMETRY; QUALITY OF LIFE; QUESTIONNAIRE; RANDOMIZED CONTROLLED TRIAL; SELF CARE;

ADAPTATION, PSYCHOLOGICAL; ADULT; AGED; AGED, 80 AND OVER; ATTITUDE TO HEALTH; AUSTRALIA; CHRONIC DISEASE; FEMALE; HEALTH STATUS INDICATORS; HUMANS; MALE; MIDDLE AGED; PATIENT EDUCATION AS TOPIC; PROGRAM EVALUATION; PSYCHOMETRICS; QUALITY OF LIFE; QUESTIONNAIRES; RESEARCH DESIGN; SELF CARE;

EID: 84863609607     PISSN: 09629343     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11136-011-9952-1     Document Type: Article

References (39)

1. Golembiewsky, R., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12, 133-157.
2. Howard, G. S., & Dailey, P. R. (1979). Response-shift bias: A source of contamination of self-report measures. Journal of Applied Psychology, 64(2), 144-150.
3. Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507-1515. (Pubitemid 29254110)
4. Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal healthrelated quality-of-life research. Social Science and Medicine, 48(11), 1531-1548. (Pubitemid 29254112)
5. Cronbach, L. J., & Furby, L. (1970). How should we measure "change"-or should we? Psychological Bulletin, 74(1), 68-80.
6. Lorig, K. R., Sobel, D. S., Stewart, A. L., Brown, B. W., Bandura, A., Ritter, P., et al. (1999). Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization. Medical Care, 37(1), 5-14.
7. Elzen, H., Slaets, J., Snijders, T., & Steverink, N. (2007). Evaluation of the chronic disease self-management program (CDSMP) among chronically ill older people in the Netherlands. Social Science and Medicine, 64, 1832-1841. (Pubitemid 46528031)
8. Lorig, K. R. (2003). Editoria-Self-management education: more than a nice extra. Medical Care, 41(6), 699-701.
9. Lorig, K. R., & Holman, H. R. (1989). Long-term outcomes of an arthritis self-management study: Effects of reinforcement efforts. Social Science and Medicine, 29(2), 221-224. (Pubitemid 19166086)
10. Gibbons, F. X. (1999). Social comparison as a mediator of response shift. Social Science and Medicine, 48(11), 1517-1530. (Pubitemid 29254111)
11. Osborne, R. H., Hawkins, M., & Sprangers, M. A. G. (2006). Change of perspective: A measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/ postintervention assessment. Arthritis Care and Research, 55(3), 458-465. (Pubitemid 43887032)
12. Sprangers, M. A. G. (1989). Response-shift bias in program evaluation. Impact Assessment Bulletin, 7, 153-166.
13. Schwartz, C., & Sprangers, M. (2010). Guidelines for improving the stringency of response shift research using the thentest. Quality of Life Research, 19, 455-464.
14. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
15. McGaw, B., & Jöreskog, K. G. (1971). Factorial invariance of ability measures in groups differing in intelligence and socioeconomic status. British Journal of Mathematical, 24, 154-168.
16. Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the "then-test". Journal of Clinical Epidemiology, 62(11), 1173-1180.
17. Howard, G. S., Schmeck, R. R., & Bray, J. H. (1979). Internal invalidity in studies employing self-report instruments: a suggested remedy. Journal of Educational Measurement, 16(2), 129-135.
18. Lorig, K., & Fries, J. (2000). The arthritis helpbook (5th ed.). Cambridge, MA: Perseus Books.
19. Lorig, K. R., González, V. M., & Laurent, D. D. (1999). The chronic disease self-management program: Leaders manual. Palo Alto: Stanford University.
20. Osteoporosis Victoria. (2001). The osteoporosis prevention and self management course-Leaders manual. Elsternwick: Osteoporosis Victoria, A division of Arthritis Victoria.
21. Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2007). The extent and breadth of benefits from participating in chronic disease self-management courses: A national patientreported outcomes survey. Patient Education and Counseling, 65(3), 351-360. (Pubitemid 46241775)
22. Osborne, R. H., Elsworth, G. R., & Whitfield, K. (2007). The health education impact questionnaire (heiQ): An outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Education and Counseling, 66, 192-201. (Pubitemid 46575176)
23. Nolte, S. (2008). Approaches to the measurement of outcomes of chronic disease self-management interventions using a self-report inventory. Melbourne: RMIT University.
24. Brown, M. B., & Forsythe, A. B. (1974). Robust tests for the equality of variances. Journal of the American Statistical Association, 69(346), 364-367.
25. Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688-701.
26. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567.
27. Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51, 201-219.
28. Lam, T. C. M., & Bengo, P. (2003). A comparison of three retrospective self-reporting methods of measuring change in instructional practice. American Journal of Evaluation, 24(1), 65-80. (Pubitemid 36316567)
29. Schwarz, N., & Strack, F. (1985). Cognitive and affective processes in judgements of subjective well-being: a preliminary model. In H. Brandstatter & E. Kirchler (Eds.), Economic psychology (pp. 439-447). Linz, Austria: R. Tauner.
30. Crowne, D. P., & Marlowe, D. (1964). The approval motive: Studies in evaluative dependence. New York, London, Sydney: Wiley.
31. Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, S. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychological Measurement, 3, 1-23.
32. Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501-517. (Pubitemid 41763973)
33. Sprangers, M. A. G., & Hoogstraten, J. (1988). On delay and reassessment of retrospective preratings. Journal of Experimental Education, 56, 148-153.
34. Norman, G. (2003). Hi! How are you? Response shift, implicit theories and differing epistemologies. Quality of Life Research, 12, 239-249. (Pubitemid 36549532)
35. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96, 341-357.
36. Nimon, K., Zigarmi, D., & Allen, J. (2011). Measures of program effectiveness based on retrospective pretest data: are all created equal? American Journal of Evaluation, 32(8), 8-28.
37. Terborg, J., & Davis, G. (1982). Evaluation of a new method of assessing change to planned job redesign as applied to Hackman and Oldham's job characteristic model. Organizational Behavior and Human Performance, 29, 112-128.
38. Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest designs. Journal of Applied Psychology, 74, 265-272.
39. Stone, G. R., & Packer, T. L. (2010). Evaluation of a rural chronic disease self-management program. Rural and Remote Health, 10(1), 1203.