Questioning the measurement precision of psychotherapy research

Psychotherapy Research

Volumn 19, Issue 4-5, 2009, Pages 374-389

  Doucette, Ann ^a,b   Wolf, Abraham W ^a,b  

^a GEORGE WASHINGTON UNIVERSITY   (United States)

^b METROHEALTH MEDICAL CENTER   (United States)

Author keywords

Classical test theory; Item response theory; Latent trait theory; Measurement; Psychotherapy outcomes

Indexed keywords

ADOLESCENT; ADULT; AGED; FEMALE; HUMAN; MALE; METHODOLOGY; MIDDLE AGED; PSYCHOLOGICAL THEORY; PSYCHOLOGY; PSYCHOMETRY; PSYCHOTHERAPY; RESEARCH; REVIEW; STATISTICS;

ADOLESCENT; ADULT; AGED; FEMALE; HUMANS; MALE; MIDDLE AGED; PSYCHOLOGICAL THEORY; PSYCHOLOGY; PSYCHOMETRICS; PSYCHOTHERAPY; RESEARCH; YOUNG ADULT;

EID: 77953156782     PISSN: 10503307     EISSN: 14684381     Source Type: Journal    
DOI: 10.1080/10503300902894422     Document Type: Article

References (71)

1. Adams, R. J., & Wilson, M. R. (1996). Formulating the Rasch model as a mixed coefficients multinomial logit. In G. Englehard & M. Wilson (Eds), Objective measurement: Theory into practice (Vol. 3, pp. 143-166). Norwood, NJ: Ablex.
2. Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1-23.
3. Andrich, D. (1978). A rating formula for ordered response categories. Psychometrika, 43, 561-573.
4. Baker, B. O., Hardyck, C., & Petrinovich, L. F. (1966). Weak measurement vs. strong statistics: An empirical critique of S. S. Stevens’ proscriptions on statistics. Educational and Psychological Measurement, 26, 291-309.
5. Balsis, S., Gleason, M. E. J., Woods, C. M., & Oltmanns, T. F. (2007). An item response theory analysis of DSM-IV personality disorder criteria across younger and older age groups. Psychology and Aging, 22(1), 171-185.
6. Bartolucci, B. (2007). A class of multidimensional IRT models for testing unidimensionality and clustering items. Psychometrika, 72(2), 141-157.
7. Becker, J., Fliege, H., Kocalevent, R.-D., Bjorner, J. B., Rose, M., Walter, O. B., et al. (2008). Functioning and validity of a computerized adaptive test to measure anxiety. Depression and Anxiety, 25(12), E182-E194.
8. Blais, M., Lenderking, W., Baer, L., DeLorell, A., Peets, K., Leahy, L., et al. (1999). Development and initial validation of a brief mental health outcome measure. Journal of Personality Assessment, 73, 359-373.
9. Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Erlbaum.
10. Bond, T., & Fox, C. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed). Mahwah, NJ: Erlbaum.
11. Burlew, A., Feaster, D., Brecht, M., & Hubbard, R. (2009). Measurement and data analysis in research addressing health disparities in substance abuse. Journal of Substance Abuse Treatment, 36(1), 25-43.
12. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The patient-reported outcome measurement information system (PROMIS). Medical Care, 45(5), S3-S11.
13. Chernyshenko, S., Oleksandr, S., Chan, K., Lee, W. C., & Drasgow, F. (2001). Effects of the test situation on item responding: Cause for concern. Journal of Applied Psychology, 86(5), 943-953.
14. Christensen, K. B., Bjorner, J. B., Kreiner, S., & Petersen, J. H. (2002). Testing unidimensionality in polytomous Rasch models. Psychometrika, 67(4), 563-574.
15. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
16. Ellis, B. B., Becker, P., & Kimmel, H. D. (1993). An item response theory evaluation of an English version of the Trier Personality Inventory. Journal of Cross-Cultural Psychology, 2, 133-148.
17. Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8, 314-349.
18. Embretson, S. E., & Hershberger, S. L. (1999). The new rules of measurement: What every psychologist and educator should know. Mahwah, NJ: Erlbaum.
19. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
20. Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58, 357-384.
21. Fischer, G. H., & Molenaar, I. W. (1995). Rasch models: Foundations, recent developments, and applications (1st ed). Berlin, Germany: Springer-Verlag.
22. Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78(2), 350-365.
23. Gibbons, R. D., Bock, D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., et al. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31(1), 4-19.
24. Gibbons, R. D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423-436.
25. Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.
26. Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (pp. 147-200). New York: Macmillan.
27. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
28. Howard, K. I., Orlinsky, D. E., & Lueger, R. J. (1995). The design of clinically relevant outcome research: Some considerations and an example. In M. Aveline & D. A. Shapiro (Eds), Research foundations for psychotherapy practice (pp. 3-47). London: Wiley.
29. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12-19.
30. Junker, B. W., & Sijtsma, K. (2001). Nonparametric item response theory in action: An overview of the special issue. Applied Psychological Measurement, 25, 211-220.
31. Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychotherapy (4th edN). New York: Wiley.
32. Lambert, M. J., Hatfield, D. R., & Vermeersch, D. A. (2001). Administration and scoring manual for the LSQ (Life Status Questionnaire). Salt Lake City, UT: OQ Measures.
33. Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.
34. Linacre, J. M. (2001). Winsteps Rasch measurement computer program (Version 3.31) [Computer software]. Chicago: Win-steps.com.
35. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.
36. Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3, 143-155.
37. Lord, F. (1953). The relation of test score to the trait underlying the test. Educational and Psychological Measurement, 13, 517-548.
38. Lord, F. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
39. Lord, F., & Novick, M. (1968). Statistical theories of mental tests. New York: Addison-Wesley.
40. Margison, F., Barkham, M., Evans, C., McGrath, G., Mellor-Clark, J., Connell, J., et al. (2000). Measurement and psychotherapy: Evidence based practice and practice based evidence. British Journal of Psychiatry, 177, 123-130.
41. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 1(34), 100-117.
42. Merbitz, C., Morris, J., & Grip, J. C. (1989). Ordinal scales and foundations of misinference. Archives of Physical Medicine and Rehabilitation, 70, 308-332.
43. Michell, J. (2001). Teaching and misteaching measurement in psychology. Australian Psychologist, 36(3), 211-217.
44. Muraki, E., & Bock, R. D. (1997). PARSCALE: IRTitem analysis and test scoring for rating-scale data. Chicago: Scientific Software International.
45. Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3(1), 1-18.
46. Pastor, D. A., & Beretvas, S. N. (2006). Longitudinal Rasch modeling in the context of psychotherapy outcomes assessment. Applied Psychological Measurement, 30(2), 100-120.
47. Poisal, J. A., Truffer, C., Smith, S., Sisko, A., Cowan, C., Keehan, S., et al. (2007). Health spending projections through 2016: Modest changes obscure Part D’s impact. Health Affairs, 26, 242-253.
48. Rasch, G. (1977). On specific objectivity: An attempt at formalizing the request for generality and validity of scientific statements. In M. Glegved (Ed.), The Danish yearbook of philosophy (pp. 58-94). Copenhagen: Munksgaard.
49. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests 1960. Chicago: University of Chicago Press. (Original work published).
50. Reckase, M. (1997). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21(1), 25-36.
51. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45 (Suppl., 1), S22-S31.
52. Reinhardt, U., Hussey, P. S., & Anderson, G. F. (2004). U.S. health care spending in an international context. Health Affairs, 23(3), 10-25.
53. Reise, S. P., & Haviland, M. G. (2005). Item response theory and measurement of clinical change. Journal of Personality Assessment, 84(3), 228-238.
54. Reise, S. P., & Henson, J. M. (2003). A discussion of modern versus traditional psychometrics as applied to personality assessment scales. Journal of Personality Assessment, 81, 93-103.
55. Roth, A., & Fonagy, P. (1996). What works for whom? A critical review of psychotherapy research. New York: Guilford Press.
56. Rouse, S. V., Butcher, J. N., & Miller, K. B. (1999). Assessment of substance abuse in psychotherapy clients: The effectiveness of the MMPI-2 substance abuse scales. Psychological Assessment, 11, 101-107.
57. Sijtsma, K., & Meijer, R. R. (2007). Nonparametric item response theory and related topics. In C. R. Rao & S. Sinharay (Eds), Handbook of statistics 26: Psychometrics (pp. 719-746). Amsterdam: Elsevier.
58. Simon, D., Schorr, G., Wirtz, M., Vodermaier, A., Caspari, C., Neuner, B., et al. (2006). Development and first validation of the Shared Decision-Making Questionnaire (SDM-Q). Patient Education and Counseling, 63(3), 319-327.
59. Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins University Press.
60. Sotsky, S. M., Glass, D. R., Shea, M. T., Pilkonis, P. A., Collins, F., Elkin, I., et al. (2006). Patient predictors of response to psychotherapy and pharmacotherapy: Findings in the NIMH Treatment of Depression Collaborative Research Program. Focus, 4(2), 278-290.
61. Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567-577.
62. Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement, 15, 22-29.
63. Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97-116.
64. Wright, B. D. (1995). 3PL or Rasch? Rasch Measurement Transactions, 9(1), 408-409.
65. Wright, B. D., & Douglas, G. A. (1977). Best procedures for sample-free item analysis. Applied Psychological Measurement, 1, 281-294.
66. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.
67. Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago: MESA Press.
68. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.
69. Yen, W. M. (2005). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23(4), 299-325.
70. Zumbo, B. D., Pope, G. A., Watson, J. E., & Hubley, A. M. (1997). An empirical test of Roskam’s conjecture about the interpretation of an ICC parameter in personality inventories. Educational and Psychological Measurement, 57, 963-969.
71. Zwick, R. (1987). Assessing the dimensionality of NAEP reading data. Journal of Educational Measurement, 24, 293-308.