Biased parameter estimates and inflated type i error rates in analysis of covariance (and analysis of partial variance) arising from unreliability: Alternatives and remedial strategies

Journal of Abnormal Psychology

Volumn 119, Issue 2, 2010, Pages 307-319

  Zinbarg, Richard E ^a   Suzuki, Satoru ^a   Uliaszek, Amanda A ^a   Lewis, Alison R ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

Analysis of covariance; Analysis of partial variance; Bias; Structural equation modeling

Indexed keywords

ANALYSIS OF COVARIANCE; ANALYSIS OF PARTIAL VARIANCE; ARTICLE; CORRELATION ANALYSIS; MENTAL DISEASE; RELIABILITY; STATISTICAL CONCEPTS; STATISTICAL MODEL; SYSTEMATIC ERROR; THEORETICAL STUDY; UNRELIABILITY;

EID: 77952923692     PISSN: 0021843X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0017552     Document Type: Article

References (69)

1. Aiken L.S., Stein J.A., Bentler P.M. Structural equation analyses of clinical subpopulation differences and comparative treatment outcomes: Characterizing the daily lives of drug addicts. Journal of Consulting and Clinical Psychology 1994, 62:488-499.
2. Allison P.D. Fixed effects regression methods for longitudinal data using SAS 2005, SAS Institute, Cary, NC.
3. Bauer D.J., Curran P.J. The integration of continuous and discrete latent variable models: Potential problems and promising opportunities. Psychological Methods 2004, 9:3-29.
4. Bernstein I.H., Teng G. Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin 1989, 105:467-477.
5. Blalock H.M. Causal inferences in nonexperimental research 1964, University of North Carolina, Chapel Hill.
6. Bollen K.A. Structural equations with latent variables 1989, Wiley, New York, NY.
7. Bollen K.A., Curran P.J. Autoregressive latent trajectory (ALT) models: A synthesis of two traditions. Sociological Methods and Research 2004, 32:336-383.
8. Browne M.W. The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology 1984, 37:1-21.
9. Campbell D.T., O'Connell E.J. Methods as diluting trait relationships rather than adding irrelevant systematic variance. New Directions for Methodology of Social and Behavioral Science: No. 12. Forms of validity in research 1982, 93-111. Jossey-Bass, San Francisco, CA. D. Brinberg, L. Kidder (Eds.).
10. Carroll R.J., Ruppert D., Stefanski L.A., Crainiceanu C.M. Measurement error in nonlinear models: A modern perspective 2006, Taylor & Francis, Boca Raton, FL. 2nd ed.
11. Cattell R.B. The scientific use of factor analysis in behavioral and life sciences 1978, Plenum Press, New York, NY.
12. Clark L.A., Watson D. Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology 1991, 100:316-336.
13. Clarke K.A. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 2005, 22:341-352.
14. Cochran W.G. Analysis of covariance: Its nature and uses. Biometrics 1957, 44:261-281.
15. Cohen J. The statistical power of abnormal-social psychological research: A review. Journal of Abnormal & Social Psychology 1962, 65:145-153.
16. Cohen J., Cohen P. Applied multiple regression/correlation analysis for the behavioral sciences 1983, Erlbaum, Hillsdale, NJ. 2nd ed.
17. Cronbach L.J. Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16:297-334.
18. DeShon R.P. A cautionary note on measurement error corrections in structural equation models. Psychological Methods 1998, 3:412-423.
19. Elashoff J.D. Analysis of covariance: A delicate instrument. American Educational Research Journal 1969, 6:383-401.
20. Fleiss J.L., Tanur J.M. The analysis of covariance in psychopathology. Psychopathology: Contributions from the social, behavioral and biological sciences 1973, 509-527. Wiley, New York, NY. M. Hammer, K. Salzinger, S. Sutton (Eds.).
21. Glynn R.J., Schneeweiss S., Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic & Clinical Pharmacology & Toxicology 2006, 98:253-259.
22. Hayduk L.A. Structural equation modeling with LISREL: Essentials and advances 1987, Johns Hopkins Press, Baltimore, MD.
23. Hertzog C., Nesselroade J.R. Assessing psychological change in adulthood: An overview of methodological issues. Psychology and Aging 2003, 18:639-657.
24. Horn J.L., McArdle J.J. A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research 1992, 18:117-144.
25. Huitema B. Analysis of covariance and alternatives 1980, Wiley, New York, NY.
26. Judd C.M., Kenny D.A. Estimating the effects of social interventions 1981, Cambridge University Press, New York, NY.
27. Kahneman D. Control of spurious association and the reliability of the controlled variable. Psychological Bulletin 1965, 64:326-329.
28. Kenny D.A. A quasi-experimental approach to assessing treatment effects in the nonequivalent control group design. Psychological Bulletin 1975, 82:345-362.
29. Kenny D.A. Correlation and causality 1979, Wiley, New York, NY.
30. Little T.D., Cunningham W.A., Shahar G., Widaman K.F. To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling 2002, 9:151-173.
31. Little T.D., Lindenberger U., Nesselroade J.R. On selecting indicators for multivariate measurement and modeling with latent variables: When " good" indicators are bad and " bad" indicators are good. Psychological Methods 1999, 4:192-211.
32. Long J.D., Trivedi P.K. Some specification tests for the linear regression model. Testing structural equation models 1993, 66-110. Sage, Newbury Park, CA. K.A. Bollen, J.S. Long (Eds.).
33. Lord F.M. Large-sample covariance analysis when the control variable is fallible. Journal of the American Statistical Association 1960, 55:307-321.
34. Lord F.M. A paradox in the interpretation of group comparisons. Psychological Bulletin 1967, 68:304-305.
35. Lord F.M. Statistical adjustments when comparing preexisting groups. Psychological Bulletin 1969, 72:336-337.
36. Mackinnon D.P., Lockwood C.M., Hoffman J., West S.G., Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychological Methods 2002, 7:83-104.
37. Marcus S.M. Using omitted variable bias to assess uncertainty in the estimation of an AIDS education treatment effect. Journal of Educational and Behavioral Statistics 1997, 22:193-201.
38. Marsh H.W. Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement 1989, 13:335-361.
39. Maxwell S.E., Delaney H.D. Designing experiments and analyzing data: A model comparison approach 2004, Erlbaum, Mahwah, NJ. 2nd ed.
40. Maxwell S.E., Delaney H.D., Manheimer J.M. ANOVA of residuals and ANCOVA: Correcting an illusion by using model comparisons and graphs. Journal of Educational Statistics 1985, 10:197-209.
41. McDonald R.A., Behson S.J., Seifert C. Strategies for dealing with measurement error in multiple regression. Journal of the Academy of Business and Economics 2005, 53:80-97.
42. Miller G.A., Chapman J.P. Misunderstanding analysis of covariance. Journal of Abnormal Psychology 2001, 110:40-48.
43. Muthén B.O. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 1984, 46:115-132.
44. Porter A.C., Raudenbush S.W. Analysis of covariance: Its model and use in psychological research. Journal of Counseling Psychology 1987, 34:383-392.
45. Preacher K.J., Hayes A.F. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers 2004, 36:717-731.
46. Reichardt C.S. The statistical analysis of data from nonequivalent group designs. Quasi-experimentation: Design & analysis issues for field settings 1979, 147-205. Houghton Mifflin, Boston, MA. T. Cook, D. Campbell (Eds.).
47. Rogosa D. Myths and methods: " Myths about longitudinal research" plus supplemental questions. The analysis of change 1995, 3-65. Erlbaum, Mahwah, NJ. J. Gottman (Ed.).
48. Rogosa D., Brandt D., Zimowski M. A growth curve approach to the measurement of change. Psychological Bulletin 1982, 90:726-748.
49. Rosenbaum P.R. Choice as an alternative to control in observational studies. Statistical Science 1999, 14:259-278.
50. Rosenbaum P.R. Observational studies 2002, Springer, New York, NY. 2nd ed.
51. Rosenbaum P.R., Rubin D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 1983, 70:41-55.
52. Rossi J. Statistical power of psychological research: What have we gained in 20 years?. Journal of Consulting and Clinical Psychology 1990, 58:646-656.
53. Rubin D.B. Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine 1997, 127:757-763.
54. Rubin D. Matched sampling for causal effects 2006, Cambridge University Press, New York, NY.
55. Sedlmeier P., Gigerenzer G. Do studies of statistical power have an effect on the power of studies?. Psychological Bulletin 1989, 105:309-316.
56. Shadish W., Cook T. Comment-design rules: More steps toward a complete theory of quasi-experimentation. Statistical Science 1999, 14:294-300.
57. Shadish W., Cook T., Campbell D. Experimental and quasi-experimental designs for generalized causal inference 2002, Houghton Mifflin, Boston, MA.
58. Sher K.J., Trull T.J. Methodological issues in psychopathology research. Annual Review of Psychology 1996, 47:371-400.
59. Shrout P.E., Bolger N. Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods 2002, 7:422-445.
60. Singer J.D., Willett J.B. Applied longitudinal data analysis: Modeling change and event occurrence 2003, Oxford University Press, New York, NY.
61. Sörbom D. An alternative to the methodology for analysis of covariance. Advances in factor analysis and structural equation models 1979, University Press of America, Lanham, MD. K.G. Jöreskog, D. Sörbom (Eds.).
62. Tomarken A.J., Waller N.G. Potential problems with " well fitting" models. Journal of Abnormal Psychology 2003, 112:578-598.
63. Vargha A., Rudas T., Delaney H.D., Maxwell S.E. Dichotomization, partial correlation, and conditional independence. Journal of Educational and Behavioral Statistics 1996, 21:264-282.
64. Wainer H. Adjusting for differential base rates: Lord's paradox again. Psychological Bulletin 1991, 109:147-151.
65. Waller N.G., Tellegen A., McDonald R.P., Lykken D.T. Exploring nonlinear models in personality assessment: Development and preliminary validation of a negative emotionality scale. Journal of Personality 1996, 64:545-576.
66. Wildt A.R., Ahtola O.T. Analysis of covariance 1978, Sage, Beverly Hills, CA.
67. Willett J. Questions and answers in the measurement of change. Review of Research in Education 1988, 15:345-422.
68. Williams R.H., Zimmerman D.W. Are simple gain scores obsolete?. Applied Psychological Measurement 1996, 20:59-69.
69. H: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika 2005, 70:123-133.