Individual influence on model selection

Psychological Methods

Volumn 17, Issue 4, 2012, Pages 582-599

  Sterba, Sonya K ^a   Pek, Jolynn ^b  

^a VANDERBILT UNIVERSITY   (United States)

^b YORK UNIVERSITY   (Canada)

Author keywords

Case influence; Experimental design; Model selection; Psychology; Sensitivity analysis

Indexed keywords

ARTICLE; BAYES THEOREM; CHI SQUARE DISTRIBUTION; DECISION SUPPORT SYSTEM; HUMAN; METHODOLOGY; PSYCHOLOGICAL MODEL; PSYCHOLOGY; STATISTICAL MODEL; THEORETICAL MODEL;

BAYES THEOREM; CHI-SQUARE DISTRIBUTION; DECISION SUPPORT TECHNIQUES; HUMANS; LIKELIHOOD FUNCTIONS; MODELS, PSYCHOLOGICAL; MODELS, THEORETICAL; PSYCHOLOGY; RESEARCH DESIGN;

EID: 84874399238     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0029253     Document Type: Article

References (93)

1. Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
2. Achenbach, T. M. (1992). Manual for the Child Behavior Checklist/2-3 and 1992 profile. Burlington, VT: University of Vermont, Department of Psychiatry.
3. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723. doi:10.1109/ TAC.1974.1100705
4. Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling (pp. 243-276). Mahwah, NJ: Erlbaum.
5. Atkinson, A. C., & Riani, M. (2008). A robust and diagnostic information criterion for selecting regression models. Journal of the Japanese Statistical Society, 38, 3-14.
6. Bauer, D. J., & Hussong, A. M. (2009). Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods, 14, 101-125. doi: 10.1037/a0015583
7. Bendre, S. M., & Kale, B. K. (1987). Masking effect on tests for outliers in normal samples. Biometrika, 74, 891-896. doi:10.1093/biomet/ 74.4.891
8. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
9. Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken, NJ: Wiley.
10. Box, G. E. (1979). Some problems of statistics and everyday life. Journal of the American Statistical Association, 74, 1-4.
11. Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230-258. doi:10.1177/ 0049124192021002005
12. Bruce, A. G., & Martin, R. D. (1989). Leave-k-out diagnostics for time series. Journal of the Royal Statistical Society, Series B, 51, 363-424.
13. Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach (2nd ed.). New York, NY: Springer-Verlag.
14. Cadigan, N. G. (1994). Computing case diagnostics for structural equation models. Retrieved from http://www.sascommunity.org/sugi/SUGI94/Sugi-94-188%20Cadigan.pdf
15. Cadigan, N. G. (1995). Local influence in structural equation models. Structural Equation Modeling, 2, 13-30. doi:10.1080/ 10705519509539992
16. Chatterjee, S., & Hadi, A. (1986). Influential observations, high leverage points, and outliers in linear regression. Statistical Science, 1, 379-393. doi:10.1214/ss/1177013622
17. Chik, Z. (2002). The effect of outliers on the performance of order selection criteria for short time series. Pakistan Journal of Applied Sciences, 2, 912-915.
18. Coffman, D. L., & Millsap, R. E. (2006). Evaluating latent growth curve models using individual fit statistics. Structural Equation Modeling, 13, 1-27. doi:10.1207/s15328007sem1301_1
19. Cohen, A. S., & Cho, S.-J. (in press). Information criteria. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of item response theory: Models, statistical tools, and applications.
20. Cohen, J., Cohen, P., West, S., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Erlbaum.
21. Cook, R. (1977). Detection of influential observations in linear regression. Technometrics, 19, 15-18. doi:10.2307/1268249
22. Cook, R. (1986). Assessment of local influence (with discussion). Journal of the Royal Statistical Society, Series B, 48, 133-169.
23. Cook, R. D., & Wang, P. C. (1983). Transformations and influential cases in regression, Technometrics, 25, 337-343.
24. Draper, N. R., & John, J. A. (1981). Influential observations and outliers in regression. Technometrics, 23, 21-26.
25. Drasgow, F., Levine, M. V., & Zickar, M. J. (1996). Optimal identification of mismeasured individuals. Applied Measurement in Education, 9, 47-64.
26. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
27. Fitzmaurice, G., Laird, N., & Ware, J. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley.
28. Greenland, S. (1989). Modeling and variable selection in epidemiological analysis. American Journal of Public Health, 79, 340-349. doi:10.2105/ AJPH.79.3.340
29. Hamaker, E. L., van Hattum, P., Kuiper, R. M., & Hoijtink, H. (2011). Model selection based on information criteria in multilevel modeling. In J. Hox & J. K. Roberts (Eds.), Handbook of advanced multilevel analysis (pp. 231-255). New York, NY: Taylor & Francis.
30. Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis. New York, NY: Wiley.
31. Hoeting, J., Raftery, A. E., & Madigan, D. (1996). A method for simultaneous variable selection and outlier identification in linear regression. Computational Statistics and Data Analysis, 22, 251-270. doi:10.1016/ 0167-9473(95)00053-4
32. Johanson, G., & Alsmadi, A. (2002). Differential person functioning. Educational and Psychological Measurement, 62, 435-443.
33. Jöreskog, K. G., & Moustaki, I. (2001). Factor analysis of ordinal vari-ables: A comparison of three approaches. Multivariate Behavioral Research, 36, 347-387. doi:10.1207/S15327906347-387
34. Kang, T., & Cohen, A. S. (2007). IRT model selection methods for dichotomous items. Applied Psychological Measurement, 31, 331-358. doi:10.1177/0146621606292213
35. Kankaras, M., Moors, G., & Vermunt, J. K. (2009). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, & J. B. Billiet (Eds.), Cross-cultural analysis: Methods and applications. Routledge.
36. Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298. doi:10.1207/S15324818AME1604_2
37. Kuha, J. (2004). AIC and BIC: Comparisons of assumptions and performance. Sociological Methods & Research, 33, 188-229. doi:10.1177/ 0049124103262065
38. Lange, K., Westlake, J., & Spence, M. (1976). Extensions to pedigree analysis: III. Variance components by the scoring method. Annals of Human Genetics, 39, 485-491. doi:10.1111/j.1469-1809.1976.tb00156.x
39. Laud, P. W., & Ibrahim, J. G. (1995). Predictive model selection. Journal of the Royal Statistical Society, Series B, 57, 247-262.
40. Le, N. D., Raftery, A. E., & Martin, R. D. (1996). Robust Bayesian model selection for autoregressive processes with additive outliers. Journal of the American Statistical Association, 91, 123-131.
41. Lee, S. Y., & Lu, B. (2003). Case-deletion diagnostics for nonlinear structural equation models, Multivariate Behavioral Research, 38, 375- 400. doi:10.1207/S15327906MBR3803_05
42. Lee, S. Y., & Wang, S. J. (1996). Sensitivity analysis of structural equation models. Psychometrika, 61, 93-108. doi:10.1007/BF02296960
43. Lee, S. Y., & Xu, L. (2003a). Case-deletion diagnostic for factor analysis model with continuous and ordinal categorical data. Sociological Methods & Research, 31, 389-419. doi:10.1177/0049124102239081
44. Lee, S. Y., & Xu, L. (2003b). On local influence analysis of full information item factor models, Psychometrika, 68, 339-360. doi:10.1007/BF02294731
45. MacCallum, R. C. (2003). Working with imperfect models. Multivariate Behavioral Research, 38, 113-139. doi:10.1207/S15327906MBR3801_5
46. Mavridis, D., & Moustaki, I. (2008). Detecting outliers in factor analysis using the forward search algorithm. Multivariate Behavioral Research, 43, 453-475. doi:10.1080/00273170802285909
47. Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Mahwah, NJ: Erlbaum.
48. McArdle, J. J. (1997). Modeling longitudinal data by latent growth curve models. In G. Marcoulides (Ed.), New statistical models with business and economic applications (pp. 359-406). Mahwah, NJ: Erlbaum.
49. McCann, L. (2006). Robust model selection and outlier detection in linear regression. Doctoral dissertation, Massachusetts Institute of Technology, Cambridge, MA.
50. Meade, A., Ellington, J. K., & Craig, S. (2004). Exploratory measurement invariance: A new method based on item response theory. Symposium presented at the 19th annual Conference for Industrial and Organizational Psychology, Chicago, IL.
51. Meijer, R. R. (2003). Diagnosing item score patterns on a test using item response theory-based person-fit statistics. Psychological Methods, 8, 72-87. doi:10.1037/1082-989X.8.1.72
52. Meijer, R. R., & Sitjsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135. doi: 10.1177/01466210122031957
53. Millsap, R. E. (2010). Testing measurement invariance using item response theory in longitudinal data: An introduction. Child Development Perspectives, 4, 5-9. doi:10.1111/j.1750-8606.2009.00109.x
54. Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515. doi:10.1207/S15327906MBR3903_4
55. Moustaki, I., Jöreskog, K. G., & Mavridis, D. (2004). Factor models for ordinal variables with covariate effects on the manifest and latent variables: A comparison of LISREL and IRT approaches. Structural Equation Modeling, 11, 487-513. doi:10.1207/s15328007sem1104_1
56. Mulaik, S. A. (2009). Linear causal modeling with structural equations. Boca Raton, FL: Chapman & Hall.
57. Mullen, M. R., Milne, G. R., & Doney, P. M. (1995). An international marketing application of outlier analysis for structural equations: A methodological note. Journal of International Marketing, 3, 45-62.
58. Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557-585. doi:10.1007/BF02296397
59. Muthén, L. K., & Muthén, B. O. (1998-2011). Mplus user's guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
60. Myung, I. J., Forster, M., & Browne, M. W. (2000). Special issue on model selection. Journal of Mathematical Psychology, 44, 1-2. doi:10.1006/ jmps.1999.1273
61. Myung, I. J., Pitt, M. A., & Kim, W. (2005). Model evaluation, testing and selection. In K. Lambert & R. Goldstone (Eds.), The handbook of cognition, pp. 422-436. London, England: Sage.
62. Neale, M. (2000). Individual fit, heterogeneity, and missing data in multigroup structural equation modeling. In T. Little, K. Schnabel, & J. Baumert (Ed.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 249-267). Mahwah, NJ: Erlbaum.
63. Neale, M. C., Boker, S. M., Xie, G., & Maes, H. H. (2003). Mx: Statistical modeling (6th ed.). Richmond, VA: Virginia Commonwealth University Department of Psychiatry.
64. Panter, A. T., & Sterba, S. K. (2011). Handbook of ethics in quantitative methodology. New York, NY: Routledge.
65. Pek, J., & MacCallum, R. C. (2011). Sensitivity analysis in structural equation models: Cases and their influence. Multivariate Behavioral Research, 46, 202-228. doi:10.1080/00273171.2011.561068
66. Poon, W.-Y., & Poon, Y. S. (2002). Influential observations in the estimation of mean vector and covariance matrix. British Journal of Mathematical and Statistical Psychology, 55, 177-192. doi:10.1348/ 000711002159644
67. Pregibon, D. (1981). Logistic regression diagnostics. The Annals of Statistics, 9, 705-724. doi:10.1214/aos/1176345513
68. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111-163. doi:10.2307/271063
69. Reise, S. P. (2000). Using multilevel logistic regression to evaluate personfit in IRT models. Multivariate Behavioral Research, 35, 543-568. doi:10.1207/S15327906MBR3504_06
70. Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27-48. doi: 10.1146/annurev.clinpsy.032408.153553
71. Reise, S. P., & Widaman, K. F. (1999). Assessing the fit of measurement models at the individual level: A comparison of item response theory and covariance structure approaches. Psychological Methods, 4, 3-21. doi:10.1037/1082-989X.4.1.3
72. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566. doi: 10.1037/0033-2909.114.3.552
73. Rensvold, R. B., & Cheung, G. W. (1999). Identification of influential cases in structural equation modeling using the jackknife method. Organizational Research Methods, 2, 293-308. doi:10.1177/ 109442819923005
74. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351-357. doi:10.2307/ 2087176
75. Rodgers, J. L. (2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65, 1-12. doi:10.1037/a0018326
76. Ronchetti, E. (1997). Robustness aspects of model choice. Statistica Sinica, 7, 327-338.
77. Ronchetti, E., Field, C., & Blanchard, W. (1997). Robust linear model selection by cross-validation. Journal of the American Statistical Association, 92, 1017-1023.
78. Sadray, S., Jonsson, E. N., & Karlsson, M. O. (1999). Likelihood-based diagnostics for influential individuals in non-linear mixed effects model selection. Pharmaceutical Research, 16, 1260-1265. doi:10.1023/A: 1014857832337
79. Savalei, V., & Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150-170. doi:10.1037/1082-989X.13.2.150
80. Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464. doi:10.1214/aos/1176344136
81. Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London, England: Sage.
82. Stevens, J. P. (1984). Outliers and influential data points in regression analysis. Psychological Bulletin, 95, 334-344. doi:10.1037/00332909.95.2.334
83. Stoel, R. D., Garre, F. G., Dolan, C., & van den Wittenboer, G. (2006). On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. Psychological Methods, 11, 439- 455. doi:10.1037/1082-989X.11.4.439
84. Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society, Series B, 39, 44-47.
85. Swaminathan, H., Hambleton, R. K., & Rogers, H. J. (2007). Assessing the fit of item response theory models. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 683-718). Amsterdam, the Netherlands: Elsevier.
86. Tanaka, Y., Watadani, S., & Moon, S. H. (1991). Influence in covariance structure analysis: With an application to confirmatory factor analysis. Communications in Statistics: Theory and Method, 20, 3805-3821. doi:10.1080/03610929108830742
87. Thissen, D., Steinberg, L., & Gerrard, M. (1986). Beyond group-mean differences: The concept of item bias. Psychological Bulletin, 99, 118- 128. doi:10.1037/0033-2909.99.1.118
88. Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York, NY: Springer.
89. Wagenmakers, E., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11, 192-196. doi:10.3758/ BF03206482
90. Xu, L., Lee, S. Y., & Poon, W. Y. (2006). Deletion measures for generalized linear mixed effects models. Computational Statistics & Data Analysis, 51, 1131-1146. doi:10.1016/j.csda.2005.11.009
91. Yang, W., Tanaka, Y., & Nakaya, J. (2006). Forward search algorithm for robust influence analysis in maximum likelihood factor analysis. International Journal of Computer Science and Network Security, 6, 43-49.
92. Zhu, H. T., & Lee, S. Y. (2003). Local influence for generalized linear mixed models, Canadian Journal of Statistics, 31, 293-309. doi: 10.2307/3316088
93. Zu, J., & Yuan, K.-H. (2010). Local influence and robust procedures for mediation analysis. Multivariate Behavioral Research, 45, 1-44. doi: 10.1080/00273170903504695