Bayesian Factor Analysis as a Variable-Selection Problem: Alternative Priors and Consequences

Multivariate Behavioral Research

Volumn 51, Issue 4, 2016, Pages 519-539

  Lu, Zhao Hua ^a   Chow, Sy Miin ^a   Loken, Eric ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

Bayesian structural equation modeling; Factor analysis; Markov chain Monte Carlo algorithms; variable selection

Indexed keywords

CONTROLLED STUDY; EXPLORATORY RESEARCH; FACTORIAL ANALYSIS; HUMAN; HUMAN EXPERIMENT; MARKOV CHAIN; ROTATION; SPIKE; STRESS; STRUCTURAL EQUATION MODELING; TEACHER; THEORETICAL MODEL; ALGORITHM; BAYES THEOREM; COMPUTER SIMULATION; DECISION MAKING; EMPLOYMENT; FACTOR ANALYSIS; MENTAL STRESS; MONTE CARLO METHOD; PEER GROUP; PSYCHOLOGY; RECEIVER OPERATING CHARACTERISTIC; REGRESSION ANALYSIS; SCHOOL TEACHER; SOCIAL SUPPORT; STATISTICAL MODEL;

ALGORITHMS; BAYES THEOREM; COMPUTER SIMULATION; DECISION MAKING; EMPLOYMENT; FACTOR ANALYSIS, STATISTICAL; HUMANS; MODELS, STATISTICAL; MONTE CARLO METHOD; PEER GROUP; REGRESSION ANALYSIS; ROC CURVE; SCHOOL TEACHERS; SOCIAL SUPPORT; STRESS, PSYCHOLOGICAL;

EID: 84975121663     PISSN: 00273171     EISSN: None     Source Type: Journal    
DOI: 10.1080/00273171.2016.1168279     Document Type: Article

References (65)

1. H.Akaike, (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723. doi:10.1109/TAC.1974.1100705
2. D.G.Altman, & P.K.Andersen, (1989). Bootstrap investigation of the stability of a cox regression model. Statistics in Medicine, 8, 771–783. doi:10.1002/sim.4780080702
3. T.Asparouhov, & B.O.Muthén, (2009). Exploratory structural equation modeling. Structural Equation Modeling:A Multidisciplinary Journal, 16, 397–438. doi:10.1080/10705510903008204
4. T.Asparouhov, B.O.Muthén, & A.J.S.Morin, (2015). Bayesian structural equation modeling with cross-loadings and residual covariances:Comments on Stromeyer et al. Journal of Management, 0149206315591075. doi:10.1177/0149206315591075
5. P.M.Bentler, (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.107.2.238
6. J.O.Berger, & M.M.Barbieri, (2004). Optimal predictive model selection. The Annals of Statistics, 32, 870–897. doi:10.1214/009053604000000238
7. P.J.Bickel, B.Li, A.B.Tsybakov, S.A.V.D.Geer, S. A. v. d., B.Yu, T.Valdés, … A.V.D.Vaart, A. v. d. (2006). Regularization in statistics. Test, 15, 271–344. doi:10.1007/BF02607055
8. M.W.Browne, (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150. doi:10.1207/S15327906MBR3601_05
9. B.M.Byrne, (1994). Burnout:Testing for the validity, replication, and invariance of causal structure across elementary, intermediate, and secondary teachers. American Educational Research Journal, 31, 645–673. doi:10.3102/00028312031003645
10. B.M.Byrne, (2012). Structural equation modeling with Mplus:Basic concepts, applications, and programming. New York, NY:Routledge, Taylor & Francis Group, LLC.
11. J.-H.Cai, X.-Y.Song, & S.-Y.Lee, (2008). Bayesian analysis of nonlinear structural equation models with mixed continuous, ordered and unordered categorical, and nonignorable missing data. Statistics and Its Interface, 1, 99–114. doi:10.4310/SII.2008.v1.n1.a9
12. H.Cham, S.G.West, Y.Ma, & L.S.Aiken, (2012). Estimating latent variable interactions with nonnormal observed data:A comparison of four approaches. Multivariate Behavioral Research, 47, 840–876. doi:10.1080/00273171.2012.732901
13. C.Chatfield, (1995). Model uncertainty, data mining and statistical inference. Journal of the Royal Statistical Society. Series A (Statistics in Society), 158, 419–466. doi:10.2307/2983440
14. S.Chatterjee, & A.S.Hadi, (2012). Regression analysis by example (5th ed.). Hoboken, NJ:John Wiley & Sons, Inc.
15. D.Draper, (1995). Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 45–97. 01394. Retrieved from http://www.jstor.org/stable/2346087
16. B.Efron, T.Hastie, I.Johnstone, & R.Tibshirani, (2004). Least angle regression. The Annals of Statistics, 32, 407–499. doi:10.1214/009053604000000067
17. J.Friedman, T.Hastie, & R.Tibshirani, (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33, 1–22. doi:10.18637/jss.v033.i01
18. A.Gelman, (1996). Inference and monitoring convergence. In W.R.Gilks, S.Richardson & D.J.Spiegelhalter (Eds.), Markov chain Monte Carlo in practice (pp. 131–143). Boca Raton, FL:CRC.
19. A.Gelman, (2006). Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper). Bayesian Analysis, 1, 515–534. doi:10.1214/06-BA117A
20. E.I.George, & R.E.McCulloch, (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889. doi:10.1080/01621459.1993.10476353
21. P.J.Green, (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 711–732. doi:10.1093/biomet/82.4.711
22. M.Hallquist, & J.Wiley, (2014). MplusAutomation:Automating Mplus model estimation and interpretation. R package version 0.6-2. Retrieved from http://CRAN.R-project.org/package=MplusAutomation
23. G.R.Hancock, (1999). A sequential Scheffé-type respecification procedure for controlling Type I error in exploratory structural equation model modification. Structural Equation Modeling:A Multidisciplinary Journal, 6, 158–168. doi:10.1080/10705519909540126
24. F.E.Harrell, (2001). Regression modeling strategies:With applications to linear models, logistic regression, and survival analysis. New York, NY:Springer-Verlag New York, Inc.
25. T.Hastie, R.Tibshirani, J.Friedman, T.Hastie, J.Friedman, & R.Tibshirani, (2009). The elements of statistical learning (2nd ed.). New York, NY:Springer Science+Business Media, LCC.
26. A.E.Hoerl, & R.W.Kennard, (1970). Ridge regression:Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67. doi:10.1080/00401706.1970.10488634
27. J.A.Hoeting, D.Madigan, A.E.Raftery, & C.T.Volinsky, (1999). Bayesian model averaging:A tutorial. Statistical Science, 14, 382–417. doi:10.1214/ss/1009212519
28. L.-T.Hu, & P.M.Bentler, (1995). Evaluating model fit. In Structural equation modeling:Concepts, issues, and applications (pp. 76–99). Thousand Oaks, CA:Sage Publications, Inc.
29. L.-T.Hu, & P.M.Bentler, (1998). Fit indices in covariance structure modeling:sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453. doi:10.1037/1082-989X.3.4.424
30. L.-T.Hu, & P.M.Bentler, (1999). Cutoff criteria for fit indexes in covariance structure analysis:Conventional criteria versus new alternatives. Structural Equation Modeling:A Multidisciplinary Journal, 6, 1–55. doi:10.1080/10705519909540118
31. J.G.Ibrahim, M.-H.Chen, & D.Sinha, (2001). Criterion-based methods for Bayesian model assessment. Statistica Sinica, 11(2), 419–444. Retrieved from:http://www.jstor.org/stable/24306870
32. H.Ishwaran, & J.S.Rao, (2005). Spike and slab variable selection:Frequentist and Bayesian strategies. The Annals of Statistics, 33, 730–773. doi:10.1214/009053604000001147
33. R.I.Jennrich, & P.F.Sampson, (1966). Rotation for simple loadings. Psychometrika, 31, 313–323. doi:10.1007/BF02289465
34. K.G.Jüreskog, (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202. doi:10.1007/BF02289343
35. K.G.Jüreskog, & D.Sürbom, (1996). LISREL 8:User's reference guide. Lincolnwood, IL:Scientific Software International, Inc.
36. K.Kamary, K.Mengersen, C.P.Robert, & J.Rousseau, (2014). Testing hypotheses via a mixture estimation model. arXiv:1412.2044 [stat]. 00004. arXiv:1412.2044. Retrieved from http://arxiv.org/abs/1412.2044
37. Y.Kano, (2002). Variable selection for structural models. Journal of Statistical Planning and Inference. C.R. Rao 80th Birthday Felicitation Volume, Part II, 108, 173–187. doi:10.1016/S0378-3758(02)00277-X
38. R.E.Kass, & A.E.Raftery, (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. doi:10.1080/01621459.1995.10476572
39. D.A.Kenny, & C.M.Judd, (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 96, 201–210. doi:10.1037/0033-2909.96.1.201
40. M.Kyung, J.Gill, M.Ghosh, & G.Casella, (2010). Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis, 5, 369–411. doi:10.1214/10-BA607
41. S.-Y.Lee, (2007). Structural equation modeling:A Bayesian approach. Chichester, West Sussex, England:John Wiley & Sons.
42. C.Leng, M.-N.Tran, & D.Nott, (2014). Bayesian adaptive lasso. Annals of the Institute of Statistical Mathematics, 66, 221–244. doi:10.1007/s10463-013-0429-6
43. Q.Li, & N.Lin, (2010). The Bayesian elastic net. Bayesian Analysis, 5, 151–170. doi:10.1214/10-BA506
44. Y.-X.Li, Y.Kano, J.-H.Pan, & X.-Y.Song, (2012). A criterion-based model comparison statistic for structural equation models with heterogeneous data. Journal of Multivariate Analysis, 112, 92–107. doi:10.1016/j.jmva.2012.05.010
45. R.C.MacCallum, M.Roznowski, & L.B.Necowitz, (1992). Model modifications in covariance structure analysis:The problem of capitalization on chance. Psychological Bulletin, 111, 490–504. doi:10.1037/0033-2909.111.3.490
46. D.Mavridis, & I.Ntzoufras, (2014). Stochastic search item selection for factor analytic models. British Journal of Mathematical and Statistical Psychology, 67, 284–303. doi:10.1111/bmsp.12019
47. T.J.Mitchell, & J.J.Beauchamp, (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83, 1023–1032. doi:10.1080/01621459.1988.10478694
48. B.O.Muthén, & T.Asparouhov, (2012). Bayesian structural equation modeling:A more flexible representation of substantive theory. Psychological Methods, 17, 313–335. doi:10.1037/a0026802
49. R.B.O'Hara, & M.J.Sillanpää, (2009). A review of Bayesian variable selection methods:What, how and which. Bayesian Analysis, 4, 85–117. doi:10.1214/09-BA403
50. X.Pang, & J.Gill, (2009). Spike and slab prior distributions for simultaneous Bayesian hypothesis testing, model selection, and prediction, of nonlinear outcomes. Unpublished manuscript. Washington University in St. Louis, St. Louis, MO. Retrieved from http://polmeth.wustl.edu/node/303
51. T.Park, & G.Casella, (2008). The Bayesian lasso. Journal of the American Statistical Association, 103, 681–686. doi:10.1198/016214508000000337
52. L.S.Pettegrew, & G.E.Wolf, (1982). Validating measures of teacher stress. American Educational Research Journal, 19, 373–396. doi:10.3102/00028312019003373
53. N.G.Polson, & J.G.Scott, (2011). Shrink globally, act locally:Sparse Bayesian regularization and prediction. In J.M.Bernardo, M.J.Bayarri, J.O.Berger, A.P.Dawid, D.Heckerman, A.F.M.Smith, & M.West (Eds.), Bayesian statistics (Vol. 9, pp. 501–538). Oxford, UK:Oxford University Press.
54. R Core Team. (2013). R:A language and environment for statistical computing. Vienna, Austria:R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
55. W.E.Saris, A.Satorra, & W.M.van der Veld, (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling:A Multidisciplinary Journal, 16, 561–582. doi:10.1080/10705510903203433
56. G.Schwarz, (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. doi:10.1214/aos/1176344136
57. S.A.Sisson, (2005). Transdimensional Markov chains:A decade of progress and future perspectives. Journal of the American Statistical Association, 100, 1077–1089. 00112. doi:10.1198/016214505000000664
58. X.-Y.Song, & S.-Y.Lee, (2006). A maximum likelihood approach for multisample nonlinear structural equation models with missing continuous and dichotomous data. Structural Equation Modeling:A Multidisciplinary Journal, 13, 325–351. doi:10.1207/s15328007sem1303_1
59. D.Sürbom, (1989). Model modification. Psychometrika, 54, 371–384. doi:10.1007/BF02294623
60. D.J.Spiegelhalter, N.G.Best, B.P.Carlin, & A.Van Der Linde, (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society:Series B (Statistical Methodology), 64, 583–639. doi:10.1111/1467-9868.00353
61. J.H.Steiger, (1990). Structural model evaluation and modification:An interval estimation approach. Multivariate Behavioral Research, 25, 173–180. doi:10.1207/s15327906mbr2502_4
62. R.Tibshirani, (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1), 267–288. Retrieved from http://www.jstor.org/stable/2346178
63. L.R.Tucker, & C.Lewis, (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. doi:10.1007/BF02291170
64. H.Zou, (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429. doi:10.1198/016214506000000735
65. H.Zou, & T.Hastie, (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society:Series B (Statistical Methodology), 67, 301–320. 03906. doi:10.1111/j.1467-9868.2005.00503.x