Parameterizations make different model selections: Empirical findings from factor analysis

Frontiers of Electrical and Electronic Engineering in China

Volumn 6, Issue 2, 2011, Pages 256-274

  Tu, Shikui ^a   Xu, Lei ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Bayesian Ying Yang learning; factor analysis; maximum likelihood; model selection; parameterizations; variational Bayes

Indexed keywords

EID: 79958741453     PISSN: 16733460     EISSN: 16733584     Source Type: Journal    
DOI: 10.1007/s11460-011-0150-2     Document Type: Article

References (32)

1. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 1974, 19(6): 716-723.
2. Schwarz G. Estimating the dimension of a model. Annals of Statistics, 1978, 6(2): 461-464.
3. Rissanen J. Modelling by the shortest data description. Automatica, 1978, 14(5): 465-471.
4. Anderson T, Rubin H. Statistical inference in factor analysis. In: Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability. 1956, 5: 111-150.
5. Fodor I K. A survey of dimension reduction techniques. Technical Report UCRL-ID-148494. 2002.
6. Jolliffe I T. Principal Component Analysis. 2nd ed. New York: Springer, 2002.
7. Tipping M E, Bishop C M. Mixtures of probabilistic principal component analyzers. Neural Computation, 1999, 11(2): 443-482.
8. Bishop C M. Variational principal components. In: Proceedings of the Ninth International Conference on Artificial Neural Networks. 1999, 509-514.
9. Ghahramani Z, Beal M J. Variational inference for Bayesian mixtures of factor analysers. Advances in Neural Information Processing System, 2000, 12: 449-455.
10. Nielsen F B. Variational approach to factor analysis and related models. Dissertation for the Master's Degree. Lyngby: Technical University of Denmark, 2004.
11. Hills S E, Smith A F. Parameterization issues in Bayesian inference. Bayesian Statistics, 1992, 4: 227-246.
12. Kass R E, Slate E H. Reparameterization and diagnostics of posterior nonnormality. Bayesian Statistics, 1992, 4: 289-305.
13. Gelman A. Parameterization and Bayesian modeling. Journal of the American Statistical Association, 2004, 99(466): 537-545.
14. Ghosh J, Dunson D B. Default prior distributions and efficient posterior computation in Bayesian factor analysis. Journal of Computational and Graphical Statistical Statistics, 2009, 18(2): 306-320.
15. Xu L. Bayesian Ying Yang system and theory as a unified statistical learning approach: (i) unsupervised and semiunsupervised learning. In: Amari S, Kassabov N, eds. Brainlike Computing and Intelligent Information Systems. Berlin: Springer-Verlag, 1997, 241-274.
16. Xu L. Bayesian Ying-Yang learning theory for data dimension reduction and determination. Journal of Computational Intelligence in Finance, 1998, 6(5): 6-18.
17. Xu L. BYY harmony learning, independent state space, and generalized APT financial analyses. IEEE Transactions on Neural Networks, 2001, 12(4): 822-849.
18. Hu X, Xu L. A comparative investigation on subspace dimension determination. Neural Networks, 2004, 17(8-9): 1051-1059.
19. Shi L, Xu L. Local factor analysis with automatic model selection: a comparative study and digits recognition application. In: Proceedings Part II of the 16th International Conference on Artificial Neural Networks. 2006, 260-269.
20. Jordan M I, Ghahramani Z, Jaakkola T S, Saul L K. An introduction to variational methods for graphical models. Machine Learning, 1999, 37(2): 183-233.
21. Beal M J. Variational algorithms for approximate Bayesian inference. Dissertation for the Doctoral Degree. London: University College London, 2003.
22. Xu L. Bayesian Ying-Yang system, best harmony learning, and five action circling. Frontiers of Electrical and Electronic Engineering in China, 2010, 5(3): 281-328.
23. Xu L. Fundamentals, challenges, and advances of statistical learning for knowledge discovery and problem solving: a BYY harmony perspective. In: Proceedings of International Conference on Neural Networks and Brain. 2005, 1: 24-55.
24. Rubin D B, Thayer D T. EM algorithm for ML factor analysis. Psychometrika, 1982, 47(1): 69-76.
25. Bozdogan H, Ramirez D E. FACAIC: model selection algorithm for the orthogonal factor model using AIC and FACAIC. Psychometrika, 1988, 53(3): 407-415.
26. Tu S, Xu L. A study of several model selection criteria for determining the number of signals. In: Proceedings of IEEE International Conference on Acoustics Speech and Signal Processing. 2010, 1966-1969.
27. Xu L. Bayesian-Kullback coupled YING-YANG machines: unified learning and new results on vector quantization. In: Proceedings of International Conference on Neural Information Processing. 1995, 977-988.
28. Xu L. Codimensional matrix pairing perspective of BYY harmony learning: hierarchy of bilinear systems, joint decomposition of data-covariance, and applications of network biology. Frontiers of Electrical and Electronic Engineering in China, 2011, 6(1): 86-119.
29. Xu L. Bayesian Ying Yang learning. Scholarpedia, 2007, 2(3): 1809.
30. Xu L. Bayesian Ying Yang system, best harmony learning, and Gaussian manifold based family. In: Zurada J, Yen G, Wang J, eds. Computational Intelligence: Research Frontiers. Berlin-Heidelberg: Springer-Verlag, 2008, 5050: 48-78.
31. Tu S, Xu L. An investigation of several typical model selection criteria for detecting the number of signals. Frontiers of Electrical and Electronic Engineering in China, 2011 (in Press).
32. Asuncion A, Newman D. UCI machine learning repository, 2007. http://www. ics. uci. edu/~mlearn/MLRepository. html.