Overview of Statistical Estimation Methods

The Oxford Handbook of Quantitative Methods: Volume 1: Foundations

Volumn 1, Issue , 2013, Pages 361-387

  Yuan, Ke Hai ^a   Schuster, Christof ^b  

^a UNIVERSITY OF NOTRE DAME   (United States)

^b JUSTUS LIEBIG UNIVERSITY   (Germany)

Author keywords

Bayes methods; bootstrap; EM algorithm; estimating equations; generalized least squares; Markov chain Monte Carlo; Maximum likelihood; Newton algorithm; pseudo maximum likelihood; robust M estimators; method

Indexed keywords

EID: 85103370281     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1093/oxfordhb/9780199934874.013.0018     Document Type: Chapter

References (102)

1. Albert, J. H. (1992). Bayesian analysis of binary and poly-chotomous response data. Journal of the American Statistical Association, 88, 669—679.
2. Baker, F. B., and Kim, S. H. (2004). Item response theory: Parameter estimation techniques (2nd ed). New York: Marcel Dekker.
3. Bartlett, M.S. (1937). Properties of sufficiency and statistical tests. Proceedings of the Royal Society of London, A160, 268-282.
4. Bentler, P. M., and Tanaka, J. S. (1983). Problems with EM algorithms for ML factor analysis. Psychometrika, 48, 247-251.
5. Berger, J. O., and Berliner, L. M. (1986). Robust Bayes and empirical Bayes analysis with ĉ-contaminated priors. Annals of Statistics, 14, 461—486.
6. Berry, D. A. (1996). Statistics: A Bayesian perspective. Belmont, CA: Duxbury Press.
7. Bock, R. D., and Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179— 197.
8. Bock, R. D., and Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443—459.
9. Bollen, K.A. (1996). An alternative two stage least squares (2SLS) estimator for latent variable equations. Psychometrika, 61, 109-12.
10. Box, G.E.P., and Tiao, G. C. (1973). Bayesian inference statistical analysis. New York: Wiley.
11. Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62-83.
12. Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 75, 33-57.
13. Cheng, Y., and Yuan, K.-H. (2010). The impact of fallible item parameter estimates on latent trait recovery. Psychometrika, 75, 280-291.
14. Carlin, B. P., and Louis, T. A. (2009). Bayesian methods for data analysis (3rd rd.). Boca Raton, FL: Chapman and Hall/CRC.
15. Casella, G., and George, E. I. (1992). Explaining the Gibbs sampler. American Statistician, 46, 167-174.
16. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society Series B, 39, 1-38.
17. Draper, N. R., and van Nostrand, C. R. (1979). Ridge regression and James-Stein estimation: Review and comments. Technometrics, 21, 451—466.
18. Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82, 171-185.
19. Efron, B. (1988). Bootstrap confidence intervals: Good or bad (with discussion)? Psychological Bulletin, 104, 293-296.
20. Efron, B., and Morris, C. (1973). Stein’s estimation rule and its competitors-An empirical Bayes approach. Journal of the American Statistical Association, 68, 117-130.
21. Efron, B. and Morris, C. (1977). Stein’s paradox in statistics. Scientific American, 236, 119-127.
22. Efron, B. and Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman and Hall.
23. Everitt, B. S. (1984). Maximum likelihood estimation of the parameters in a mixture of two univariate normal distributions; a comparison of different algorithms. Statistician, 33, 205-215.
24. Everitt, B. S. (1987). Introduction to optimization methods and their application in statistics. London: Chapman and Hall.
25. Ferguson, T. S. (1967). Mathematical statistics: A decision theoretic approach. New York: Academic Press.
26. Geary, R. C. (1947). Testingfor normality. Biometrika, 34, 209-242.
27. Gelfand, A. E., and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal ofthe American Statistical Association, 85, 398—409.
28. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004). Bayesian data analysis (2nd ed.). Boca Raton, FL: Chapman and Hall/CRC.
29. Geman, S., and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Mechine Intelligence, 721-741.
30. Godambe, V. P. (1960). An optimum property of regular maximum likelihood estimation. Annals of Mathematical Statistics, 31, 1208-1211.
31. Gong, G., and Samaniego, F. J. (1981). Pseudo maximum likelihood estimation: theory and applications, Annals of Statistics, 9, 861-869.
32. Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 215-231.
33. Gourieroux, C., Monfort, A., and Trognon, A. (1984). Pseudo maximum likelihood methods: Theory. Econometrica, 52, 681-700.
34. Green, P. J. (1984). Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives (with discussion). Journal of the Royal Statistical Society B, 46, 149-192.
35. Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Annals of Statistics, 8, 586-597.
36. Harville, D.A. (1977). Maximum likelihood approaches to variance component estimation and to related problems (with discussions). Journal of the American Statistical Association, 72, 320-340.
37. Hayashi, K., and Yuan, K.-H. (2003). Robust Bayesian factor analysis. Structural Equation Modeling, 10, 525-533.
38. Hoaglin, D. C., Mosteller, F., and Tukey, J. W. (1983). Understanding Robust and Exploratory Data Analysis. Wiley.
39. Hoaglin, D. C., Mosteller, F., and Tukey, J. W. (1983). Understanding robust and exploratory data analysis. New York: Wiley.
40. Hoerl, A. E., and Kennard, R. W (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55-67.
41. Holland, P. W., and Welsch, R. E. (1977). Robust regression using iteratively reweighted least-squares. Communications in Statistics-Theory and Methods, A6, 813-827.
42. Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (pp. 221-233). Berkeley, CA: University of California Press.
43. Huber, P. J. (1981). Robust statistics. New York: Wiley.
44. James, W., and Stein, C. (1961).Estimation with quadratic loss. In Proceedings ofthe fourth Berkeley Symposium on Mathematical Statistics and Probability (pp. 361-379). Berkeley, CA: University of California Press.
45. Jamshidian, M., and Bentler, P. M. (1999). Using complete data routines for ML estimation of mean and covariance structures with missing data. Journal Educational and Behavioral Statistics, 23, 21-41.
46. Jamshidian, M., and Jennrich, R. I. (1997). Acceleration of the EM algorithm by using Quasi-Newton methods. Journal of the Royal Statistical Society B, 59, 569-587.
47. Jöreskog, K. G. (1994). On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika, 59, 381-390.
48. Kano, Y., Berkane, M., and Bentler, P. M. (1993). Statistical inference based on pseudo-maximum likelihood estimators in elliptical populations. Journal of the American Statistical Association, 88, 135-143.
49. Kelley, C. T. (2003). Solving nonlinear equations with Newton’s method. Philadelphia: SIAM.
50. Klein, A., and Moosbrugger, H. (2000). Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika, 65, 457-474.
51. Laird, N. M., and Louis, T. A. (1987). Empirical Bayes confidence intervals based on bootstrap samples (with discussion). Journal of the American Statistical Association, 82, 739-757.
52. Lee S.-Y. (2007). Structural equation modeling: A Bayesian approach. Chichester: Wiley.
53. Lee, Y., and Nelder, J. A. (1996). Hierarchical generalized linear models (with discussion). Journal of the Royal Statistical Society B, 58, 619-678.
54. Lee, S.-Y., Poon, W.-Y., and Bentler, P. M. (1995). A two-stage estimation of structural equation models with continuous and polytomous variables. British Journal of Mathematical and Statistical Psychology, 48, 339-358.
55. Lee, S.-Y., and Song, X.-Y. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39, 653-686.
56. Lee, S.-Y., and Zhu, H.-T. Maximum likelihood estimation of nonlinear structural equation models. Psychometrika, 67, 189-210.
57. Liang, K. Y., and Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22.
58. Little, R.J.A. (1988). Robust estimation of the mean and covariance matrix from data with missing values. Applied Statistics, 37, 23-38.
59. Little, R. J. A., and Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: Wiley.
60. Maydeu-Olivares, A. (2006). Limited information estimation and testing of discretized multivariate normal structural models. Psychometrika, 71, 57-77.
61. McCullagh, P., and Nelder, J. A. (1989). Generalized linear models (2nd ed.). London: Chapman and Hall.
62. Mammen, E. (1992). When does bootstrap work? New York: Springer.
63. Maronna, R. A. (1976). Robust M-estimators of multivariate location and scatter. Annals of Statistics, 4, 51-67.
64. Maronna, R. A., Martin, R. D., and Yohai, V. J. (2006). Robust statistics: Theory and methods. New York: Wiley.
65. McLachlan, G. J., and Krishnan, T. (2008). The EM algorithm and extensions (2nd ed.). New York: Wiley.
66. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166.
67. Mislevy, R. J., and Bock, R. D. (1982). Biweight estimates of latent ability. Educational and Psychological Measurement, 42, 725-737.
68. Muthén, B., and Satorra, A. (1995). Technical aspects of Muthén’s LISCOMP approach to estimation of latent variable relations with a comprehensive measurement model. Psychometrika, 60, 489-503.
69. Nelder, J. A. and Lee, Y. (1992). Likelihood, quasi-likelihood and pseudo likelihood: Some comparisons. Journal of the Royal Statistical Society B, 54, 273-284.
70. Neyman, J., and Scott, E. L. (1948). Consistent estimates based on partially consistent observations. Econometrika, 16, 1-32.
71. Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44, 443-460.
72. Papp, R. (2002). What are the advantage of MCMC based inference in latent variable models? Statistica Neerlandica, 56, 2-22.
73. Parke, W. R. (1986). Pseudo maximum likelihood estimation: the asymptotic distribution, Annals of Statistics, 14, 355-357.
74. Patterson, H. D., and Thompson, R. (1971). Recovery of interblock information when block sizes are unequal. Biometrika, 58, 545-554.
75. Patz, R. J., and Junker, B. W. (1999). A straightforward approach to Markov Chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146-178.
76. Pawitan, Y. (2001). In all likelihood: Statistical modelling and inference using likelihood. New York: Oxford University Press.
77. Poon, W.-Y., and Lee, S.-Y. (1987). Maximum likelihood estimation of multivariate polyserial and polychoric correlation coefficient. Psychometrika, 52, 409-430.
78. Rasmussen, J. (1987). Estimating correlation coefficients: Bootstrap and parametric approaches. Psychological Bulletin, 101, 136-139.
79. Rubin, D. B., and Thayer, D. T. (1982). EM algorithms for ML factor analysis. Psychometrika, 47, 69-76.
80. Schuster, C., and Yuan, K.-H. (2011). Robust estimation oflatent ability in item response models. Journal of Educational and Behavioral Statistics, 36, 720-735.
81. Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate distribution. In Proceedings of the third Berkeley symposium on mathematical statistics and probability, vol. I, (pp. 197-206). Berkeley and Los Angeles: University of California Press.
82. Tanner, M. A. (1996). Tools for statistical inference: Methods for the exploration of posterior distributions and likelihood functions (3rd ed.). New York: Springer-Verlag.
83. Tanner, M. A., and Wong, W. (1987). The Calculation of Posterior Distributions by Data Augmentation (with discussion). Journal of the American Statistical Association, 82, 528-550.
84. Tatsuoka, K. K., and Tatsuoka, M. M. (1982). Detection of aberrant response patterns and their effect on dimensionality. Journal of Educational Statistics, 7, 215-231.
85. Wainer, H., and Wright, B. D. (1980). Robust estimation of ability in the Rasch model. Psychometrika, 45, 373-391.
86. White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1-25.
87. Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.). Burlington, MA: Academic Press.
88. Yuan, K.-H. (2009). Normal distribution based pseudo ML for missing data: With applications to mean and covariance structure analysis. Journal of Multivariate Analysis, 100, 1900-1918.
89. Yuan, K.-H., and Bentler, P. M. (1998). Structural equation modeling with robust covariances. Sociological Methodology, 28, 363-396.
90. Yuan, K.-H., and Bentler, P. M. (2000). Robust mean and covariance structure analysis through iteratively reweighted least squares. Psychometrika, 65, 43-58.
91. Yuan, K.-H., and Bentler, P. M. (2010). Consistency of normal distribution based pseudo maximum likelihood estimates when data are missing at random. American Statistician, 64, 263-267.
92. Yuan, K.-H., Bentler, P. M., and Chan, W. (2004). Structural equation modeling with heavy tailed distributions. Psychometrika, 69, 421—436.
93. Yuan, K.-H., Bentler, P. M., and Zhang, W. (2005). The effect of skewness and kurtosis on mean and covariance structure analysis: The univariate case and its multivariate implication. Sociological Methods and Research, 34, 249-258.
94. Yuan, K.-H., and Bushman, B.J. (2002). Combining standardized mean differences using the method of maximum likelihood. Psychometrika, 67, 589—607.
95. Yuan, K.-H., and Chan, W. (2005). On nonequivalence of several procedures of structural equation modeling. Psychometrika, 70, 791—798.
96. Yuan, K.-H., and Chan, W. (2008). Structural equation modeling with near singular covariance matrices. Computational Statistics and Data Analysis, 52, 4842-4858.
97. Yuan, K.-H., and Hayashi, K. (2006). Standarderrorsincovariance structure models: Asymptotics versus bootstrap. British Journal of Mathematical and Statistical Psychology, 59, 397—417.
98. Yuan, K.-H., and Jennrich, R. I. (1998).A symptotics of estimating equations under natural conditions. Journal of Multivariate Analysis, 65, 245—260.
99. Yuan, K.-H., and Jennrich, R. I. (2000). Estimating equations with nuisance parameters: Theory and applications. Annals of the Institute of Statistical Mathematics, 52, 343—350.
100. Yuan, K.-H., Wu, R., and Bentler, P. M. (2011). Ridge structural equation modeling with correlation matrices for ordinal and continuous data. British Journal of Mathematical and Statistical Psychology, 64, 107—133.
101. Yuan, K.-H., and Zhong, X. (2008). Outliers, leverage observations and influential cases in factor analysis: Minimizing their effect using robust procedures. Sociological Methodology, 38, 329—368.
102. Zu, J., and Yuan, K.-H. (2010). Local influence and robust procedures for mediation analysis. Multivariate Behavioral Research, 45, 1-44.