Logistic regression in data analysis: An overview

International Journal of Data Analysis Techniques and Strategies

Volumn 3, Issue 3, 2011, Pages 281-299

  Maalouf, Maher ^a  

^a University of Oklahoma   (United States)

Author keywords

Classification; Data mining; Imbalanced data; Logistic regression; LR; Rare events

Indexed keywords

EID: 84856707986     PISSN: 17558050     EISSN: 17558069     Source Type: Journal    
DOI: 10.1504/IJDATS.2011.041335     Document Type: Review

References (69)

1. Agresti, A. (2002) Categorical Data Analysis, 2nd ed., Wiley-Interscience.
2. Agresti, A. (2007) An Introduction to Categorical Data Analysis, Wiley-Interscience.
3. Amemiya, T. (1985) Advanced Econometrics, Harvard University Press.
4. Ben-Akiva, M. and Lerman, S. (1985) Discrete Choice Analysis: Theory and Application to Travel Demand, The MIT Press.
5. Berk, R. (2008) Statistical Learning from a Regression Perspective, 1st ed., Springer.
6. Berthold, M.R. and Hand, D.J. (Eds.) (2010) Intelligent Data Analysis, 2nd ed., Springer.
7. Bi, Y. and Jeske, D.R. (2010)'The efficiency of logistic regression compared to normal discriminant analysis under class-conditional classification noise', Journal of Multivariate Analysis, Vol. 101, No. 7, pp.1622-1637.
8. Bishop, C.M. (2006) Pattern Recognition and Machine Learning, Springer.
9. Brazdil, P., Giraud-Carrier, C., Soares, C. and Vilalta, R. (2008) Metalearning: Applications to Data Mining, 1st ed., Springer.
10. Cameron, A.C. and Trivedi, P.K. (2005) Microeconometrics: Methods and Applications, Cambridge University Press.
11. Canu, S. and Smola, A.J. (2005)'Kernel methods and the exponential family', in ESANN, pp.447-454.
12. Collett, D. (2003) Modelling Binary Data, 2nd ed., Chapman & Hall/CRC.
13. Collins, M., Schapire, R.E. and Singer, Y. (2002)'Logistic regression, adaboost and bregman distances', Machine Learning, Vol. 48, pp.253-285. (Pubitemid 34247580)
14. Cordeiro, G. and Barroso, L. (2007)'A third-order bias corrected estimate in generalized linear models', TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Vol. 16, No. 1, pp.76-89.
15. Cordeiro, G.M. and McCullagh, P. (1991)'Bias correction in generalized linear models', Journal of Royal Statistical Society, Vol. 53, No. 3, pp.629-643.
16. Cosslett, S.R. (1981) Structural Analysis of Discrete Data and Econometric Applications, The MIT Press, Cambridge.
17. Cowan, G. (1998) Statistical Data Analysis, Oxford University Press.
18. Cox, D.R. and Hinkley, D.V. (1979) Theoretical Statistics, Chapman & Hall/CRC.
19. Cox, D.R. and Snell, E.J. (1968)'A general definition of residuals', Journal of the Royal Statistical Society, Vol. 30, No. 2, pp.248-275.
20. Cramer, J.S. (2003) Logit Models From Economics and other Fields, Cambridge University Press.
21. Efron, B. and Tibshirani, R.J. (1994) An Introduction to the Bootstrap, Chapman & Hall/CRC.
22. Fan, W., Stolfo, S.J., Zhang, J. and Chan, P.K. (1999)'Adacost: misclassification cost-sensitive boosting', in Proceedings to the 16th International Conference on Machine Learning, Morgan Kaufmann, pp.97-105.
23. Firth, D. (1993)'Bias reduction of maximum likelihood estimates', Biometrika, Vol. 80, pp.27-38.
24. Freund, Y. and Schapire, R.E. (1999)'A brief introduction to boosting', in Proceedings of the 16th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, pp.1401-1406.
25. Friedman, J., Hastie, T. and Tibshirani, R. (2000)'Additive logistic regression: a statistical view of boosting', The Annals of Statistics, Vol. 38, No. 2, pp.337-374.
26. Garthwaite, P., Jolliffe, I. and Byron, J. (2002) Statistical Inference, Oxford University Press.
27. Hastie, T., Tibshirani, R. and Friedman, J. (2009) The Elements of Statistical Learning, 2nd ed., Springer Verlag.
28. Heinze, G. and Schemper, M. (2001)'A solution to the problem of monotone likelihood in COX regression', Biometrics, Vol. 57, pp.114-119. (Pubitemid 32200144)
29. Hilbe, J.M. (2009) Logistic Regression Models, Chapman & Hall/CRC.
30. Horton, N.J. and Kleinman, K.P. (2007)'Much ado about nothing: a comparison of missing data methods and software to fit incomplete data regression models', The American Statistician, Vol. 61, No. 1, pp.79-90.
31. Hosmer, D.W. and Lemeshow, S. (2000) Applied Logistic Regression, 2nd ed., Wiley.
32. Imbens, G.W. (1992)'An efficient method of moments estimator for discrete choice models with choice-based sampling', Econometrica, September, Vol. 60, No. 5, pp.1187-1214.
33. Imbens, G.W. and Lancaster, T. (1996)'Efficient estimation and stratified sampling', Journal of Econometrics, Vol. 74, pp.289-318. (Pubitemid 126375525)
34. Jaakkola, T. and Haussler, D. (1999) Probabilistic Kernel Regression Models.
35. Joshi, M.V., Agarwal, R.C. and Kumar, V. (2002)'Predicting rare classes: can boosting make any weak learner strong?', in KDD'02: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, ACM, pp.297-306.
36. Joshi, M.V., Kumar, V. and Agarwal, R.C. (2001)'Evaluating boosting algorithms to classify rare classes: comparison and improvements', in ICDM'01: Proceedings of the 2001 IEEE International Conference on Data Mining, Washington, DC, USA, pp.257-264.
37. Karsmakers, P., Pelckmans, K. and Suykens, J.A.K. (2007)'Multi-class kernel logistic regression: a fixed-size implementation', International Joint Conference on Neural Networks, pp.1756-1761.
38. Keerthi, S.S., Duan, K.B., Shevade, S.K. and Poo, A.N. (2005)'A fast dual algorithm for kernel logistic regression', Mach. Learn., Vol. 61, Nos. 1-3, pp.151-165. (Pubitemid 43049366)
39. King, G. and Zeng, L. (2001)'Logistic regression in rare events data', Political Analysis, Vol. 9, pp.137-163.
40. Kleinbaum, D.G., Kupper, L.L., Nizam, A. and Muller, K.E. (2007) Applied Regression Analysis and Multivariable Methods, 4th ed., Duxbury Press.
41. Koh, K., Kim, S. and Boyd, S. (2007)'An interior-point method for large-scale ℓ1-regularized logistic regression', Journal of Machine Learning Research, Vol. 8, pp.1519-1555. (Pubitemid 47210273)
42. Komarek, P. (2004)'Logistic regression for data mining and high-dimensional classification', PhD thesis, Carnegie Mellon University.
43. Komarek, P. and Moore, A. (2005a)'Making logistic regression a core data mining tool: a practical investigation of accuracy, speed, and simplicity', Technical report, Carnegie Mellon University.
44. Komarek, P. and Moore, A. (2005b)'Making logistic regression a core data mining tool with TR-IRLS', Proceedings of the Fifth IEEE Conference on Data Mining.
45. Lewis, J.M., Lakshmivarahan, S. and Dhall, S. (2006) Dynamic Data Assimilation: A Least Squares Approach, Cambridge University Press.
46. Lin, C., Weng, R.C. and Keerthi, S.S. (2007)'Trust region Newton methods for large-scale logistic regression', Proceedings of the 24th International Conference on Machine Learning.
47. Long, J.S. and Freese, J. (2005) Regression Models for Categorical Dependent Variables Using Stata, 2nd ed., Stata Press.
48. Maalouf, M. and Trafalis, T.B. (2008)'Kernel logistic regression using truncated Newton method', in C.H. Dagli, D.L. Enke, K.M. Bryden, H. Ceylan and M. Gen (Eds.): Intelligent Engineering Systems through Artificial Neural 20 M. Maalouf Networks, Vol. 18, pp.455-462, New York, NY, USA, ASME Press.
49. Maalouf, M. and Trafalis, T.B. (2011)'Robust weighted kernel logistic regression in imbalanced and rare events data', Computational Statistics & Data Analysis, Vol. 55, No. 1, pp.168-183.
50. Maimon, O. and Rokach, L. (Eds.) (2005) Data Mining and Knowledge Discovery Handbook, Springer.
51. Maiti, T. and Pradhan, V. (2008)'A comparative study of the bias corrected estimates in logistic regression', Statistical Methods in Medical Research, Vol. 17, No. 6, pp.621-634.
52. Malouf, R. (2002)'A comparison of algorithms for maximum etropy parameter estimation', in Proceedings of Conference on Natural Language Learning, Vol. 6.
53. Manski, C.F. and Lerman, S.R. (1977)'The estimation of choice probabilities from choice based samples', Econometrica, Vol. 45, No. 8, pp.1977-1988.
54. McCullagh, P. and Nelder, J. (1989) Generalized Linear Model, Chapman and Hall/CRC.
55. Milgate, M., Eatwell, J. and Newman, P.K. (Eds.) (1990) Econometrics, W.W. Norton & Company.
56. Minka, T.P. (2003)'A comparison of numerical optimizers for logistic regression', Technical report, Department of Statistics, Carnegie Mellon University.
57. Mitsa, T. (2010) Temporal Data Mining, Chapman & Hall/CRC.
58. Pal, S.K. and Mitra, P. (2004) Pattern recognition Algorithms for Data Mining, 1st ed., Chapman and Hall/CRC.
59. Palepu, K. (1986)'Predicting takeover targets: a methodological and empirical analysis', Journal of Accounting and Economics, Vol. 8, pp.3-35.
60. Park, M.Y. and Hastie, T. (2008)'Penalized logistic regression for detecting gene interactions', Biostatistics, Vol. 9, No. 1, pp.30-50. (Pubitemid 350274580)
61. Ramalho, E.A. and Ramalho, J.J.S. (2007)'On the weighted maximum likelihood estimator for endogenous stratified samples when the population strata probabilities are unknown', Applied Economics Letters, Vol. 14, pp.171-174. (Pubitemid 46322886)
62. Rossi, R.J. (2009) Applied Biostatistics for the Health Sciences, 1st ed., Wiley.
63. Roth, V. (2001)'Probabilistic discriminative kernel classifiers for multi-class problems', in Proceedings of the 23rd DAGM-Symposium on Pattern Recognition, Springer-Verlag, London, UK, pp.246-253. (Pubitemid 33347119)
64. Schein, A.I. and Ungar, L.H. (2007)'Active learning for logistic regression: an evaluation', Machine Learning, Vol. 68, No. 3, pp.235-265. (Pubitemid 47312489)
65. Vapnik, V. (1995) The Nature of Statistical Learning, Springer, NY.
66. Visa, S. and Ralescu, A. (2005)'Issues in mining imbalanced data sets - a review paper', in Proceedings of the Sixteen Midwest Artificial Intelligence and Cognitive Science Conference, pp.67-73.
67. Wang, S. and Wang, T. (2001)'Precision of warm's weighted likelihood for a polytomous model in computerized adaptive testing', Applied Psychological Measurement, Vol. 25, No. 4, pp.317-331. (Pubitemid 33395812)
68. Xie, Y. and Manski, C.F. (1989)'The logit model and response-based samples', Sociological Methods & Research, Vol. 17, pp.283-302.
69. Zadrozny, B. (2004)'Learning and evaluating classifiers under sample selection bias', in ICML'04: Proceedings of the 21st International Conference on Machine Learning, ACM, New York, NY, USA, p.114.