Robust weighted kernel logistic regression in imbalanced and rare events data

Computational Statistics and Data Analysis

Volumn 55, Issue 1, 2011, Pages 168-183

  Maalouf, Maher ^a   Trafalis, Theodore B ^a  

^a USA   (United States)

Author keywords

Classification; Endogenous sampling; Kernel methods; Logistic regression; Truncated Newton

Indexed keywords

CLASSIFICATION; ENDOGENOUS SAMPLING; KERNEL METHODS; LOGISTIC REGRESSION; TRUNCATED NEWTON;

LEARNING SYSTEMS; NEWTON-RAPHSON METHOD; NUMERICAL METHODS; PATTERN RECOGNITION;

REGRESSION ANALYSIS;

EID: 77956394683     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2010.06.014     Document Type: Article

References (65)

1. T. Amemiya Advanced Econometrics 1985 Harvard University Press
2. Asuncion, A., Newman, D.J., 2007. UCI machine learning repository, University of California, Irvine. School of Information and Computer Sciences. http://www.ics.uci.edu/~mlearn/MLRepository.html.
3. Bai, S.B., Wang, J., Zhang, F.Y., Pozdnoukhov, A., Kanevski, M., 2008. Prediction of landslide susceptibility using logistic regression: a case study in Bailongjiang river basin, China. In: Fuzzy Systems and Knowledge Discovery, Fourth International Conference on, vol. 4, pp. 647651.
4. M. Ben-Akiva, and S. Lerman Discrete Choice Analysis: Theory and Application to Travel Demand 1985 The MIT Press
5. R. Berk Statistical Learning from a Regression Perspective 1st edition 2008 Springer
6. Busser, B., Daelemans, W., Bosch, A., 1999. Machine learning of word pronunciation: the case against abstraction. In: Proceedings of the Sixth European Conference on Speech Communication and Technology, Eurospeech99, pp. 21232126.
7. A.C. Cameron, and P.K. Trivedi Microeconometrics: Methods and Applications 2005 Cambridge University Press
8. Canu, S., Smola, A.J., 2005. Kernel methods and the exponential family. In: ESANN, pp. 447454.
9. Chang, C.-C., Lin, C.-J., 2001. LIBSVM: a library for support vector machines. Software available at: http://www.csie.ntu.edu.tw/cjlin/libsvm.
10. P.K. Chan, and S.J. Stolfo Toward scalable learning with non-uniform class and cost distributions: a case study in credit card fraud detection Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining 1998 AAAI Press 164 168
11. G. Cowan Statistical Data Analysis 1998 Oxford University Press
12. J.S. Cramer Logit Models from Economics and Other Fields 2003 Cambridge University Press
13. N. Cristianini, and J. Shawe-Taylor An Introduction to Support Vector Machines and other kernel-based learning methods 2000 Cambridge University Press
14. Drummond, C., Holte, R.C., 2003. C4.5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling, pp. 18.
15. M.V.D. Eeckhaut, T. Vanwalleghem, J. Poesen, G. Govers, G. Verstraeten, and L. Vandekerckhove Prediction of landslide susceptibility using rare events logistic regression: a case-study in the Flemish Ardennes (Belgium) Geomorphology 76 34 2006 392 410
16. B. Efron, and R.J. Tibshirani An Introduction to the Bootstrap 1994 Chapman & Hall/CRC
17. F. Ferraty, and P. Vieu Nonparametric Functional Data Analysis: Theory and Practice 1st edition 2006 Springer
18. R.P. Gorman, and T.J. Sejnowski Analysis of hidden units in a layered network trained to classify sonar targets Neural Networks 1 1988 75 89
19. Haberman, S.J., 1976. Generalized residuals for log-linear models. In: Proceedings of the 9th International Biometrics Conference, Boston.
20. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning 2001 Springer-Verlag
21. G. Heinze, and M. Schemper A solution to the problem of monotone likelihood in cox regression Biometrics 57 2001 114 119
22. D.W. Hosmer, and S. Lemeshow Applied Logistic Regression 2nd edition 2000 Wiley
23. G.W. Imbens, and T. Lancaster Efficient estimation and stratified sampling Journal of Econometrics 74 1996 289 318
24. T. Jaakkola, and D. Haussler Probabilistic Kernel Regression Models 1999
25. Japkowicz, N., 2000. Learning from imbalanced data sets: a comparison of various strategies, Tech. Rep. University of DalTech/Dalhousie.
26. D. Jensen, and P.R. Cohen Multiple comparison in induction algorithms Machine Learning 38 2000 309 338
27. Karsmakers, P., Pelckmans, K., Suykens, J.A.K., 2007. Multi-class kernel logistic regression: a fixed-size implementation. In: International Joint Conference on Neural Networks. pp. 17561761.
28. L.J. Keele Semiparametric Regression for the Social Sciences 2008 Wiley
29. G. King, and L. Zeng Explaining rare events in international relations International Organization 55 3 2001 693 715
30. G. King, and L. Zeng Improving forecast of state failure World Politics 53 4 2001 623 658
31. G. King, and L. Zeng Logistic regression in rare events data Political Analysis 9 2001 137 163
32. Komarek, P., 2004. Logistic regression for data mining and high-dimensional classification. Ph.D. Thesis. Carnegie Mellon University.
33. Komarek, P., Moore, A., 2005. Making logistic regression a core data mining tool: a practical investigation of accuracy, speed, and simplicity. Tech. Rep. Carnegie Mellon University.
34. M. Kubat, R.C. Holte, and S. Matwin Machine learning for the detection of oil spills in satellite radar images Machine Learning 1998 195 215
35. L.A. Kurgan, K.J. Cios, R. Tadeusiewicz, M. Ogiela, and L.S. Goodenday Knowledge discovery approach to automated cardiac spect diagnosis Artificial Intelligence in Medicine 32 2 2001 149 169
36. V. Lakshmanan, G. Stumpf, and A. Witts A neural network for detecting and diagnosing tornadic circulations using the mesocyclone detection and near storm environment algorithms 21st International Conference on Information Processing Systems 2005 American Meteorological Society San Diego, CA CD-ROM, J52.2
37. J.M. Lewis, S. Lakshmivarahan, and S. Dhall Dynamic Data Assimilation: A Least Squares Approach 2006 Cambridge University Press
38. C.-J. Lin, R.C. Weng, and S.S. Keerthi Trust region Newton methods for large-scale logistic regression ICML '07: Proceedings of the 24th International Conference on Machine Learning Corvalis, Oregon 2007 ACM New York, NY, USA 561 568
39. M. Maalouf, and T.B. Trafalis Kernel logistic regression using truncated Newton method C.H. Dagli, D.L. Enke, K.M. Bryden, H. Ceylan, M. Gen, Intelligent Engineering Systems Through Artificial Neural Networks, Vol. 18 2008 ASME Press New York, NY, USA 455 462
40. O. Maimon, L. Rokach, Data Mining and Knowledge Discovery Handbook 2005 Springer
41. T. Maiti, and V. Pradhan A comparative study of the bias corrected estimates in logistic regression Statistical Methods in Medical Research 17 6 2008 621 634
42. Maloof, M.A., 2003. Learning when data sets are imbalanced and when costs are unequal and unknown. In: ICML-2003 Workshop on Learning from Imbalanced Data Sets II.
43. Malouf, R., 2002. A comparison of algorithms for maximum entropy parameter estimation. In: Proceedings of Conference on Natural Language Learning, vol. 6.
44. C.F. Manski, and S.R. Lerman The estimation of choice probabilities from choice based samples Econometrica 45 8 1977 1977 1988
45. P. McCullagh, and J. Nelder Generalized Linear Model 1989 Chapman and Hall/CRC
46. M. Milgate, J. Eatwell, P.K. Newman, Econometrics 1990 W. W. Norton & Company
47. Minka, T.P., 2003. A comparison of numerical optimizers for logistic regression. Tech. Rep. Department of Statistics, Carnegie Mellon University.
48. M.Y. Park, and T. Hastie Penalized logistic regression for detecting gene interactions Biostatistics 9 1 2008 30 50
49. Prati, R.C., Batista, G.E.A.P.A., Monard, M.C., 2004. Learning with class skews and small disjuncts. In: SBIA, pp. 296306.
50. J. Quigley, T. Bedford, and L. Walls Estimating rate of occurrence of rare events with empirical Bayes: a railway application Reliability Engineering & System Safety 92 5 2007 619 627
51. J. Ramsay, and B.W. Silverman Functional Data Analysis 2nd edition 2005 Springer
52. C. Seiffert, T.M. Khoshgoftaar, J.V. Hulse, and A. Napolitano Mining data with rare events: a case study ICTAI '07: Proceedings of the 19th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2007, vol. 2 2007 IEEE Computer Society Washington, DC, USA 132 139
53. J. Shawe-Taylor, and N. Cristianini Kernel Methods for Pattern Analysis 2004 Cambridge University Press
54. V.G. Sigillito, S.P. Wing, L.V. Hutton, and K.B. Baker Classification of radar returns from the ionosphere using neural networks Johns Hopkins APL Technical Digest 10 1989 262 266
55. J.W. Smith, J.E. Everhart, W.C. Dickson, and R.S. Johannes Using the adap learning algorithm to forecast the onset of diabetes mellitus Proceedings of the Symposium on Computer Applications and Medical Care 1988 IEEE Computer Society Press
56. Trafalis, T.B., Ince, H., Richman, M.B., 2003. Tornado detection with support vector machines. International Conference on Computational Science. pp. 289298.
57. P. Tsoucas Rare events in series of queues Journal of Applied Probability 29 1992 168 175
58. J. Van-Hulse, T.M. Khoshgoftaar, and A. Napolitano Experimental perspectives on learning from imbalanced data ICML '07: Proceedings of the 24th International Conference on Machine Learning 2007 ACM New York, NY, USA 935 942
59. V. Vapnik The Nature of Statistical Learning 1995 Springer NY
60. S. Wang, and T. Wang Precision of warm's weighted likelihood for a polytomous model in computerized adaptive testing Applied Psychological Measurement 25 4 2001 317 331
61. Wang, Y., Witten, I.H., 2002. Modeling for optimal probability prediction. In: ICML, pp. 650657.
62. G.M. Weiss Mining with rarity: a unifying framework SIGKDD Explorations Newsletter 6 1 2004 7 19
63. G.M. Weiss, and H. Hirsh Learning to predict extremely rare events AAAI Workshop on Learning from Imbalanced Data Sets 2000 AAAI Press 64 68
64. Y. Xie, and C.F. Manski The logit model and response-based samples Sociological Methods & Research 17 1989 283 302
65. B. Zadrozny Learning and evaluating classifiers under sample selection bias ICML '04: Proceedings of the Twenty-First International Conference on Machine learning 2004 ACM New York, NY, USA 114