Logistic regression by means of evolutionary radial basis function neural networks

IEEE Transactions on Neural Networks

Volumn 22, Issue 2, 2011, Pages 246-263

  Gutiérrez, Pedro Antonio ^a   Hervás Martínez, César ^a   Martínez Estudillo, Francisco J ^a  

^a UNIVERSITY OF CÓRDOBA   (Spain)

Author keywords

Artificial neural networks; classification; evolutionary algorithms; evolutionary programming; logistic regression; radial basis function neural networks

Indexed keywords

ARTIFICIAL NEURAL NETWORK; CLASSIFICATION; EVOLUTIONARY PROGRAMMING; LOGISTIC REGRESSION; RADIAL BASIS FUNCTION NEURAL NETWORKS;

CLASSIFIERS; COMPUTER PROGRAMMING; EVOLUTIONARY ALGORITHMS; MACHINE DESIGN; MAXIMUM LIKELIHOOD ESTIMATION; NEURAL NETWORKS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; REGRESSION ANALYSIS; VECTOR SPACES;

RADIAL BASIS FUNCTION NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER PROGRAM; MATHEMATICAL COMPUTING; METHODOLOGY; REGRESSION ANALYSIS; STATISTICAL MODEL; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; LOGISTIC MODELS; MATHEMATICAL COMPUTING; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS; SOFTWARE DESIGN; SOFTWARE VALIDATION;

EID: 79951675527     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2093537     Document Type: Article

References (57)

1. A. K. Jain, R. P. Duin, and J. Mao, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 4-37, Jan. 2000. (Pubitemid 32132953)
2. T. J. Hastie and R. J. Tibshirani, "Nonparametric regression and classification. Part II: Nonparametric classification," in From Statistics to Neural Networks: Theory and Pattern Recognition Applications (Computer and System Sciences), vol. 136, V. Cherkassky, J. H. Friedman, and H. Wechsler, Eds. New York: Springer-Verlag, 1996, pp. 70-82.
3. J. Friedman, "Multivariate adaptive regression splines (with discussion)," Ann. Stat., vol. 19, no. 1, pp. 1-141, 1991.
4. C. Hervás-Martínez and F. Martínez-Estudillo, "Logistic regression using covariates obtained by product-unit neural network models," Pattern Recognit., vol. 40, no. 1, pp. 52-64, Jan. 2007. (Pubitemid 44486291)
5. C. Hervás-Martínez, F. J. Martínez-Estudillo, and M. Carbonero-Ruz, "Multilogistic regression by means of evolutionary product-unit neural networks," Neural Netw., vol. 21, no. 7, pp. 951-961, Sep. 2008.
6. I. T. Nabney, "Efficient training of RBF networks for classification," Int. J. Neural Syst., vol. 14, no. 3, pp. 201-208, Jun. 2004.
7. B. Krishnapuram, L. Carin, M. A. Figueiredo, and A. J. Hartemink, "Sparse multinomial logistic regression: Fast algorithms and generalization bounds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 6, pp. 957-968, Jun. 2005. (Pubitemid 40889845)
8. C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), 1st ed. New York: Springer-Verlag, 2006.
9. A. Asuncion and D. Newman. (2007). UCI Machine Learning Repository [Online]. Available: http://www.ics.uci.edu/∼mlearn/MLRepository.html
10. M. J. Orr, "Optimising the widths of radial basis functions," in Proc. 5th Brazilian Symp. Neural Netw., Belo Horizonte, Brazil, Dec. 1998, pp. 26-29.
11. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis functions," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
12. K. J. Hunt, D. Sbarbaro, R. Zbikowski, and P. J. Gawthrop, "Neural networks for control systems-A survey," Automatica, vol. 28, no. 6, pp. 1083-1112, Nov. 1992. (Pubitemid 23587964)
13. J. B. Gomm and D. L. Yu, "Selecting radial basis function network centers with recursive orthogonal least squares training," IEEE Trans. Neural Netw., vol. 11, no. 2, pp. 306-314, Mar. 2000.
14. Z. Uykan and C. Guzelis, "Input-output clustering for determining centers of radial basis function network," in Proc. Eur. Conf. Circuit Theory Design, vol. 2. 1997, pp. 435-439.
15. O. Buchtala, M. Klimek, and B. Sick, "Evolutionary optimization of radial basis function classifiers for data mining applications," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 35, no. 5, pp. 928-947, Oct. 2005. (Pubitemid 41444578)
16. L. N. de Castro, E. R. Hruschka, and R. J. G. B. Campello, "An evolutionary clustering technique with local search to design RBF neural network classifiers," in Proc. IEEE Int. Joint Conf. Neural Netw., vol. 3. Jul. 2004, pp. 2083-2088. (Pubitemid 40006579)
17. Z. Q. Zhao and D. S. Huang, "A mended hybrid learning algorithm for radial basis function neural networks to improve generalization capability," Appl. Math. Model., vol. 31, no. 7, pp. 1271-1281, Jul. 2007.
18. S. Chen, Y. Wu, and B. L. Luk, "Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1239-1243, Sep. 1999.
19. J. González, I. Rojas, J. Ortega, H. Pomares, F. J. Fernández, and A. F. Días, "Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation," IEEE Trans. Neural Netw., vol. 14, no. 6, pp. 1478- 1495, Nov. 2003.
20. B. A. Whitehead and T. D. Choate, "Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction," IEEE Trans. Neural Netw., vol. 7, no. 4, pp. 869-880, Jul. 1996. (Pubitemid 126776360)
21. B. A. Whitehead, "Genetic evolution of radial basis function coverage using orthogonal niches," IEEE Trans. Neural Netw., vol. 7, no. 6, pp. 1525-1528, Nov. 1996. (Pubitemid 126780494)
22. B. A. Whitehead and T. D. Choate, "Evolving space-filling curves to distribute radial basis functions over an input space," IEEE Trans. Neural Netw., vol. 5, no. 1, pp. 15-23, Jan. 1994.
23. M. J. Embrechts, B. Szymanski, and M. Sternickel, "Introduction to scientific data mining: Direct kernel methods and applications," in Computationally Intelligent Hybrid Systems, S. J. Ovaska, Ed. New York: Wiley, 2004, ch. 10, pp. 317-362.
24. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, 1st ed. Cambridge, U.K.: Cambridge Univ. Press, 2000.
25. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 1st ed. Cambridge, MA: MIT Press, 2001.
26. B. Boser, I. Guyon, and V. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. ACM Workshop Comput. Learn. Theory, Pittsburgh, PA, 1992, pp. 144-152. (Pubitemid 23615470)
27. C. Cortes and V. Vapnik, "Support-vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, Sep. 1995.
28. T.-F. Wu, C.-J. Lin, and R. C. Weng, "Probability estimates for multiclass classification by pairwise coupling," J. Mach. Learn. Res., vol. 5, pp. 975-1005, 2004.
29. S. S. Keerthi, K. B. Duan, S. K. Shevade, and A. N. Poo, "A fast dual algorithm for kernel logistic regression," Mach. Learn., vol. 61, nos. 1-3, pp. 151-165, Nov. 2005. (Pubitemid 43049366)
30. N. Lawrence, M. Seeger, and R. Herbrich, "Fast sparse Gaussian process methods: The informative vector machine," in Advances in Neural Information Processing Systems, vol. 15. Cambridge, MA: MIT Press, 2001, pp. 625-632.
31. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, Jun. 2001. (Pubitemid 33687203)
32. H. Chen, P. Tino, and X. Yao, "Probabilistic classification vector machines," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 901-914, Jun. 2009.
33. C.-W. Hsu and C.-J. Lin, "A comparison of methods for multi-class support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002. (Pubitemid 34475042)
34. L. Narlikar and A. J. Hartemink, "Sequence features of DNA binding sites reveal structural class of associated transcription factor," Bioinformatics, vol. 22, no. 2, pp. 157-163, Jan. 2006. (Pubitemid 43205401)
35. J. Liu, J. Chen, and J. Ye, "Large-scale sparse logistic regression," in Proc. 15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Paris, France, 2009, pp. 547-556.
36. N. Subrahmanya and Y. Shin, "Sparse multiple kernel learning for signal processing applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 788-798, May 2010.
37. Y.-J. Oyang, S.-C. Hwang, Y.-Y. Ou, C.-Y. Chen, and Z.-W. Chen, "Data classification with radial basis function network based on a novel kernel density estimation algorithm," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 225-236, Jan. 2005. (Pubitemid 40241923)
38. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques (Data Management Systems), 2nd ed. San Mateo, CA: Morgan Kaufmann, 2005.
39. S. le Cessie and J. van Houwelingen, "Ridge estimators in logistic regression," Appl. Stat., vol. 41, no. 1, pp. 191-201, 1992.
40. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization. San Francisco, CA: Academic, 1982.
41. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Stat., vol. 38, no. 2, pp. 337-374, 2000.
42. N. Landwehr, M. Hall, and E. Frank, "Logistic model trees," Mach. Learn., vol. 59, nos. 1-2, pp. 161-205, May 2005. (Pubitemid 40890416)
43. P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Netw., vol. 5, no. 1, pp. 54-65, Jan. 1994.
44. X. Yao, "Global optimization by evolutionary algorithms," in Proc. 2nd Aizu Int. Symp. Parallel Algorithms/Architecutre Synthesis, Aizu- Wakamatsu, Japan, 1997, pp. 282-291.
45. A. C. Martínez-Estudillo, C. Hervás-Martínez, F. J. Martínez-Estudillo, and N. García-Pedrajas, "Hybridization of evolutionary algorithms and local search by means of a clustering method," IEEE Trans. Syst., Man Cybern. B: Cybern., vol. 36, no. 3, pp. 534-545, Jun. 2006. (Pubitemid 43811312)
46. F. J. Martínez-Estudillo, C. Hervás-Martínez, P. A. Gutiérrez, and A. C. Martínez-Estudillo, "Evolutionary product-unit neural networks classifiers," Neurocomputing, vol. 72, nos. 1-2, pp. 548-561, Dec. 2008.
47. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Francisco, CA: Academic, 1999.
48. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing," Science, vol. 220, no. 4598, pp. 671-680, May 1983.
49. I. Rechenberg, Evolutionsstrategie: Optimierung Technischer Systeme Nach Prinzipien der Biologischen Evolution. Stuttgart, Germany: Frommann-Holzboog, 1973.
50. Y. Freund and R. E. Schapire, "Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 148-156.
51. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning. New York: Springer-Verlag, 2001.
52. S. Ventura, C. Romero, A. Zafra, J. A. Delgado, and C. Hervas, "JCLEC: A Java framework for evolutionary computation," Soft Comput., vol. 12, no. 4, pp. 381-392, Oct. 2008. (Pubitemid 350041999)
53. C.-C. Chang and C.-J. Lin. (2001). LIBSVM: A Library for Support Vector Machines [Online]. Available: http://www.csie.ntu.edu.tw/∼cjlin/libsvm
54. S. South, J. Qi, and D. P. Lusch, "Optimal classification methods for mapping agricultural tillage practices," Remote Sens. Environ., vol. 91, no. 1, pp. 90-97, May 2004. (Pubitemid 38583644)
55. J. Peña-Barragán, F. Léopez-Granados, M. Jurado-Expéosito, and L. García-Torres, "Spectral discrimination of ridolfia segetum and sunflower as affected by phenological stage," Weed Res., vol. 46, no. 1, pp. 10-21, Feb. 2006. (Pubitemid 43382110)
56. J. Demšar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, 2006. (Pubitemid 43022939)
57. M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings," Ann. Math. Stat., vol. 11, no. 1, pp. 86-92, Mar. 1940.