Learning ensembles of neural networks by means of a Bayesian artificial immune system

IEEE Transactions on Neural Networks

Volumn 22, Issue 2, 2011, Pages 304-316

  Castro, Pablo A Dalbem ^a   Von Zuben, Fernando Joś ^a  

^a UNIVERSITY OF CAMPINAS   (Brazil)

Author keywords

Artificial immune system; Bayesian networks; classification problems; combinatorial optimization; ensemble of neural networks

Indexed keywords

ARTIFICIAL IMMUNE SYSTEM; AUTOMATIC CONTROL; BAYESIAN; BAYESIAN FRAMEWORKS; CANDIDATE SOLUTION; CLASSIFICATION PROBLEMS; COMPARATIVE ANALYSIS; ENSEMBLE OF NEURAL NETWORKS; ESTIMATION OF DISTRIBUTION ALGORITHMS; JOINT DISTRIBUTIONS; POPULATION SIZES; PROBABILISTIC MODELS; SEARCH ALGORITHMS;

AUTOMATION; CLONING; COMBINATORIAL OPTIMIZATION; CONTROL; DISTRIBUTED PARAMETER NETWORKS; EVOLUTIONARY ALGORITHMS; HEURISTIC ALGORITHMS; IMMUNOLOGY; INFERENCE ENGINES; INTELLIGENT NETWORKS; MATHEMATICAL OPERATORS; NEURAL NETWORKS; POPULATION STATISTICS; PROBABILITY DISTRIBUTIONS; SPEECH RECOGNITION;

BAYESIAN NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; CLASSIFICATION; COMPUTER SIMULATION; HUMAN; IMMUNITY; METHODOLOGY; PHYSIOLOGY; STANDARD; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; CLASSIFICATION; COMPUTER SIMULATION; HUMANS; IMMUNE SYSTEM PHENOMENA; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 79951681587     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2096823     Document Type: Article

References (52)

1. L. K. Hansen and P. Salamon, "Neural network ensembles," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 10, pp. 993-1001, Oct. 1990.
2. A. Krogh and J. Vedelsby, "Neural network ensembles, cross validation, and active learning," in Advances in Neural Information Processing System, vol. 7. Cambridge, MA: MIT Press, 1994, pp. 231-238.
3. M. Perrone and L. N. Cooper, "When networks disagree: Ensemble method for neural networks," in Neural Networks for Speech and Image Processing, R. Mammone, Ed. London, U.K.: Chapman & Hall, 1993, pp. 126-142.
4. N. García-Pedrajas and C. Fyfe, "Construction of classifier ensembles by means of artificial immune systems," J. Heuristics, vol. 14, no. 3, pp. 285-310, Jun. 2008. (Pubitemid 351534266)
5. N. García-Pedrajas and C. Fyfe, "Immune network based ensembles," Neurocomputing, vol. 70, nos. 7-9, pp. 1155-1166, Mar. 2007. (Pubitemid 46336774)
6. R. Pasti and L. N. De Castro, "Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity," Inf. Sci., vol. 179, no. 10, pp. 1441-1453, Apr. 2009.
7. R. Pasti, L. N. De Castro, G. P. Coelho, and F. J. Von Zuben, "Neural network ensembles: Immune-inspired approaches to the diversity of components," Natural Comput., vol. 9, no. 3, pp. 625-653, Sep. 2010.
8. R. Pasti and L. N. De Castro, "The influence of diversity in an immunebased algorithm to train MLP networks," in Proc. 6th Int. Conf. Artif. Immune Syst., 2007, pp. 71-82.
9. B. Barbosa, L. Bui, H. Abbass, L. Aguirre, and A. Braga, "Evolving an ensemble of neural networks using artificial immune systems," in Proc. 7th Int. Conf. Simul. Evol. Learn., 2008, pp. 121-130.
10. X. Zhang, S. Wang, T. Shan, and L. Jiao, "Selective SVMs ensemble driven by immune clonal algorithm," in Proc. EvoWorkshops, 2005, pp. 325-333. (Pubitemid 41273947)
11. F. O. De França, G. P. Coelho, P. A. D. Castro, and F. J. Von Zuben, "Conceptual and practical aspects of the aiNet family of algorithms," Int. J. Natural Comput. Res., vol. 1, no. 1, pp. 1-35, 2010.
12. L. N. De Castro and F. J. Von Zuben, "Learning and optimization using the clonal selection principle," IEEE Trans. Evol. Comput., vol. 6, no. 3, pp. 239-251, Jun. 2002. (Pubitemid 34867248)
13. J. H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Cambridge, MA: MIT Press, 1992.
14. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
15. P. A. D. De Castro and F. J. Von Zuben, "BAIS: A Bayesian artificial immune system for the effective handling of building blocks," Inf. Sci., vol. 179, no. 10, pp. 1426-1440, Apr. 2009.
16. P. A. D. De Castro and F. J. Von Zuben, "Feature subset selection by means of a Bayesian artificial immune system," in Proc. 8th Int. Conf. Hybrid Intell. Syst., Barcelona, Spain, Sep. 2008, pp. 561-566.
17. P. A. D. Castro and F. J. Von Zuben, "MOBAIS: A Bayesian artificial immune system for multi-objective optimization," in Proc. 7th Int. Conf. Artif. Immune Syst., Phuket, Thailand, Aug. 2008, pp. 48-59.
18. P. A. D. De Castro and F. J. Von Zuben, "Multi-objective Bayesian artificial immune system: Empirical evaluation and comparative analyses," J. Math. Model. Algorithms, vol. 8, no. 2, pp. 151-173, 2009.
19. P. A. D. De Castro and F. J. Von Zuben, "Multi-objective feature selection using a Bayesian artificial immune system," Int. J. Intell. Comput. Cybern., vol. 3, no. 2, pp. 235-256, 2010.
20. G. Brown, J. Wyatt, R. Harris, and X. Yao, "Diversity creation methods: A survey and categorisation," Inf. Fusion, vol. 6, no. 1, pp. 5-20, Mar. 2005.
21. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, Aug. 1996. (Pubitemid 126724382)
22. R. E. Schapire, "The strength of weak learnability," Mach. Learn., vol. 5, no. 2, pp. 197-227, Jun. 1990.
23. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998. (Pubitemid 128741345)
24. Y. Liu and X. Yao, "Ensemble learning via negative correlation," Neural Netw., vol. 12, no. 10, pp. 1399-1404, Dec. 1999.
25. Y. Liu and X. Yao, "Simultaneous training of negatively correlated neural networks in an ensemble," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 29, no. 6, pp. 716-725, Dec. 1999. (Pubitemid 32211610)
26. Y. Liu, X. Yao, and T. Higuchi, "Evolutionary ensembles with negative correlation learning," IEEE Trans. Evol. Comput., vol. 4, no. 4, pp. 380-387, Nov. 2000. (Pubitemid 32075189)
27. M. M. Islam, X. Yao, and K. Murase, "A constructive algorithm for training cooperative neural network ensembles," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 820-834, Jul. 2003.
28. H. Chen and X. Yao, "Regularized negative correlation learning for neural network ensembles," IEEE Trans. Neural Netw., vol. 20, no. 12, pp. 1962-1979, Dec. 2009.
29. H. Chen and X. Yao, "Multiobjective neural network ensembles based on regularized negative correlation learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 12, pp. 1738-1751, Feb. 2010.
30. D. Dasgupta, "Advances in artificial immune systems," IEEE Comput. Intell. Mag., vol. 1, no. 4, pp. 40-49, Nov. 2006.
31. S. Sarafijanovic and J.-Y. Le Boudec, "An artificial immune system approach with secondary response for misbehavior detection in mobile ad hoc networks," IEEE Trans. Neural Netw., vol. 16, no. 5, pp. 1076-1087, Sep. 2005. (Pubitemid 41444613)
32. L. N. De Castro and J. Timmis, "An artificial immune network for multimodal function optimization," in Proc. World Congr. Comput. Intell., vol. 1. Honolulu, HI, 2002, pp. 699-704.
33. G. P. Coelho and F. J. Von Zuben, "Omni-aiNet: An immune-inspired approach for omni optimization," in Proc. 5th Int. Conf. Artif. Immune Syst., Oeiras, Portugal, Sep. 2006, pp. 294-308. (Pubitemid 44744019)
34. V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1999.
35. P. A. D. Castro, G. P. Coelho, M. F. Caetano, and F. J. Von Zuben, "Designing ensembles of fuzzy classification systems: An immuneinspired approach," in Proc. 4th Int. Conf. Artif. Immune Syst., Banff, AB, Canada, Aug. 2005, pp. 469-482. (Pubitemid 41472744)
36. Y. Wu and C. Fyfe, "Exploratory data analysis with artificial immune systems," Evol. Intell., vol. 1, no. 2, pp. 159-169, 2008.
37. F. V. Jensen, An Introduction to Bayesian Networks. London, U.K.: UCL Press, 1996.
38. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
39. G. F. Cooper and E. Herskovits, "A Bayesian method for the induction of probabilistic networks from data," Mach. Learn., vol. 9, no. 4, pp. 309-347, Oct. 1992.
40. M. Henrion, "Propagating uncertainty in Bayesian networks by probabilistic logic sampling," in Proc. 2nd Annu. Conf. Uncert. Artif. Intell., vol. 2. 1988, pp. 149-163.
41. X. Yao, "Evolving artificial neural networks," Proc. IEEE, vol. 87, no. 9, pp. 1423-1447, Sep. 1999. (Pubitemid 30503424)
42. E. M. Iyoda and F. J. Von Zuben, "Hybrid neural networks: An evolutionary approach with local search," Integr. Comput.-Aided Eng., vol. 9, no. 1, pp. 57-72, Jan. 2002. (Pubitemid 34080649)
43. Y. Liu and X. Yao, "Evolutionary design of artificial neural networks with different nodes," in Proc. IEEE Int. Conf. Evol. Comput., Nagoya, Japan, May 1996, pp. 670-675.
44. M. F. Møller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Netw., vol. 6, no. 4, pp. 525-533, 1993.
45. P. J. Angeline, G. M. Saunders, and J. P. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Netw., vol. 5, no. 1, pp. 54-65, Jan. 1994.
46. L. I. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy," Mach. Learn., vol. 51, no. 2, pp. 181-207, May 2003.
47. D. B. Skalak, "The sources of increased accuracy for two proposed boosting algorithms," in Proc. AAAI Integr. Multiple Learn. Models Workshop, 1996, pp. 1-6.
48. A. Frank and A. Asuncion. (2010). UCI Machine Learning Repository [Online]. Available: http://archive.ics.uci.edu/ml.
49. N. García-Pedrajas, C. Hervás-Martínez, and D. Ortiz-Boyer, "Cooperative coevolution of artificial neural network ensembles for pattern classification," IEEE Trans. Evol. Comput., vol. 9, no. 3, pp. 271-302, Jun. 2005. (Pubitemid 40871133)
50. D. Whitley, "The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best," in Proc. 3rd Int. Conf. Genetic Algorithms, 1989, pp. 116-121.
51. M. Pelikan, D. E. Goldberg, and E. Cantú-Paz, "BOA: The Bayesian optimization algorithm," in Proc. Genetic Evol. Comput. Conf., vol. 1. Orlando, FL, 1999, pp. 525-532.
52. B. M. Broom and D. Subramanian, "Computational methods for learning Bayesian networks from high-throughput biological data," in Bayesian Inference for Gene Expression and Proteomics. Cambridge, U.K.: Cambridge Univ. Press, 2006.