Novel layered clustering-based approach for generating ensemble of classifiers

IEEE Transactions on Neural Networks

Volumn 22, Issue 5, 2011, Pages 781-792

  Rahman, Ashfaqur ^a   Verma, Brijesh ^a  

^a CENTRAL QUEENSLAND UNIVERSITY   (Australia)

Author keywords

Cluster oriented ensemble classifier; committee of experts; ensemble classifiers; multiple classifier systems

Indexed keywords

BASE CLASSIFIERS; CLASSIFICATION RESULTS; COMMITTEE OF EXPERTS; DATA SETS; ENSEMBLE CLASSIFIERS; ENSEMBLE OF CLASSIFIERS; INDIVIDUAL CLASSIFIERS; MAJORITY VOTING; MULTIPLE CLASSIFIER SYSTEMS; MULTIPLE LAYERS; NOVEL CONCEPT; OVERLAPPING PATTERNS; TEST PATTERN;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; COMPUTER PROGRAM; HUMAN; MATHEMATICAL COMPUTING; MATHEMATICAL PHENOMENA; METHODOLOGY; STANDARD; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; HUMANS; MATHEMATICAL COMPUTING; MATHEMATICAL CONCEPTS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SOFTWARE VALIDATION; STATISTICS AS TOPIC;

EID: 79955824962     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2118765     Document Type: Article

References (48)

1. T. Windeatt, "Accuracy/diversity and ensemble MLP classifier design," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1194-1211, Sep. 2006. (Pubitemid 46445178)
2. H. Zouari, L. Heutte, and Y. Lecourtier, "Controlling the diversity in classifier ensembles through a measure of agreement," Pattern Recognit., vol. 38, no. 11, pp. 2195-2199, Nov. 2005. (Pubitemid 41214656)
3. R. Polikar, "Ensemble based systems in decision making," IEEE Circuits Syst. Mag., vol. 6, no. 3, pp. 21-45, Sep. 2006.
4. T. Windeatt, "Diversity/accuracy and ensemble classifier design," in Proc. 17th Int. Conf. Pattern Recognit., vol. 3. Aug. 2004, pp. 454-457.
5. L. I. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy," Mach. Learn., vol. 51, no. 2, pp. 181-207, 2003.
6. L. Rokach, O. Maimon, and I. Lavi, "Space decomposition in data mining: A clustering approach," in Proc. Int. Symp. Methodol. Intell. Syst., 2003, pp. 24-31. (Pubitemid 37331851)
7. J. Xiuping and J. A. Richards, "Cluster-space classification: A fast knearest neighbour classification for remote sensing hyperspectral data," in Proc. IEEE Workshop Adv. Tech. Anal. Remot. Sens. Data, Oct. 2003, pp. 407-410.
8. L. I. Kuncheva, "Clustering-and-selection model for classifier combination," in Proc. Int. Conf. Knowl.-Based Intell. Eng. Syst. All. Technol. 2000, pp. 185-188.
9. L. I. Kuncheva, "Switching between selection and fusion in combining classifiers: An experiment," IEEE Trans. Syst., Man, Cybern. Part B: Cybern., vol. 32, no. 2, pp. 146-156, Apr. 2002. (Pubitemid 34249373)
10. B. Tang, M. I. Heywood, and M. Shepherd, "Input partitioning to mixture of experts," in Proc. Int. Joint Conf. Neural Netw., Honolulu, HI, May 2002, pp. 227-232. (Pubitemid 34647632)
11. G. Nasierding, G. Tsoumakas, and A. Z. Kouzani, "Clustering based multi-label classification for image annotation and retrieval," in Proc. IEEE Int. Conf. Syst., Man Cybern., San Antonio, TX, Oct. 2009, pp. 4514-4519.
12. H. Cevikalp and R. Polikar, "Local classifier weighting by quadratic programming," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1832-1838, Oct. 2008.
13. L. Nanni and A. Lumini, "Evolved feature weighting for random subspace classifier," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 363-366, Feb. 2008. (Pubitemid 351359301)
14. L. I. Kuncheva, J. C. Bezdek, and R. Duin, "Decision templates for multiple classifier fusion: An experimental comparison," Pattern Recognit., vol. 34, no. 2, pp. 299-314, Feb. 2001. (Pubitemid 32871876)
15. A. H. R. Ko, R. Sabourin, A. de S. Britto, and L. Oliveira, "Pairwise fusion matrix for combining classifiers," Pattern Recognit., vol. 40, no. 8, pp. 2198-2210, Aug. 2007. (Pubitemid 46574774)
16. N. M. Wanas, R. A. Dara, and M. S. Kamel, "Adaptive fusion and cooperative training for classifier ensembles," Pattern Recognit., vol. 39, no. 9, pp. 1781-1794, Sep. 2006. (Pubitemid 43867570)
17. O. R. Terrades, E. Valveny, and S. Tabbone, "Optimal classifier fusion in a non-Bayesian probabilistic framework," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1630-1644, Sep. 2009.
18. R. Maclin and J. W. Shavlik, "Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks," in Proc. Int. Joint Conf. Artif. Intell., 1995, pp. 524-530.
19. T. Yamaguchi, K. J. Mackin, E. Nunohiro, J. G. Park, K. Hara, K. Matsushita, M. Ohshiro, and K. Yamasaki, "Artificial neural network ensemble-based land-cover classifiers using MODIS data," Artif. Life Robot., vol. 13, no. 2, pp. 570-574, 2009.
20. T. G. Dietterich, "Ensemble methods in machine learning," in Proc. 1st Int. Workshop Multiple Class. Syst., 2000, pp. 1-15.
21. K. J. Cherkauer, "Human expert-level performance on a scientific image analysis task by a system using combined artificial neural networks," in Proc. Work. Notes AAAI Workshop Integr. Multiple Learn. Models, 1996, pp. 15-21.
22. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998. (Pubitemid 128741345)
23. A. Bertoni, R. Folgieri, and G. Valentini, "Bio-molecular cancer prediction with random subspace ensembles of support vector machines," Neurocomputing, vol. 63, pp. 535-539, Jan. 2005. (Pubitemid 40103878)
24. L. I. Kuncheva, J. J. Rodriguez, C. O. Plumpton, D. E. Linden, and S. J. Johnston, "Random subspace ensembles for fMRI classification," IEEE Trans. Med. Imag., vol. 29, no. 2, pp. 531-542, Feb. 2010.
25. G. Martínez-Muñoz, A. Sánchez-Martínez, D. Hernández-Lobato, and A. Suarez, "Class-switching neural network ensembles," Neurocomputing, vol. 7, nos. 13-15, pp. 2521-2528, Aug. 2008.
26. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, no. 1, pp. 263-286, Jan. 1995.
27. H. Parvin, H. Alizadeh, and B. Minaei-Bidgoli, "Using clustering for generating diversity in classifier ensemble," Int. J. Digital Content Technol. Appl., vol. 3, no. 1, pp. 51-57, Mar. 2009.
28. R. Neumayer, "Clustering based ensemble classification for spam filtering," in Proc. 6th Workshop Data Anal., 2006, pp. 11-22.
29. M. I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, no. 2, pp. 181-214, Mar. 1994.
30. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, Aug. 1996. (Pubitemid 126724382)
31. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, Oct. 2001. (Pubitemid 32933532)
32. L. Breiman, "Pasting small votes for classification in large databases and on-line," Mach. Learn., vol. 36, nos. 1-2, pp. 85-103, 1999.
33. G. Martínez-Muñoz, D. Hernández-Lobato, and A. Suarez, "An analysis of ensemble pruning techniques based on ordered aggregation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 245-259, Feb. 2009.
34. L. Chen and M. S. Kamel, "A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems," Pattern Recognit., vol. 42, no. 5, pp. 629-644, May 2009.
35. S. Eschrich and L. O. Hall, "Soft partitions lead to better learned ensembles," in Proc. Annu. Meet. North Amer. Fuzzy Inf. Process. Soc., 2002, pp. 406-411.
36. R. E. Schapire, "The strength of weak learnability," Mach. Learn., vol. 5, no. 2, pp. 197-227, Jun. 1990.
37. H. Drucker, C. Cortes, L. D. Jackel, Y. LeCun, and V. Vapnik, "Boosting and other ensemble methods," Neural Comput., vol. 6, no. 6, pp. 1289-1301, Nov. 1994.
38. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119-139, Aug. 1997. (Pubitemid 127433398)
39. N. García-Pedrajas, "Constructing ensembles of classifiers by means of weighted instance selection," IEEE Trans. Neural Netw., vol. 20, no. 2, pp. 258-277, Feb. 2009.
40. J. J. Rodríguez and J. Maudes, "Boosting recombined weak classifiers," Pattern Recognit. Lett., vol. 29, no. 8, pp. 1049-1059, Jun. 2008. (Pubitemid 351522343)
41. D. Parikh and R. Polikar, "An ensemble-based incremental learning approach to data fusion," IEEE Trans. Syst., Man, Cybern. Part B: Cybern., vol. 37, no. 2, pp. 437-450, Apr. 2007. (Pubitemid 46523231)
42. M. D. Muhlbaier, A. Topalis, and R. Polikar, "Learn++ .NC: Combining ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes," IEEE Trans. Neural Netw., vol. 20, no. 1, pp. 152-168, Jan. 2009.
43. UCI Machine Learning Database. (2009, Oct. 6) [Online]. Available: http://www.archive.ics.uci.edu/ml/
44. R. Kohavi and D. H. Wolpert, "Bias plus variance decomposition for zero-one loss functions," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 275-283.
45. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA data mining software: An update," ACM SIGKDD Explor. Newslett., vol. 11, no. 1, pp. 10-18, Nov. 2009.
46. J. Demsar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, Jan. 2006. (Pubitemid 43022939)
47. D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures. London, U.K.: Chapman & Hall, 2000.
48. L. Nanni and A. Lumini, "Fuzzybagging: A novel ensemble of classifiers," Pattern Recognit., vol. 39, no. 3, pp. 488-490, Mar. 2006. (Pubitemid 41808629)