Using metaheuristics to optimize the combination of classifier and cluster ensembles

Integrated Computer-Aided Engineering

Volumn 22, Issue 3, 2015, Pages 229-242

  Coletta, Luiz F S ^a   Hruschka, Eduardo R ^a   Acharya, Ayan ^b   Ghosh, Joydeep ^b  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

^b The University of Texas at Austin   (United States)

Author keywords

Classification; clustering; ensembles; evolutionary algorithms; metaheuristics

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; ECONOMIC AND SOCIAL EFFECTS; EVOLUTIONARY ALGORITHMS; HEURISTIC ALGORITHMS; OPTIMIZATION; PARAMETER ESTIMATION;

CLASSIFICATION AND CLUSTERING; CLUSTERING; COMBINATION OF CLASSIFIERS; ENSEMBLES; META HEURISTICS; NUMBER OF ITERATIONS; OPTIMIZATION PROCEDURES; PARAMETER OPTIMIZATION;

C (PROGRAMMING LANGUAGE);

EID: 84931081651     PISSN: 10692509     EISSN: 18758835     Source Type: Journal    
DOI: 10.3233/ICA-150485     Document Type: Article

References (85)

1. A. Acharya, E. Hruschka, J. Ghosh and S. Acharyya, C3E: A framework for combining ensembles of classifiers and clusterers, in: Multiple Classifier Systems, volume 6713 of Lecture Notes in Computer Science, Springer Berlin Heidelberg, (2011), 269-278.
2. A. Acharya, E.R. Hruschka, J. Ghosh and S. Acharyya, An optimization framework for combining ensembles of classifiers and clusterers with applications to nontransductive semisupervised learning and transfer learning, ACM Trans Knowl Discov Data 9(1) (2014), 1-35.
3. M. Ahmadlou and H. Adeli, Enhanced probabilistic neural network with local decision circles: A robust classifier, Integrated Computer-Aided Engineering 17(3) (2010), 197-210.
4. T. Bäck, D.B. Fogel and Z. Michalewicz, Handbook of evolutionary computation, IOP Publishing, Bristol, UK, 1997.
5. T. Bäck, F. Hoffmeister and H.P. Schwefel, A survey of evolution strategies, in: Proceedings of the Fourth International Conference on Genetic Algorithms (1991), 2-9.
6. T. Bäck and H.-P. Schwefel, An overview of evolutionary algorithms for parameter optimization, Evol Comput 1 (1993), 1-23.
7. A. Banerjee, S. Merugu, I.S. Dhillon and J. Ghosh, Clustering with bregman divergences, J Mach Learn Res 6 (2005), 1705-1749.
8. S. Basu, I. Davidson and K. Wagstaff, Constrained clustering: Advances in algorithms, theory, and applications, Chapman & Hall/CRC Press, 2008.
9. J. Bergstra and Y. Bengio, Random search for hyperparameter optimization, J Mach Learn Res 13 (2012), 281-305.
10. S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found Trends Mach Learn 3(1) (2011), 1-122.
11. J. Brest, S. Greiner, B. Boskovic, M. Mernik and V. Zumer, Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems, Evolutionary Computation, IEEE Transactions on 10(6) (2006), 646-657.
12. R. Campello and E. Hruschka, A fuzzy extension of the silhouette width criterion for cluster analysis, Fuzzy Sets and Systems 157(21) (2006), 2858-2875.
13. B.R. Campomanes-Álvarez, O. Cordón and S. Damas, Evolutionary multi-objective optimization for mesh simplification of 3D open models, Integrated Computer-Aided Engineering 20(4) (2013), 375-390.
14. F. Caraffini, F. Neri, B. Passow and G. Iacca, Re-sampled inheritance search: High performance despite the simplicity, Soft Computing 17(12) (2013), 2235-2256.
15. F. Caraffini, F. Neri and L. Picinali, An analysis on separability for memetic computing automatic design, Information Sciences 265 (2014), 1-22.
16. T. Cheng, P. Li, S. Zhu and D. Torrieri, M-cluster and x-ray: Two methods for multi-jammer localization in wireless sensor networks, Integrated Computer-Aided Engineering 21(1) (2014), 19-34.
17. th Brazilian Computational Intelligence (BRICS-CCI CBIC), 2013 BRICS Congress on (2013), 483-488.
18. K.W. De Bock and D.V.d. Poel, An empirical evaluation of rotation-based ensemble classifiers for customer churn prediction, Expert Systems with Applications 38(10) (2011), 12293-12301.
19. J. Demšar, Statistical comparisons of classifiers over multiple data sets, J Mach Learn Res 7 (2006), 1-30.
20. T.G. Dietterich, Ensemble methods in machine learning, in: International Workshop on Multiple Classifier Systems, Springer-Verlag, (2000), 1-15.
21. M.A. Duval-Poo, J. Sosa-Garcia, A. Guerra-Gandon, S. Vega-Pons and J. Ruiz-Shulcloper, A new classifier combination scheme using clustering ensemble, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, volume 7441 of Lecture Notes in Computer Science, Springer, Berlin Heidelberg, 2012.
22. Eberhart and Y. Shi, Particle swarm optimization: developments, applications and resources, in: Evolutionary Computation, 2001. Proceedings of the 2001 Congress on 1 (2001), 81-86.
23. A.E. Eiben and J.E. Smith, Introduction to evolutionary computing (Natural Computing Series), Springer, 2008.
24. P. Fleming and R. Purshouse, Evolutionary algorithms in control systems engineering: A survey, Control Engineering Practice 10(11) (2002), 1223-1241.
25. E. Forcael, V. González, F. Orozco, S. Vargas, A. Pantoja and P. Moscoso, Ant colony optimization model for tsunamis evacuation routes, Computer-Aided Civil and Infrastructure Engineering 29 (2014), 723-737.
26. A. Frank and A. Asuncion, UCI Machine Learning Repository [http://archive.ics.uci.edu/ml], University of California, Irvine, School of Information and Computer Sciences. 2010.
27. A.A. Freitas, A survey of evolutionary algorithms for data mining and knowledge discovery, Springer-Verlag Inc., New York, USA, 2003.
28. H. Gan, N. Sang, R. Huang, X. Tong and Z. Dan, Using clustering analysis to improve semi-supervised classification, Neurocomputing 101 (2013), 290-298.
29. J. Gao, F. Liang, W. Fan, Y. Sun and J. Han, A graph-based consensus maximization approach for combining multiple supervised and unsupervised models, Knowledge and Data Engineering, IEEE Transactions on 25(1) (2013), 15-28.
30. R. Ghaemi, N. Sulaiman, H. Ibrahim and N. Mustapha, A Survey: Clustering ensembles techniques, Proceedings of World Academy of Science, Engineering and Technology 38 (2009).
31. J. Ghosh and A. Acharya, Cluster ensembles, Wiley Interdisc Rew: Data Mining and Knowledge Discovery 1(4) (2011), 305-315.
32. S. Ghosh-Dastidar and H. Adeli, Wavelet-clustering-neural network model for freeway incident detection, Computer-Aided Civil and Infrastructure Engineering 18(5) (2003), 325-338.
33. N. Gonçalves, J. Nikkilä and R. Vigário, Self-supervised mri tissue segmentation by discriminative clustering, International Journal of Neural Systems 24(1) (2014), 1450004.
34. N. Hansen, The CMA evolution strategy: A comparing review, in: Towards a New Evolutionary Computation, volume 192 of Studies in Fuzziness and Soft Computing, Springer Berlin Heidelberg, (2006), 75-102.
35. N. Hansen and A. Ostermeier, Completely derandomized self-adaptation in evolution strategies, Evol Comput 9(2) (2001), 159-195.
36. F. Hejazi, I. Toloue, M. Jaafar and J. Noorzaei, Optimization of earthquake energy dissipation system by genetic algorithm, Computer-Aided Civil and Infrastructure Engineering 28(10) (2013), 796-810.
37. P. Hore, L. Hall and D. Goldgof, A scalable framework for cluster ensembles, Pattern Recognition 42(5) (2009), 676-688.
38. E.R. Hruschka, R.J. Campello and L.N.D. Castro, Evolving clusters in gene-expression data, Information Sciences 176(13) (2006), 1898-1927.
39. W.-Y. Hsu, Application of quantum-behaved particle swarm optimization to motor imagery EEG classification, International Journal of Neural Systems 23(6) (2013), 1350026.
40. C. Igel, No free lunch theorems: Limitations and perspectives of metaheuristics, in: Theory and Principled Methods for the Design of Metaheuristics, Natural Computing Series, Springer, Berlin Heidelberg, (2014), 1-23.
41. A.K. Jain, Data clustering: 50 years beyond k-means, Pattern Recognition Letters 31(8) (2010), 651-666.
42. L. Jia, Y. Wang and L. Fan, Multiobjective bilevel optimization for production-distribution planning problems using hybrid genetic algorithm, Integrated Computer-Aided Engineering 21(1) (2014), 77-90.
43. X. Jiang and H. Adeli, Fuzzy clustering approach for accurate embedding dimension identification in chaotic time series, Integrated Computer-Aided Engineering 10(3) (2003), 287-302.
44. X. Jiang and H. Adeli, Clustering-neural network models for freeway work zone capacity estimation, International Journal of Neural Systems 14(3) (2004), 147-163.
45. J. Kennedy and R. Eberhart, Particle swarm optimization, in: Neural Networks, 1995. Proceedings, IEEE International Conference on 4 (1995), 1942-1948.
46. S. Kirkpatrick, C.D. Gelatt and M.P. Vecchi, Optimization by simulated annealing, Science, Number 4598 220, 4598 (1983), 671-680.
47. V.S. Kodogiannis, M. Amina and I. Petrounias, A clusteringbased fuzzy wavelet neural network model for short-term load forecasting, International Journal of Neural Systems 23(5) (2013), 1350024.
48. L.I. Kuncheva, Combining pattern classifiers: Methods and algorithms, Wiley-Interscience, Hoboken, NJ, 2004.
49. P.J.M. Laarhoven and E.H.L. Aarts, eds, Simulated annealing: Theory and applications, Kluwer Academic Publishers, Norwell, MA, USA, 1987.
50. J. Liang, A. Qin, P. Suganthan and S. Baskar, Comprehensive learning particle swarm optimizer for global optimization of multimodal functions, Evolutionary Computation, IEEE Transactions on 10(3) (2006), 281-295.
51. J.M. Luna, J.R. Romero, C. Romero and S. Ventura, Reducing gaps in quantitative association rules: A genetic programming free-parameter algorithm, Integrated Computer-Aided Engineering 21(4) (2014), 321-337.
52. H.D. Menéndez, D.F. Barrero and D. Camacho, A genetic graph-based approach for partitional clustering, International Journal of Neural Systems 24(3) (2014), 1430008.
53. M. Molina-García, J. Calle-Sánchez, C. González-Merino, A. Fernández-Durán and J.I. Alonso, Design of in-building wireless networks deployments using evolutionary algorithms, Integrated Computer-Aided Engineering 21(4) (2014), 367-385.
54. N. Nguyen and R. Caruana, Consensus clusterings, ICDM '07: Proceedings of the 2007 Seventh IEEE International Conference on Data Mining 1 (2007).
55. N.C. Oza and K. Tumer, Classifier ensembles: Select real-world applications, Information Fusion 9(1) (2008), 4-20.
56. E.C. Pedrino, V.O. Roda, E.R.R. Kato, J.H. Saito, M.L. Tronco, R.H. Tsunaki, O. Morandin, Jr. and M.C. Nicoletti, A genetic programming based system for the automatic construction of image filters, Integrated Computer-Aided Engineering 20(3) (2013), 275-287.
57. F. Peng and Y. Ouyang, Optimal clustering of railroad track maintenance jobs, Computer-Aided Civil and Infrastructure Engineering 29(4) (2014), 235-247.
58. K. Price, Differential evolution: A fast and simple numerical optimizer, in: Fuzzy Information Processing Society, 1996. NAFIPS. 1996 Biennial Conference of the North American, IEEE, (1996), 524-527.
59. A.K. Qin, V.L. Huang and P. Suganthan, Differential evolution algorithm with strategy adaptation for global numerical optimization, Evolutionary Computation, IEEE Transactions on 13(2) (2009), 398-417.
60. A. Rahman and B. Verma, Cluster-based ensemble of classifiers, Expert Systems 30(3) (2013), 270-282.
61. I. Rechenberg, Evolutionsstrategie: Optimierung technischer systeme nach prinzipien der biologischen evolution, number 15 in Problemata, Frommann-Holzboog Verlag, Stuttgart-Bad Cannstatt, 1973.
62. O. Reyes, C. Morell and S. Ventura, Evolutionary feature weighting to improve the performance of multi-label lazy algorithms, Integrated Computer-Aided Engineering 21(4) (2014), 339-354.
63. M. Rizzi, M. D'Aloia and B. Castagnolo, A supervised method for microcalcification cluster diagnosis, Integrated Computer-Aided Engineering 20(2) (2013), 157-167.
64. F. Samadzadegan, B. Bigdeli and P. Ramzi, A multiple classifier system for classification of LIDAR remote sensing data using multi-class SVM, in: Multiple Classifier Systems, Springer, (2010), 254-263.
65. H.P. Schwefel, Numerical optimization of computer models, John Wiley & Sons, Inc., Chichester, 1981.
66. th International Conference on Evolutionary Programming VII, EP '98, Springer-Verlag, London, UK, (1998), 591-600.
67. R.G.F. Soares, H. Chen and X. Yao, Semisupervised classification with cluster regularization, Neural Networks and Learning Systems, IEEE Transactions on 23(11) (2012), 1779-1792.
68. R. Storn and K. Price, Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces, Jornal of Global Optimization 11(4) (1997), 341-359.
69. A. Strehl and J. Ghosh, Cluster ensembles - a knowledge reuse framework for combining multiple partitions, Journal of Machine Learning Research 3 (2002), 583-617.
70. A. Topchy, A. Jain and W. Punch, Clustering ensembles: Models of consensus and weak partitions, Pattern Analysis and Machine Intelligence, IEEE Transactions on 27(12) (2005), 1866-1881.
71. K. Tumer and J. Ghosh, Analysis of decision boundaries in linearly combined neural classifiers, Pattern Recognition 29(2) (1996), 341-348.
72. B. Twala, Multiple classifier application to credit risk assessment, Expert Systems with Applications 37(4) (2010), 3326-3336.
73. B. Verma and A. Rahman, Cluster-oriented ensemble classifier: Impact of multicluster characterization on ensemble classifier learning, Knowledge and Data Engineering, IEEE Transactions on 24(4) (2012), 605-618.
74. J. Vesterstrom and R. Thomsen, A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, in: Evolutionary Computation. CEC 2004. Congress on 2 (2004), 1980-1987.
75. E.D. Wandekokem, E. Mendel, F. Fabris, M. Valentim, R.J. Batista, F.M. Varejao and T.W. Rauber, Diagnosing multiple faults in oil rig motor pumps using support vector machine classifier ensembles, Integrated Computer-Aided Engineering 18(1) (2011), 61-74.
76. nd edition, Morgan Kaufmann, 2005.
77. J.-W. Wu, J.C. Tseng and W.-N. Tsai, A hybrid linear text segmentation algorithm using hierarchical agglomerative clustering and discrete particle swarm optimization, Integrated Computer-Aided Engineering 21(1) (2014), 35-46.
78. R. Xia, C. Zong and S. Li, Ensemble of feature sets and classification algorithms for sentiment classification, Information Sciences 181(6) (2011), 1138-1152.
79. th ACM International Conference on Multimedia, MM '11, ACM, New York, NY, USA, (2011), 1153-1156.
80. nd edition, Luniver Press, 2010.
81. X.-S. Yang, Swarm-based metaheuristic algorithms and nofree-lunch theorems, in: Theory and New Applications of Swarm Intelligence, InTech, (2012), 1-17.
82. X. Yu and M. Gen, General discussion on performance evaluation, in: Introduction to Evolutionary Algorithms, Springer Science & Business Media, (2010), 101-115.
83. Z. Zeng, J. Xu, S. Wu and M. Shen, Antithetic method-based particle swarm optimization for a queuing network problem with fuzzy data in concrete transportation systems, Computer- Aided Civil and Infrastructure Engineering 29(10) (2014), 771-800.
84. Z.-H. Zhou, When semi-supervised learning meets ensemble learning, Frontiers of Electrical and Electronic Engineering in China 6(1) (2011), 6-16.
85. Z.-H. Zhou, Ensemble methods: Foundations and algorithms, CRC Press, 2012.