Negative correlation ensemble learning for ordinal regression

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 11, 2013, Pages 1836-1849

  Fernández Navarro, Francisco ^a   Gutiérrez, Pedro Antonio ^b   Hervás Martínez, Casar ^b   Yao, Xin ^c  

^a EUROPEAN SPACE AGENCY   (Netherlands)

^b UNIVERSITY OF CÓRDOBA   (Spain)

^c UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

Negative correlation learning (NCL); neural network ensembles; ordinal regression; threshold methods

Indexed keywords

ENSEMBLE LEARNING; GENERALIZATION PERFORMANCE; NEGATIVE CORRELATION; NEGATIVE CORRELATION LEARNING; NETWORK THRESHOLDS; NEURAL NETWORK ENSEMBLES; ORDINAL REGRESSION; THRESHOLD METHODS;

NEURAL NETWORKS;

REGRESSION ANALYSIS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; PROCEDURES; REGRESSION ANALYSIS; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS;

EID: 84886953869     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2268279     Document Type: Article

References (58)

1. T.-S. Lim, W.-Y. Loh, and Y.-S. Shih, "A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms," Mach. Learn., vol. 40, no. 3, pp. 203-228, 2000
2. D. Kibler, D. W. Aha, and M. K. Albert, "Instance-based prediction of real-valued attributes," Comput. Intell., vol. 5, no. 2, pp. 51-57, 1989
3. R. Arens, "Learning SVM ranking functions from user feedback using document metadata and active learning in the biomedical domain," in Preference Learning, J. Fürnkranz and E. Hüllermeier, Eds. New York, NY, USA: Springer-Verlag, 2010, pp. 363-383
4. W. Chu and Z. Ghahramani, "Preference learning with Gaussian processes," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 137-144 (Pubitemid 43183326)
5. K.-J. Kim and H. Ahn, "A corporate credit rating model using multi-class support vector machines with an ordinal pairwise partitioning approach," Comput. Oper. Res., vol. 39, no. 8, pp. 1800-1811, 2012
6. R. Caruana, S. Baluja, and T. Mitchell, "Using the future to sort out the present: Rankprop and multitask learning for medical risk evaluation," in Proc. Adv. Neural Inf. Process. Syst., 1996, pp. 959-965
7. K. Crammer and Y. Singer, "Pranking with ranking," in Proc. NIPS, 2001, pp. 641-647
8. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender, "Learning to rank using gradient descent," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 89-96 (Pubitemid 43183320)
9. W. Chu and Z. Ghahramani, "Gaussian processes for ordinal regression," J. Mach. Learn. Res., vol. 6, pp. 1019-1041, Jan. 2005
10. R. Herbrich, T. Graepel, and K. Obermayer, "Support vector learning for ordinal regression," in Proc. Int. Conf. Artif. Neural Netw., 1999, pp. 97-102 (Pubitemid 30519921)
11. W. Chu and S. S. Keerthi, "Support vector ordinal regression," Neural Comput., vol. 19, no. 3, pp. 792-815, 2007
12. S. Agarwal and D. Roth, "Learnability of bipartite ranking functions," in Proc. 18th Annu. Conf. Learn. Theory, 2005, pp. 16-31
13. S. Kramer, G. Widmer, B. Pfahringer, and M. D. Groeve, "Prediction of ordinal classes using regression trees," in Proc. 12th Int. Symp. Found. Intell. Syst., 2000, pp. 426-434
14. H. Zhang, L. Jiang, and J. Su, "Augmenting naive bayes for ranking," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 1020-1027
15. E. Frank and M. Hall, "A simple approach to ordinal classification," in Proc. ECML, 2001, pp. 145-156 (Pubitemid 33331065)
16. J. S. Cardoso and J. F. Pinto da Costa, "Learning to classify ordinal data: The data replication method," J. Mach. Learn. Res., vol. 8, pp. 1393-1429, Sep. 2007 (Pubitemid 47210277)
17. A. Shashua and A. Levin, "Ranking with large margin principle: Two approaches," in Advances in Neural Information Processing Systems 15
18. [Neural Information Processing Systems, NIPS, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA, USA: MIT Press, 2002, pp. 937-944
19. L. Li and H.-T. Lin, "Ordinal regression by extended binary classification," in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 865-872
20. C.-W. Seah, I. Tsang, and Y.-S. Ong, "Transductive ordinal regression," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1074-1086, Sep. 2012
21. R. Herbrich, T. Graepel, and K. Obermayer, "Large margin rank boundaries for ordinal regression," in Advances Large Margin Classifiers. Cambridge, MA, USA: MIT Press, 2000, pp. 115-132
22. P. McCullagh, "Regression models for ordinal data," J. R. Stat. Soc., Ser. B, Methodol., vol. 42, no. 2, pp. 109-142, 1980
23. J. Verwaeren, W. Waegeman, and B. De Baets, "Learning partial ordinal class memberships with kernel-based proportional odds models," Comput. Stat. Data Anal., vol. 56, no. 4, pp. 928-942, 2012
24. W. Chu and S. S. Keerthi, "New approaches to support vector ordinal regression," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 145-152 (Pubitemid 43183327)
25. Y. Liu and X. Yao, "Negatively correlated neural networks can produce best ensembles," Austral. J. Intell. Inf. Process. Syst., vol. 4, no. 3, pp. 176-185, 1997
26. A. Omari and A. R. Figueiras-Vidal, "Feature combiners with gategenerated weights for classification," IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 1, pp. 158-163, Jan. 2013
27. W. Waegeman and L. Boullart, "An ensemble of weighted support vector machines for ordinal regression," Int. J. Comput. Syst. Sci. Eng., vol. 3, no. 1, pp. 47-51, 2009
28. H.-T. Lin and L. Li, "Large-margin thresholded ensembles for ordinal regression: Theory and practice," in Proc. 17th Int. Conf. Algorithmic Learn. Theory, 2006, pp. 319-333 (Pubitemid 44705637)
29. H.-T. Lin and L. Li, "Combining ordinal preferences by boosting," in Proc. ECML/PKDD 2009, Sep., pp. 69-83
30. K. Tumer and J. Ghosh, "Analysis of decision boundaries in linearly combined neural classifiers," Pattern Recognit., vol. 29, no. 2, pp. 341-348, 1996 (Pubitemid 126397840)
31. K. Tumer and J. Ghosh, "Error correlation and error reduction in ensemble classifiers," Connection Sci., vol. 8, no. 3, pp. 385-404, 1996
32. Y. Liu and X. Yao, "Ensemble learning via negative correlation," Neural Netw., vol. 12, no. 10, pp. 1399-1404, 1999
33. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996 (Pubitemid 126724382)
34. Y. Freund and R. E. Schapire, "Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn.. 1996, pp. 148-156
35. Z. S. Chan and N. Kasabov, "Fast neural network ensemble learning via negative-correlation data correction," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1707-1710, Nov. 2005 (Pubitemid 41709665)
36. G. Brown, J. Wyatt, R. Harris, and X. Yao, "Diversity creation methods: A survey and categorisation," J. Inf. Fusion, vol. 6, no. 1, pp. 5-20, 2005
37. K. Crammer and Y. Singer, "Online ranking by projecting," Neural Comput., vol. 17, no. 1, pp. 145-175, 2005 (Pubitemid 40305886)
38. W. Chu and S. S. Keerthi, "Support vector ordinal regression," Neural Comput., vol. 19, no. 3, pp. 792-815, 2007
39. P. A. Gutiérrez, C. Hervas-Martínez, and F. J. Martínez-Estudillo, "Logistic regression by means of evolutionary radial basis function neural networks," IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 246-263, Feb. 2011
40. M. J. Mathieson, "Ordinal models for neural networks," in Proc. 3rd Int. Conf. Neural Netw. Capital Markets, 1996, pp. 523-536
41. F. Fernández-Navarro, C. Hervás-Martínez, P. A. Gutierrez, and M. Carboreno, "Evolutionary q-gaussian radial basis functions neural networks for multi-classification," Neural Netw., vol. 24, no. 7, pp. 779-784, 2011
42. F. Fernández-Navarro, C. Hervás-Martínez, M. Cruz, P. A. Gutierrez, and A. Valero, "Evolutionary q-gaussian radial basis function neural network to determine the microbial growth/no growth interface of Staphylococcus aureus," Appl. Soft Comput., vol. 11, no. 3, pp. 3012-3020, 2011
43. F. Fernández-Navarro, C. Hervás-Martínez, J. Sánchez-Monedero, and P. A. Gutierrez, "MELM-GRBF: A modified version of the extreme learning machine for generalized radial basis function neural networks," Neurocomputing, vol. 74, no. 16, pp. 2502-2510, 2011
44. A. Chandra and X. Yao, "Ensemble learning using multi-objective evolutionary algorithms," J. Math. Model. Algorithms, vol. 5, no. 4, pp. 417-445, 2006 (Pubitemid 44371147)
45. A. Chandra and X. Yao, "Evolving hybrid ensembles of learning machines for better generalisation," Neurocomputing, vol. 69, nos. 7-9, pp. 686-700, 2006 (Pubitemid 43230374)
46. T. Windeatt, "Accuracy/diversity and ensemble MLP classifier design," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1194-1211, Sep. 2006 (Pubitemid 46445178)
47. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems. New York, NY, USA: Winston, 1977
48. A. Asuncion and D. Newman, (2007). UCI Machine Learning Repository
49. [Online]. Available: http://www.ics.uci.edu/mlearn/MLRepository.html
50. S. Baccianella, A. Esuli, and F. Sebastiani, "Evaluation measures for ordinal regression," in Proc. 2009 9th Int. Conf. Intell. Syst. Design Appl., Dec., pp. 283-287
51. M. Kendall, Rank Correlation Methods. London, U.K.: Griffin, 1948
52. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The weka data mining software: An update," SIGKDD Explorations Newslett., vol. 11, no. 1, pp. 10-18, 2009
53. D. J. C. Mackay, "Bayesian methods for backpropagation networks," in Models of Neural Networks III. New York, NY, USA: Springer-Verlag, 1994, ch. 6, pp. 211-254
54. J. Dem?ar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, Jan. 2006
55. M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings," Anna. Math. Stat., vol. 11, no. 1, pp. 86-92, 1940
56. H. Chen and X. Yao, "Regularized negative correlation learning for neural network ensembles," IEEE Trans. Neural Netw., vol. 20, no. 12, pp. 1962-1979, Dec. 2009
57. S. Wang and X. Yao, "Relationships between diversity of classification ensembles and single-class performance measures," IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 206-219, Jan. 2013
58. S. Wang and X. Yao, "Multi-class imbalance problems: Analysis and potential solutions," IEEE Trans. Syst., Man Cybern., Part-B, Cybern., vol. 42, no. 4, pp. 1119-1130, Aug. 2012.