Learning ensemble classifiers via restricted Boltzmann machines

Pattern Recognition Letters

Volumn 36, Issue 1, 2014, Pages 161-170

  Zhang, Chun Xia ^a   Zhang, Jiang She ^a   Ji, Nan Nan ^a   Guo, Gao ^b  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b XI'AN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Bagging; Deep learning; Diversity; Ensemble classifier; Majority voting; Restricted Boltzmann machine

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); DATA MINING; DEEP LEARNING; FEATURE EXTRACTION; LEARNING SYSTEMS; TREES (MATHEMATICS); VIRTUAL REALITY;

BAGGING; DIVERSITY; ENSEMBLE CLASSIFIERS; MAJORITY VOTING; RESTRICTED BOLTZMANN MACHINE;

LEARNING ALGORITHMS;

EID: 84893100083     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2013.10.009     Document Type: Article

References (54)

1. Bauer, E., Kohavi, R., 1999. An empirical comparison of voting classification algorithms: bagging, boosting, and variants. Mach. Learn. 36 (1-2), 105-139.
2. Baumgartner, D., Serpen, G., 2012. A design heuristic for hybrid classification ensembles in machine learning. Intell. Data Anal. 16 (2), 233-246.
3. Bengio, Y., 2009. Learning deep architectures for AI. Found. Trends Mach. Learn. 2 (1), 1-127.
4. Bengio, Y., 2012. Practical recommendations for gradient-based training of deep architectures. In: Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, Vol. 7700. Springer, Berlin, pp. 437-478.
5. Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., 2007. Greedy layer-wise training of deep networks. Advances in Neural Information Processing System, Vol. 19, pp. 153-160.
6. Bengio, Y., Chapados, N., Delalleau, O., Larochelle, H., Saint-Mleux, X., Hudon, C, Louradour, J., 2012. Detonation classification from acoustic signature with the restricted Boltzmann machine. Comput. Intell. 28 (2), 261-288.
7. Breiman, L, 1996. Bagging predictors. Mach. Learn. 24 (2), 123-140. (Pubitemid 126724382)
8. Breiman, L, 2001. Random forests. Mach. Learn. 45 (1), 5-32. (Pubitemid 32933532)
9. Breiman, L., Friedman, J., Olshen, R., Stone, C, 1984. Classification and Regression Trees. Chapman & Hall, CRC Press, New York.
10. Brügge, K., Fischer, A., Igel, C, 2013. The flip-the-state transition operator for restricted Boltzmann machines. Mach. Learn. 93 (1), 53-69.
11. Cai, X.G., Hu, S., Lin, X.L, 2012. Feature extraction using restricted Boltzmann machine for stock price prediction. In: Proc. IEEE Int. Conf. Comput. Sci. Autom. Eng., pp. 80-83.
12. Cheng, J., Liu, Q.S., Lu, H.Q., Chen, Y.W., 2006. Ensemble learning for independent component analysis. Pattern Recognit. 39 (1), 81-88. (Pubitemid 41525566)
13. Crawford, M., Kim, W., 2009. Manifold learning for multi-classifier systems via ensembles. In: Benediktsson, J.A., Kitller, J., Roli, F. (Eds.), MCS, Lecture Notes in Computer Science, Vol. 5519. Springer, Berlin, pp. 519-528.
14. De Bock, K.W., Poel, D.V., 2011. An empirical evaluation of rotation-based ensemble classifiers for customer churn prediction. Expert Syst. Appl. 38 (10), 12293-12301.
15. Demšar, J., 2006. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. 7, 1-30. (Pubitemid 43022939)
16. Duin, R.P.W., Juszczak, P., Paclik, P., Pekalska, E., Ridder, D., Tax, D.M.J., Verzakov, S., 2007. PRTools4: a matlab toolbox for pattern recognition. Delft University of Technology, Delft.
17. Džeroski, S., Zenko, B., 2004. Is combining classifiers with stacking better than selecting the best one? Mach. Learn. 54 (3), 255-273.
18. Fischer, A., Igel, C, 2012. An introduction to restricted Boltzmann machines. In: Alvarez et al. (Eds.), CIARP, Lecture Notes in Computer Science, Vol. 7441. Springer, Berlin, pp. 14-36.
19. Fischer, A., Igel, C, 2014. Training restricted Boltzmann machines: an introduction. Pattern Recognit. 47 (1), 25-39.
20. Frank, A., Asuncion A., 2010. UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml.
21. Freund, Y., Schapire, R.E., 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Sys. Sci. 55 (1), 119-139. (Pubitemid 127433398)
22. García-Pedrajas, N., García-Osorio, C, Fyfe, C, 2007. Nonlinear boosting projections for ensemble construction. J. Mach. Learn. Res. 8, 1-33. (Pubitemid 46155122)
23. Hinton, G.E., 2002. Training products of experts by minimizing contrastive divergence. Neural Comput. 14 (8), 1771-1800.
24. Hinton, G.E., 2010. A practical guide to training restricted Boltzmann machines. Department of Computer Science, University of Toronto, Montreal.
25. Hinton, G.E., Salakhutdinov, R.R., 2006. Reducing the dimensionality of data with neural networks. Science 313, 504-507. (Pubitemid 44148451)
26. Hinton, G.E., Osindero, S., Teh, Y.W., 2006. A fast learning algorithm for deep belief nets. Neural Comput. 18 (7), 1527-1554. (Pubitemid 44024729)
27. Ho, T.K., 1998. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20 (8), 832-844. (Pubitemid 128741345)
28. Hothorn, T., Lausen, B., 2003. Double-bagging: combining classifiers by bootstrap aggregation. Pattern Recognit. 36 (6), 1303-1309. (Pubitemid 36230219)
29. Kuncheva, L.I., 2004. Combining Pattern Classifiers, Methods and Algorithms. Wiley Interscience, John Wiley & Sons, New Jersey.
30. Larochelle, H., Mandel, M., Pascanu, R., Bengio, Y, 2012. Learning algorithms for the classification restricted Boltzmann machine. J. Mach. Learn. Res. 13, 643-669.
31. Loeff, N., Forsyth, D., Ramachandran, D., 2008. ManifoldBoost: stagewise function approximation for fully-, semi-, and un-supervised learning. In: Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, pp. 600-607.
32. Plumpton, CO., Kuncheva, LI., Oosterhof, N.N., Jognston, S.J., 2012. Naive random subspace ensemble with linear classifiers for real-time classification of fMRI data. Pattern Recognit. 45 (5), 2101-2108.
33. Polikar, R., 2006. Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 6 (3), 21-45.
34. Rodríguez, J.J., Kuncheva, LI., Alonso, C.J., 2006. Rotation forest: a new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell. 28 (10), 1619-1630. (Pubitemid 46405081)
35. Rodríguez, J.J., Díez-Pastor, J.F., Garcia-Osorio, C, 2011. Ensembles of decision trees for imbalanced data. In: Sansone, C, Kittler, J., Roli, F. (Eds.), MCS, Lecture Notes in Computer Science, Vol. 6713. Springer, Berlin, pp. 76-85.
36. Rokach, L, 2009. Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography. Comput. Stat. Data Anal. 53 (12), 4046-4072.
37. Rokach, L, 2010. Ensemble-based classifiers. Artif. Intell. Rev. 33 (1-2), 1-39.
38. Sammut, C, Webb, G.I., 2011. Encyclopedia of Machine Learning. Springer-Verlag, New Work.
39. Sun, Z.L., Zheng, C.H., Gao, Q.W., Zhang, J., Zhang, D.X., 2012. Tumor classification using eigengene-based classifier committee learning algorithm. IEEE Signal Process Lett. 19 (8), 455-458.
40. Tieleman, T., 2008. Training restricted Boltzmann machines using approximations to the likelihood gradient. In: Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, pp. 1064-1071.
41. Tieleman, T., Hinton, G.E., 2009. Using fast weights to improve persistent contrastive divergence. In: Proc. 26th Int. Conf. Mach. Learn., Montreal, Canada, pp. 1033-1040.
42. Ting, K.M., Low, B.T., Witten, LH., 1999. Learning from batched data: models combination vs data combination. J. Knowl. Inf. Syst. 1 (1), 83-106.
43. Tran, T., Phung, D.Q., Venkatesh, S., 2011. Mixed-variate restricted Boltzmann machines. J. Mach. Learn. Res. 20, 213-229.
44. Wang, G.W., Zhang, C.X., Zhuang, J., 2012. An application of classifier combination methods in hand gesture recognition. Math. Prob. Eng. Article ID 346951, 17 pages.
45. Wang, Z., Jie, W.B., Chen, S.C., Gao, D.Q., 2013. Random projection ensemble learning with multiple empirical kernels. Knowledge-Based Syst. 37, 388-393.
46. Webb, G.I., 2000. Multiboosting: a technique for combining boosting and wagging. Mach. Learn. 40 (2), 159-196.
47. Welling, M., Rosen-Zvi, M., Hinton, G.E., 2005. Exponential family harmoniums with an application to information retrieval. Advances in Neural Information Processing System, Vol. 17. Cambridge, MIT Press, pp. 1481-1488.
48. Zhang, C.X., Duin, R.P.W., 2009. An empirical study of a linear regression combiner on multi-class data sets. In: Benediktsson, J.A., Kitller, J., Roli, F. (Eds.), MCS, Lecture Notes in Computer Science, Vol. 5519. Springer, Berlin, pp. 478-487.
49. Zhang, C.X., Duin, R.P.W., 2011. An experimental study of one- and two-level classifier fusion for different sample sizes. Pattern Recognit. Lett. 32 (14), 1756-1767.
50. Zhang, C.X., Zhang, J.S., 2010. A variant of rotation forest for constructing ensemble classifiers. Pattern Anal. Appl. 13 (1), 59-77.
51. Zhang, Y., Zhou, Z.H., 2010. Cost-sensitive face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32 (10), 1758-1769.
52. Zhang, M.L., Zhou, Z.H., 2013. Exploring unlabeled data to enhance ensemble diversity. Data Min. Knowl. Discovery 26 (1), 98-129.
53. Zhou, Z.H., 2012. Ensemble Methods: Foundations and Algorithms. Chapman & Hall/CRC, Boca Raton.
54. Zhu, D., 2010. A hybrid approach for efficient ensembles. Decis. Support Syst. 48 (3), 480-487.