Improving deep neural network ensembles using reconstruction error

Proceedings of the International Joint Conference on Neural Networks

Volumn 2015-September, Issue , 2015, Pages

  Huang, Wenhao ^a   Hong, Haikun ^a   Bian, Kaigui ^a   Zhou, Xiabing ^a   Song, Guojie ^a   Xie, Kunqing ^a  

^a Peking University   (China)

Author keywords

Silicon

Indexed keywords

CLASSIFICATION (OF INFORMATION); ERRORS; IMPORTANCE SAMPLING; SILICON;

BOOTSTRAP SAMPLINGS; CONFIDENCE INTERVAL; DEEP NEURAL NETWORKS; GENERALIZATION CAPABILITY; NEURAL NETWORK ENSEMBLES; PREDICTION INTERVAL; RECONSTRUCTION ERROR; STRUCTURE ENSEMBLES;

NETWORK LAYERS;

EID: 84951084176     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2015.7280524     Document Type: Conference Paper

References (24)

1. P. S. A. Krogh, "Learning with ensembles: How over-fitting can be useful," Advances in neural iriformation processing systems, vol. 8, p. 190, 1996.
2. L. Rokach, "Ensemble-based classifiers," ArtijicialIntelligence Review, vol. 33, no. 1-2, pp. 1-39, 2010.
3. Z.-H. Zhou, J. Wu, and W Tang, "Ensembling neural networks: many could be better than all," Artijicial intelligence, vol. 137, no. I, pp. 239-263, 2002.
4. L. Breiman, "Bagging predictors," Machine learning, vol. 24, no. 2, pp. 123-140, 1996.
5. B. Efron and R. J. Tibshirani, An introduction to the bootstrap. CRC press, 1994, vol. 57.
6. Y Freund, "Boosting a weak learning algorithm by majority," Iriformation and computation, vol. 121, no. 2, pp. 256-285, 1995.
7. A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25, 2012, pp. 1106-1114.
8. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
9. D. Erhan, Y Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio, "Why does unsupervised pre-training help deep learning" The Journal of Machine Learning Research, vol. 11, pp. 625-660, 2010.
10. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," arXiv preprint arXiv:1207. 0580, 2012.
11. Y Bengio, "Learning deep architectures for ai," Foundations and trends@ in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
12. H. Larochelle, Y Bengio, J. Louradour, and P. Lamblin, "Exploring strategies for training deep neural networks," The Journal of Machine Learning Research, vol. 10, pp. 1-40, 2009.
13. G. E. Hinton, S. Osindero, and Y-W Teh, "A fast learning algorithm for deep belief nets," Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006.
14. Y Bengio, P Lamblin, D. P opovici, and H. Larochelle, "Greedy layer-wise training of deep networks," Advances in neural iriformation processing systems, vol. 19, p. 153, 2007.
15. P Vincent, H. Larochelle, L. Lajoie, Y Bengio, and P-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," The Journal of Machine Learning Research, vol. 9999, pp. 3371-3408, 2010.
16. S. Lange and M. Riedmiller, "Deep auto-encoder neural networks in reinforcement learning," in Neural Networks (JCNN), The 20iO international Joint Coriference on. IEEE, 2010, pp. 1-8.
17. D. Erhan, P-A. Manzagol, Y Bengio, S. Bengio, and P. Vincent, "The difficulty of training deep architectures and the effect of unsupervised pre-training," in International Conference on ArtijicialIntelligence and Statistics, 2009, pp. 153-160.
18. J. G. Hwang and A. A. Ding, "Prediction intervals for artificial neural networks," Journal of the American Statistical Association, vol. 92, no. 438, pp. 748-757, 1997.
19. A. Khosravi, S. Nahavandi, D. Creighton, and A. F. Atiya, "Comprehensive review of neural network-based prediction intervals and new advances," Neural Networks, IEEE Transactions on, vol. 22, no. 9, pp. 1341-1356, 2011.
20. G. E. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for lvcsr using rectified linear units and dropout," in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 8609-8613.
21. S. Wang and C. Manning, "Fast dropout training," in Proceedings of the 30th international Coriference on Machine Learning (ICML-J3), 2013, pp. 118-126.
22. Y Zheng, F. Liu, and H.-P. Hsieh, "U-air: when urban air quality inference meets big data," in Proceedings of the i9th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2013, pp. 1436-1444.
23. Y Zheng, X. Chen, Q. Jin, Y Chen, X. Qu, X. Liu, E. Chang, W-Y Ma, Y Rui, and W Sun, "A cloud-based knowledge discovery system for monitoring fine-grained air quality," MSR-TR-2014-40, Tech. Rep., 2014.
24. S. Rifai, P Vincent, X. Muller, X. Glorot, and Y Bengio, "Contractive auto-encoders: Explicit invariance during feature extraction," in Proceedings of the 28th International Conference on Machine Learning (ICML-ll), 2011, pp. 833-840