Regularisation methods for neural network model averaging

Engineering Applications of Artificial Intelligence

Volumn 41, Issue , 2015, Pages 128-138

  Didcock, Nico ^a   Jakubek, Stefan ^a   Kögeler, Hans Michael ^b  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b AVL LIST GMBH   (Austria)

Author keywords

Empirical risk; Information criteria; Model ensembles; Neural networks; Nonlinear regression

Indexed keywords

COMPLEX NETWORKS; ECONOMIC AND SOCIAL EFFECTS; FORECASTING; MODEL STRUCTURES; RISK ASSESSMENT;

COMPLEX DATA STRUCTURES; EMPIRICAL RISKS; INFORMATION CRITERION; MODEL ENSEMBLES; NEURAL NETWORK MODEL; NON-LINEAR REGRESSION; PREDICTION PERFORMANCE; WEIGHT CALCULATION;

NEURAL NETWORKS;

EID: 84926345841     PISSN: 09521976     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.engappai.2015.02.005     Document Type: Article

References (37)

1. Akaike, Hirotugu, 1973. Information theory and an extension of the maximum likelihood principle. Second International Symposium on Information Theory, pp. 267-281.
2. U. Anders, and O. Korn Model selection in neural networks Neural Netw. 12 2 1999 309 323
3. J. Ashburner, and K.J. Friston Nonlinear spatial normalization using basis functions Human Brain Map. 7 4 1999 254 266
4. C.M. Bishop Curvature-driven smoothing a learning algorithm for feedforward networks IEEE Trans. Neural Netw. 4 5 1993 882 884
5. L. Breiman Bagging predictors Mach. Learn. 24 2 1996 123 140
6. S. Boyd, and L. Vandenberghe Convex Optimization 2009 Cambridge University Press Cambridge
7. S.T. Buckland, K.P. Burnham, and N.H. Augustin Model selection an integral part of inference Biometrics 53 2 1997 603 618
8. K.P. Burnham, and D.R. Anderson Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2002 Springer New York City
9. G. Claeskens, and N.L. Hjort Model Selection and Model Averaging vol. 330 2008 Cambridge University Press Cambridge
10. B. Efron The estimation of prediction error J. Am. Stat. Assoc. 99 467 2004
11. S. Geman, E. Bienenstock, and R. Doursat Neural networks and the bias/variance dilemma Neural Comput. 4 1 1992 1 58
12. F. Girosi, M. Jones, and T. Poggio Regularization theory and neural networks architectures Neural Comput. 7 2 1995 219 269
13. C. Hametner, C. Mayr, and S. Jakubek Dynamic NOx emission modelling using local model networks Int. J. Engine Res. 15 8 2014 928 933 10.1177/1468087414523281
14. B.E. Hansen Least squares model averaging Econometrica 75 4 2007 1175 1189
15. D.C. Hoaglin, and R.E. Welsch The hat matrix in regression and ANOVA Am. Stat. 32 1 1978 17 22
16. K. Hornik, M. Stinchcombe, and H. White Multilayer feedforward networks are universal approximators Neural Netw. 2 5 1989 359 366
17. S. Jakubek, and C. Hametner Identification of neurofuzzy models using GTLS parameter estimation IEEE Trans. Syst. Man Cybern. Part B: Cybern. 39 5 2009 1121 1133
18. S. Jakubek, and N. Keuth A local neuro-fuzzy network for high-dimensional models and optimization Eng. Appl. Artif. Intel. 19 6 2006 705 717
19. D.A. Jones Statistical analysis of empirical models fitted by optimization Biometrika 70 1 1983 67 88
20. Kotsiantis, S.B.; Kanellopoulos, D.; Zaharakis, I.D.; 2006. Bagged averaging of regression models. In: Artificial Intelligence Applications and Innovations, Springer, US, pp. 53-60.
21. Li, K.C.; 1987. Asymptotic optimality for Cp, CL, cross-validation and generalized cross-validation: discrete index set. Ann. Stat. 958-975.
22. C.L. Mallows Some comments on Cp Technometrics 15 4 1973 661 675
23. Moody, J.; 1994. Prediction risk and architecture selection for neural networks. In: From Statistics to Neural Networks, Springer, Berlin Heidelberg, pp. 147-165.
24. N. Murata, S. Yoshizawa, and S.I. Amari Network information criterion-determining the number of hidden units for an artificial neural network model IEEE Trans. Neural Netw. 5 6 1994 865 872
25. O. Nelles Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models 2001 Springer New York City
26. R. Nishii Asymptotic properties of criteria for selection of variables in multiple regression Ann. Stat. 12 2 1984 758 765
27. G. Schwarz Estimating the dimension of a model Ann. Stat. 6 2 1978 461 464
28. R. Shibata An optimal selection of regression variables Biometrika 68 1 1981 45 54
29. C.Y. Sin, and H. White Information criteria for selecting possibly misspecified parametric models J. Econom. 71 1 1996 207 225
30. Stein, C.M.; 1981. Estimation of the mean of a multivariate normal distribution. Ann. Stat.; 1135-1151.
31. Stone, M.; 1974. Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. Ser. B (Methodol.), 111-147.
32. T. Tasdizen, R. Whitaker, P. Burchard, and S. Osher Geometric surface processing via normal maps ACM Trans. Graph. (TOG) 22 4 2003 1012 1033
33. A. Tsanas, M.A. Little, P.E. McSharry, and L.O. Ramig Accurate telemonitoring of Parkinsons disease progression by noninvasive speech tests IEEE Trans. Biomed. Eng. 57 4 2010 884 893
34. P. Tüfekci Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods Int. J. Electr. Power Energy Syst. 60 2014 126 140
35. H. White Learning in artificial neural networks a statistical perspective Neural Comput. 1 4 1989 425 464
36. J. Ye On measuring and correcting the effects of data mining and model selection J. Am. Stat. Assoc. 93 441 1998 120 131
37. I.C. Yeh Modeling of strength of high-performance concrete using artificial neural networks Cement Concr. Res. 28 12 1998 1797 1808