MLPs (mono-layer polynomials and multi-layer perceptrons) for nonlinear modeling

Journal of Machine Learning Research

Volumn 3, Issue , 2003, Pages 1383-1398

  Rivals, Isabelle ^a   Personnaz, Léon ^a  

^a PSL RESEARCH UNIVERSITY   (France)

Author keywords

Additive procedure; Approximate leave one out scores; Confidence intervals; Input variable selection; Jacobian matrix conditioning; Model approval; Model selection; Neural networks; Orthogonalization procedure; Overfitting avoidance; Polynomials; Statistical tests

Indexed keywords

CONFIDENCE INTERVAL; INPUT VARIABLE SELECTION; JACOBIANS; LEAVE-ONE-OUT; MODEL SELECTION; ORTHOGONALIZATION; OVERFITTING;

COMPLEX NETWORKS; JACOBIAN MATRICES; NEURONS; POLYNOMIALS; STATISTICAL TESTS;

NEURAL NETWORKS;

EID: 0346825283     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (26)

1. U. Anders. Neural network pruning and statistical hypotheses tests. In Progress in Connectionnist-Based Information Systems (Addendum), Proceedings of ICONIP.97. Springer, Berlin: 1-4, 1997.
2. U. Anders and O. Korn. Model selection in neural networks. Neural Networks 12: 309- 323, 1999.
3. A. Antoniadis, J. Berruyer and R. Carmona. Régression non linéaire et applications. Economica, 1992.
4. D. M. Bates and D. G. Watts. Nonlinear regression analysis and its applications. John Wiley & Sons, 1988.
5. S. Chen and S. A. Billings. Prediction-error estimation algorithm for non-linear outputaffine systems. Int. Journal of Control 47(1): 309-332, 1988.
6. S. Chen, A. Billings and W. Luo. Orthogonal least-squares methods and their application to non-linear system identification. Int. Journal of Control 50(5): 1873-1896, 1989.
7. N. R. Draper and H. Smith. Applied regression analysis. John Wiley & Sons inc., New York, 1998.
8. G. H. Golub and C.F. Van Loan. Matrix computations. John Hopkins University Press, Baltimore, 1983.
9. B. Hassibi, D. Stork, G. Wolff and T. Watanabe. Optimal brain surgeon: Extensions and performance comparisons. In Advances in Neural Information Processing Systems 6. Morgan Kaufman, San Mateo, CA: 263-270, 1994.
10. T. Hastie, R. Tibshirani and J. Friedman. The elements of statistical learning. Springer, New York, 2001.
11. T.-Y. Kwok and D.-Y. Yeung. Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks 8(3): 630-645, 1997a.
12. T.-Y. Kwok and D.-Y. Yeung. Objective functions for training new hidden units in constructive neural networks. IEEE Transactions on Neural Networks 8(5): 1131- 1148, 1997b.
13. Y. Le Cun, J. S. Denker and S. A. Solla. Optimal brain damage. In D. S. Touretzki (Ed.), Advances in Neural Information Processing Systems, vol. 2. Morgan Kaufmann, San Mateo, CA: 598-605, 1990.
14. I. J. Leontaritis and S. A. Billings. Model selection and validation methods for non-linear systems. Int. J. Control 45(1): 311-341, 1988.
15. J. Moody. Prediction risk and architecture selection for neural networks. In V. Cherkassy, J. H. Friedman and H. Wechsler (eds.), From statistics to neural networks: theory and pattern recognition applications, NATO ASI Series, Springer Verlag, 1994.
16. I. Rivals and L. Personnaz. Construction of confidence intervals in neural modeling using a linear Taylor expansion. Proceedings of the International Workshop on Advanced Black-Box Techniques for Nonlinear Modeling, 8-10 July 1998, Leuwen: 17-22, 1998.
17. I. Rivals and L. Personnaz. On cross-validation for model selection. Neural Computation 11(4): 863-870, 1999.
18. I. Rivals and L. Personnaz. Construction of confidence intervals for neural networks based on least squares estimation. Neural Networks 13(1): 463-484, 2000a.
19. I. Rivals and L. Personnaz. A statistical procedure for determining the optimal number of hidden neurons of a neural model. Proceedings of the Second ICSC Symposium on Neural Computation NC'2000, Berlin-Germany, 2000b.
20. I. Rivals and L. Personnaz. Neural network construction and selection in nonlinear modeling. IEEE Transactions on Neural Networks, to appear 2003a.
21. I. Rivals and L. Personnaz. Jacobian conditioning analysis for model validation. Neural Computation, to appear 2003b.
22. G. A. F. Seber. Linear regression analysis.Wiley, New York, 1977.
23. G. A. F. Seber and C. Wild. Nonlinear regression. Wiley, New York, 1989.
24. H. Stoppiglia. Méthodes statistiques de sélection de modèles neuronaux; applications financières et bancaires. Thèse de Doctorat de l.Université Paris 6, 1997.
25. V. Vapnik. Estimation of dependences based on empirical data. Springer Verlag, New York, 1982.
26. J.-P. Vila, V.Wagner and P. Neveu. Bayesian nonlinear model selection and neural networks: a conjugate prior approach. IEEE Transactions on Neural Networks 11(2): 265-278, 1999.