The "weight smoothing" regularization of MLP for Jacobian stabilization

IEEE Transactions on Neural Networks

Volumn 10, Issue 6, 1999, Pages 1502-1510

  Aires, Filipe ^a   Schmitt, Michel ^b   Chedin, Alain ^a   Scott, Noëlle ^a  

^a ECOLE POLYTECHNIQUE   (France)

^b PSL RESEARCH UNIVERSITY   (France)

Author keywords

Ill posed problems; Inverse problems; MLP; Neural jacobians; Regularization

Indexed keywords

APPROXIMATION THEORY; INVERSE PROBLEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; PROBLEM SOLVING;

JACOBIAN STABILIZATION;

MULTILAYER NEURAL NETWORKS;

EID: 0033355287     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.809096     Document Type: Article

References (22)

1. F. Aires, R. Armante, A. Chédin, and N. Scott, "Surface and atmospheric temperature retrieval with the high resolution interferometer iasi," in AMS Conf., Paris, France, 1998, pp. 181-186.
2. C. Bishop, "Curvature-driven smoothing: A learning algorithm for feedforward networks," IEEE Trans. Neural Networks, vol. 4, pp. 882-884, 1993.
3. _, "Training with noise is equivalent to tikhonov regularization," Neural Comput., vol. 7, no. 1, pp. 108-116, 1995.
4. _, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1996.
5. A. Chédin, N. Scott, C. Wahiche, and P. Moulinier, "The improved initialization inversion method: A high-resolution physical method for temperature retrievals from tiros-n series," J. Clim. Appl. Meteor., vol. 24, pp. 128-143, 1985.
6. S. Colombano, M. Compton, and M. Bualat, "Goal directed model inversion: Adaptation to unexpected model changes," in Neuro-Nimes, 269-278, 1991.
7. H. Drucker and Y. Le Cun, "Improving generalization performance using double backpropagation," IEEE Trans. Neural Networks, vol. 3, pp. 991-997, 1992.
8. J. Escobar, Base de données pour la restitution de paramètres atmosphériques à l'échelle globale; étude sur l'inversion par réseaux de neurones des données des sondeurs verticaux atmosphériques satellitaires présents et à venir, Ph.D. dissertation, Université Denis Diderot (Pans VII), 1991.
9. J. Escobar, A. Chédin, F. Cheruy, and N. Scott, "Réseaux de neurones multicouches pour la restitution de variables thermodynamiques atmosphériques à l'aide de sondeurs verticaux satellitaires," CRAS, vol. 317, no. II, pp. 911-918, 1993.
10. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias-variance dilema," Neural Comput., vol. 1, no. 4, pp. 1-58, 1992.
11. F. Girossi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol. 7, pp. 219-269, 1995.
12. M. Jordan and D. Rumelhart, "Forward models: Supervised learning with a distal teacher," Cognitive Sci., vol. 16, pp. 307-354, 1992.
13. J. Kindermann and A. Linden, "Inversion of neural networks by gradient descent," Parallel Computing, 270-286, 1990.
14. J.-W. Lee and J.-H. Oh, "Hybrid learning of mapping and its Jacobian in multilayer neural networks," Neural Comput., vol. 9, pp. 937-958, 1997.
15. M. Nashed and G. Wahba, "Generalized inverses in reproducing kernel spaces: an approach to regularization of linear operator equations," SIAM J. Math. Anal., vol. 5, no. 6, pp. 974-987, 1974.
16. D. Phillips, J. Assoc. Comput. Machinery, vol. 9, pp. 84-97, 1962.
17. T. Poggio and F. Girosi, "Networks for approximation and learning," in Proc. IEEE, vol. 78, pp. 1481-1497, 1990.
18. C. Rodgers, "Retrieval of atmospheric temperature and composition from remote measurements of thermal radiation," Rev. Geophys. Space Phys., vol. 14, no. 4, pp. 609-624, Nov. 1976.
19. D. Rumelhart, G. Hinton, and R. Williams, Learning Representations by Backpropagating Error, vol. I. Cambridge, MA: MIT Press, 1986, ch. 8, pp. 318-362.
20. N. Scott and A. Chédin, "A fast line-by-line method for atmospheric absorption computations: The automatized atmospheric absorption atlas," J. Appl. Meteor., vol. 20, pp. 802-812, 1981.
21. A. Tikhonov and V. Arsenin, Solutions of Ill-Posed Problems. Washington, D.C.: V. H. Vinsten, 1977.
22. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.