On "natural" learning and pruning in multilayered perceptrons

Neural Computation

Volumn 12, Issue 4, 2000, Pages 881-901

  Heskes, Tom ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; DECISION TREE; NEUROSURGERY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DECISION TREES; NEURAL NETWORKS (COMPUTER); NEUROSURGERY;

EID: 0034167148     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015637     Document Type: Article

References (19)

1. Almeida, L., & Silva, F. (1992). Adaptive decorrelation. In I. Aleksander & J. Taylor (Eds.), Artificial neural networks, 2 (pp. 149-156). Amsterdam: North-Holland.
2. Amari, S. (1985). Differential-geometric methods in statistics. New York: Springer-Verlag.
3. Amari, S. (1998). Natural gradient works efficiently in learning. Neural Computation, 10, 251-276.
4. Ampazis, N., Perantonis, S., & Taylor, J. (1999). Dynamics of multilayer networks in the vicinity of temporary minima. Neural Networks, 12, 43-58.
5. Castellano, G., Fanelli, A., & Pelillo, M. (1997). An iterative pruning algorithm for feedforward neural networks. IEEE Transactions on Neural Networks, 8, 519-531.
6. Egmont-Petersen, M., Talmon, J., Hasman, A., & Ambergen, A. (1998). Assessing the importance of features for multi-layer perceptrons. Neural Networks, 11, 623-635.
7. Gelman, A., Carlin, J., Stern, H., & Rubin, D. (1995). Bayesian data analysis. London: Chapman & Hall.
8. Hassibi, B., & Stork, D. (1993). Second order derivatives for network pruning: Optimal brain surgeon. In S. Hanson, J. Cowan, & L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 164-171). San Mateo, CA: Morgan Kaufmann.
9. Heskes, T., & Coolen, J. (1997). Learning in two-layered networks with correlated examples. Journal of Physics A, 30, 4983-4992.
10. Hondou, T., & Sawada, Y. (1994). Analysis of learning processes of chaotic time series by neural networks. Progress in Theoretical Physics, 91, 397-402.
11. Kivinen, J., & Warmuth, M. (1997). Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132, 1-63.
12. Laar, P. v. d., & Heskes, T. (1999). Pruning using parameter and neuronal metrics. Neural Computation, 11, 977-993.
13. Luenberger, D. (1984). Linear and nonlinear programming. Reading, MA: Addison-Wesley.
14. Pedersen, M., Hansen, L., & Larsen, J. (1996). Pruning with generalization based weight saliencies: γOBD, γOBS. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 521-527). Cambridge, MA: MIT Press.
15. Rattray, M., Saad, D., & Amari, S. (1998). Natural gradient descent for on-line learning. Physical Review Letters, 81, 5461.
16. Saad, D., & Solla, S. (1995). Exact solution for on-line learning in multilayer neural networks. Physical Review Letters, 74, 4337.
17. Stahlberger, A., & Riedmiller, M. (1997). Fast network pruning and feature extraction using the unit-OBS algorithm. In M. Mozer, M. Jordan & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 655-661). Cambridge, MA: MIT Press.
18. Wiegerinck, W., & Heskes, T. (1996). How dependencies between successive examples affect on-line learning. Neural Computation, 8, 1743-1765.
19. Yang, H., & Amari, S. (1998). The efficiency and robustness of natural gradient descent learning rule. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 385-391). Cambridge, MA: MIT Press.