Second-order learning algorithm with squared penalty term

Advances in Neural Information Processing Systems

Volumn , Issue , 1997, Pages 627-633

  Saito, Kazumi ^a   Nakano, Ryohei ^a  

^a NTT Communication Science Laboratories   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE PERFORMANCE; GENERALIZATION PERFORMANCE; PENALTY FACTOR; PENALTY TERM; SECOND ORDERS;

LEARNING ALGORITHMS;

EID: 0039816623     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. Battiti, R. (1992) First-and second-order methods for learning between steepest descent and Newton's method. Neural Computation 4(2):141-166.
2. Bishop, CM. (1995) Neural networks for pattern recognition. Clarendon Press.
3. Hanson, S.J. & Pratt, L. Y. (1989) Comparing biases for minimal network construction with back-propagation. In D. S. Touretzky (ed.), Advances in Neural Processing Systems, Volume 1, pp. 177-185. San Mateo, CA: Morgan Kaufmann.
4. Hinton, G.E. (1987) Learning translation invariant recognition in massively parallel networks. In J. W. de Bakker, A. J. Nijman and P. C. Treleaven (eds.), Proceedings PARLE Conference on Parallel Architectures and Languages Europe, pp. 1-13. Berlin: Springer-Verlag.
5. Ishikawa, M. (1990) A structural learning algorithm with forgetting of link weight. Tech. Rep. TR-90-7, Electrotechnical Lab. Tsukuba-City, Japan.
6. MacKay, D.J.C. (1992) Bayesian interpolation. Neural Computation 4(3):415-447.
7. Møller, M.F. (1993) A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6(4):525-533.
8. Poggio, T. &; Girosi, F. (1990) Regularization algorithms for learning that are equivalent to multilayer networks. Science 247:978-982.
9. Saito, K. &; Nakano, R. (1997) Partial BFGS update and efficient step-length calculation for three-layer neural networks. Neural Computation 9(1):239-257 (in press).
10. Silva, F.M. & Almeida, L.B. (1990) Speeding up backpropagation. In R. Eckmiller (ed.), Advanced Neural Computers, pp. 151-160. Amsterdam: North-Holland.
11. Stone, M. (1978) Cross-validation: A review. Operationsforsch. Statist. Ser. Statistics B 9(1):111-147.
12. Williams, P.M. (1995) Bayesian regularization and pruning using a Laplace prior. Neural Computation 7(1):117-143.