MDL regularizer: A new regularizer based on the MDL principle

IEEE International Conference on Neural Networks - Conference Proceedings

Volumn 3, Issue , 1997, Pages 1833-1838

  Saito, K ^a   Nakano, R ^a  

^a NTT Communication Science Laboratories   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

GENERALIZATION ERROR; GENERALIZATION PERFORMANCE; MDL PRINCIPLE; MINIMUM DESCRIPTION LENGTH; PRIOR DISTRIBUTION; REGRESSION PROBLEM; REGULARIZATION METHODS; SECOND ORDERS;

ARTIFICIAL INTELLIGENCE; COGNITIVE SYSTEMS; APPROXIMATION THEORY; ERROR ANALYSIS; LEARNING ALGORITHMS; REGRESSION ANALYSIS; VECTORS;

NEURAL NETWORKS;

MAXIMUM LIKELIHOOD WEIGHT VECTOR; MINIMUM DESCRIPTION LENGTH (MDL) REGULARIZER;

EID: 0030675039     PISSN: 10987576     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICNN.1997.614177     Document Type: Conference Paper

References (17)

1. H. Akaike. A new look at the statistical model identification. IEEE Trans. Autom. Contor., AC-19:716-723, 1973.
2. P. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. Technical report, Australian National University, 1996.
3. C. Bishop. Neural networks for pattern recognition. Oxford Press, 1995.
4. B. Efron and R. Tibshirani. An introduction to the bootstrap. Chapman & Hall, New York, 1993.
5. P. Gill, W. Murray, and M. Wright. Practical optimization. Academic Press, London, 1981.
6. S. Hanson and L. Pratt. Comparing biases for minimal network construction with back-propagation. In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 1, pages 177-185, San Mateo, CA, 1989. Morgan Kaufmann.
7. G. Hinton. Learning translation invariant recognition in massively parallel networks. In J. de Bakker, A. Nij-man, and P. Treleaven, editors, Proc. PARLE Conference on Parallel Architectures and Languages Europe, pages 1-13, Berlin, 1987. Springer-Verlag.
8. M. Ishikawa. A structural learning algorithm with forgetting of link weight. Technical report, Tech. Rep. TR-90-7, Electrotechnical Lab. Tsukuba-City, Japan, 1990.
9. D. MacKay. Bayesian interpolation. Neural Computation, 4(3):415-447, 1992.
10. B. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6(1):147-160, 1994.
11. T. Poggio and F. Girosi. Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247:978-982, 1990.
12. J. Rissanen. Stochastic complexity in statistical inquiry. World Scientific, 1989.
13. D. Rumelhart, G. Hinton, and R. Williams. Learning internal representations by error propagation. In D. Rumelhart and J. McClellandnd, editors, Parallel Distributed Processing, pages 318-362. MIT Press, 1986.
14. K. Saito and R. Nakano. Partial BFGS update and efficient step-length calculation for three-layer neural networks. Neural Computation, 9(1):123-141, 1997.
15. K. Saito and R. Nakano. Second-order learning algorithm with squared penalty term. In Advances in Neural Information Processing Systems, volume 9, 1997.
16. M. Stone. Cross-validation: A review. Operations-forsch. Statist. Ser. Statistics B, 9(1):111-147, 1978.
17. P. Williams. Bayesian regularization and pruning using a Laplace prior. Neural Computation, 7(1):117-143, 1995.