Tempering Backpropagation Networks: Not All Weights are Created Equal

NIPS 1995: Proceedings of the 8th International Conference on Neural Information Processing Systems

Volumn , Issue , 1995, Pages 563-569

  Schraudolph, Nicol N ^a   Sejnowski, Terrence J ^b  

^a Evotec BioSystems GmbH   (Germany)

^b SALK INSTITUTE FOR BIOLOGICAL STUDIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; LEARNING ALGORITHMS; NETWORK LAYERS;

BACKPROPAGATION LEARNING ALGORITHM; BP (BACK-PROPAGATION) NETWORK; ERROR SIGNAL; LEARNING RATES; MODEL SIGNALS; NETWORK STRUCTURES; PERFORMANCE; SINGLE VECTORS; STRUCTURAL INFORMATION; WEIGHT PARAMETERS;

TEMPERING;

EID: 85156239724     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. Amari, S.-I. (1995). Learning and statistical inference. In Arbib, M. A., editor, The Handbook of Brain Theory and Neurai Networks, pages 522-526. MIT Press, Cambridge.
2. Battiti, T (1992). First- and second-order methods for learning: Between steepest descent and Newton's method. Neural Computation, 4 (2):141-166.
3. Haykin, S. (1994). Neural Networks: A Comprehensive Foundation. Macmillan, New York.
4. Hinton, G. (1986). Learning distributed representations of concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society, pages 1-12, Amherst 1986. Lawrence Erlbaum, Hillsdale.
5. Jacobs, R. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295-307.
6. Krogh, A., Thorbergsson, G., and Hertz, J. A. (1990). A cost function for internal representations. In Touretzky, D. S., editor, Advances in Neural Information Processing Systems, volume 2, pages 733-740, Denver, CO, 1989. Morgan Kaufmann, San Mateo.
7. LeCun, Y. (1993). Efficient learning & second-order methods. Tutorial given at the NIPS Conference, Denver, CO.
8. LeCun, Y, Kanter, L, and Solla, S. A. (1991). Second order properties of error surfaces: Learning time and generalization. In Lippmann, R. P., Moody, J. E., and Touretzky, D. S., editors, Advances in Neural Information Processing Systems, volume 3, pages 918-924, Denver, CO, 1990. Morgan Kaufmann, San Mateo.
9. O'Reilly, R. C. (1996). Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8.
10. Plaut, D., Nowlan, S., and Hinton, G. (1986). Experiments on learning by back propagation. Technical Report CMU-CS-86-126, Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA.