Convergence suppression and divergence facilitation: Minimum and joint use of hidden units by multiple outputs

Neural Networks

Volumn 10, Issue 2, 1997, Pages 353-367

  Yasui, Syozo ^a  

^a KYUSHU INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

auto encoder; backpropagation; classifier; hidden unit; modularization; multiple outputs; pruning; structural compatibility

Indexed keywords

ALGORITHMS; BACKPROPAGATION; LEARNING SYSTEMS; OPTIMIZATION;

AUTO ENCODER; CLASSIFIER; HIDDEN UNITS; MODULARIZATION; MULTIPLE OUTPUTS; PRUNING; STRUCTURAL COMPATIBILITY;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; PRIORITY JOURNAL;

EID: 0031105479     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00055-X     Document Type: Article

References (20)

1. Baker, E. B., Mlodzik, M., & Rubin, G. M. (1990). Spacing differentiation in the developing Drophila eye: A fibrinogen-related lateral inhibitor encoded by scabrous. Science, 250, 1370-1376.
2. Doe, C. Q., & Goodman, C. S. (1985). Early events in insect neurogenesis. II: The role of cell interactions and cell lineage in the determination of neuronal precursor cells. Developmental Biology, 111, 206-219.
3. Hirose, Y., Yamashita, K., & Hijiya, S. (1991). Back propagation algorithm which varies the number of hidden units. Neural Networks, 4, 61-66.
4. Ishikawa, M. (1989). A structural learning algorithm with forgetting of link weights. Proceedings International Joint Conference on Neural Networks (Vol. 2, p. 626). Washington, DC.
5. Ishikawa, M., & Uchida, H. (1992). A structural learning of neural networks based on an entropy criterion. Proceedings IEEE & INNS International Joint Conference on Neural Networks (IJCNN), (Vol. 2, pp. 375-380). Beijing.
6. Kobayashi, H., Takahashi, M., Yagi, T., & Shimmi, T. (1993). Image processing regularization filters on layered architecture. Neural Networks, 6, 327-350.
7. Krogh, A., & Hertz, J. A. (1992). A simple weight decay can improve generalization. In E. Moody, S. J. Hanson, & R. P. Lippmann (Eds.), Advances in neural information processing systems, (Vol. 4, pp. 951-957). Morgan Kaufman.
8. Kruschke, J. K. (1988). Creating local and distributed bottlenecks in hidden layers of back-propagation networks. In D. Touretzky, G. Hinton, & T. Sejnowski (Eds.), Proceedings 1988 Connectionist Models Summer School (pp. 120-126).
9. LeCun, Y., Denker, J. S., & Solla, S. A. (1990). Optimal brain damage. In D. S. Touretzky (Ed.), Advances in neural information processing systems (Vol. 2, pp. 598-605). Morgan Kaufman.
10. Malinowski, A., Miller, D. A., & Zurada, J. M. (1995). Reconciling training and weight suppression: New guidelines for pruning-emcient training. Proceedings World Congress on Neural Networks (pp. 724-728). Washington, DC.
11. Mozar, M., & Smolensky, P. (1989). Skeletonization: A technique for trimming the fat from a network via relevance assessment. In D. S. Touretzky (Ed.), Advances in neural information processing systems (Vol. 1, pp. 107-115). Morgan Kaufman.
12. Plaut, D. C., Nowlan, S. J., & Hinton, G. E. (1996). Experiments on learning by back propagation (Carnegie-Mellon University CMU-CS-86-126).
13. Ramachandran, S., & Pratt, L. Y. (1992). Information measure based skeletonization. In J. E. Moody, S. T. Hanson, & R. P. Lippman (Eds.), Advances in neural information processing systems (Vol. 4, pp. 1080-1087). Morgan Kaufman.
14. Reed, R. (1993). Pruning algorithms - A survey. IEEE Transactions on Neural Networks, 4, 740-747.
15. Rumelhart, D., Hinton, E., & Williums, R. J. (1986). Learning representations by back-propagation errors. Nature, 323, 533-536.
16. Weigend, D. E., Rumelhart, D. E., & Huberman, B. A. (1990). Back-propagation, weight-elimination and time series prediction. In D. Touretzky, J. Elman, T. Sejnowski, & G. Hinton (Eds.), Proceedings 1990 Connectionist Models Summer School (pp. 105-116).
17. Yasui, S. (1992). A new method to remove redundant connections in backpropagation neural networks: Introduction of 'parametric lateral inhibition fields'. Proceedings IEEE & INNS International Joint Conference on Neural Networks (IJCNN) (Vol. 2, pp. 360-367). Beijing.
18. Yasui, S. (1994). Convergence suppression and divergence facilitation for pruning multi-output backpropagation networks. Proceedings 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing (pp. 137-139). Iiuzuka.
19. Yasui, S., Malinowski, A., & Zurada, J. M. (1995). Convergence suppression and divergence facilitation: New approach to prune hidden layer and weights of feedforward neural networks. Proceedings IEEE International Symposium on Circuits and Systems, ISCAS'95 (Vol. 1, pp. 121-124). Seattle.
20. Yasui, S., Furukawa, T., Yamada, M., & Saito, T. (1996). Plasticity of center-surround opponent receptive fields in real and artificial neural systems of vision. In D. S. Touretzky, M. C. Mozer, & M. E. Haselno (Eds.), Advances in neural information processing systems (Vol. 8, pp. 159-165). Morgan Kaufman.