A Framework for Improved Training of Sigma-Pi Networks

IEEE Transactions on Neural Networks

Volumn 6, Issue 4, 1995, Pages 893-903

  Heywood, Malcolm ^a   Noakes, Peter ^a  

^a UNIVERSITY OF ESSEX   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BACKPROPAGATION; COMPUTER SIMULATION; ELECTRIC NETWORK TOPOLOGY; LEARNING SYSTEMS; REDUNDANCY; SET THEORY;

BACKPROPAGATION LEARNING RULE; DYNAMIC WEIGHT PRUNING ALGORITHM; OPTIMAL MULTILAYER PERCEPTRONS; SIGMA PI NETWORKS; TRAINING SET;

NEURAL NETWORKS;

EID: 0029343601     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.392251     Document Type: Article

References (31)

1. D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing, Explorations in the Macrostructure of Cognition., vol. 1. Cambridge, MA: MIT Press, 1986.
2. M. L. Minsky and S. Papert, Perceptrons. Cambridge, MA: MIT Press, 1969.
3. C. L. Giles and T. Maxwell, “Learning, invariance and generalization in high-order neural networks,” Appl. Optics, vol. 26, no. 23, pp. 4972-4978, Dec. 1987.
4. M. I. Heywood, Univ. Essex, Dep. Electronic Syst. Eng., internal progress rep. 1, 1992.
5. T. Maxwell, C. L. Giles, and Y. C. Lee, “Generalization in neural networks: The contiguity problem,” in Proc. 1st IEEE IJCNN, vol. 2, 1987, pp. 41-46.
6. T. Maxwell and C. L. Giles, “Transformation invariance using high-order correlations in neural net architectures,” in Proc. IEEE Int. Conf. Syst., Man Cybern., 1986, pp. 627-632.
7. L. Spirkovska and M. B. Reid, “Coarse-coded higher-order neural networks for PSRI object recognition,” IEEE Trans. Neural Networks, vol. 4, no. 2, pp. 276-283, Mar. 1993.
8. R. W. Duren, “Efficient second order neural network architecture for orientation invariant character recognition,” Ph.D. dissertation, Southern Methodist Univ., Dallas, TX, 1991.
9. M. I. Heywood, Univ. Essex, Dep. Electronic Syst. Eng., internal progress report 2, Sep. 1992.
10. C. H. Chang and J. Y. Chaung, “Backpropagation algorithm in higher order neural network,” in Proc. IEEE IJCNN, Baltimore, vol. 3, 1992, p. 551.
11. H. Yang and C. C. Guest, “Higher order neural networks with reduced numbers of interconnection weights,” in Proc. IEEE IJCNN, vol. 3, June 1991, pp. 281-286.
12. V. S. Stometta and B. A. Hubermann, “An improved three-layer backpropagation algorithm,” in Proc. IEEE IJCNN, vol. 2, 1987, pp. 637-643.
13. M. C. Monzer and P. Smolensky, “Skeletonisation: A technique for trimming the fat from a network via reliance assessment,” in Advances in Neural Network Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1989, pp. 107-115.
14. Y. Le Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in Neural Information Processing Systems, vol. 2. San Mateo, CA: Morgan Kaufmann, 1990, pp. 598-605.
15. B. Hassibi, D. G. Stork, and G. J. Wolff, “Optimal brain surgeon and general network pruning,” in Proc. IEEE ICNN, San Francisco, vol. 1, 1993, pp. 293-299.
16. S. J. Hanson and L. Y. Pratt, “Comparing basis for minimal network construction with back propagation,” in Advances in Neural Information Processing Systems, vol. 1. San Mateo, CA: Morgan Kaufmann, 1989, 177-185.
17. A. S. Weigand, D. E. Rumelhart, and B. A. Huberman, “Generalization by weight elimination with application to forecasting,” Advances in Neural Information Processing Systems, vol. 3. San Mateo, CA: Morgan Kaufmann, 1991, pp. 875-882.
18. H. Qin and Z. He, “Variable step BP algorithm which prunes away redundant connections dynamically,” in Proc. IEEE IJCNN, vol. 2, Nov. 1992, pp. 441-444.
19. S. Stalin and T. V. Sreenias, “Vectorised backpropagation and automatic pruning for MLP network optimization,” in Proc. IEEE ICNN, San Fransisco, vol. 3, Mar. 1993, 1427-1432.
20. M. I. and P. D. Noakes, “Simple addition to backpropagation learning for dynamic weight pruning, sparse network extraction and faster learning,” in Proc. IEEE ICNN, San Francisco, vol. 2, Mar. 1993, pp. 620-625.
21. “Directed product term selection in Sigma-Pi networks,” in Proc. IEEE ICNN, Orlando, 1994.
22. “Product term selection in a Sigma-Pi framework,” Univ. Essex, Dep. Electronic Syst. Eng., Report No. heywm2, Dec. 1993.
23. M.I. Heywood, “A Practical Framework for Training Sigma—Pi Networks with an Application in Rotational Invariant Object Recognition,” Ph.D. dissertation, Dept. Elec. Syst. Eng., Univ. Essex, Colchester, UK, 1994.
24. D. S. Yang, J. H. Garrett, and D. S. Shaw, “A comparative study of neural networks that can perform transformation invariant pattern recognition,” in Proc. IEEE IJCNN, Beijing, vol. 1, 1992, pp. 664-669.
25. R. J. Schalkoff, Digital Image Processing and Computer Vision. New York: Wiley, 1989.
26. M.-K. Hu, “Visual pattern recognition by moment invariants,” IRE Trans. Information Theory, vol. 8, no. 2, pp. 179-189, Feb. 1962.
27. P. Raveendran, S. Omatu, and W. W. Yue, “Performance of generated moments by neural network to recognize invariant image representation,” in Proc. IEEE IJCNN, Beijing, vol. 1, 1992, pp. 680-694.
28. M. R. Teague, “Image analysis via the general theory of moments,” J. Optical Soc. Amer., vol. 70, no. 8, pp. 920-930., Aug. 1980.
29. A. Khotanzad and J.-H. Lu, “Classification of invariant image representations using a neural network,” IEEE Trans. Acoustics, Speech Signal Process., vol. 38, no. 6, pp. 1028-1038, June 1990.
30. C.-H. Teh and R. T. Chin, “On image analysis by the methods of moments,” IEEE Trans. Pattern Anal. Machine Intell., vol. 10, no. 4, pp. 496-512, July 1988.
31. A. Estevez Pablo, Okabe Yoichi, “Training the piecewise linear higher order neural network through error backpropagation,” in Proc. IEEE IJCNN, Singapore, vol. 1, 1991, pp. 711-716.