Feedforward neural network design with tridiagonal symmetry constraints

IEEE Transactions on Signal Processing

Volumn 48, Issue 5, 2000, Pages 1446-1454

  Dumitras, Adriana ^a   Kossentini, Faouzi ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Feedforward neural network; Symmetry constraints; Tridiagonal

Indexed keywords

EID: 0003194588     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/78.839989     Document Type: Article

References (69)

1. D. E. Rumelhart, J. L. McClelland, and FDP Research Group, Parallel Distributed Processing. Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1987, vol. 1 and 2.
2. C. T. Harston, A. Maren, and R. M. Pap, Handbook of Neural Computing Applications. New York: Academic, 1990.
3. R. Hecht-Nielsen, Neurocomputing. Reading, MA: Addison-Wesley, 1991.
4. E. B. Baum and D. Haussler, What size net gives valid generalization?, in Advances in Neural Information Processing Systems, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, vol. 1.
5. J. M. Zurada, Introduction to Artificial Neural Systems. San Francisco, CA: West, 1992.
6. R. P. Lippmann, An introduction to computing with neural nets, IEEE Acoust., Speech, Signal Processing Mag., vol. 7, pp. 4-22, Apr. 1987.
7. K. Hagiwara and S. Usui, Numerical experiments on the information criteria for layered feedforward neural nets, in Proc. Int. Conf. Artificial Neural Networks, vol. 1, Sorrento, Italy, 1994, pp. 493-496.
8. H. Akaike, Fitting autoregressive models for prediction, Ann. Inst. Stat. Math., vol. 21, pp. 243-247, 1989.
9. B. H. Jansen, Time series analysis by means of linear modeling, in Digital Biosignal Processing, R. Weitkunat, Ed. New York: Elsevier, 1991, pp. 157-179.
10. M. Cottrell, B. Girard, Y Girard, M. Mangeas, and C. Muller, SSM: A statistical stepwise method for weight elimination, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 601-604.
11. J. Larsen and L. K. Hansen, Generalization performance of regularized neural network models, in Proc. Int. Conf. Artificial Neural Networks, J. Vlontzos, J. N. Hwang, and E. Wilson, Eds. Ermioni, Greece, 1994.
12. N. Murata, S. Yoshizawa, and S. I. Amari, Network information criterion-determining the number of hidden units for an artificial network model, Dept. Math. Eng. Inform. Phys., Faculty Eng., Univ. Tokyo, Preprint, 1992.
13. H. Gish, Maximum likelihood training of neural networks, in Artificial Intelligence Frontiers in Statistics, D. J. Hand, Ed. London, U.K.: Chapman & Hall, 1993, pp. 241-255.
14. M. Lehtokangas, J. Saarinen, and P. Huuhta, Neural network modeling and prediction of multivariate time series using predictive MDL principle, in Proc. Int. Conf. Artificial Neural Networks, Amsterdam, The Netherlands, 1993.
15. _, Neural network prediction of nonlinear time series using predictive MDL principle, in Proc. IEEE Workshop Nonlinear Digital Signal Process., Tampere, Finland, 1993, pp. 7.2.2.1-7.2.2.6.
16. Y. H. Hu, Configuration of feedforward multilayer perceptron neural networks, Dept. Elect. Comput. Eng., Univ. Wisconsin-Madison, Preprint, 1993.
17. T. M. Nahban and A. Y. Zomaya, Toward generating neural network structures for function approximation, Neural Networks, vol. 7, no. 1, pp. 89-99, 1994.
18. T.-Y Kwok and D.-Y Yeung, Constructive feedforward neural networks for regression problems: A survey, Hong Kong Univ. Sci. Technol., Tech. Rep. HKUST-CS 95-43, 1995.
19. E. Fiesler, Comparative bibliography of ontogenic neural networks, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 793-796.
20. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
21. E. Alpaydin, Grow and learn: An incremental method for category learning, in Proc. Int. Neural Network Conf., Paris, France, 1990.
22. F. M. Frattale Mascioli, G. Martinelli, and G. Lazzaro, Comparison of constructive algorithms for neural networks, in Proc. Int Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 731-734.
23. S. E. Fahlman and C. Lebiere, The cascade-correlation learning architecture, School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-90-100, 1991.
24. G. P. Drago, Cascade correlation convergence theorem, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 573-576.
25. M. Frean, The Upstart algorithm: A method for constructing and training feedforward neural networks, Neural Comput., vol. 2, pp. 198-209, 1990.
26. I. K. Sethi, Entropy nets: From decision trees to neural networks, Proc. IEEE, vol. 78, pp. 1605-1613, 1990.
27. R. P. Brent, Fast training algorithms for multilayer neural nets, IEEE Trans. Neural Networks, vol. 2, pp. 346-353, May 1991.
28. R. A. Jacobs, Task decomposition through competition in a modular connectionist architecture, Ph.D. dissertation, Univ. Mass., Amherst, 1990.
29. J. M. Lange, H.-M. Voigt, and D. Wolf, Task decomposition and correlations in growing artificial neural networks, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 735-738.
30. M. T. Tham, Z. Wang, C. Di Massimo, and J. Morris, A procedure for determining the topology of multilayer feedforward neural networks, Neural Networks, vol. 7, no. 2, pp. 291-300, 1994.
31. R. Reed, Pruning algorithms-A survey, IEEE Trans. Neural Networks, vol. 4, pp. 740-747, Sept. 1996.
32. J. Gorodkin, L. K. Hansen, A. Krogh, C. Svarer, and O. Winther, A quantitative study of pruning by optimal brain damage, Electron. Inst., Tech. Univ. Denmark, Lyngby, Preprint, 1993.
33. J. R. Movellan and J. K. Kruschke, Benefits of the gain: Speeded learning and minimal hidden layers in back propagation networks, IEEE Trans. Syst., Man, Cybern., vol. 21, pp. 273-280, Jan. 1991.
34. J. S. Denker, Y Le Cun, and S. A. Solla, Optimal brain damage, in Advances in Neural Information Processing Systems, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990.
35. L. K. Hansen, C. E. Rasmussen, C. Svarer, and J. Larsen, Adaptive regularization, in Proc. IEEE Workshop Neural Networks Signal Process., Ermioni, Greece, 1994.
36. T. Ciblas, F. F. Souli, P. Gallinari, and S. Raudys, Variable selection with optimal cell damage, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 327-330.
37. B. Hassibi and D. G. Stork et al, Second order derivatives for network pruning: Optimal brain surgeon,in Proc. NIPS, S. J. Hansone/aJ.,Eds.. San Mateo, CA, 1993.
38. E. D. Karnin, A simple procedure for pruning back-propagation trained neural networks, IEEE Trans. Neural Networks, vol. 1, pp. 239-242, Apr. 1990.
39. L. K. Hansen and C. E. Rasmussen, Pruning from adaptive regularization, Electron. Inst., Tech. Univ. Denmark, Lyngby, Preprint, 1993.
40. P. M. Williams, Bayesian regularization and pruning using a Laplace prior, Neural Comput., vol. 7, pp. 117-143, 1995.
41. S. Haykin, Neural Networks-A Comprehensive Foundation. New York: MacMillan, 1994.
42. M. W. Pedersen and L. K. Hansen, Controlled growth of cascade correlation nets, in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, 1994, pp. 780-797.
43. Y. Chauvin, A back-propagation algorithm with optimal use of hidden units, in Advances in Neural Information Processing Systems, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, vol. 1.
44. D. Psaltis, C. Ji, and R. R. Snapp, Generalizing smoothness constraints from discrete samples, Neural Comput., vol. 2, no. 2, pp. 188-197, 1990.
45. S. Yasui, A. Malinowski, and J. M. Zurada, Convergence suppression and divergence facilitation: New approach to prune hidden layer and weights of feedforward neural networks, in P roc. IEEE Int. Symp. Circuits Syst., Seattle, WA, 1995, pp. 121-124.
46. M. C. Mozer and P. Smolensky, Skeletonization: A technique for trimming the fat from a network via relevance assessment, in Advances in Neural Information Processing Systems, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989.
47. Q. Xue, Y. H. Hu, and W. J. Tompkins, Structural simplification of a feedforward, multilayer perceptron artificial neural network, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Toronto, Ont., Canada, 1991.
48. S. Y Kung and Y H. Hu, A Frobenius approximation reduction method (FARM) for determining optimal number of hidden units, in Proc. IEEE Conf. Neural Networks, vol. 2, Seattle, WA, 1991, pp. 163-168.
49. K. Yamashita, Y Hirose, and S. Hijiya, Backpropagation algorithm which varies the number of hidden units, Neural Networks, no. 4, pp. 61-66, 1991.
50. A. G. Ivanhnenko, The group method of data handling-A rival of stochastic approximation, Sov. Automat. Contr, vol. 1, pp. 43-55, 1968.
51. R. L. Barron, Learning networks improve computer-aided prediction and control, Comput. Des., vol. 1, pp. 65-70, Aug. 1975.
52. B. Fritzke, Growing cell structures, a self-organizing network for unsupervised and supervised learning, Neural Networks, vol. 7, no. 9, pp. 1441-1460, Aug. 1994.
53. A. Dumitras., A. T. Murgan, and V. Läzärescu, A growing-decreasing method for designing neural filters, in Proc. IEEE Workshop Nonlinear Signal Image Process., vol. 2, Neos Marmaras, Greece, 1995, pp. 579-582.
54. D. W. Thomas, Y Q. Chen, and M. S. Nixon, Generating-shrinking algorithm for learning arbitrary classification, Neural Networks, vol. 7, no. 9, pp. 1477-1489, Aug. 1994.
55. E. B. Bartlett, Dynamic node architecture learning: An information theoretic approach, Neural Networks, vol. 7, no. 1, pp. 129-140, 1994.
56. M. Hintz-Madsen, L. K. Hansen, and J. Larsen et al., Design and evaluation of neural classifiers-Application to skin lesion classification, in Proc. IEEE Workshop Neural Networks Signal Process., 1995, pp. 484-493.
57. J. Hawkins and R. Allen, The Oxford Encyclopedic English Dictionary. Oxford, U.K.: Clarendon, 1994.
58. N.S.Cardell, W. H. Joerding, and Y Li, Symmetry constraints for feedforward network models of gradient systems, IEEE Trans. Neural Networks, vol. 6, pp. 1249-1254, Sept. 1995.
59. J. Shawe-Taylor, Symmetries and discriminability in feedforward neural networks, IEEE Trans. Neural Networks, vol. 4, pp. 816-826, Sept. 1993.
60. R. Yang, L. Yin, M. Gabbouj, J. Astola, and Y Neuvo, Optimal weighted median filtering under symmetry constraints, IEEE Trans. Signal Processing, vol. 43, pp. 591-603, Mar. 1995.
61. V. K. Madisetti, VLSI Digital Signal Processors-An Introduction to Rapid Prototyping and Design Synthesis. New York: IEEE , 1995.
62. R. J. Higgins, Digital Signal Processing in VLSI. Englewood Cliffs, NJ: Prentice-Hall, 1990.
63. P. Lapsley, J. Bier, A. Shoham, and E. A. Lee, DSP Processor Fundamentals. Piscataway, NJ: IEEE, 1997.
64. K. Parhi, VLSI Digital Signal Processing Systems: Design and Implementation. New York: Wiley, 1998.
65. H. H. Thodberg, A review of Bayesian neural networks with an application to infrared spectroscopy, IEEE Trans. Neural Networks, vol. 7, pp. 56-72, Jan. 1996.
66. K. Diamantaras and S.-Y Kung, Multilayer neural networks for reduced-rank approximation, IEEE Trans. Neural Networks, vol. 5, pp. 684-697, Sept. 1994.
67. A. Dumitras. and V. Läzärescu, MOBD: A neural network algorithm with symmetry constraints, in Proc. Third Int. Conf. Signal Process., vol. 2, Beijing, China, 1996, pp. 1429-1432.
68. A. A. Ghorbani and V. C. Bhavsar, Incremental communication for multilayer neural networks, IEEE Trans. Neural Networks, vol. 6, pp. 1375-1385, Nov. 1995.
69. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing. New York: Wiley, 1993.