Use of periodic and monotonic activation functions in multilayer feedforward neural networks trained by extended Kalman filter algorithm

IEE Proceedings: Vision, Image and Signal Processing

Volumn 149, Issue 4, 2002, Pages 217-224

  Wong, K W ^a   Leung, C S ^a   Chang, S J ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; COMPUTER ARCHITECTURE; COMPUTER SIMULATION; FUNCTIONS; KALMAN FILTERING; PIECEWISE LINEAR TECHNIQUES;

EXTENDED KALMAN FILTER ALGORITHM; HANDWRITTEN DIGIT RECOGNITION; LOGIC FUNCTION; MONOTONIC ACTIVATION FUNCTIONS; PARITY GENERATION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0036697709     PISSN: 1350245X     EISSN: None     Source Type: Journal    
DOI: 10.1049/ip-vis:20020515     Document Type: Article

References (29)

1. FUKUSHIMA, K., MIYAKE, S., and ITO, T.: Neocognitron: a neural-network model for a mechanism of visual pattern recognition' ANDERSON, J.A. ROSENFELD, E. (Eds.): (MIT Press, Cambridge, MA, 1988)
2. SEJNOWSKI, T.J., and ROSENBERG, C.R.: 'Parallel networks that learn to pronounce English text', Complex Syst., 1987, 1, pp. 145-168
3. LIPPMANN, R.P.: 'Review of neural networks for speech recognition', Neural Comput., 1989, 1, pp. 1-38
4. TESAURO, G., HE, Y., and AHMAD, S.: 'Asymptotic convergence of backpropagation', Neural Comput., 1989, 1, pp. 382-391
5. HUANG, G., and BABRI, H.A.: 'Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions', IEEE Trans. Neural Netw., 1998, 9, pp. 224-229
6. DAWSON, M.R.W., and SCHOPFLOCHER, D.P.: 'Modifying the generalized delta rule to train networks of non-monotonic processors for pattern classification', Connect. Sci., 1992, 4, (1), pp. 19-31
7. McCAUGHAN, D.B.: 'On the properties of periodic perceptrons'. Proceedings of ICNN, Houston, TX, USA, 1997, pp. 188-193
8. FILLIATRE, B., and RACCA, R.: 'Multi-threshold neurones perceptron', Process. Lett., 1996, (4), pp. 39-4
9. WONG, K.W., and LEUNG, C.S.: 'On-line successive synthesis of wavelet networks', Neural Process. Lett., 1998, (7), pp. 91-100
10. LIAO, X.F., WONG, K.W., LEUNG, C.S., and WU, Z.F.: 'Hopf bifurcation and chaos in a single delayed neuron equation with nonmonotonic activation function', Chaos Solitons Fractals, 2001, 12, (8), pp. 1535-1547
11. SINGHAL, S., and WU, L.: 'Training feedforward networks with the extended Kalman algorithm'. Proceedings of IEEE International Conference on Acoustics, Speech and signal processing, Glasgow, UK, 1989, pp. 1187-1190
12. IIGUNL, Y., SAKAI, H., and TOKUMARU, H.: 'Real-time learning algorithm for a multilayered neural network based on the extended Kalman filter', IEEE Trans. Signal Process., 1992, 40, pp. 959-967
13. RUSSELL, R.: 'Pruning algorithm - a survey', IEEE Trans. Neural. Netw., 1993, 4, pp. 740-747
14. YASUI, S.: 'Convergence suppression and divergence facilitation: minimum and joint use of hidden units by multiple output', Neural Netw., 1997, 10, pp. 353-367
15. LEUNG, C.S., WONG, K.W., SUM, P.F., and CHAN, L.W.: 'On-line training and pruning for recursive least square algorithms', Electron. Lett., 1996, 32, pp. 2152-2153
16. WONG, A.S.Y., WONG, K.W., and LEUNG, C.S.: 'Recursive least squares approach to combining principal and minor components analyses', Electron. Lett., 1998, 34, pp. 1074-1076
17. BURGES, C.J.C.: 'A tutorial on support vector machines for pattern recognition', Data Min. Knowl. Discov., 1998, 2, pp. 121-167
18. WALLIS, G.: 'Using spatio-temporal correlations to learn invariant object recognition', Neural Netw., 1996, 9, (9), pp. 1513-1519
19. LECUN, Y., BOSER, B., DENKER, J.S., HENDERSON, D., HOWARD, R.E., HUBBARD, W., and JACKEL, L.D.: 'Backpropagation applied to handwritten zip code recognition', Neural Comput., 1989, 1, pp. 541-551
20. CHANG, S.J., WONG, K.W., and LEUNG, C.S.: 'Combining DEKF Algorithm and Trace Rule for Fast On-Line Invariance Extraction and Recognition', Pattern Recognit. Lett., 2000, 21, (6-7), pp. 493-501
21. WEIGEND, A.S., HUBERMAN, B.A., and RUMELHART, D.E.: 'Predicting the future: A connectionist approach', Int. J. Neural Syst., 1990, 1, pp. 193-209
22. SVARER, C., HANSEN, L.K., and LARSEN, J.: 'On design and evaluation of tapped-delay neural network architectures', Proceedings of IEEE International Conference on Neural networks, San Francisco, 1993, pp. 46-51
23. SUM, J., LEUNG, C.S., YOUNG, G.H., and KAN, W.K.: 'On the Kalman filtering method in neural-network training and pruning', IEEE Trans. Neural Netw., 1999, 10, (1), pp. 161-166
24. ENGELBRECHT, A.P.: 'A new pruning heuristic based on variance analysis of sensitivity information', IEEE Trans. Neural Netw., 2001, 12, (6), pp. 1386-1399
25. LE CUN, Y., DENKER, J.S., and SOLLA, S.A.: 'Optimal brain damage', in TOURETZKEY, D.S. (Ed.): 'Advances in Neural Inf. Processing Systems 2 (NIPS 89)' (Morgan Kaufmann, 1990), pp. 598-605
26. HASSIBI, B., and STORK, D.G.: 'Second-order derivatives for network pruning: optimal brain surgeon', in HANSON, S.J. COWAN, J.D. GILES, C.L., (Eds.): 'Advances in neural information processing' (Morgan Kaufmann, 1993), pp. 164-171
27. PEARLMUTTER, B.A.: 'Fast exact multiplication by the Hessian', Neural Comput., 1994, 6, pp. 147-160
28. DECO, G., FINNOFF, W., and ZIMMERMANN, H.G.: 'Unsupervised mutual information criterion for elimination of overtraining in supervised multilayer networks', Neural Comput., 1995, 7, pp. 86-107
29. TONG, H., and LIM, K.S.: 'Threshold autoregression, limit cycles and cyclical data', J. Roy. Stat. Soc. B, 1980, 42, pp. 245