A fault-tolerant regularizer for RBF networks

IEEE Transactions on Neural Networks

Volumn 19, Issue 3, 2008, Pages 493-507

  Leung, Chi Sing ^a   Sum, John Pui Fai ^b  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b NATIONAL CHUNG HSING UNIVERSITY   (Taiwan)

Author keywords

Kullback Leibler divergence; Node open fault; Regularization

Indexed keywords

COMPUTATIONAL COMPLEXITY; FAULT TOLERANT COMPUTER SYSTEMS; GAUSSIAN DISTRIBUTION; LEARNING ALGORITHMS;

KULLBACK-LEIBLER DIVERGENCE; NODE OPEN FAULT; OBJECTIVE FUNCTION;

RADIAL BASIS FUNCTION NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; HUMAN; LEARNING; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; HUMANS; LEARNING; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER);

EID: 40949101789     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.912320     Document Type: Article

References (37)

1. G. R. Bolt, "Fault models for artificial neural networks," in Proc. Int. Joint Conf. Neural Netw., Singapore, 1991, pp. 1373-1378.
2. G. R. Bolt, J. Austin, and G. Morgan, "Fault tolerant multi-layer perceptron networks," Tech. Rep. YCS-92-180, 1992.
3. A. F. Murray and P. J. Edwards, "Enhanced MLP performance annd fault tolerance resulting from synaptic weight noise during training," IEEE Trans. Neural Netw., vol. 5, no. 5, pp. 792-802, Sep. 1994.
4. C. T. Chiu, K. Mehrotra, C. K. Mohan, and S. Ranka, "Modifying training algorithms for improved fault tolerance," in Proc. Int. Conf. Neural Netw., 1994, vol. 4, pp. 333-338.
5. D. S. Phatak and I. Koren, "Complete and partial fault tolerance of feedforward neural nets," IEEE Trans. Neural Netw., vol. 6, no. 2, pp. 446-456, Mar. 1995.
6. Z. H. Zhou, S. F. Chen, and Z. Q. Chen, "Improving tolerance of neural networks against multi-node open fault," in Proc. Int. Joint Conf. Neural Netw., 2001, vol. 3, pp. 1687-1692.
7. Z. H. Zhou and S. F. Chen, "Evolving fault-tolerant neural networks," Neural Comput. Appl., vol. 11, pp. 156-160, 2003.
8. M. D. Emmerson and R. I. Damper, "Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application," IEEE Trans. Neural Netw., vol. 4, no. 5, pp. 788-793, Sep. 1993.
9. C. H. Sequin and R. D. Clay, "Fault tolerance in feedforward artificial neural networks," Neural Netw., vol. 4, pp. 111-141, 1990.
10. S. Cavalieri and O. Mirabella, "A novel learning algorithm which improves the partial fault tolerance of multilayer neural networks," Neural Netw., vol. 12, pp. 91-106, 1999.
11. C. Neti, M. H. Schneider, and E. D. Young, "Maximally fault tolerance neural networks," IEEE Trans. Neural Netw., vol. 3, no. 1, pp. 14-23, Jan. 1992.
12. D. Deodhare D, M. Vidyasagar, and S. S. Keerth, "Sythesis of fault-tolerant feedforward neural networks using minimax optimization," IEEE Trans. Neural Netw., vol. 9, no. 5, pp. 891-900, Sep. 1998.
13. D. Simon and H. El-Sherief, "Fault-tolerance training for optimal interpolative nets," IEEE Trans. Neural Netw., vol. 6, no. 6, pp. 1531-1535, Nov. 1995.
14. D. S. Phatak and E. Tchernev, "Synthesis of fault tolerant neural networks," Proc. Int. Joint Conf. Neural Netw., pp. 1475-1480, 2002.
15. O. J. L. Mark, "Regularization in the selection of radial basis function centers," Neural Comput., vol. 7, pp. 606-623, 1995.
16. X. Hong, "A fast identification algorithm for box-cox transformation based radial basis function neural network," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 1064-1069, Jul. 2006.
17. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006.
18. C. Constantinopoulos and A. Likas, "An incremental training method for the probabilistic RBF network," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 966-974, Jul. 2006.
19. B. S. Lin, F. C. Chong, and F. Lai, "Higher-order-statistics-based radial basis function networks for signal enhancement," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 823-832, May 2007.
20. J. E. Moddy, "Note on generalization, regularization, and architecture selection in nonlinear learning systems," in Proc. 1st IEEE-SP Workshop Neural Netw. Signal Process., 1991, pp. 1-11.
21. D. S. Phatak, "Relationship between fault tolerance, generalization and the Vapnik-Cervonenkis (VC) dimension of feedforward ANNs," Proc. Int. Joint Conf. Neural Netw., vol. 1, pp. 705-709, 1999.
22. N. Kamiura, Y. Taniguchi, T. Isokawa, and N. Matsui, "An improvement in weight-fault tolerance of feedforward neural networks," in Proc. Appl. Telecommun. Symp., 2001, pp. 359-364.
23. C. S. Leung, A. C. Tsoi, and L. W. Chan, "Two regularizers for recursive least square algorithms in feedforward multilayered neural networks," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1314-1332, Nov. 2001.
24. C. M. Bishop, "Training with noise is equivalent to Tikhonov regularization," Neural Comput., vol. 7, no. 1, pp. 108-116, 1995.
25. Y. Xu, K. W. Wong, and C. S. Leung, "Generalized RLS approach to the training of neural networks," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 19-34, Jan. 2005.
26. S. I. Amari, N. Murata, K. R. Muller, M. Finke, and H. H. Yang, "Asymptotic statistical theory of overtraining and cross-validation," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 996-985, Sep. 1997.
27. S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, "Sparse modelling using orthogonal forward regression with press statistic and regularization," IEEE Trans. Syst. Man. Cybern. B, Cybern., vol. 34, no. 2, pp. 898-911, Apr. 2004.
28. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
29. J. Moody, "A smoothing regularizer for feedforward and recurrent neural networks," Neural Comput., vol. 8, pp. 461-489, 1996.
30. D. J. C. Mackay, "A practical Bayesian framework for backprop networks," Neural Comput., vol. 4, pp. 448-472, 1992.
31. C. S. Leung, G. H. Young, J. Sum, and W. K. Kan, "On the regularization of forgetting recursive least square," IEEE Trans. Neural Netw., vol. 10, no. 6, pp. 1482-1486, Nov. 1999.
32. S. Chen, "Local regularization assisted orthogonal least squares regression," Neurocomputing, pp. 559-585, 2006.
33. M. E. Tipping, "The relevance vector machine," in Advances in Neural Information Processing Systems 12. Cambridge, MA: MIT Press, 2000, pp. 652-658.
34. V. Cherkassky and Y. Ma, "Multiple model regression estimation," IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 785-798, Jul. 2005.
35. V. Vapnik, S. Golowich, and A. Smola, "Support vector method for function approximation, regression estimation, and signal processing," in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997, pp. 281-287.
36. S. Singh, "Noise impact on time-series forecasting using an intelligent pattern matching technique," Pattern Recognit., vol. 32, pp. 1389-1398, 1999.
37. E. W. M. Lee, C. P. Lim, R. K. K. Yuen, and S. M. Lo, "A hybrid neural network model for noisy data regression," IEEE Trans. Syst. Man Cybern. B., Cybern., vol. 34, no. 2, pp. 951-960, Apr. 2004.