Feedforward sigmoidal networks - Equicontinuity and fault-tolerance properties

IEEE Transactions on Neural Networks

Volumn 15, Issue 6, 2004, Pages 1350-1366

  Chandra, Pravin ^a   Singh, Yogesh ^a  

^a GURU GOBIND SINGH INDRAPRASTHA UNIVERSITY   (India)

Author keywords

Equicontinuity; Fault tolerance; Feedforward artificial neural networks (FFANNs); Function sets; Sigmoidal networks

Indexed keywords

APPROXIMATION THEORY; ERROR ANALYSIS; MATRIX ALGEBRA; THEOREM PROVING; VECTORS;

EQUICONTINUITY; FAULT TOLERANCE PROPERTIES; FEEDFORWARD ARTIFICIAL NEURAL NETWORKS; FUNCTION SETS; SIGMOIDAL NETWORKS;

FEEDFORWARD NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; EVALUATION; FEEDBACK SYSTEM; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; FEEDBACK; LOGISTIC MODELS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 9244245241     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.831198     Document Type: Article

References (47)

1. N. K. Bose and P. Liang, Neural Network Fundamentals with Graphs, Algorithms, and Applications. New Delhi, India: Tata McGraw-Hill, 1998.
2. S. Haykin, Neural Networks: A Comprehensive Foundation. Englewood Cliffs, NJ: Prentice-Hall, 1999.
3. F. M. Ham and I. Kostanic, Principles of Neurocomputing for Science and Engineering, Singapore: McGraw-Hill, 2001.
4. B. Widrow and M. A. Lehr, "30 years of adaptive neural networks: Perceptron, madaline, and backpropagation," Proc. IEEE, vol. 78, pp. 1415-1442, 1990.
5. A. R. Gallant and H. White, "There exists a neural network that does not make avoidable mistakes," in Proc. Second Int. Joint Conf. Neural Networks, vol. I, 1988, pp. 593-606.
6. G. Cybenko, "Approximation by superposition of a sigmoidal function," Math. Contr., Signal, and Syst., vol. 5, pp. 233-243, 1989.
7. K. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Networks, vol. 2, pp. 183-192, 1989.
8. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, pp. 359-366, 1989.
9. K. Hornik, "Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks," Neural Networks, vol. 3, pp. 551-560, 1990.
10. M. Stinchcombe and H. White, "Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights," in Proc. IJCNN, vol. III, 1990, pp. 7-16.
11. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Networks, vol. 4, pp. 251-257, 1991.
12. A. R. Barron, "Universal approximation bounds for superposition of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
13. n ) by multilayer feedforward networks and related problems," IEEE Trans. Neural Networks, vol. 6, pp. 25-30, 1995.
14. M. B. Stinchcombe, "Neural network approximation of continuous functionals and continuous functions on compactifications," Neural Networks, vol. 12, pp. 467-477, 1999.
15. J.I. Castro, C. J. Mantas, and J. M. Bentez, "Neural networks with a continuous squashing function in the output are universal approximators," Neural Networks, vol. 13, pp. 561-563, 2000.
16. W. Cheney and W. Light, A Course in Approximation Theory. Pacific Grove, CA: Brooks/Cole, 2000.
17. B. DasGupta and G. Schnitger, "The power of approximating: A comparision of activation functions," in Advances in Neural Information Processing Systems, C. L. Giles, S. J. Hanson, and J. D. Cowan, Eds. San Mateo, CA: Morgan Kaufmann, 1993, vol. 5, pp. 615-622.
18. K. Hornik, M. Stinchcombe, H. White, and P. Auer, "Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives,", Tech. Rep. NC-TR-95-004, 1995. Neuro-COLT.
19. H. Mhasker, "Neural networks for optimal approximation of smooth and analytic functions," Neural Comput., vol. 8, pp. 164-177, 1996.
20. M. Schmitt. (2000) Lower bounds on the complexity of approximating continuous functions by sigmoidal neural networks. [Online]. Available: http://www.neurocolt.com
21. T. Poggio and E Girosi, "Networks and the Best Approximation Property," Artificial Intelligence Laboratory, Mass. Inst. Technol., MA, Tech. Rep. AI Memo no. 1164, 1989.
22. P. Chandra and Y. Singh, Bounded weight sigmoidal FFANN's have the best approximation property, in IEEE Trans. Neural Networks, 2003. submitted.
23. P. Chandra, Closure property of function sets represented by feedforward sigmoidal networks, in Int. J. Neural Systems of Neural Systems, 2003. submitted.
24. G. Bolt, "Fault Tolerant Multi-Layer Perceptron Networks," Advanced Computer Architecture Group, Dept. Computer Sci., Univ. York, U.K., Tech. Rep. YCS 180, 1992.
25. Y. Singh and P. Chandra, "A class +l sigmoidal activation functions for FFANNs," J. Economic Dyn. Contr., vol. 28, pp. 183-187, 2003.
26. J. Jost, Postmodern Analysis. Berlin, Germany: Springer-Verlag, 1998.
27. G. F. Simmons, Topology and Modern Analysis. Tokyo, Japan: McGraw-Hill/Kogakusha, 1964.
28. W. Rudin, Principles of Mathematical Analysis, 2nd ed. Tokyo, Japan: McGraw-Hill/Kogakusha, 1964.
29. W. Rudin, Real and Complex Analysis. New Delhi, India: Tata-McGraw-Hill, 1966.
30. M. J. Carter, "The illusion of fault tolerance of neural nets for pattern recognition and signal processing," in Proc. Tech. Session on Fault Integrated System. Durham, 1988.
31. T. R. Damarla and P. K. Bhagat, "Fault tolerance of neural networks," in IEEE Proc. 1989 SOUTHEASTCON, 1989, pp. 328-331.
32. D. S. Phatak and I. Koren, "Complete and partial fault tolerance of feed-forward neural nets," IEEE Trans. Neural Networks, vol. 6, pp. 446-456, 1995.
33. B. E. Segee and M. J. Carter, "Comparative fault-tolerance of parallel distributed processing networks," IEEE Trans. Comput., vol. 43, pp. 1323-1329, 1994.
34. J.I. Minnix, "Fault tolerance of backpropagation neural networks trained with noisy data," in IJCNN'91, vol. 1, 1991, pp. 703-708.
35. K. Matasuoka, "Noise injection into inputs in backpropagation learning," IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 436-440, 1992.
36. L. Holmstrom and P. Koistinen, "Using additive noise in backpropagation training," IEEE Trans. Neural Networks, vol. 3, pp. 24-28, 1992.
37. A. F. Murray and P. J. Edwards, "Synaptic weight noise during MLP training: Enhanced MLP performance and fault tolerance resulting from synaptic noise during training," IEEE Trans. Neural Networks, vol. 5, pp. 792-802, 1994.
38. P. Chandra and Y. Singh, "Fault tolerance of feedforward artificial neural networks - A framework of study," in IJCNN'03, 2003, pp. 489-494.
39. P. Chandra, "Regularization and feedforward artificial neural network training with noise," in IJCNN'03, 2003, pp. 2366-2371.
40. G. Z. An, "The effect of adding noise during backpropagation training on a generalization performance," Neural Comput., vol. 8, pp. 643-674, 1996.
41. K. Jim, C. L. Giles, and B. G. Home, "An analysis of noise in recurrent neural networks: Convergence and generalization," IEEE Trans. Neural Networks, vol. 7, pp. 1424-1439, 1996.
42. C. H. Sequin and R. D. Clay, "Fault tolerance in artificial neural networks," in IJCNN'90, vol. 1, 1990, pp. 703-708.
43. Z.-H. Zhou and S.-F. Chen, Evolving fault-tolerant neural networks, in Neural Computing and Applications, 2003. in press.
44. N. C. Hammadi and H. Ito, "A learning algorithm for fault tolerant feedforward networks," IEICE Trans. Inform. Syst., vol. E80-D, pp. 21-27, 1997.
45. C.-T. Chin, K. Mehrotra, C. K. Mohan, and S. Ranka, "Training techniques to obtain fault tolerant neural networks," in Proc. Int. Symp. Fault Tolerant Computing, Austin, TX, 1994, pp. 360-369.
46. N. C. Hammadi and H. Ito, "On the activation function and fault tolerance in feedforward artificial neural networks," IEICE Trans. Inform. Syst., vol. E81-D, pp. 66-72, 1998.
47. M. D. Emmerson and R. I. Damper, "Determining and improving the fault tolerance of multilayer perceptrons in a pattern recognition application" IEEE Trans. Neural Networks, vol. 4, pp. 788-793, 1993.