Radial Basis Function network learning using localized generalization error bound

Information Sciences

Volumn 179, Issue 19, 2009, Pages 3199-3217

  Yeung, Daniel S ^a   Chan, Patrick P K ^a,b   Ng, Wing W Y ^a  

^a SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Learning objective function; Localized generalization error bound; Radial Basis Function network; Regularization; Sensitivity

Indexed keywords

GENERALIZATION CAPABILITY; GENERALIZATION ERROR BOUNDS; HIDDEN NEURONS; LEARNING OBJECTIVE FUNCTION; LOCALIZED GENERALIZATION ERROR BOUND; LOCALIZED GENERALIZATION ERROR MODELS; LOCALIZED GENERALIZATION ERRORS; OBJECTIVE FUNCTIONS; PATTERN CLASSIFICATION PROBLEMS; RADIAL BASIS FUNCTIONS; RBF NETWORK; REGULARIZATION; REGULARIZATION METHODS; SENSITIVITY; TESTING ACCURACY; TIKHONOV REGULARIZATION; TRAINING ERRORS; TRAINING SAMPLE; WEIGHT DECAY;

ATTITUDE CONTROL; ELECTRON MULTIPLIERS; OPTIMIZATION;

RADIAL BASIS FUNCTION NETWORKS;

EID: 67651001563     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2009.06.001     Document Type: Article

References (43)

1. Bishop C. Neural Networks for Pattern Recognition (1995), Clarendon Press, Oxford
2. M.M. Brizzotti, A.C.P.L.F. Carvalho, The influence of clustering techniques in the RBF networks generalization, in: Proceedings of the Seventh International Conference on Image Processing and Its Applications, vol. 1, 1999, pp. 87-92.
3. Broomhead D., and Lowe D. Multivariable functional interpolation and adaptive networks. Complex Systems 2 (1988) 321-355
4. Bruzzone L., and Prieto D.F. A technique for the selection of kernel-function parameters in RBF neural networks for classification of remote-sensing images. IEEE Transactions on Volume Geoscience and Remote Sensing 37 2 (1999) 1179-1184
5. Caruana R., Lawrence S., and Giles C.L. Overfitting in neural networks: backpropagation, conjugate gradient, and early stopping. Advance in Neural Information Processing Systems 13 (2000) 402-408
6. P.P.K. Chan, W.W.Y. Ng, D.S. Yeung, Active learning using localized generalization error of candidate sample as criterion, in: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 4, 2005, pp. 3604-3609.
7. Chen Z., and Haykin S. On different facets of regularization theory. Neural Computation 14 12 (2002) 2791-2846
8. Cherkassky V., and Mulier F. Learning From Data (1998), Wiley, New York
9. Cherkassky V., Shao X., Mulier F.M., and Vapnik V.N. Model complexity control for regression using VC generalization bounds. IEEE Transactions Neural Network 10 5 (1999) 1075-1089
10. M.C.F. De Castro, F.C.C. De Castro, D.S. Arantes, RBF neural networks with centers assignment via Karhunen-Loeve transform, in: Proceedings of the International Joint Conference on Neural Networks, vol. 2, 1999, pp. 1265-1270.
11. Hastie T., Tibshirani R., and Friedman J. The Element of Statisitcal Learning (2001), Springer-Verlag, New York
12. Haykin S. Neural Networks a Comprehensive Foundation (1994), MacMillan, New York
13. Kiernan L., Mason J.D., and Warwick K. Robust initialisation of gaussian radial basis function networks using partitioned k-means clustering. Electronics Letters 32 7 (1996) 671-673
14. T. Kohonen, The self-organizing map, Proceedings of the IEEE 78 (9) (1990) 1464-1480.
15. M.A. Kon, L. Plaskota, Complexity of regularization RBF networks, in: Proceedings of the International Joint Conference on Neural Networks, vol. 1, 2001, pp. 342-346.
16. B. Krzyiak, T. Linder, Radial basis function networks and nonparametric classification: complexity regularization and the rates of convergence, in: Proceedings of the International Conference on Pattern Recognition, vol. 4, 1996, pp. 650-653.
17. Kubat M. Decision trees can initialize radial-basis function networks. IEEE Transactions on Neural Networks 9 5 (1998) 813-821
18. L. Ma, A. Wahab, C. Quek, A modified generalized RBF model with em-based learning algorithm for medical applications, in: Proceedings of the 19th IEEE Symposium on Computer-Based Medical Systems, 2006, pp. 291-296.
19. M.W. Mak, K.W. Cho, Genetic evolution of radial basis function centers for pattern classification, in: Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 1, 1998, pp. 669-673.
20. Meghabghab G., and Kandel A. Stochastic simulations of web search engines: RBF versus second-order regression models. Information Sciences 159 1 (2004) 1-28
21. Mendoza O., Melin P., and Licea G. A hybrid approach for image recognition combining type-2 fuzzy logic, modular neural networks and the Sugeno integral. Information Sciences 179 13 (2009) 2078-2101
22. Montazer G.A., Sabzevari R., and Ghorbani F. Three-phase strategy for the OSD learning method in RBF neural networks. Neurocomputing 72 7-9 (2009) 1797-1802
23. J.E. Moody, Note on generalization, regularization and architecture selection in nonlinear learning systems, in: Proceedings of the 1991 IEEE Workshop of Neural Networks for Signal Processing, 1991, pp. 1-10.
24. Moody J.E., and Darken C. Fast learning networks of locally-tuned processing units. Neural Computation 1 (1989) 289-303
25. Musavi M.T., Ahmed W., Chan K.H., Faris K.B., and Hummels D.M. On the training of radial basis function classifiers. Neural Network 5 (1992) 595-603
26. Ng W.W.Y., Yeung D.S., Firth M., Tsang E.C.C., and Wang X.Z. Feature selection using localized generalization error for supervised classification problems using RBFNN. Pattern Recognition 41 12 (2008) 3706-3719
27. Ng W.W.Y., Yeung D.S., Wang D., Tsang E.C.C., and Wang X.Z. Localized generalization error of Gaussian-based classifiers and visualization of decision boundaries. Soft Computating 11 4 (2007) 375-381
28. Y. Ng, Feature selection, L1 vs. L2 regularization, and rotational invariance, in: Proceedings of the Twenty-first International Conference on Machine Learning, vol. 64, 2004, pp. 78-85.
29. T. Poggio, F. Girosi, A theory of networks for approximation and learning, A.I. Memo No. 1140, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1989.
30. T. Poggio, F. Girosi, Networks for approximation and learning, Proceedings of the IEEE 78 (9) (1990) pp. 1481-1497.
31. Schwenker F., Kestler H.A., and Palm G. Three learning phases for radial-basis-function networks. Neural Networks 14 4-5 (2001) 439-458
32. Sohn I., and Ansari N. A novel algorithm to configure RBF networks. International Conference on Neural Networks 3 (1997) 1809-1814
33. Tikhonov N., and Arsenin V.A. Solutions of Ill-posed Problems (1977), Winston & Sons, Washington
34. M. Vogt, Combination of radial basis function neural networks with optimized learning vector quantization, in: Proceedings of the IEEE International Conference on Neural Networks, vol. 3, 1993, pp. 1841-1846.
35. Watanabe S. Algebraic analysis for non-identificable learning machines. Neural Computation 13 (2001) 899-933
36. West D., and Dellana S. Diversity of ability and cognitive style for group decision processes. Information Sciences 179 5 (2009) 542-558
37. Xue H., Chen S., and Zeng X. Classifier learning with a new locality regularization method. Pattern Recognition 41 5 (2008) 1496-1507
38. D.S. Yeung, W.W.Y. Ng, Feature selection using generalization error for supervised classification problem, in: Proceedings of the International Conference on Instrumentation, Control and Information Technology, 2005, pp. 41-46.
39. Yeung D.S., Ng W.W.Y., Wang D., Tsang E.C.C., and Wang X.Z. Localized generalization error model and its application to architecture selection for radial basis function neural network. IEEE Transactions on Neural Networks 18 5 (2007) 1294-1305
40. J.H. Yoo, I.K. Sethi, Design of radial basis function networks using decision trees, in: Proceedings of the IEEE International Conference on Neural Networks, vol. 3, 1995, pp. 1269-1272.
41. Young P.M., and Trevor H. L1-regularization path algorithm for generalized linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 69 4 (2007) 659-677
42. http://archive.ics.uci.edu/ml/.
43. http://ida.first.fraunhofer.de/homepages/ida/.