Learning algorithms for RBF functions and subspace based functions

Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques

Volumn , Issue , 2009, Pages 60-94

  Xu, Lei ^a,b  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b PEKING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956476076     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4018/978-1-60566-766-9.ch003     Document Type: Chapter

References (83)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 714-723.
2. Akaike, H. (1981). Likelihood of a model and information criteria. Journal of Econometrics, 16, 3-14.
3. Amari, S. I., Cichocki, A., & Yang, H. (1996), A new learning algorithm for blind separation of sources, In Touretzky, Mozer, & Hasselmo (Eds.), Advances in Neural Information Processing System 8, MIT Press, 757-763.
4. Bell, A., & Sejnowski, T. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129-1159.
5. Botros, S. M., & Atkeson, C. G. (1991), Generalization properties of radial basis function, In Lippmann, Moody, & Touretzky (eds), Advances in Neural Information Processing System 3, Morgan Kaufmann Pub., 707-713.
6. Bozdogan, H. (1987). Model Selection and Akaike's Information Criterion: The general theory and its analytical extension. Psychometrika, 52, 345-370.
7. Broomhead, D. S., & Lowe, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321-323.
8. Cavanaugh, J. (1997). Unifying the derivations for the Akaike and corrected Akaike information criteria. Statistics & Probability Letters, 33, 201-208.
9. Chang, P. R., & Yang, W. H. (1997). Environment-adaptation mobile radio propagation prediction using radial basis function neural networks. IEEE Transactions on Vehicular Technology, 46, 155-160.
10. Chen, S., Cowan, C. N., & Grant, P. M. (1991). Orthogonal least squares learning algorithm for Radial basis function networks. IEEE Transactions on Neural Networks, 2, 302-309.
11. Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995). The Helmholtz machine. Neural Computation, 7(5), 889-904.
12. Devroye, L. (1981). On the almost everywhere convergence of nonparametric regression function estimates. Annals of Statistics, 9, 1310-1319.
13. Devroye, L. (1987). A Course in Density Estimation. Boston: Birkhauser.
14. Er, M. J. (2002). Face recognition with radial basis function (RBF) neural networks. IEEE Transactions on Neural Networks, 13(3), 697-710.
15. Fang, S. C., Gao, D. Y., Shue, R. L., & Wu, S. Y. (2008). Canonical dual approach for solving 0-1 quadratic programming problems. Journal of Industrial and Management Optimization, 4(1), 125-142.
16. Ghahramani, Z., & Beal, M. J. (2000). Variational inference for Bayesian mixtures of factor analysers. In Solla, Leen, & Muller (eds), Advances in Neural Information Processing Systems 12, MIT Press, 449-455.
17. Girosi, F., & Poggio, T. (1990). Networks and the best approximation property. Biological Cybernetics, 63(3), 169-176.
18. Guerra, F. A., & Coelho, L. S. (2008). Multi-step ahead nonlinear identification of Lorenz's chaotic system using radial basis neural network with learning by clustering and particle swarm optimization. Chaos, Solitons, and Fractals, 35(5), 967-979.
19. Hartman, E. J., Keeler, J. D., & Kowalski, J. M. (1990). Layered neural networks with Gaussian hidden units as universal approximations. Neural Computation, 2, 210-215.
20. Heinen, T. (1996). Latent class and discrete latent trait models: Similarities and differences. Housand Oaks, California: Sage.
21. Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. N. (1995). The wake-sleep algorithm for unsupervised learning neural networks. Science, 268, 1158-1160.
22. Hinton, G. E., & Zemel, R. S. (1994), Autoencoders, minimum description length and Helmholtz free energy, In Cowan, Tesauro, & Alspector (eds), Advances in Neural Information Processing Systems 6, Morgan Kaufmann Pub., 449-455.
23. Isaksson, M., Wisell, D., & Ronnow, D. (2005). Wide-band dynamic modeling of power amplifiers using radial-basis function neural networks. IEEE Transactions on Microwave Theory and Techniques, 53(11), 3422-3428.
24. Jaakkola, T. S. (2001), Tutorial on variational approximation methods, in Opper & Saad (eds), Advanced Mean Field Methods: Theory and Practice, MIT press, 129-160.
25. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
26. Jordan, M., Ghahramani, Z., Jaakkola, T., & Saul, L. (1999). Introduction to variational methods for graphical models. Machine Learning, 37, 183-233.
27. Jordan, M. I., & Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181-214.
28. Jordan, M. I., & Xu, L. (1995). Convergence Results for The EM Approach to Mixtures of Experts Architectures. Neural Networks, 8, 1409-1431.
29. Karami, A., & Mohammadi, M. S. (2008). Radial basis function neural network for power system load-flow. International Journal of Electrical Power & Energy Systems, 30(1), 60-66.
30. Kardirkamanathan, V., Niranjan, M., & Fallside, F. (1991), Sequential adaptation of Radial basis function neural networks and its application to time-series prediction, In Lippmann, Moody, & Touretzky (eds), Advances in Neural Information Processing System 3, Morgan Kaufmann Pub., 721-727.
31. Konishi, S., Ando, T., & Imoto, S. (2004). Bayesian information criteria and smoothing parameter selection in radial basis function networks. Biometrika, 91(1), 27-43.
32. Lee, J. (1999). A practical radial basis function equalizer. IEEE Transactions on Neural Networks, 10, 450-455.
33. Lin, G. F., & Chen, L. H. (2004). A non-linear rainfall-runoff model using radial basis function network. Journal of Hydrology (Amsterdam), 289, 1-8.
34. Ma, S., Ji, C., & Farmer, J. (1997). An Efficient EM-based Training Algorithm for Feedforward Neural Networks. Neural Networks, 10(2), 243-256.
35. MacKay, D. (2003), Information Theory, Inference, and Learning Algorithms, Cambridge University Press.
36. Mackey, D. (1992). A practical Bayesian framework for backpropagation. Neural Computation, 4, 448-472.
37. Mai-Duy, N., & Tran-Cong, T. (2001). Numerical solution of differential equations using multiquadric radial basis function networks. Neural Networks, 14(2), 185-199.
38. McLachlan, G. J., & Geoffrey, J. (1997), The EM Algorithms and Extensions, Wiley.
39. Mel, B. W., & Omohundro, S. M. (1991), How receptive field parameters affect neural learning. In Lippmann, Moody, & Touretzky (eds), Advances in Neural Information Processing System 3, Morgan Kaufmann Pub., 757-763.
40. Moody, J., & Darken, C. (1989). Fast learning in networks of locally-tuned processing units. Neural Computation, 1, 281-294.
41. Neath, A. A., & Cavanaugh, J. E. (1997). Regression and Time Series model selection using variants of the Schwarz information criterion. Communications in Statistics A, 26, 559-580.
42. Nowlan, S. J. (1990), Max likelihood competition in RBF networks, TR. CRG-Tr-90-2, U. of Toronto, Dept. of Computer Science.
43. Park, J., & Sandberg, I. W. (1993). Universal approximation using radial-basis-function networks. Neural Computation, 5, 305-316.
44. Poggio, T., & Girosi, F. (1990). Networks for approximation and learning, Networks for approximation and learning. Proceedings of the IEEE, 78, 1481-1497.
45. Powell, M. J. D. (1987), Radial basis functions for multivariable interpolation: a review, in Mason & Cox (Eds.), Algorithms for Approximation, Oxford: Clarendon Press.
46. Reddy, R., & Ganguli, R. (2003). Structural damage detection in a helicopter rotor blade using radial basis function neural networks. Smart Materials and Structures, 12, 232-241.
47. Redner, R. A., & Walker, H. F. (1984). Mixture densities, maximum likelihood, and the EM algorithm. SIAM Review, 26, 195-239.
48. Rissanen, J. (1986). Stochastic complexity and modeling. Annals of Statistics, 14(3), 1080-1100.
49. Rissanen, J. (1989), Stochastic Complexity in Statistical Inquiry, World Scientific: Singapore.
50. Rockafellar, R. (1972), Convex Analysis, Princeton University Press.
51. Rumelhart, D. E., Hintont, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533-536.
52. Salah, A. A., & Alpaydin, E. (2004), Incremental mixtures of factor analyzers, Proc. the 17th International Conference on Pattern Recognition, 23-26 Aug. 2004, Cambridge, UK, Vol.1, 276-279.
53. Sarimveis, H., Doganis, P., & Alexandridis, A. (2006). classification technique based on radial basis function neural networks. Advances in Engineering Software, 37(4), 218-221.
54. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
55. Shi, L. (2008), Bayesian Ying-Yang harmony learning for local factor analysis: a comparative investigation, In Tizhoosh & Ventresca (eds), Oppositional Concepts in Computational Intelligence, SpringerVerlag, 209-232.
56. Specht, D. F. (1990). Probabilistic neural networks. Neural Networks, 3, 109-118.
57. Stone, M. (1978). Cross-validation: A review. Math. Operat. Statist., 9, 127-140.
58. Sun, K., Tu, S. K., Gao, D. Y., & Xu, L. (2009), Canonical Dual Approach to Binary Factor Analysis. In T. Adali et al. (Eds.), ICA 2009 (LNCS 5441), pp 346-353.
59. Treier, S., & Jackman, S. (2002), Beyond factor analysis: modern tools for social measurement. Presented at the 2002 Annual Meetings of the Western Political Science Association and the Midwest Political Science Association.
60. Vapnik, V. (1995), The Nature Of Statistical Learning Theory, Springer.
61. Vapnik, V. (2006), Estimation of Dependences Based on Empirical Data, Springer.
62. Wallace, C., & Freeman, P. (1987). Estimation and inference by compact coding. Journal of the Royal Statistical Society. Series A (General), 49(3), 240-265.
63. Wallace, C. S., & Boulton, D. M. (1968). An information measure for classification. The Computer Journal, 11, 185-194.
64. Xu, L. (1995), Bayesian-Kullback coupled YING-YANG machines: unified learning and new results on vector quantization, Proc.ICONIP95, Oct 30-Nov.3, 1995, Beijing, pp 977-988. A further version in NIPS8, D.S. Touretzky, et al (Eds.), MIT Press, 444-450.
65. Xu, L. (1998). RBF nets, mixture experts, and Bayesian Ying-Yang learning. Neurocomputing, 19(1-3), 223-257.
66. Xu, L. (2001). Best Harmony, Unified RPCL and Automated Model Selection for Unsupervised and Supervised Learning on Gaussian Mixtures, ME-RBF Models and Three-Layer Nets. International Journal of Neural Systems, 11(1), 3-69.
67. Xu, L. (2001). BYY harmony learning, independent state space and generalized APT financial analyses. IEEE Transactions on Neural Networks, 12(4), 822-849.
68. Xu, L. (2002). BYY harmony learning, structural RPCL, and topological self-organizing on unsupervised and supervised mixture models. Neural Networks, 15, 1125-1151.
69. Xu, L. (2004a). Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor auto-determination. IEEE Transactions on Neural Networks, 15, 885-902.
70. Xu, L. (2004b). Temporal BYY encoding, Markovian state spaces, and space dimension determination. IEEE Transactions on Neural Networks, 15, 1276-1295.
71. Xu, L (2005), Fundamentals, Challenges, and Advances of Statistical Learning for Knowledge Discovery and Problem Solving: A BYY Harmony Perspective, Keynote talk. Proc. Of Intl. Conf. on Neural Networks and Brain, Oct. 13-15, 2005, Beijing, China, Vol. 1, 24-55.
72. Xu, L. (2007a), Bayesian Ying Yang Learning, Scholarpedia 2(3):1809, Retrieved from http://scholar-pedia.org/article/Bayesian_Ying_Yang_learning.
73. Xu, L. (2007b), Rival penalized competitive learning, Scholarpedia 2(8):1810, Retrieved from http://www.scholarpedia.org/article/Rival_penalized_competitive_learning
74. Xu, L. (2007c), A trend on regularization and model selection in statistical learning: a Bayesian Ying Yang learning perspective, In Duch & Mandziuk (eds.), Challenges for Computational Intelligence, Springer-Verlag, 365-406.
75. Xu, L. (2007d). A unified perspective and new results on RHT computing, mixture based learning, and multi-learner based problem solving. Pattern Recognition, 40, 2129-2153.
76. Xu, L. (2008a), Bayesian Ying Yang System, Best Harmony Learning, and Gaussian Manifold Based Family, In Zurada et al (eds.) Computational Intelligence: Research Frontiers, WCCI2008 Plenary/ Invited Lectures, LNCS5050, 48-78.
77. Xu, L. (2008b). (in press). Machine learning problems from optimization perspective, A special issue for CDGO 07. Journal of Global Optimization.
78. Xu, L. (2008c), Independent Subspaces, in Ramón, Dopico, Dorado & Pazos (Eds.), Encyclopedia of Artificial Intelligence, IGI Global (IGI) publishing company, 903-912.
79. Xu, L., Jordan, M. I., & Hinton, G. E. (1994), A Modified Gating Network for the Mixtures of Experts Architecture, Proc. of WCNN94, San Diego, CA, 405-410.
80. Xu, L., Jordan, M. I., & Hinton, G. E. (1995), An Alternative Model for Mixtures of Experts, In Tesauro, Touretzky & Leen (eds), Advances in Neural Information Processing Systems 7, MIT Press, 633-640.
81. Xu, L., Krzyzak, A., & Oja, E. (1993). Rival Penalized Competitive Learning for Clustering Analysis, RBF net and Curve Detection. IEEE Trans. Neural Networks, 4(4), 636-649. An early version on Proc. 1992 IJCNN, Nov.3-6, 1992, Beijing, 665-670.
82. Xu, L., Krzyzak, A., & Yuille, A. L. (1994). On Radial Basis Function Nets and Kernel Regression: Statistical Consistency, Convergence Rates and Receptive Field Size. Neural Networks, 7(4), 609-628.
83. Yuille, A. L., & Grzywacz, N. M. (1989). A mathematical analysis of the motion coherence theory. International Journal of Computer Vision, 3, 155-175.