Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Information Sciences

Volumn 177, Issue 10, 2007, Pages 2049-2074

  Lima, Clodoaldo A M ^a   Coelho, André L V ^b   Von Zuben, Fernando J ^a  

^a UNIVERSITY OF CAMPINAS   (Brazil)

^b UNIVERSITY OF FORTALEZA   (Brazil)

Author keywords

Hybridization; Kernels; Mixtures of experts; Nonlinear systems identification; Support vector machines

Indexed keywords

APPROXIMATION THEORY; COMPUTER SIMULATION; MATHEMATICAL MODELS; MIXTURES; RISK MANAGEMENT; SUPPORT VECTOR MACHINES;

HYBRIDIZATION; HYBRIDIZING MIXTURES; KERNELS; MIXTURE OF EXPERTS (ME); NONLINEAR SYSTEMS IDENTIFICATION; RISK MINIMIZATION;

EXPERT SYSTEMS;

EID: 33847706511     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2007.01.009     Document Type: Article

References (38)

1. Abe S. Support Vector Machines for Pattern Classification (2005), Springer, London
2. Cao L. Support vector machines experts for time series forecasting. Neurocomputing 51 (2003) 321-339
3. Cherkassky V., and Ma Y. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Networks 17 (2004) 687-693
4. Cherkassky V., and Mulier F. Learning from Data: Concepts, Theory and Methods (1998), Wiley, New York
5. A.L.V. Coelho, C.A.M. Lima, F.J. Von Zuben, Hybrid genetic training of gated mixtures of experts for nonlinear time series forecasting, in: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Washington, DC, 2003, pp. 4625-4630.
6. Collobert R., Bengio S., and Bengio Y. A parallel mixture of SVMs for very large scale problems. Neural Computation 14 (2002) 1105-1114
7. Cortes C., and Vapnik V. Support-vector networks. Machine Learning 20 (1995) 207-273
8. Cristianini N., and Shawe-Taylor J. An Introduction to Support Vector Machines (2000), Cambridge University Press, Cambridge, UK
9. Dempster A.P., Laird N.M., and Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39 (1977) 1-38
10. J. Fritsh, Modular neural networks for speech recognition, Master Thesis, Carnegie Mellon University & University of Karlsrube, 1996.
11. S. Gunn, Support vector machines for classification and regression, Technical Report ISIS-1-98, Image Speech & Intelligent Systems Group, University of Southampton, 1998.
12. Haykin S. Neural Networks-A Comprehensive Foundation (1999), Prentice-Hall, Upper Saddle River, NJ
13. Jacobs R. Bias/Variance analyses of mixtures-of-experts architectures. Neural Computation 9 (1997) 369-383
14. Jacobs R., Jordan M., Nowlan S., and Hinton G. Adaptive mixtures of local experts. Neural Computation 3 (1991) 79-87
15. Jordan M., and Jacobs R. Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6 (1994) 181-214
16. J.T.-Y. Kwok, Support vector mixture for classification and regression problems, in: Proceedings of the 14th International Conference on Pattern Recognition, Brisbane, 1998, pp. 255-258.
17. C.A.M. Lima, A.L.V. Coelho, F.J. Von Zuben, Mixture of experts applied to nonlinear dynamic systems identification: A comparative study, in: Proceedings of the VII Brazilian Symposium on Neural Networks, 2002, pp. 162-167.
18. C.A.M. Lima, A.L.V. Coelho, F.J. Von Zuben, Ensembles of support vector machines for regression problems, in: Proceedings of the International Joint Conference on Neural Networks, Hawaii, USA, 2002, pp. 2381-2386.
19. McLachlan G.J., and Basford K.E. Mixture Models: Inference and Applications to Clustering (1988), Marcel Deckker Inc., New York
20. P. Moerland, Some methods for training mixtures of experts, Technical Report IDIAP-Com 97-05, IDIAP Research Institute, 1997.
21. P. Moerland, Classification using localized mixtures of experts, in: Proceedings of International Conference on Artificial Neural Networks, Edinburgh, UK, 1999, pp. 838-843.
22. Narendra K.S., and Mukhopadhyay S. Intelligent control using neural networks. IEEE Control Systems 12 (1992) 11-18
23. Narendra K.S., and Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1 (1990) 4-27
24. Ramamurti V., and Ghosh J. Structurally adaptive modular networks for nonstationary environments. IEEE Transactions on Neural Networks 10 (1999) 152-160
25. A. Rida, A. Labbi, C. Pellegrini, Local experts combination through density decomposition, in: Proceedings of Seventh International Workshop on Artificial Intelligence and Statistics, Morgan Kaufmann, 1999, pp. 130-136.
26. In: Ruano A.E. (Ed). Intelligent Control Systems Using Computational Intelligence Techniques (2005), IEE Press
27. In: Sharkey A. (Ed). Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems (1999), Springer, London
28. Schölkopf B., and Smola A. Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond (2002), MIT Press Cambridge, MA
29. Shawe-Taylor J., and Cristianini N. Kernel Methods for Pattern Analysis (2004), Cambridge University Press, Cambridge
30. G.F. Smits, E.M. Jordaan, Improved SVM regression using mixture of kernels, in: Proceedings of the International Joint Conference on Neural Networks, Hawaii, USA, 2002, pp. 2785-2790.
31. Srivastava A.N., Su R., and Weigend A.S. Data mining for features using scale-sensitive gated experts. IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (1999) 1268-1279
32. Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least Squares Support Vector Machines (2002), World Scientific Publishers
33. Tikhonov A.N., and Arsenim V.Y. Solutions of Ill-posed Problems (1977), W.H. Winston
34. R. Torres, R. Salas, H. Allende, C. Moraga, Robust expectation maximization learning algorithm for mixture of experts, in: Proceedings of the 7th International Work-Conference on Artificial and Natural Neural Networks, Lecture Notes in Computer Science, vol. 2686, 2003, pp. 238-245.
35. Vapnik V.N. The Nature of Statistical Learning Theory (1995), Springer, New York
36. Vapnik V.N. Statistical Learning Theory (1998), Wiley, New York
37. Weigend A.S., Mangeas M., and Srivastava A.N. Nonlinear gated experts for time series: Discovering regimes and avoiding overfitting. International Journal of Neural Systems 6 (1995) 373-399
38. Xu L., Jordan M.I., and Hinton G.E. An alternative model for mixtures of experts. In: Tesauro G., Touretzky D.S., and Leen T.K. (Eds). Advances in Neural Information Processing Systems (1995), MIT Press 633-640