Nonlinear system identification: From multiple-model networks to Gaussian processes

Engineering Applications of Artificial Intelligence

Volumn 21, Issue 7, 2008, Pages 1035-1055

  Gregorčič, Gregor ^a   Lightbody, Gordon ^b  

^a AVL LIST GMBH   (Austria)

^b UNIVERSITY COLLEGE CORK   (Ireland)

Author keywords

Gaussian processes; Local model network; Network structure; Nonlinear system identification; Radial basis function network

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONTROL SYSTEMS; FEEDFORWARD NEURAL NETWORKS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); IDENTIFICATION (CONTROL SYSTEMS); NONLINEAR SYSTEMS; RADIAL BASIS FUNCTION NETWORKS; TRELLIS CODES; VEGETATION;

CURSE OF DIMENSIONALITY; GAUSSIAN PROCESS MODELS; GAUSSIAN PROCESSES; LOCAL MODEL NETWORK; LOCAL MODEL NETWORKS; LOCAL MODELING; MULTIPLE MODELING; NETWORK STRUCTURE; NEURAL NETWORK MODELLING; NON-PARAMETRIC; NONLINEAR SYSTEM IDENTIFICATION; RADIAL BASIS FUNCTION NETWORK;

NEURAL NETWORKS;

EID: 52949150838     PISSN: 09521976     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.engappai.2007.11.004     Document Type: Article

References (109)

1. Abonyi J. Fuzzy Model Identification for Control (2003), Birkhäuser, Boston
2. Andersen H.C., Lotfi A., and Westphal L.C. Comments on "Functional equivalence between radial basis function networks and fuzzy inference systems". IEEE Transactions on Neural Networks 9 6 (1998) 1529-1531
3. Aoyama, A., Doyle, F.J., Venkatasubramanian, V., 1995. A fuzzy neural-network approach for nonlinear process control. Engineering Application Artificial Intelligence 8(5), 483-498.
4. Ažman K., and Kocijan J. Application of Gaussian processes for black-box modelling of biosystems. ISA Transactions 46 4 (2007) 443-457
5. Babuška R., and Verbruggen H. Neuro-fuzzy methods for nonlinear system identification. Annual Reviews in Control 27 (2003) 73-85
6. Bailer-Jones, C.A.L., Sabin, T.J., MacKay, D.J.C., Withers, P., 1997. Prediction of deformed and annealed microstructures using Bayesian neural networks and Gaussian processes. In: Chandra, T., Leclair, S.R., Meech, J.A., Verma, B., Smith, M., Balachandran, B. (Eds.), Proceedings of the Australasia Pacific Forum on Intelligent Processing and Manufacturing of Materials, pp. 913-919.
7. Billings S.A. Identification of nonlinear systems-a survey. Proceedings of IEE, Part D: Control Theory and Applications 127 6 (1980) 272-285
8. Bishop, C.M., Tipping, M.E., 2003. Bayesian regression and classification. In: Suykens, J., Horvath, G., Basu, S., Micchelli, C., Vandewalle, J. (Eds.), Advances in Learning Theory: Methods, Models and Applications, vol. 190, NATO Science Series III: Computer and Systems Sciences. IOS Press, pp. 267-285.
9. Boukhris A., Mourot G., and Ragot J. Non-linear dynamic system identification: a multi-model approach. International Journal of Control 72 7/8 (1999) 591-604
10. Brown M.D., Lightbody G., and Irwin G.W. Nonlinear internal model control using local model networks. IEE Proceedings: Control Theory and Applications 144 6 (1997) 505-514
11. Chen S., Cowan C.F.N., and Grant P.M. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Transactions on Neural Networks 2 2 (1991) 302-309
12. DeVeaux R.D., Psichogios D.C., and Ungar L.H. A comparison of two nonparametric estimation schemes: Mars and neural networks. Computers in Chemical Engineering 17 8 (1993) 819-837
13. Dodd T.J., and Harris C.J. Identification of nonlinear time series via kernels. International Journal of System Science 33 9 (2002) 737-750
14. Economou C.G., Morari M., and Palsson B.O. Internal model control: extension to non-linear systems. Industrial Engineering Chemical Process Design and Development 25 (1986) 403-411
15. Faul, S., Gregorčič, G., Boylan, G., Marnane, W., Lightbody, G., Connolly, S., 2006. Gaussian process modelling as an indicator of neonatal seizure. In: Proceedings of the Third IASTED International Conference on Signal Processing, Pattern Recognition and Applications (SPPRA 2006). Innsbruck, Austria, pp. 177-182.
16. Faul S., Gregorčič G., Boylan G., Marnane W., Lightbody G., and Connolly S. Gaussian process modeling of EEG for the detection of neonatal seizures. IEEE Transactions on Biomedical Engineering 54 12 (2007) 2151-2162
17. Foss B.A., Johansen T.A., and Sørensen A.V. Nonlinear predictive control using local models-applied to a batch fermentation process. Control Engineering Practice 3 3 (1995) 389-396
18. Gibbs, M.N., 1997. Bayesian Gaussian processes for regression and classification. Ph.D. Thesis, University of Cambridge.
19. Gibbs M., and MacKay D. Variational Gaussian process classifiers. IEEE Transactions on Neural Networks 11 6 (2000) 1458-1464
20. Girard A., Rasmussen C., Candela J.Q., and Murray-Smith R. Multiple-step ahead prediction for non linear dynamic systems-a Gaussian process treatment with propagation of the uncertainty. In: Becker S., Thrun S., and Obermayer K. (Eds). Advances in Neural Information Processing Systems (2002), MIT Press, Cambridge, MA 545-552
21. Godfrey, K.R., 1991. Introduction to binary signals used in system identification. In: IEE International Conference on Control, Edinburgh, UK, pp. 161-166.
22. Gollee H., Murray-Smith D.J., and Jarvis J.C. A non-linear approach to modelling of electrically stimulated skeletal muscle. IEEE Transactions on Biomedical Engineering 48 4 (2001) 406-415
23. Gregorčič, G., 2004. Data-based modelling of nonlinear systems for control. Ph.D. Thesis, University College Cork.
24. Gregorčič, G., Lightbody, G., 2003. Internal model control based on a Gaussian process prior model. In: Proceedings of the 2003 American Control Conference, Denver, CO, USA, pp. 4981-4986.
25. Gregorčič G., and Lightbody G. Gaussian process approaches to nonlinear modelling for control. In: Ruano A.E. (Ed). Intelligent Control Systems Using Computational Intelligence Techniques, IEE Control Engineering Series 70 (2005), IEE, London, UK 177-217 (Chapter 6)
26. Gregorčič G., and Lightbody G. Local model network identification with Gaussian processes. IEEE Transactions on Neural Networks 18 9 (2007) 1404-1423
27. Hunt K.J., and Johansen T.A. Design and analysis of gain-scheduled control using local controller networks. International Journal of Control 66 5 (1997) 619-651
28. Hunt K.J., and Sbarbaro D. Neural networks for nonlinear internal model control. Proceedings of IEE, Part D: Control Theory and Applications 138 (1991) 431-438
29. Hunt K.J., Haas R., and Murray-Smith R. Extending the functional equivalence of radial basis function networks and fuzzy inference systems. IEEE Transactions on Neural Networks 7 3 (1996) 776-781
30. Hunt K.J., Haas R., and Kalkkuhl J.C. Local controller network for autonomous vehicle steering. Control Engineering Practice 4 8 (1996) 1045-1051
31. Hunt K.J., Kalkkuhl J.C., Fritz H., and Johansen T.A. Constructive empirical modelling of longitudinal vehicle dynamics with local model networks. Control Engineering Practice 4 2 (1996) 167-178
32. Hunt K.J., Johansen T.A., Kalluhl J., Fritz H., and Göttsche T. Speed control design for an experimental vehicle using a generalized gain scheduling approach. IEEE Transactions on Control Systems Technology 8 3 (2000) 381-395
33. Isaaks E.H., and Srivastava R.M. Applied Geostatistics (1998), Oxford University Press, Oxford
34. Jang J.S.R., and Sun C.T. Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Transactions on Neural Networks 4 1 (1993) 156-159
35. Jang J.-S.R., and Sun C.-T. Neuro-fuzzy modeling and control. Proceedings of the IEEE 83 (1995) 378-406
36. Johansen, T.A., 1995. On the optimality of the Takagi-Sugeno-Kang fuzzy inference mechanism. In: Proceedings of the IEEE Conference on Fuzzy Systems, vol. 1, Yokohama, Japan, pp. 97-102.
37. Johansen, T.A., Babuška, R., 2002. On multi-objective identification of Takagi-Sugeno fuzzy model parameters. In: International Federation of Automatic Control, 15th IFAC Triennial World Congress, Barcelona, Spain, pp. 847-860.
38. Johansen T.A., and Foss B.A. A NARMAX model representation for adaptive control based on local models. Modeling Identification and Control 13 (1992) 25-39
39. Johansen T.A., and Foss B.A. Constructing NARMAX models using ARMAX models. International Journal of Control 58 5 (1993) 1125-1153
40. Johansen T.A., and Foss B.A. Identification of non-linear system structure and parameters using regime decomposition. Automatica 31 (1995) 321-326
41. Johansen T.A., and Foss B.A. Operating regime based process modeling and identification. Computers and Chemical Engineering 21 (1997) 159-176
42. Johansen T.A., and Foss B.A. Orbit-operating regime based modeling and identification toolkit. Control Engineering Practice 6 (1998) 1277-1286
43. Johansen T.A., Shorten R., and Murray-Smith R. On the interpretation and identification of Takagi-Sugeno fuzzy models. IEEE Transactions on Fuzzy Systems 8 (2000) 297-313
44. Johansen T.A., Hunt K., and Petersen I. Gain scheduled control of a solar power plant. Control Engineering practice 8 9 (2000) 1011-1022
45. Jordan M.I., and Bishop C.M. Neural networks. In: Tucker A.B. (Ed). CRC Handbook of Computer Science (1997), CRC Press, Boca Raton, FL, USA 536-556
46. Kocijan, J., Leith, D.J., 2004. Derivative observations used in predictive control. In: Proceedings of the 12th IEEE Mediterranean Electrotechnical Conference, MELECON 2004, Dubrovnik, Croatia, pp. 379-382.
47. Kocijan J., Žunič G., Strmčnik S., and Vrančić D. Fuzzy gain-scheduling control of a gas-liquid separation plant implemented on a PLC. International Journal of Control 75 14 (2002) 1082-1091
48. Kocijan, J., Banko, B., Likar, B., Girard, A., Murry-Smith, R., Rasmussen, C.A., 2003a. A case based comparison of identification with neural network and Gaussian process models. In: Ruano, A.E. (Ed.), Proceedings of the IFAC International Conference on Intelligent Control Systems and Signal Processing, ICONS 2003, Faro, Portugal, pp. 137-142.
49. Kocijan, J., Murray-Smith, R., Rasmussen, C.E., Likar, B., 2003b. Predictive control with Gaussian process models. In: Proceedings of the IEEE Region 8 Eurocon 2003: Computer as a Tool, Ljubljana, Slovenia, pp. 352-356.
50. Kocijan, J., Girard, A., Banko, B., Murray-Smith, R., 2003c. Dynamic systems identification with Gaussian processes. In: Proceedings of the Fourth Mathmod, Vienna, Austria, pp. 776-784.
51. Kocijan, J., Murray-Smith, R., Rasmussen, C.E., Girard, A., 2004. Gaussian process model based predictive control. In: Proceedings of the 2004 American Control Conference, Boston, MA, USA, pp. 2214-2218.
52. Larimore, W.E., 1997. Optimal reduced rank modeling, prediction, monitoring, and control using canonical variate analysis. In: Proceedings of the IFAC International Symposium on Advanced Control of Chemical Processes, Banff, Canada, pp. 61-66.
53. Leith D.J., and Leithead W.E. Gain-scheduled controller design: an analytic framework directly incorporating non-equilibrium plant dynamics. International Journal of Control 70 2 (1998) 249-269
54. Leith D.J., and Leithead W.E. Gain-scheduled and nonlinear systems: dynamic analysis by velocity-based linearization families. International Journal of Control 70 2 (1998) 289-317
55. Leith D.J., and Leithead W.E. Analytic framework for blended multiple model systems using linear local models. International Journal of Control 72 7/8 (1999) 605-619
56. Leith, D.J., Leithead, W.E., Murray-Smith, R., 2001. Nonlinear structure identification: a Gaussian process/velocity-based approach 〈http://www.hamilton.ie/publications/1002728180_file(pdf.ps.zip).structid.pdf〉.
57. Lightbody G., and Irwin G.W. Nonlinear control structures based on embedded neural system models. IEEE Transactions on Neural Networks 8 (1997) 553-567
58. Ljung L. System Identification: Theory for the User. second ed (1999), Prentice-Hall, Upper Saddle River, NJ
59. Lundgren, A., Sjoberg, J., 2003. Gaussian processes framework for validation of linear and nonlinear models. In: 13th IFAC Symposium on System Identification SYSID 2003, Rotterdam, The Netherlands, pp. 67-72.
60. MacKay, D.J.C., 1998a. Introduction to Gaussian processes. In: Bishop, C.M. (Ed.), Neural Networks and Machine Learning. NATO ASI Series. Kluwer Academic Press, pp. 133-166.
61. MacKay D.J.C. Introduction to Monte Carlo methods. In: Jordan M.I. (Ed). Learning in Graphical Models. NATO Science Series (1998), Kluwer Academic Press 175-204
62. MacKay D.J.C. Comparison of approximate methods for handling hyperparameters. Neural Computation 11 5 (1999) 1035-1068
63. MacKay D.J.C. Information Theory, Inference and Learning Algorithms (2003), Cambridge University Press, Cambridge
64. McLoone S., and Irwin G. Non-linear optimization of RBF networks. International Journal of Systems Science 29 2 (1998) 179-189
65. McLoone, S., 2000. Nonlinear identification using local model networks. Ph.D. Thesis, Queens University of Belfast.
66. McLoone S.C., Irwin G.W., and McLoone S.F. Constructing networks of continuous-time velocity-based models. IEE Proceedings-Control Theory and Applications 148 5 (2001) 397-405
67. McLoone, S.C., Irwin, G.W., McLoone, S.F., 2002. On Gaussian weighting functions for velocity-based local model networks. In: Proceedings of the Irish Signals and Systems Conference ISSC2002, Cork, Ireland, pp. 135-140.
68. von Mises R. Mathematical Theory of Probability and Statistics (1964), Academic Press, New York
69. Morris A.J., Montague G.A., and Willis M.J. Artificial neural networks: studies in process modeling and control. Chemical Engineering Research and Design 72 (1994) 3-19
70. Murray-Smith, R., 1992. A fractal radial basis function network for modeling. In: International Conference on Automation, Robotics and Computer Vision, vol. 1, Singapore, pp. NW-2.6.0-NW-2.6.5.
71. Murray-Smith, R., Girard, A., 2001. Gaussian process priors with ARMA noise models. In: Proceedings of the Irish Signals and Systems Conference, Maynooth, Ireland, pp. 147-152.
72. Murray-Smith, R., Gollee, H., 1994. A constructive learning algorithm for local model networks. In: IEEE Workshop on Computer-Intensive Methods in Control and Signal Processing, Prague, Czech Republic, pp. 21-29.
73. Murray-Smith R., and Hunt K.J. Local model architectures for nonlinear modelling and control. In: Hunt K.J., Irwin G.R., and Warwick K. (Eds). Neural Network Engineering in Dynamic Control Systems, Advances in Industrial Control (1995), Springer, Berlin 61-82
74. Murray-Smith, R., Johansen, T.A., 1995. Local learning in local model networks. In: Proceedings of the Fourth IEE International Conference on Artificial Neural Networks, Cambridge, UK, pp. 40-46.
75. In: Murray-Smith R., and Johansen T.A. (Eds). Multiple Model Approaches to Modelling and Control (1997), Taylor & Francis, London
76. Murray-Smith, R., Sbarbaro, D., 2002. Nonlinear adaptive control using nonparametric Gaussian process models. In: International Federation of Automatic Control, 15th IFAC Triennial World Congress, Barcelona, Spain, pp. 934-939.
77. Murray-Smith, R., Johansen, T.A., Shorten, R., 1999. On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures. In: Proceedings of the European Control Conference, Karlsruhe, Germany, pp. BA-B14.
78. Murray-Smith, R., Sbarbaro, D., Rasmussen, C.E., Girard, A., 2003. Adaptive, cautious, predictive control with Gaussian process priors. In: de Hof, P.V., Wahlberg, B., Weiland, S. (Eds.), Proceedings of the 13th IFAC Symposium on System Identification, Rotterdam, The Netherlands, pp. 1195-1200.
79. Nahas E.P., Henson M.A., and Seborg D.E. Non-linear internal model control strategy for neural network models. Computers and Chemical Engineering 16 (1992) 1039-1057
80. Narendra K.S., and Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1 1 (1990) 4-27
81. Neal, R.M., 1997. Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Technical Report 9702, Department of Statistics, University of Toronto.
82. Nelles, O., 1999. Nonlinear system identification with local linear neuro-fuzzy models. Ph.D. Thesis, Technische Universität Darmstadt.
83. Nelles, O., Fink, A., Isermann, R., 2000. Local linear model trees (lolimot) toolbox for nonlinear system identification. In: 12th IFAC Symposium on System Identification (SYSID), Santa Barbara, USA, pp. 845-850.
84. Nise N. Control Systems Engineering. third ed (2000), Wiley, New York
85. O'Hagan A. Curve fitting and optimal design for prediction (with discussion). Journal of the Royal Statistical Society B 40 1 (1978) 1-42
86. Platt, J.C., Burges, C.J.C., Swenson, S., Weare, C., Zheng, A., 2002. Learning a Gaussian process prior for automatically generating music playlists. In: Advances in Neural Information Processing Systems, pp. 1425-1432.
87. Pollard J.F., Broussard M.R., Garrison D.B., and San K.Y. Process identification using neural networks. Computers in Chemical Engineering 16 4 (1992) 253-270
88. Rasmussen, C.E., 1996. Evaluation of Gaussian processes and other methods for non-linear regression. Ph.D. Thesis, University of Toronto.
89. Rasmussen C.E., and Williams C.K.I. Gaussian Processes for Machine Learning (2006), The MIT Press, Cambridge, MA
90. Reed R. Pruning algorithms-a survey. IEEE Transactions on Neural Networks 4 5 (1999) 740-747
91. Richalet J. Industrial applications of model based predictive control. Automatica 29 5 (1993) 1251-1274
92. Sbarbaro, D., Murray-Smith, R., 2003. Self-tuning control of non-linear systems using Gaussian process prior models. Technical Report TR-2003-143, Glasgow University.
93. Seborg, D.E., 1994a. Experience with nonlinear control and identification strategies. In: International Conference on Control, vol. 2, IEE, Coventry, UK, pp. 879-886.
94. Seborg D.E. A perspective on advanced control strategies for process control. Modeling Identification and Control 15 3 (1994) 179-189
95. Seeger, M., 2003. Bayesian Gaussian process models: PAC-Bayesian generalisation error bounds and sparse approximations. Ph.D. Thesis, University of Edinburgh.
96. Shorten, R., Murray-Smith, R., 1994. On normalising radial basis function networks. In: Proceedings of the Fourth Irish Neural Network Conference, University College Dublin, Ireland, pp. 213-217.
97. Shorten R., Murray-Smith R., and Bjørgan R. On the interpretation of local models in blended multiple model structures. International Journal of Control 72 7/8 (1999) 620-628
98. Stark J., Broomhead D.S., Davies M.E., and Huke J. Delay embeddings of forced systems: II. Stochastic forcing. Journal of Nonlinear Science 13 6 (2003) 519-577
99. Strang G. Linear Algebra and its Applications. third ed (1988), Harcourt College Publishers, Harcourt
100. Takagi T., and Sugeno M. Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems, Man and Cybernetics SMC-15 1 (1985) 116-132
101. Tancret F., Bhadeshia H.K.D.H., and MacKay D.J.C. Comparison of artificial neural networks with Gaussian processes to model the yield strength of nickel-based superalloys. The Iron and Steel Institute of Japan 39 10 (1999) 1020-1026
102. Ungar, L.H., Johnson, T., de Veaux, R.D., 1994. Radial basis function neural networks for process control. In: CIMPRO Proceedings, pp. 357-364.
103. Vivarelli F., and Williams C.K.I. Discovering hidden features with Gaussian processes regression. In: Kearns M.S., Solla S.A., and Cohn D.A. (Eds). Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II (1999), MIT Press, Cambridge, MA, USA 613-619
104. ∞ robust controllers for a high performance fighter aircraft under a herbst-like manoeuvre. International Journal of Control 72 7/8 (1999) 740-754
105. Williams, C.K.I., 1997. Regression with Gaussian Processes, Mathematics of Neural Networks: Models, Algorithms and Applications. Kluwer.
106. Williams, C.K.I., 1998. Prediction with Gaussian processes: from linear regression to linear prediction and beyond. Technical Report NCRG/97/012, Aston University, Birmingham, UK.
107. Williams C.K.I., and Barber D. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 12 (1998) 1342-1351
108. Williams C.K.I., and Rasmussen C.E. Gaussian processes for regression. In: Touretzky M.E.H., and Mozer M.C. (Eds). Advances in Neural Information Processing Systems, vol. 8 (1996), MIT Press, Cambridge, MA 514-520
109. Zadeh L. Fuzzy sets. Information and Control 8 (1965) 338-353