Model selection approaches for non-linear system identification: A review

International Journal of Systems Science

Volumn 39, Issue 10, 2008, Pages 925-946

  Hong, X ^a   Mitchell, R J ^a   Chen, S ^b   Harris, C J ^b   Li, K ^c   Irwin, G W ^c  

^a UNIVERSITY OF READING   (United Kingdom)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^c QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Adaptive learning; Control engineering; Cross validation; Model generalisation; Model selection; System identification

Indexed keywords

ALGORITHMS; APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; EIGENVALUES AND EIGENFUNCTIONS; FOOD PROCESSING; IDENTIFICATION (CONTROL SYSTEMS); INDUSTRIAL APPLICATIONS; LEARNING SYSTEMS; LINEAR SYSTEMS; MATHEMATICAL MODELS; MULTILAYER NEURAL NETWORKS; NONLINEAR SYSTEMS; PARAMETER ESTIMATION; SUPPORT VECTOR MACHINES; TIME VARYING SYSTEMS; VECTORS;

BASIC RESEARCH; BAYESIAN; CONVERGENCE CONDITIONS; CONVEX OPTIMISATION; CROSS-VALIDATION (CV); DATA-SETS; EXPERIMENTAL DESIGN (ED); FUNDAMENTAL PROBLEMS; INPUT SELECTION; KERNEL MODELS; MACHINE-LEARNING; MINIMAL MODELING; MODEL CONSTRUCTION; MODEL GENERALISATION; MODEL SELECTIONS; NON LINEAR MODELING; NON LINEAR SYSTEMS; NON-LINEAR SYSTEM IDENTIFICATION; OBSERVATIONAL DATA; ON-LINE SYSTEM IDENTIFICATION; PARAMETER MODELLING; REGULARISATION; RESEARCH AREA (RA); STRUCTURAL RISK; SUPPORT VECTOR REGRESSION (SVR); SYSTEMATIC (CO); UNIVERSAL APPROXIMATION;

LEARNING ALGORITHMS;

EID: 47749106824     PISSN: 00207721     EISSN: 14645319     Source Type: Journal    
DOI: 10.1080/00207720802083018     Document Type: Article

References (213)

1. Adeney, K.M., and Korenberg, M.J. (2000), "Iterative Fast Orthogonal Search Algorithm for MDL-based Training of Generalised Single-layer Network," Neural Networks, 13, 787-799.
2. Aguirre, L.A., Coelho, M.C.S., and Correa, M.V. (2005), "On the Interpretation and Practice of Dynamical Differences Between Hammerstein and Wiener Models," IEE Proc. and Control Theory Applications, 152(4), 349-356.
3. Aguirre, L.A., Donoso-Garcia, P.F., and Santos-Filho, R. (2000), "Use of a Priori Information in the Identification of Global Nonlinear Models-a Case Study using a Buck Converter," IEEE Transactions on Circuits and Systems-I: Fundamental Theory and Applications, 47(7), 1081-1085.
4. Aguirre, L.A., Lopes, R.A.M., Amaral, G.F.V., and Letellier, C. (2004), "Constraining the topology of Neural Networks to Ensure Dynamics with Symmetry Properties," Physical Review E, 69, 026701-1-026701-11.
5. Akaike, H. (1974), "A New Look at the Statistical Model Identification," IEEE Transactions on Automatic Control, AC-19, 716-723.
6. Allen, D.M. (1974), "The Relationship Between Variable Selection and Data Augumentation and a Method for Prediction," Technometrics, 16, 125-127.
7. 3 Particulate-reinforced alsi10mg Alloy Composites Prepared by using Stir Casting Process," Journal of Composite Materials, 40(9), 779-796.
8. Aronszajin, N. (1950), "Theory of Reproducing Kernels," Transactions on American Mathematical Society, 68, 337-404.
9. Aström, K.J., and Eyhhoff, P. (1971), "System Identification - a Survey," Automatica, 7, 123-162.
10. Aström, K.J., and Wittenmark, B. (1989), Adaptive Control, MS: Addison Wesley.
11. Atkinson, A.C., and Donev, A.N. (1992), Optimum Experimental Designs, Oxford: Clarendon Press.
12. Bai, E.W. (1998), "An Optimal Two-stage Identification Algorithm for Hammerstein-wiener Nonlinear Systems," Automatica, 34, 333-338.
13. Bai, E.W. (2004), "Decoupling the Linear and Nonlinear Parts in Hammerstein Model Identification," Automatica, 40, 671-676.
14. Barron, A.R. (1984), "Predicted Square Error. A Criterion for Automatic Model Selection," in SelfOrganizing Methods in Modeling, ed. S. Farlow, Cambridge, MA: MIT Press.
15. Bartlett, P.L. (2003), "Reproducing Kernel Hubert Spaces," Lecture Notes, CS281B/Stat241B, Statistical Learning Theory, Berkeley, USA: The Computer Science Division, University of California.
16. Basso, M., Giarre, L., Groppi, S., and Zappa, G. (2005), "NaRX Models of an Industrial Power Plant Gas Turbine," IEEE Transactions on Control Systems Technology, 13(4), 599-604.
17. Battiti, R. (1994), "Using Mutual Information for Selecting Features in Supervised Neural Net Learning," IEEE Transactions on Neural Networks, 5(4), 537-550.
18. Billings, S.A., and Wei, H.L. (2005), "The Wavelet-narmax Representation: A Hybrid Model Structure Combining Polynomial Models with Multiresolution Wavelet Decompositions," International Journal of Systems Science, 36(3), 137-152.
19. Billings, S.A., and Voon, W.S.F. (1987), "Piecewise Linear Identification of Nonlinear Systems," International Journal of Control, 46, 215-235.
20. Billings, S.A., Chen, S., and Backhouse, R.J. (1989), "The Identification of Linear and Nonlinear Models of a Turbocharged Automative Diesel Engine," Mechanical Systems and Signal Processings, 3(2), 123-142.
21. Bohlin, T. (1971), "On the Problem of Ambiguities in Maximum Likelihood Identification," Automatica, 7, 199-200.
22. Bohlin, T. (2006), Practical Grey-box Process Identification: Theory and Applications, New York: Springer.
23. Bossley, K.M. (1997), Neurofuzzy Modelling Approaches in System Identification, Ph.D. thesis, University of Southampton, Dept. of ECS.
24. Bowden, GJ., Dandy, G.C., and Maier, H.R. (2005), "Input Determination for Neural Network, Models in Water Resources Applications. Part 1-background and Methodology," Journal of Hydrology, 301, 75-92.
25. Box, G.E.P., and Jenkins, G.M. (1976), Time Series Analysis: Forecasting and Control, San Francisco: Holden-Day Inc.
26. Bradford, J.R., and Westhead, D.R. (2005), "Improved Prediction of Protein-protein Binding Sites using a Support Vector Machines Approach," Bioinformatics, 21(8), 1487-1494.
27. Broomhead, D.S., and Lowe, D. (1988), "Multivariable Functional Interpolation and Adaptive Networks," Complex Systems, 2, 321-355.
28. Brown, M., and Harris, C.J. (1994), Neurofuzzy Adaptive Modelling and Control, Heme! Hempstead: Prentice Hall.
29. Brown, M.D., Flynn, D., and Irwin, G.W. (2002), "Multiple Model Nonlinear Control of Synchronous Generators," Transactions on Inst MC, Special Issue on Electrical Power Generation, 24(3), 215-230.
30. Brown, M.D., Irwin, G.W., and Lightbody, G. (1997), "Nonlinear Internal Model Control using Local Model Networks with Application to a pH Process," Proc. IFAC nt. Symp. on Advanced Control of Chemical Processes, AdChem 97, Canada: Banff, pp. 13-18.
31. Burges, C.J. (1996), "Simplified Support Vector Decision Rules," Proc. International Conference on Machine Learning, pp. 71-77.
32. Chen, S. (2002), "Locally Regularised Orthogonal Least Squares Algorithm for the Construction of Sparse Kernel Regression Models," Proceedings of 6th Int. Cof. Signal Processing, China: Beijing, pp. 1229-1232.
33. Chen, S., Billings, S.A., and Luo, W. (1989), "Orthogonal Least Squares Methods and their Applications to Nonlinear System Identification," International Journal of Control, 50, 1873-1896.
34. Chen, S.S., Donoho, D.L., and Saunders, M.A. (1998), "Atomic Decomposition by Basis Pursuit," SIAM Journal on Scientific Computing, 20(1), 33-61.
35. Chen, S., Hong, X., and Harris, C.J. (2003b), "Sparse Multioutput Radial Basis Function Network Construction using Combined Locally Regularised Orthogonal Least Square and D-optimality Experimental Design," IEE Proceedings Control Theory and Applications, 150(2), 139-146.
36. Chen, S., Hong, X., and Harris, C.J. (2003a), "Sparse Kernel Regression Modelling using Combined Locally Regularised Orthogonal Least Squares and D-optimality Experimental Design," IEEE Transactions on Automatic Control, 48(6), 1029-1036.
37. Chen, S., Hong, X., and Harris, C.J. (2005a), "Orthogonal forward Selection for Constructing the Radial Basis Function Network with Tunable Nodes," Proceedings of 2005 Int. Conf. Intelligent Computing, Hefei, China, pp. 777-786.
38. Chen, S., Wu, Y., and Luk, B.L. (1999), "Combined Genetic Algorithm Optimization and Regularized Orthogonal Least Squares Learning for Radial Basis Function Networks," IEEE Transactions on Neural Networks, 10, 1239-1243.
39. Chen, S., Gibson, G.J., Cowan, C.F., and Grant, P.M. (1991), "Reconstruction of Binary Signals using an Adaptive Radial-basis-function Equalizer," Signal Processing, 20(1), 77-93.
40. Chen, S., Hong, X., Harris, C.J., and Sharkey, P.M. (2004), "Sparse Modelling using Orthogonal forward Regression with PRESS Statistic and Regularizaron," IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, 34(2), 898-911.
41. Chen, S., Hong, X., Wang, X.X., and Harris, C.J. (2005b), "Identification of Nonlinear Systems using Generalised Kernel Models," IEEE Transactions on Control Systems Technology, 13(3), 401-411.
42. Chen, S., Wolfgang, A., Harris, C.J., and Hanzo, L. (2007), "Symmetric Kernel Detector for Multiple-antenna Aided Beamforming Systems," 2007 International Joint Conference on Neural Networks (IJCNN).
43. Chiras, N., Evans, C., and Rees, D. (2001), "Nonlinear Gas Turbine Modelling using Narmax Structures," IEEE Transactions on Instrumentation and Measurement, 50(4), 893-898.
44. Clarkson, T. (1999), "Applications of Neural Networks in Telecommunications," Proceedings of the ERUDIT Workshop on Application of Computational Intelligence Techniques in Telecommunication, London: Imperial College.
45. Cohn, D.A., Ghahramani, Z., and Jordan, M.I. (1996), "Active Learning with Statistical Models," Journal of Artificial Intelligence Research, 4, 129-145.
46. Connally, P., Li, K., and Irwin, G.W. (2005), "Two Applications of Eng-genes Bases Nonlinear Identification," in Proceedings of the 16th IFAC World Congress, Prague.
47. Connally, P., Li, K., and Irwin, G.W. (2007), "Integrated Structure Selection and Parameter Optimisation for Eng-genes Neural Models," Neurocomputing. (in press) doi:10.1016/j.neucom.2007.06.005.
48. Cybenko, G. (1989), "Approximations by Superpositions of Sigmoidal Functions," Mathematics of Control, Signals, and Systems, 2, 303-314.
49. Daubechies, I. (1992), Ten Lectures on Wavelets, Philadelphia, PA: SIAM.
50. Debnath, L., and Mikusinski, P. (1998), Introduction to Hubert Spaces with Applications, San Diego, CA: Academic Press.
51. Donoho, D.L. (1995), "De-noising by Soft-thresholding," IEEE Transactions on Information Theory, 41(3), 613-627.
52. Downs, T., Gates, K., and Masters, A. (2001), "Exact Simplification of Support Vector Solutions," Journal of Machine Learning Research, 2, 293-297.
53. Drezet, P.M.L., and Harrison, R.F. (1998), "Support Vector Machines for System Identification," Proceedings of UKACC International Conference on Control, Vol. 1, UK: Swansea, pp. 688-692.
54. Dybowski, R., and Gant, V. (2001), Clinical Applications of Artificial Neural Networks, Cambridge: Cambridge University Press.
55. Engel, Y., Mannor, S., and Meir, R. (2004), "The Kernel Recursive Least Squares Algorithm," IEEE Transactions on Signal Processing, 52(8), 2275-2285.
56. Espinoza, M., Suykens, J.A.K., and De Moor, B. (2005a), "Imposing Symmetry in Least Squares Support Vector Machines Regression," Proc. Joint 44th IEEE Conf. Decision and Control and European Control Conf. 2005, Seville, Spain, pp. 5716-5721.
57. Espinoza, M., Suykens, J.A.K., and De Moor, B. (2005b), "Kernel Based Partially Linear Models and Nonlinear Identification," IEEE Transactions on Automatic Control, 50(10), 1602-1606.
58. Evgeniou, T., Pontil, M., and Poggio, T. (2000), "Regularization Networks and Support Vector Machine," Advances in Computational Mathematics, 13(1), 1-50.
59. Eykhoff, P. (1974), System Identification - Parameter and State Estimation, Chichester: J. Wiley.
60. Fabri, S.G., and Kadirkamanathan, V. (2001), Functional Adaptive Control: An Intelligent Systems Approach, London, Berlin, Heidelberg: Springer.
61. Faller, W.E., and Schreck, S.J. (1996), "Neural Networks: Applications and Opportunities in Aeronautics," Progress in Aerospace Sciences, 32(5), 433-456.
62. Flood, I., and Kartam, N. (1998), "Artificial Neural Networks for Civil Engineers: Advanced Features and Applications," Handbook of Intelligent Control, Reston, VA: American Society of Civil Engineers.
63. Friedman, J.H. (1991), "Multivariate Adaptive Regression Splines," The Annals of Statistics, 19(1), 1-141.
64. Friedman, J.H., and Stuetzle, W. (1981), "Projection Pursuit Regression," Journal of the American Statistical Association, 76(376), 817-823.
65. Funahashi, K. (1989), "On the Approximate Realization of Continuous Mapping by Neural Networks," Neural Networks, 2, 183-192.
66. Gamero, L.G., Armentano, R.L., Barra, J.G., Simon, A., and Levenson, J. (2001), "Identification of Arterial Wall Dynamics in Conscious Dogs," Experimental Physiology, 86, 519-528.
67. Gao, Y., and Er, M.J. (2003), "Online Adaptive Fuzzy Neural Identification and Control of a Class of Mimo Nonlinear Systems," IEEE Transactions on Fuzzy Systems, 11(4), 462-477.
68. Gao, J.B., Harris, C.J., and Gunn, S.R. (2001), "On a Class of Support Vector Kernels Based on Frames in Function Hubert Spaces," Neural Computations, 13, 1975-1994.
69. Garson, G.D. (1991), "A Comparison of Neural Network and Expert Systems Algorithms with Common Multivariate Procedures for analysis of Social Science Data," Social Science Computer Review, 9(3), 399-434.
70. Ge, S.S., Hang, C.C., Lee, T.H., and Zhang, T. (2001), Stable Adaptive Neural Network Control, Boston, MA: Kluwer Academic.
71. Geman, S., Bienenstock, E., and Doursat, R. (1992), "Neural Networks and the Bias/VAriance Dilemma," Neural Computation, 4, 1-58.
72. Gevers, M. (2005), "Identification for Control: From the Early Achievements to the Revival of Experiment Design," European Journal of Control, 11, 1-18.
73. Girosi, F. (1998), "An Equivalence Between Sparse Approximation and Support Vector Machines," Neural Computation, 10(6), 1455-1480.
74. Girosi, F., and Poggio, T. (1990), "Networks and the Best Approximation Property," Biological Cybernetics, 63, 169-176.
75. Glass, J.W., and Franchek, M.A. (1999), "NaRMAX Modelling and Robust Control of Internal Combustion Engines," International Journal of Control, 72(4), 289-304.
76. Goethals, I., Pelckmans, K., Suykens, J.A.K., and De Moor, B. (2005), "Identification of MIMO Hammerstein Models using Least Squares Support Vector Machines," Automatica, 41, 1263-1272.
77. Goethals, I., Van Gestel, T., Suykens, J., Van Dooren, P., and De Moor, B. (2003), "Identification of Positive Real Models in Subspace Identification by using Regularization," IEEE Transactions on Automatic Control, 48(10), 1843-1847.
78. Goodwin, G.C., and Sin, K.S. (1984), Adaptive Filtering Prediction and Control, Englewood Cliffs, NJ: Prentice Hall.
79. Govindhasamy, J.J., McLoone, S.F., Irwin, G.W., French, J.J., and Doyle, R. (2005), "Neural Modelling, Control and Optimisation of an industrial Grinding Process," Control Engineering Practice, 13(10), 1243-1258.
80. Gray, G.J., Murray-Smith, D.J., Li, Y., and Sharman, K.C. (1996), "Nonlinear Model Structure Identification using Genetic Programming," in Late Breaking Papers at the Genetic Programming 1996 Conference Stanford University July 28-31, 1996, ed. John R. Koza, CA, USA: Stanford Bookstore, Stanford University, pp. 32-37.
81. Greblicki, W. (1989), "Nonparametric Orthogonal Series Identification of Hammerstein Systems," International Journal of Systems Science, 20, 2355-2367.
82. Green, P.J., and Silverman, B.W. (1994), Nonparametric Regression and Generalized Linear Models, London: Chapman and Hall.
83. Harris, C.J., Hong, X., and Gan, Q. (2002), Adaptive Modelling, Estimation and Fusion from Data: A Neurofuzzy Approach, Heidelberg: Springer-Verlag.
84. Harrison, R.F., and Kennedy, R.L. (2005), "Artificial Neural Network Models for Prediction of Acute Coronary Syndromes using Clinical Data from the Time of Presentation," Annals of Emergency Medicine, 46, 431-439.
85. Hastie, T.J., and Tibshirani, R.J. (1996), Generalised Additive Models, London: Chapman and Hall.
86. Hastie, T.J., Tibshirani, R.J., and Friedman, J. (2002), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, New York: Springer.
87. Hoegaerts, L., Suykens, J.A.K., Vandewalle, J., and De Moor, B. (2005), "Subset Based Least Squares Subspace Regression in Rkhs," Neurocomputing, 63, 293-323.
88. Hoerl, A.E., and Kennard, R.W. (1970), "Ridge Regression: Biased Estimation for Nonorthogonal Problems," Technometrics, 12(1), 55-67.
89. Hong, W.C. (2006), "The Application of Support Vector Machines to forecast tourist Arrivals in Barbados: An Empirical Study," International Journal of Management, 23(2), 375-385.
90. Hong, X., and Chen, S. (2005), "M-estimator and D-Optimality Model Construction using Orthogonal Forward Regression," IEEE Transactions on System, Man, and Cybernetics, Part B, 35(1), 155-162.
91. Hong, X., and Harris, C.J. (2001a), "Neurofuzzy Design and Model Construction of Nonlinear Dynamical Processes from Data," IEE Proceedings Control Theory and Applications, 148(6), 530-538.
92. Hong, X., and Harris, C.J. (2001b), "Variable Selection Algorithm for the Construction of Mimo Operating Point Dependent Neurofuzzy Network," IEEE Transactions on Fuzzy Systems, 9(1), 88-101.
93. Hong, X., and Harris, C.J. (2003), "Experimental Design and Model Construction Algorithms for Radial Basis Function Networks," International Journal of Systems Science, 34(14-15), 733-745.
94. Hong, X., and Mitchell, R.J. (2007), "Backward Elimination Model Construction for Regression and Classification using Leave one Out Criteria," International Journal of Systems Science, 38(2), 101-113.
95. Hong, X., Harris, C.J., Brown, M., and Chen, S. (2004), "Backward Elimination Methods for Associative Memory Network Pruning," International Journal of Hybrid Intelligent Systems, 1(1-2), 90-98.
96. Hong, X., Sharkey, P.M., and Warwick, K. (2003), "Automatic Nonlinear Predictive Model Construction using forward Regression and the PRESS Statistic," IEE Proceedings and Control Theory Applications, 150(3), 245-254.
97. Hornik, K., Stinchcombe, M., and White, H. (1989), "Multilayer Feedforward Networks are Universal Approximator," Neural Networks, 2, 359-366.
98. Huang, G.B., Saratchandran, P., and Sundararaja, N. (2004a), "An Efficient Sequential Learning Algorithm for Growing and Pruning RBF (GAP-RBF)networks," IEEE Transactions on Systems, Man and Cybernetics, Part B, 34(6), 2284-2292.
99. Huang, G.B., Zhu, Q.Y., and Siew, C.K. (2004b), "Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks," Proceedings of International Joint Conference on Neural Networks (IJCNN2004), Vol. 2, Hungary: Budapest, pp. 985-990.
100. Huang, S.N., Tan, K.K., and Lee, T.H. (2006), "Nonlinear Adaptive Control of interconnected Systems using Neural Networks," IEEE Transactions on Neural Network, 17(1), 243-246.
101. Huber, P.J. (1981), Robust Statistics, New Jersey: J. Wiley.
102. Iplikci, S. (2006), "Support Vector Machines-based Generalized Predictive Control," International Journal of Robust and Non-Linear Control, 16, 834-862.
103. Irwin, G.W., Warwick, K., and Hunt, K.J. (1995), Neural Network Applications in Control, London: IEE.
104. James, W., and Stein, C. (1961), "Estimation with Quadratic Loss," Proceedings Fourth Berkeley Symp. Math. Statist. Prob., 1, 311-319.
105. Juditsky, A., Hjalmarsson, H., Benveniste, A., Delyon, B., Ljung, L., Sjöberg, J., and Zhang, Q. (1995), "Nonlinear Black-box Modelling in System Identification: Mathematical Foundations," Automatica, 31(12), 1725-1752.
106. Jurado, F. (2004), "Experience with Non-linear Model Identification for Fuel Cell Plants," Fuel Cells, 5, 105-114.
107. Jurado, F. (2005), "Non-linear Modeling of Micro-turbines using NARX Structures on the Distribution Feeder," Energy Conversion and Management, 46(3), 385-401.
108. Kadirkamanathan, V., and Niranjan, M. (1993), "A Function Estimation Approach to Sequential Learning with Neural Networks," Neural Computation, 5(6), 954-975.
109. Kalkkuhl, J., Hunt, K.J., and Fritz, H. (1999), "Fern-based Neural-network Approach to Nonlinear Modeling with Application to Longitudinal Vehicle Dynamics Control," IEEE Transactions on Neural Networks, 10(4), 885-897.
110. Kanevski, M., Parkin, R., Pozdnukhov, A., Timonin, V., Maignan, M., Yatsalo, B., and Canu, S. (2004), "Environmental Data Mining and Modelling Based on Machine Learning Algorithms and Geostatistics," Journal of Environmental Modelling and Software, 19, 845-855.
111. Karayiannis, N.B., Mukherjee, A., Glover, J.R., Ktonas, P.Y., Frost, Jr, J.D., Hrachovy, R.A., and Mizrahi, E.M. (2006), "Detection of Pseudosinusoidal Epileptic Seizure Segments in the Neonatal Eeg by Cascading a Rule-based Algorithm with a Neural Network," IEEE Transactions on Biomedical Engineering, 53(4), 633-641.
112. Kavli, T. (1993), "AsMOD - an Algorithm for Adaptive Spline Modelling of Observation Data," International Journal of Control, 58(4), 947-967.
113. Khan, M.S., and Coulibaly, P. (2006), "Application of Support Vector Machine in Lake Water Level Prediction," Journal of Hydrologic Engineering, 11(3), 199-205.
114. Knight, K., and Fu, W.J. (2000), "Asymptotics for Lasso-type Estimators," Annals of Statistics, 28, 1356-1378.
115. Korenberg, M.J. (1988), "Identifying Nonlinear Difference Equation and Functional Expansion Representations: The Fast Orthogonal Algorithm," Annals of Biomedical Engineering, 16, 123-142.
116. Kwak, N. (2002), "Input Feature Selection by Mutual Information Based on Parzen Window," IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(12), 1667-1671.
117. Lee, K.L., and Billings, S.A. (2002), "Time Series Prediction using Support Vector Machines, the Orthogonal, and the Regularised Orthogonal Least Squares Algorithms," International Journal of Systems Science, 33, 811-821.
118. Leontaritis, I.J., and Billings, S.A. (1985), "Input-output Parametric Models for Nonlinear Systems - Part 1: Deterministic Nonlinear Systems; Part 2: Stochastic Nonlinear Systems," International Journal of Control, 4.1(1), 303-344.
119. Leung, H., Lo, T., and Wang, S. (2001), "Prediction of Noisy Chaotic Time Series using an Optimal Radial Basis Function Neural Network," IEEE Transactions on Neural Networks, 12(5), 1163-1172.
120. Leva, A., and Piroddi, L. (2002), "Narx-based Technique for the Modelling of Magneto-rheological Damping Devices," Smart Materials Structures, 11, 79-88.
121. Lewis, F.L., and Parisini, T. (1998), "Guest Editorial: Neural Network Feedback with Guaranteed Stability," International Journal of Control, 70(3), 337-339.
122. Li, K. (2005), "Eng-genes: A New Genetic Modelling Approach for Nonlinear Dynamic Systems," in Proceedings of the 16th IFAC World Congress, Prague.
123. Li, K., and Peng, J.X. (2006), "System Oriented Neural Networks - Problem formulation, Methodology and Application," International Journal of Pattern Recognition and Artificial Intelligence, 20(2), 143-158.
124. Li, K., Peng, J., and Bai, E.W. (2006), "A Two-stage Algorithm for Identification of Nonlinear Dynamic Systems," Automatica, 42(7), 1189-1197.
125. Li, K., Peng, J.X., and Irwin, G.W. (2005), "A Fast Nonlinear Model Identification Method," IEEE Transactions on Automatic Control, 50(8), 1211-1216.
126. Li, K., Thompson, S., and Peng, J. (2004), "Modelling and Prediction of NOx Emission in a Coal-fired Power Generation Plant," Control Engineering Practice, 12, 707-723.
127. Lin, B., Lin, B., Chong, F., and Lai, F. (2007), Higher-order- Statistics-based Radial basis Function Networks for Signal Enhancement, 18(3), 823-832.
128. Liu, G.P. (2001), Nonlinear Identification and Control-A Neural Network Approach, London: Springer.
129. Liu, G.P., Kadirkamanathan, V., and Billings, S.A. (1999), "Variable Neural Networks for Adaptive Control of Nonliear Systems," IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, 29(1), 34-43.
130. Ljung, L. (1987), System Identification: Theory for the User, New Jersey: Prentice Hall.
131. Ljung, L., and Vicino, A. (2005), "Special Issue on Identification," IEEE Transactions on Automatic Control, 50(10), 1477-1634.
132. Ljung, L., and Söderström, T. (1983), Theory and Practice of Recersive Identification, Cambridge, MA: The MIT Press.
133. Lorito, F. (1999), "Identification of a Grey-box Model of Nonlinear Current Transformers for Simulation Purposes," Control Engineering Practice, 6(11), 1331-1339.
134. Luh, G.C., and Cheng, W.C. (2004), "Identification of Immune Models for Fault Detection," Proceedings Instn Mech. Engrs Part I:J Systems and Control Engineering, 218, 353-367.
135. Luo, W., Morris, A.J., Karim, M.N., Martin, E.B., and Hong, T. (1996), "Online Identification of a Ph Waste Water Neutralisation Process using Time-varying Nonlinear Arx Models," in IEEE International Conference on Systems, Man, and Cybernetics, 1996, Vol. 1. 14-17 Oct., pp. 107-112.
136. Ma, Q., and Wang, J. (2000), "Application of Bayesian Neural Networks to Protein Sequence Classification," in ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 305-309.
137. MacKay, D.J.C. (1991), Bayesian Methods for Adaptive Models. PhD thesis, USA: California Institute of Technology.
138. Mackay, D.J. (1997), "Gaussian Processes: A Replacement for Supervised Neural Networks," Lecture notes at NIPS 1997, http://www.ra.phy.cam.ac.uk/mackay
139. Mallat, S.G. (1989), "A theory for Multiresolutional Signal Decomposition: The Wavelet Representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 674-693.
140. Mallat, S.G., and Zhang, Z. (1993), "Matching Pursuits with Time-frequency Dictionaries," IEEE Transactions on Signal Processing, 41(12), 3397-3415.
141. Mandic, D.P., and Chambers, J.A. (2001), Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures, and Stability, Chichester: J Wiley.
142. Mao, K.Z., and Billings, S.A. (1999), "Variable Selection in Nonlinear Systems Modelling," Mechanical Systems and Signal Processing, 13(2), 351-366.
143. Mao, K.Z., and Billings, S.A. (2000), "Multi-directional Model Validity Tests for Nonlinear System Identification," International Journal of Control, 73, 132-143.
144. Mao, K.Z., Billings, S.A., and Zhu, Q.M. (1999), "A Regularised Least Squares Algorithm for Nonlinear Rational Model Identification," International Journal of Systems Science, 30, 455-465.
145. Markovsky, I., Willems, J.C., Rapisarda, P., and de Moor, B.L.M. (2005), "Algorithms for Deterministic Balanced Subspace Identification," Automatica, 41, 755-766.
146. Marose, R.A. (1990), "A Financial Neural-network Application," AI Expert, 5(5), 50-53.
147. Marquardt, D.W. (1970), "Generalised inverse, Ridge Regression, Biased Linear Estimation and Nonlinear Estimation," Technometrics, 12(3), 591-612.
148. McGinnity, S., and Irwin, G.W. (1996), "Nonlinear State Estimation using Fuzzy Local Linear Models," International Journal of Systems Science, 28(7), 643-656.
149. Mercer, J. (1909), "Functions of Positive and Negative Type and their Connection with the Theory of Integral Equations," Philos. Transactions on Roy. Soc. London, A 209, 415-446.
150. Miller, A.J. (1990), Subset Selection in Regression, Boca Raton, Florida: Chapman and Hall/CRC.
151. Minoux, M. (1986), Mathematical Programming: Theory and Algorithms, New York: Wiley.
152. Mitra, S., and Hayashi, Y. (2006), "Bioinformatics with Soft Computing," IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 36(5), 616-635.
153. Moody, J. (1994), "Prediction Risk and Architecture Selection for Neural Networks," in From Statistics to Neural Networks: Theory and Pattern Recognition Applications, ed. V. Cherkassky, Berlin, New York: Springer-Verlag.
154. Müller, K.R., Mika, S., Ratsch, G., Tsuda, K., and Schölkopf, B. (2001), "An introduction to Kernel Based Learning Algorithms," IEEE Transactions on Neural Networks, 12(2), 181-201.
155. Murray-Smith, R. (1994), "A Local Model Network Approach to Nonlinear Modelling," PhD thesis., Dept. of Computer Science, Glasgow: University of Strathclyde.
156. Murray-Smith, R., and Johansen, T.A. (1997), Multiple Model Approaches to Modelling and Control, London: Taylor and Francis.
157. Myers, R.H., and Montgomery, D.C. (1995), Response Surface Methodology: Process and Product Design using Designed Experimetns, New York: Wiley.
158. Neal, R.M. (1996), Bayesian Learning for Neural Networks, New York: Springer-Verlag.
159. Nishii, R. (1984), "Asymptotic Properties of Criteria for selection of Variables in Mutiple Regression," The Annals of Statistics, 12(2), 758-765.
160. Nunnari, G., Dorling, S., Schlink, U., Cawled, G., Foxall, R., and Chatterton, T. (2004), "Modelling SO2 Concentration at a Point with Statistical Approaches," Environmental Modelling & Software, 19(10), 887-905.
161. Orr, M.J.L. (1995), "Regularisation in the Selection of Radial Basis Function Centers," Neural Computation, 7(3), 954-975.
162. Park, D.C., El-Sharkawi, M.A., and Marks, R.J. II (1991), "An Adaptively Trained Neural Network," IEEE Transactions on Neural Networks, 2(3), 334-345.
163. Parlitz, U., Hornstein, A., Engster, D., Al-Bender, F., Lampaert, V., Tjahjowidodo, T., Fassois, S.D., Rizos, D., Wong, C.X., Worden, K., and Manso, G. (2004), "Identification of Pre-sliding Friction Dynamics," Chaos, 14(2), 420-430.
164. Pelckmans, K., Suykens, J.A.K., and De Moor, B. (2005), Neurocomputing, 64, 137-159.
165. Peng, H., Ozaki, T., Toyodac, Y., Shioyad, H., Nakanoe, K., Haggan-Ozakif, V., and Mori, M. (2004), "Modelling and Prediction of NOx Emission in a Coal-fired Power Generation Plant," Control Engineering Practice, 12, 191-203.
166. Peng, J., Li, K., and Huang, D.S. (2006), "A Hybrid forward Algorithm for RBF Neural Network Construction," IEEE Transactions on Neural Networks, 17(6), 1439-1451.
167. Piatt, J. (1991), "A Resource Allocating Network for Function interpolation," Neural Computation, 3(2), 213-225.
168. Platt, J. (1999), "Fast Training of Support Vector Machines using Sequential Minimal Optimization," in Advances in Kernel Methods - Support Vector Learning, eds. C.J.C. Burges, B. Schölkopf, and A.J. Smola, Cambridge, MA: MIT Press, pp. 185-208.
169. Plutowski, M., and White, H. (1993), "Selecting Concise Training Sets from Clean Data," IEEE Transactions on Neural Netwoks, 4, 305-318.
170. Powell, M.J.D. (1985), "Radial Basis Functions for Multivariable interpolation: A Review," in Algorithms for Approximation, eds. J.C. Mansonn, and M.G. Cox, Oxford: Oxford University Press, pp. 143-167.
171. Priestley, M.B. (1981), Spectral Analysis and Time Series, London, New York: Academic Press.
172. Raghavan, H., Gopaluni, R.B., Shah, S., Pakpahan, J., Patwardhan, R., and Robson, C. (2005), "Gray-box Identification of Dynamic Models for the Bleaching Operation in a Pulp Mill," Journal of Process Control, 15(4), 451-468.
173. Rao, G.P. (2006), "Identification of Continuous-time Systems," IEE Proceedings - Control Theory Applications, 153(2), 185-220.
174. Ribeiro, B. (2005), "Support Vector Machines for Quality Monitoring in a Plastic Injection Molding Process," IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 35(3), 401-410.
175. Rojas, C.R., Welsh, J.S., Goodwin, G.C., and Feuer, A. (2007), "Robust Optimal Experiment Design for System Identification," Automatica, 43, 993-1008.
176. Ruano, A.E. (2005), Intelligent Control Systems using Computational Intelligence Techniques, London: IEE Publishing.
177. Sanner, R., and Slotine, J. (1992), "Gaussian Networks for Direct Adaptive Control," IEEE Transactions on Neural Networks, 3(6), 837-863.
178. Sargantanis, I.G., and Karim, M.N. (2004), "Variable Structure NARX Models: Application to Dissolved-Oxygen Bioprocess," AIChE Journal, 45(9), 2034-2045.
179. Shevade, S.K., Keerthi, S.S., Bhattacharyya, C., and Murthy, K.R.K. (2000), "Improvements to the SMO Algortihms for SVM Regression," IEEE Transactions on Neural Networks, 11(5), 1188-1193.
180. Sindelář, R., and Babuška, R. (2004), "Input Selection for Nonlinear Regression Models," IEEE Transactions on Fuzzy Systems, 12(5), 688-696.
181. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P., Hjalmarsson, H., and Juditsky, A. (1995), "Nonlinear Black-box Modelling in System Identification: A Unified Overview," Automatica, 31(12), 1691-1724.
182. Smola, A.J. (1998), Learning with Kernels. Ph.D. thesis, Germany: Informatik der Technischen Universitate Berlin.
183. Smola, A.J., and Schölkopf, B. (1998), "A Tutorial on Support Vector Regression," NeuroCOLT Technical Report NC-TR-98-030, London: Royal Holloway College, University of. http://www.kernel-machines.org/tutorial. html
184. Söderström, T. (2006), "Errors-in-variables Methods in System Identification," Proc. 14th IFAC Symposium on System Identification, Australia: Newcastle.
185. Söderström, T., and Stoica, P. (1989), System Identification, New York: Prentice Hall.
186. Söderström, T., and Stoica, P. (1990), "On Covariance Function Tests Used in System Identification," Automatica, 26, 125-133.
187. Söderström, T., Van den Hof, P., Wahlberg, B., and Weiland, S. (2005), "Special Issue on Data-based Modelling and System Identification," Automatica, 41(3), 357-562.
188. Soumelidis, M.I., and Stobart, R.K. (2006), "Dynamic Modelling of Three-way Catalysts using Non-linear Identification Techniques," Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 220(7), 595-605.
189. Stewart, P., Fleming, P.J., and MacKenzie, S.A. (2003), "Real-time Simulation and Control Design by the Response Surface Methodology and Design of Experiments," International Journal of Systems Science, 34(14-15), 837-850.
190. Stone, M. (1974), "Cross Validatory Choie and Assessment of Statistical Predictions," Journal of the Royal Statistical Society, Series B, 36, 117-147.
191. Sugiyama, M., and Ogawa, H. (2001), "Incremental Projection Learning for Optimal Generalization," Neural Networks, 14(1), 67-78.
192. Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., and Vandewalle, J. (2002), Least Squares Support Vector Machines, Singapore: World Scientific.
193. Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso," Journal of Royal Statistical Society. Series B, 58(1), 267-288.
194. Tothill, R.W., Kowalczyk, A., Rischin, D., Bousioutas, A., Haviv, I., van Laar, R.K., Waring, P.M., Zalcberg, J., Ward, R., Biankin, A.V., Sutherland, R.L., Henshall, S.M., Fong, K., Pollack, J.R., Bowtell, D.D.L., and Holloway, A.J. (2005), "An Expression-based Site of Origin Diagnostic Method Designed for Clinical Application to Cancer of Unknown Origin," Cancer Research, 65(10), 4031-4040.
195. Townsend, S., and Irwin, G.W. (2001), "Nonlinear Model Based Predictive Control using Multiple Local Models," in Nonlinear Predictive Control: Theory and Practice, eds. B. Kouvaritakis, and M. Cannon, London: IEE, Control Engineering. Book Series 61.
196. Tsang, K.M., and Chan, W.L. (2005), "Adaptive Control of Power Factor Correction Converter using Nonliear System. Identification," IEE Proceedings Electric Power Applications, 152(3), 627-633.
197. Vanderbei, R.J. (1999), "LOQO: An Interior Point Code for Quadratic Programming," Optimization Methods and Software, 11(1-4), 451-484.
198. Vapnik, V. (1995), The Nature of Statictical Learning Theory, New York: Springer-Verlag.
199. Vapnik, V. (1998), Statistical Learning Theory, New York: J. Wiley.
200. Vemuri, A., Polycarpou, M., and Diakourtis, S. (1998), "Neural Network Based Fault Detection and Accommodation in Robotic Manipulators," IEEE Transactions on Robotics and Automation, 14(2), 342-348.
201. Wahba, G. (1990), "Spline Models for Observational Data," Society for Industrial and Applied Mathematics.
202. Wang, L., and Mendel, J.M. (1992), "Fuzzy Basis Functions, Universal Approximation, and Orthogonal Least-squares Learning," IEEE Transactions on Neural Networks, 5, 807-814.
203. Wilson, D.J.H., Irwin, G.W., and Lightbody, G. (1999), "RbF Principal Surfaces for Process Monitoring," IEEE Trans on Neural Networks, 10(6), 1424-1434.
204. Xia, X., and Li, K. (2007), "A New Score Correlation analysis Multi-class Support Vector Machine for Microarray," in 2007 International Joint Conference on Neural Networks, Florida, Orlendo.
205. Xie, N., and Leung, H. (2005), "Blind Equalization using a Predictive Radial Basis Function Neural Network," IEEE Transactions on Neural Networks, 16(3), 709-720.
206. Yingwei, L., Sundararajan, N., and Saratchandran, P. (1998), "Performance Evaluation of a Sequential Minimal Radial Basisfunction (RBF) Neural Network Learning Algorithm," IEEE Transactions on Neural Networks, 9(2), 308-318.
207. Young, P.C. (1984), Recursive Estimation and Time Series Analysis, Berlin: Springer-Verlag.
208. Zhang, Q. (1993), "Using Wavelets Network in Nonparametric Estimation," IEEE Transactions on Neural Networks, 8(2), 1997.
209. Zhang, L.F., Zhu, Q.M., and Longden, A. (2007), "A Set of Novel Correlation Tests for Nonlinear System Variables," International Journal of Systems Science, 38(1), 47-60.
210. Zheng, G.L., and Billings, S.A. (1996), "Radial Basis Function Network Configuration using Mutual Information and the Orthogonal Least Squares Algorithm," Neural Networks, 9, 1619-1637.
211. Zheng, G.L., and Billings, S.A. (1999), "Qualitative Validation and Generalization in Non-linear System Identification," International Journal of Control, 72(17), 1592-1608.
212. Zheng, G.L., and Billings, S.A. (2002), "Effects of Overparameterisation in Nonlinear System Identification and Neural Networks," International Journal of Systems Science, 33(5), 331-349.
213. Zhong, Z.D., Zhu, X.J., and Cao, G.Y. (2006), "Modeling a PEMFC by a Support Vector Machine," Journal of Power Sources, 160(1), 293-298.