Comparison of statistical and machine learning methods in modelling of data with multicollinearity

International Journal of Modelling, Identification and Control

Volumn 18, Issue 4, 2013, Pages 295-312

  Garg, Akhil ^a   Tai, Kang ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

ANN; Artificial neural network; Factor analysis; Genetic programming; Machine learning; Multicollinearity; PCA; Principal component analysis; Regression; Statistics

Indexed keywords

FACTOR ANALYSIS; FUZZY LOGIC; FUZZY NEURAL NETWORKS; GENETIC ALGORITHMS; GENETIC PROGRAMMING; LEARNING SYSTEMS; LEAST SQUARES APPROXIMATIONS; MACHINE LEARNING; MULTIVARIANT ANALYSIS; NEURAL NETWORKS; RADIAL BASIS FUNCTION NETWORKS; REGRESSION ANALYSIS; STATISTICS;

MACHINE LEARNING METHODS; MULTICOLLINEARITY; PARTIAL LEAST SQUARE (PLS); PRINCIPAL COMPONENT REGRESSION; RADIAL BASIS FUNCTIONS; REGRESSION; STEPWISE REGRESSION; VARIABLE REDUCTION METHOD;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84876995897     PISSN: 17466172     EISSN: 17466180     Source Type: Journal    
DOI: 10.1504/IJMIC.2013.053535     Document Type: Article

References (43)

1. Affenzeller, M. and Winkler, S. (2009) Genetic Algorithms and Genetic Programming: Modern Concepts and Practical Applications, Chapman & Hall/CRC, Boca Raton, Florida.
2. Baziar, M.H., Jafarian, Y., Shahnazari, H., Movhed, V. and Amin Tutunchian, M. (2011) 'Prediction of strain energy-based liquefaction resistance of sand-silt mixtures: An evolutionary approach', Computers & Geosciences, Vol. 37, No. 11, pp.1883-1893.
3. Bendjama, H., Boucherit, M.S., Bouhouche, S. and Bast, J. (2011) 'Fault detection and diagnosis using principal component analysis. Application to low pressure lost foam casting process', International Journal of Modelling, Identification and Control, Vol. 14, Nos. 1/2, pp.102-111.
4. Bradley, E., Easley, M. and Stolle, R. (2001) 'Reasoning about nonlinear system identification', Artificial Intelligence, Vol. 133, No. 2, pp.139-188.
5. Buckinx, W., Verstrateten, G. and Van Den Poel, D. (2007) 'Predicting customer loyalty using the internal transactional database', Expert Systems with Applications, Vol. 32, No. 1, pp.125-134.
6. Buyukbingol, E., Sisman, A., Akyildiz, M., Alpaslan, F.N. and Adejare, A. (2007) 'Adaptive neuro-fuzzy inference system (ANFIS): A new approach to predictive modeling in QSAR applications: A study of neuro-fuzzy modeling of PCP-based NMDA receptor antagonists', Bioorganic & Medicinal Chemistry, Vol. 15, No. 12, pp.4265-4282.
7. Can, B. and Heavey, C. (2011) 'Comparison of experimental designs for simulation-based symbolic regression of manufacturing systems', Computers & Industrial Engineering, Vol. 61, No. 3, pp.447-462.
8. Castillo, F., Kordon, A. and Smits, G. (2007) 'Robust Pareto front genetic programming parameter selection based on design of experiments and industrial data', in Genetic Programming Theory and Practice IV, Vol. 5, pp.149-166, New York.
9. Castillo, F., Kordon, A., Smits, G., Christenson, B. and Dickerson, D. (2006) 'Pareto front genetic programming parameter selection based on design of experiments and industrial data', GECCO '06 Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, ACM, pp.1613-1620.
10. Chetouani, Y. (2010) 'Use of a neural-network-based approach for a reliable modelling of a distillation column', International Journal of Modelling, Identification and Control, Vol. 11, Nos. 1/2, pp.71-78.
11. Chong, I.G. and Jun, C.H. (2005) 'Performance of some variable selection methods when multicollinearity is present', Chemometrics and Intelligent Laboratory Systems, Vol. 78, No. 2, pp.103-112.
12. Daszykowski, M., Serneels, S., Kaczmarek, K., Van Espen, P., Croux, C. and Walczak, B. (2007) 'TOMCAT: A MATLAB toolbox for multivariate calibration techniques', Chemometrics and Intelligent Laboratory Systems, Vol. 85, No. 2, pp.269-277.
13. Dorugade, A. and Kashid, D. (2010) 'Alternative method for choosing ridge parameter for regression', Applied Mathematical Sciences, Vol. 4, No. 9, pp. 447-456.
14. Feldt, R. and Nordin, P. (2000) 'Using factorial experiments to evaluate the effect of genetic programming parameters', Genetic Programming, Proceedings of EuroGP'2000, Vol. 1802, pp.271-282.
15. Gou, Z. and Fyfe, C. (2004) 'A canonical correlation neural network for multicollinearity and functional data', Neural Networks, Vol. 17, No. 2, pp.285-293.
16. Hiden, H.G. (1998) Data-based Modelling using Genetic Programming, PhD thesis, Department of Chemical and Process Engineering, University of Newcastle, UK.
17. Hinchliffe, M., Hiden, H., Mckay, B., Willis, M., Tham, M. and Barton, G. (1996) 'Modelling chemical process systems using a multi-gene genetic programming algorithm', Late Breaking Paper, GP'96, Stanford, USA, pp.28-31.
18. Hoerl, R.W., Schuenemeyer, J.H. and Hoerl, A.E. (1986) 'A simulation of biased estimation and subset selection regression techniques', Technometrics, Vol. 28, No. 4, pp.369-380.
19. Iba, H. and Sasaki, T. (1996) 'Using genetic programming to predict financial data', Proceedings of Congress on Evolutionary Computation (CEC), IEEE. Vol. 1.pp.244-251.
20. Kankar, P.K., Sharma, S.C. and Harsha, S.P. (2012) 'Vibration-based fault diagnosis of a rotor bearing system using artificial neural network and support vector machine', International Journal of Modelling, Identification and Control, Vol. 15, No. 3, pp.185-198.
21. Kashid, D. and Kulkarni, S. (2002) 'A more general criterion for subset selection in multiple linear regression', Communications in Statistics-Theory and Methods, Vol. 31, No. 5, pp.795-811.
22. Kordon, A., Castillo, F., Smits, G. and Kotancheck, M. (2006) 'Application issues of genetic programming in industry', in Genetic Programming Theory and Practice III, Vol. 9, pp.241-258, NY, USA.
23. Li, J.G. and Geng, T. (2010) 'Tumor classification based on partial least square regression', International Conference on Biomedical engineering and Computer Science (ICBECS), IEEE, pp.1-6.
24. Lu, M. and Xue, H. (2005) '24-hour electrical load predictions by the complementary use of PLS and neural network', Proceedings of Intelligent Systems and Control, ACTA Press, Vol. 497.
25. Ma, C.Y., Buontempo, F.V. and Wang, X.Z. (2011) 'Inductive data mining: Automatic generation of decision trees from data for QSAR modelling and process historical data analysis', International Journal of Modelling, Identification and Control, Vol. 12, Nos. 1/2, pp.101-106.
26. Manly, B.F.J. (2005) Multivariate Statistical Methods: A Primer, Chapman & Hall/CRC, Boca Raton, Florida.
27. Meng, F., Wu, X., Ding, X. and Zhen, R. (2012) 'The research of study methods for pattern recognition', International Journal of Modelling, Identification and Control, Vol. 16, No. 3, pp.229-233.
28. Montgomery, D.C., Peck, E.A., Vining, G.G. and Vining, J. (2001) Introduction to Linear Regression Analysis, Wiley, New York.
29. Myers, R.H., Montgomery, D.C. and Anderson-Cook, C.M. (2009) Response Surface Methodology: Process and Product Optimization Using Designed Experiments, John Wiley & Sons Inc., Hoboken, New Jersey.
30. Neter, J., Wasserman, W., Kutner, M.H. and Li, W. (1996) Applied Linear Statistical Models, Irwin, Columbus, Ohio.
31. Paris, G., Robilliard, D. and Fonlupt, C. (2002) 'Applying boosting techniques to genetic programming', Lecture Notes in Computer Science, 2002, Vol. 2310/2002, pp.889-918.
32. Park, T.C., Kim, U.S., Kim, L.H., Jo, B.W. and Yeo, Y.K. (2010) 'Heat consumption forecasting using partial least squares, artificial neural network and support vector regression techniques in district heating systems', Korean Journal of Chemical Engineering, Vol. 27, No. 4, pp.1063-1071.
33. Ping, J., Bronson, C., Zartman, K. and Dobermann, R. (2004) 'Identification of relationships between cotton yield, quality, and soil properties', Agronomy Journal, Vol. 96, No. 6, pp.1588-1597.
34. Sall, J., Creighton, L. and Lehman, A. (2005) JMP Start Statistics: A Guide to Statistics and Data Analysis using JMP and JMP IN Software, Sas Inst., Cary, North Carolina.
35. Saxen, H., Pettersson, F. and Gunturu, K. (2007) 'Evolving nonlinear time-series models of the hot metal silicon content in the blast furnace', Materials and Manufacturing Processes, Vol. 22, No. 5, pp.577-584.
36. Searson, D.P., Leahy, D.E. and Willis, M.J. (2010) 'GPTIPS: An open source genetic programming toolbox for multigene symbolic regression', International MultiConference of Engineers and Computer Scientists 2010.
37. Serneels, S., Croux, C., Filzmoser, P. and Van Espen, P.J. (2005) 'Partial robust M-regression', Chemometrics and Intelligent Laboratory Systems, Vol. 79, No. 2, pp.55-64.
38. Singh, K.P., Ojha, P., Malik, A. and Jain, G. (2009) 'Partial least squares and artificial neural networks modeling for predicting chlorophenol removal from aqueous solution', Chemometrics and Intelligent Laboratory Systems, Vol. 99, No. 2, pp.150-160.
39. Vignon, M. (2011) 'Inference in morphological taxonomy using collinear data and small sample sizes: Monogenean sclerites (Platyhelminthes) as a case study', Zoologica Scripta, Vol. 40, No. 3, pp.306-316.
40. Vohland, M. and Emmerling, C. (2011) 'Determination of total soil organic C and hot water-extractable C from VIS-NIR soil reflectance with partial least squares regression and spectral feature selection techniques', European Journal of Soil Science, Vol. 62, No. 4, pp.598-606.
41. Wang, B., Yao, M. and Zhu, R. (2011) 'Gene expression programming with multiple chromosomes', International Journal of Modelling, Identification and Control, Vol. 14, No. 4, pp.235-241.
42. Wang, R., Wang, C., Guo, Z. and Tang, J. (2008) 'Forecast of railway freight ton-kilometers based on semi-parametric regression', Journal of Southwest Jiaotong University, Vol. 43, No. 1, pp.96-100.
43. Xu, J., Zhang, M. and Wang, Y. (2010) 'Neural networks modelling and generalised predictive control for an autonomous underwater vehicle', International Journal of Modelling, Identification and Control, Vol. 11, Nos. 1/2, pp.79-86.