A methodology for training set instance selection using mutual information in time series prediction

Neurocomputing

Volumn 141, Issue , 2014, Pages 236-245

  Stojanović, Miloš B ^a,c   Božić, Miloš M ^b   Stanković, Milena M ^b   Stajić, Zoran P ^b  

^a 23 Dorda Stratimirovica   (Serbia)

^b UNIVERSITY OF NI   (Serbia)

^c Bulevar doktora Zorana indića 29/5 ^*  (Serbia)

Author keywords

Instance selection; Mutual information; Time series prediction

Indexed keywords

ARTIFICIAL INTELLIGENCE; TIME SERIES;

ERROR PROPAGATION; INSTANCE SELECTION; LONG-TERM TIME SERIES PREDICTIONS; MACHINE LEARNING PROBLEM; MUTUAL INFORMATIONS; PRE-PROCESSING STEP; RECURSIVE PREDICTION; TIME SERIES PREDICTION;

FORECASTING;

ARTICLE; CONTROLLED STUDY; EXPERIMENTAL STUDY; FORECASTING; INFORMATION; INFORMATION PROCESSING; LEARNING ALGORITHM; LEAST SQUARE SUPPORT VECTOR MACHINE; MATHEMATICAL COMPUTING; METHODOLOGY; MUTUAL INFORMATION; PREDICTION; PREDICTIVE VALUE; PRIORITY JOURNAL; QUALITY CONTROL; RECURSIVE PREDICTION; STATISTICAL MODEL; SUPPORT VECTOR MACHINE; TIME SERIES ANALYSIS; TRAINING SET INSTANCE SELECTION; UNCERTAINTY; VALIDATION PROCESS;

EID: 84901603905     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.03.006     Document Type: Article

References (57)

1. Kim K. Financial time series forecasting using support vector machines. Neurocomputing 2003, 55:307-319.
2. Bermolen P., Rossi D. Support vector regression for link load prediction. Comput. Netw. 2009, 53:191-201.
3. Amjady N. Short-term hourly load forecasting using time-series modeling with peak load estimation capability. IEEE Trans. Power Syst. 2001, 16:798-805.
4. Papalexopoulos A.D., Hesterberg T.C. A regression-based approach to short-term system load forecasting. IEEE Trans. Power Syst. 1990, 5:1535-1547.
5. Box G.E.P., Jenkins G. Time Series Analysis, Forecasting and Control 1976, Holden-Day, San Francisco.
6. Holt C.C. Forecasting seasonals and trends by exponentially weighted moving averages. Int. J. Forecast. 2004, 20:5-10.
7. Zhang G., Patuwo B.E., Hu M.Y. Forecasting with artificial neural networks: The state of the art. Int. J. Forecast. 1998, 14:35-62.
8. Sapankevych N.I., Sankar R. Time series prediction using support vector machines: a survey. IEEE Comput. Intell. Mag. 2009, 4:24-38.
9. Nguyen H.H., Chan C.W. Multiple neural networks for a long term time series forecast. Neural Comput. Appl. 2004, 13:90-98.
10. Kourentzes N., Crone S.F. Forecasting high-frequency time series with neural networks-an analysis of modelling challenges from increasing data frequency. Int. Conf. Data Min. 2008.
11. S.F. Crone, N. Kourentzes, Input-variable specification for neural networks-an analysis of forecasting low and high time series frequency, in: Proceedings of International Joint Conference on Neural Networks, 2009, pp. 619-626.
12. Zhang J., Yim Y., Yang J. Intelligent selection of instances for prediction functions in lazy learning algorithms. Artif. Intell. Rev. 1997, 11:175-191.
13. Tolvi J. Genetic algorithms for outlier detection and variable selection in linear regression models. Soft Comput. 2004, 8:527-533.
14. Guillen A., Herrera L.J., Rubio G., Pomares H., Lendasse A., Rojas I. New method for instance or prototype selection using mutual information in time series prediction. Neurocomputing 2010, 73:2030-2038.
15. Candela J.Q., Sugiyama M., Schwaighofer A., Lawrence N.D. Dataset Shift in Machine Learning 2009, The MIT Press, Cambridge, MA.
16. Storkey A.J., Sugiyama M. Mixture Regression for Covariate Shift. Adv. Neural Inf. Process. Syst. 2007, 19:1337-1344.
17. Torres J.G.M., Raeder T., Rodriguez R.A., Chawla N.V., Herrera F. A unifying view on dataset shift in classification. Pattern Recogn. 2012, 45:521-530.
18. Cohn D.A., Ghahramani Z., Jordan M.I. Active learning with statistical models. J. Artif. Intell. Res. 1996, 4:129-145.
19. V.S. Iyengar, C. Apte, T. Zhang, Active learning using adaptive resampling, in: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2000 pp. 91-98.
20. Ludl M.C., Lewandowski A. Adaptive machine learning in delayed feedback domains by selective relearning. Appl. Artif. Intell. 2008, 22:543-557.
21. Shimodaira H. Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plann. Inference 2000, 90:227-244.
22. Aha D.W. Tolerating noisy, irrelevant and novel attributes in instance-based learning algorithms. Int. J. Man-Mach. Stud. 1992, 36:267-287.
23. Wilson D.R., Martinez T. Reduction techniques for instance based learning algorithms. Mach. Learn. 2000, 38:257-286.
24. Guo Y., Schuurmans D. Discriminative batch mode active learning. Adv. Neural Inf. Process. Syst. (NIPS) 2008, 593-600.
25. H. Ishibuchi, T. Nakashima, M. Nii, Learning of neural networks with GA-based instance selection, in: Proceedings of the IFSA World Congress and 20th NAFIPS International Conference 4, 2001 pp. 2102-2107.
26. Sebban M., Nock R., Lallich S. Stopping criterion for boosting-based data reduction techniques: from binary to multiclass problems. J. Mach. Learn. Res. 2002, 3:863-865.
27. V.B. Zubek, T.G. Dietterich, Pruning improves heuristic search for cost-sensitive learning, in: Proceedings of the International Conference on Machine Learning, 2002, pp. 27-34.
28. Yu1 K., Xu1 X., Ester M., Kriegel H.P. Feature weighting and instance selection for collaborative filtering: an information-theoretic approach. Knowl. Inf. Syst. 2003, 5:201-224.
29. Lopez J.A.O., Ochoa J.A.C., Trinidad J.F.M., Kittler J. A review of instance selection methods. Artif. Intell. Rev. 2010, 34:133-143.
30. N. Jankowski, M. Grochowski, Comparison of instances seletion algorithms I, in: Proceedings of the Artificial Intelligence and Soft Computing-ICAISC 2004, 2004, pp. 598-603.
31. Zhang J., Yim Y.S., Yang J. Intelligent selection of instances for prediction functions in lazy learning algorithms. Artif. Intell. Rev. 1997, 11:175-191.
32. Z. Huang, M.L. Shyu, k-NN based LS-SVM framework for long-term time series prediction, in: Proceedings of the 2010 IEEE International Conference on Information Reuse and Integration, 2010, pp. 69-74.
33. Shyu M.L., Huang Z. Long-term time series prediction using k-NN based LS-SVM framework with multi-value integration. Recent Trends in Information Reuse and Integration 2012, 191-209. Springer.
34. A. Guillén, L. Herrera, G. Rubio, Instance or prototype selection for function approximation using mutual information, in: Proceedings of the European Symposium on Time Series Prediction-ESTSP 2008, 2008, pp. 67-75.
35. Stojanović M., Božić M., Stanković M., Stajić Z. Adaptive least squares support vector machines method for short-term load forecasting based on mutual information for inputs selection. Int. Rev. Electr. Eng. 2012, 7:3574-3585.
36. Batina L., Gierlichs B., Prouff E., Rivain M., Standaert F.X., Charvillon N.V. Mutual information analysis: a comprehensive study. J. Cryptol. 2010, 24:269-291.
37. Sorjamaa A., Hao J., Reyhani N., Ji Y., Lendasse A. Methodology for long-term prediction of time series. Neurocomputing 2007, 70:2861-2869.
38. Rossi F., Lendasse A., Francois D., Wertz V., Verleysen M. Mutual information for the selection of relevant variables in spectrometric nonlinear modeling. Chemom. Intell. Lab. Syst. 2006, 80:215-226.
39. L.J. Herrera, H. Pomares, I. Rojas, M. Verleysen, A. Guillen, Effective input variable selection for function approximation, in: Proceedings of the International Conference on Artificial Neural Networks-ICANN 2006, 2006, pp. 41-50.
40. Doquire G., Verleysen M. Feature selection with missing data using mutual information estimators. Neurocomputing 2012, 90:3-11.
41. Moddemeijer R. On estimation of entropy and mutual information of the continuous distributions. Signal Process. 1989, 16:233-248.
42. Moon Y., Rajagopalan B., Lall U. Estimation of mutual information using kernel density estimators. Phys. Rev. E 1995, 52:2318-2321.
43. Kraskov A., Stgbauer H., Grassberger P. Estimating mutual information. Phys. Rev. E 2004, 69:066138.
44. http://www.klab.caltech.edu/~kraskov/MILCA/.
45. Stogbauer H., Kraskov A., Astakhov S.A., Grassberger P. Least dependent component analysis based on mutual information. Phys. Rev. E 2004, 70:066123.
46. François D., Rossi F., Wertz V., Verleysen M. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing 2007, 70:1276-1288.
47. Herrera L.J., Pomares H., Rojas I., Guillen A., Prieto A., Valenzuela O. Recursive prediction for long term time series forecasting using advanced models. Neurocomputing 2007, 70:2870-2880.
48. Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., Vandewalle J. Least Squares Support Vector Machines 2002, World Scientific, Singapore.
49. Arlot S., Celisse A. A survey of cross-validation procedures for model selection. Stat. Surv. 2010, 4:40-79.
50. http://www.neural-forecasting-competition.com/NN5/datasets.htm.
51. Taieb S.B., Bontempi G., Atiya A., Sorjamaa A. review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition. Expert Syst. Appl. 2012, 39:7067-7083.
52. Tashman L.J. Out-of-sample tests of forecasting accuracy: an analysis and review. Int. J. Forecast. 2000, 16:437-450.
53. Crone S.F., Kourentzes N. Feature selection for time series prediction-a combined filter and wrapper approach for neural networks. Neurocomputing 2010, 73:1923-1936.
54. http://www.esat.kuleuven.be/sista/lssvmlab/.
55. Dešmar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.
56. Garcia S., Herrera F. An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons. J. Mach. Learn. Res. 2008, 9:2677-2694.
57. http://www.neural-forecasting-competition.com/NN5/results.htm.