A survey on multi-output regression

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery

Volumn 5, Issue 5, 2015, Pages 216-233

  Borchani, Hanen ^a   Varando, Gherardo ^b   Bielza, Concha ^b   Larrañaga, Pedro ^b  

^a AALBORG UNIVERSITY   (Denmark)

^b UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

OPEN SYSTEMS; PETROLEUM RESERVOIR EVALUATION; REGRESSION ANALYSIS; SURVEYS;

ADAPTATION METHODS; EVALUATION MEASURES; MULTI-OUTPUT; ON STATE; PROBLEM TRANSFORMATIONS; REAL-WORLD PROBLEM; REGRESSION METHOD;

OPEN SOURCE SOFTWARE;

EID: 84939243215     PISSN: 19424787     EISSN: 19424795     Source Type: Journal    
DOI: 10.1002/widm.1157     Document Type: Review

References (74)

1. Aho T, Ženko B, Džeroski S, Elomaa T. Multi-target regression with rule ensembles. J Mach Learn Res 2009, 373:2055-2066.
2. Appice A, Džeroski S. Stepwise induction of multi-target model trees. In: Proceedings of the Eighteenth European Conference on Machine Learning, Warsaw, Poland, 2007, 502-509. Springer Verlag.
3. Kocev D, Džeroski S, White MD, Newell GR, Griffioen P. Using single- and multi-target regression trees and ensembles to model a compound index of vegetation condition. Ecol Model 2009, 220:1159-1168.
4. Spyromitros-Xioufis E, Groves W, Tsoumakas G, Vlahavas I. Multi-label classification methods for multi-target regression, arXiv preprint arXiv:1211.6581, 2012, 1159-1168. Cornell University Library.
5. Tsoumakas G, Spyromitros-Xioufis E, Vrekou A, and Vlahavas I. Multi-target regression via random linear target combinations. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Nancy, France, 2014, 225-240. Springer Verlag.
6. Breiman L, Friedman JH. Predicting multivariate responses in multiple linear regression. J R Stat Soc Series B Stat Methodol 1997, 59:3-54.
7. Brown PJ, Zidek JV. Adaptive multivariate ridge regression. Ann Stat 1980, 8:64-74.
8. Haitovsky Y. On multivariate ridge regression. Biometrika 1987, 74:563-570.
9. Micchelli CA, Pontil M. On learning vector-valued functions. Neural Comput 2005, 17:177-204.
10. Similä T, Tikka J. Input selection and shrinkage in multiresponse linear regression. Comput Stat Data Anal 2007, 52:406-422.
11. Madjarov G, Kocevb D, Gjorgjevikja D, Džeroskib S. An extensive experimental comparison of methods for multi-label learning. Pattern Recogn 2012, 45:705-727.
12. Tsoumakas G, Katakis I. Multi-label classification: an overview. Int J Data Warehouse Min 2007, 3:1-13.
13. Zhang M, Zhou Z. A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 2014, 26:1819-1837.
14. Bielza C, Li G, Larrañaga P. Multi-dimensional classification with Bayesian networks. Int J Approx Reason 2011, 52:705-727.
15. Burnham AJ, MacGregor JF, Viveros R. Latent variable multivariate regression modeling. Chemometr Intell Lab 1999, 48:167-180.
16. Kuznar D, Mozina M, BratkoI I. Curve prediction with kernel regression. In: Proceedings of the ECML/PKDD 2009 Workshop on Learning from Multi-Label Data, Bled, Slovenia, 2009, 61-68.
17. Han Z, Liu Y, Zhao J, Wang W. Real time prediction for converter gas tank levels based on multi-output least square support vector regressor. Control Eng Pract 2012, 20:1400-1409.
18. Tuia D, Verrelst J, Alonso L, Pérez-Cruz F, Camps-Valls G. Multioutput support vector regression for remote sensing biophysical parameter estimation. IEEE Geosci Remote Sens Lett 2011, 8:804-808.
19. Sánchez-Fernández M, de-Prado-Cumplido M, Arenas-Garca J, Pérez-Cruz F. SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems. IEEE Trans Signal Process 2004, 52:2298-2307.
20. Baxter J. A Bayesian/information theoretic model of learning to learn via multiple task sampling. Mach Learn 1997, 28:7-39.
21. Caruana R. Multitask learning. Mach Learn 1997, 28:41-75.
22. Ben-David S, Schuller R. Exploiting task relatedness for multiple task learning. In: Proceedings of the Sixteenth Annual Conference on Learning Theory, Washington, DC, 2003, 567-580. Springer Verlag.
23. Jalali A, Sanghavi S, Ruan C, Ravikumar PK. A dirty model for multi-task learning. In: Proceedings of the Advances in Neural Information Processing Systems 23, Vancouver, Canada, 2010, 964-972.
24. Marquand AF, Williams SCR, Doyle OM, Rosa MJ. Full Bayesian multi-task learning for multi-output brain decoding and accommodating missing data. In: Proceedings of the 2014 International Workshop on Pattern Recognition in Neuroimaging, Tübingen, Germany, 2014, 1-4. IEEE Press.
25. Hoerl AE, Kennard RW. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 1970, 12:55-67.
26. Drucker H, Burges CJC, Kaufman L, Smola A, Vapnik V. Support vector regression machines. In: Proceedings of the Advances in Neural Information Processing Systems 9, Denver, CO, 1997, 155-161.
27. Breiman L. Bagging predictors. Mach Learn 1997, 24:123-140.
28. Friedman JH. Stochastic gradient boosting. Comput Stat Data Anal 2002, 38:367-378.
29. Godbole S, Sarawagi S. Discriminative methods for multi-labeled classification. In: Proceedings of the Eighth Pacific-Asia Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 2004, 22-30. Springer Verlag.
30. Wolpert DH. Stacked generalization. Neural Netw 1992, 5:241-259.
31. Read J, Pfahringer B, Holmes G, Frank E. Classifier chains for multi-label classification. Mach Learn 2011, 85:333-359.
32. Zhang W, Liu X, Ding Y, Shi D. Multi-output LS-SVR machine in extended feature space. In: Proceedings of the 2012 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications, Tianjin, China, 2012, 130-134.
33. Izenman AJ. Reduced-rank regression for the multivariate linear model. J Multivar Anal 1975, 5:248-264.
34. van der Merwe A, Zidek JV. Multivariate regression analysis and canonical variates. Can J Stat 1980, 8:27-39.
35. Abraham Z, Tan P, Perdinan P, Winkler J, Zhong S, Liszewska M. Position preserving multi-output prediction. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Prague, Czech Republic, 2013, 320-335. Springer Verlag.
36. Vazquez E, Walter E. Multi-output support vector regression. In: Proceedings of the Thirteen IFAC Symposium on System Identification, Rotterdam, The Netherlands, 2003, 1820-1825.
37. Chilès JP, Delfiner P. Geostatistics: Modeling Spatial Uncertainty. Wiley Series in Probability and Statistics. Wiley; 1999.
38. Brudnak M. Vector-valued support vector regression. In: Proceedings of the 2006 International Joint Conference on Neural Networks, Vancouver, Canada, 2006, 1562-1569. IEEE Press.
39. Deger F, Mansouri A, Pedersen M, Hardeberg JY. Multi- and single-output support vector regression for spectral reflectance recovery. In: Proceedings of the Eighth International Conference on Signal Image Technology and Internet Based Systems, Sorrento, Italy, 2012, 139-148. IEEE Press.
40. Cai F, Cherkassky V. SVM+ regression and multi-task learning. In: Proceedings of the 2009 International Joint Conference on Neural Networks, Atlanta, GA, 2009, 418-424. IEEE Press.
41. Liu G, Lin Z, Yu Y. Multi-output regression on the output manifold. Pattern Recogn 2009, 42:2737-2743.
42. Kennedy J, Eberhart R. Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, Perth, Western Australia, 1995, 2-8. IEEE Press.
43. Xu S, An X, Qiao X, Zhu L, Li L. Multi-output least-squares support vector regression machines. Pattern Recogn Lett 2013, 34:1078-1084.
44. Baldassarre L, Rosasco L, Barla A, Verri A. Multi-output learning via spectral filtering. Mach Learn 2012, 87:259-301.
45. Evgeniou T, Pontil M. Regularized multi-task learning. In: Proceedings of the 2004 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, 2004, 109-117. ACM Press.
46. Evgeniou T, Micchelli CA, Pontil M. Learning multiple tasks with kernel methods. J Mach Learn Res 2005, 6:615-637.
47. Álvarez MA, Rosasco L, Lawrence ND. Kernels for vector-valued functions: a review. Found Trends Mach Learn 2012, 4:195-266.
48. Struyf J, Džeroski S. Constraint based induction of multi-objective regression trees. In: Proceedings of the Fifth International Workshop on Knowledge Discovery in Inductive Databases, Berlin, Germany, 2006, 222-233. Springer Verlag.
49. De'ath G. Multivariate regression trees: a new technique for modeling species-environment relationships. Ecology 2002, 83:1105-1117.
50. Breiman L, Friedman JH, Stone CJ, Olshen RA. Classification and Regression Trees. Chapman & Hall/CRC; 1984.
51. Malerba D, Esposito F, Ceci M, Appice A, Pontil M. Top-down induction of model trees with regression and splitting nodes. IEEE Trans Pattern Anal Mach Intell 2004, 26:612-625.
52. Kocev D, Vens C, Struyf J, Džeroski S. Tree ensembles for predicting structured outputs. Pattern Recogn 2012, 46:817-833.
53. Breiman L. Random forests. Mach Learn 2001, 45:5-32.
54. Ikonomovska E, Gama J, Džeroski S. Incremental multi-target model trees for data streams. In: Proceedings of the 2011 ACM Symposium on Applied Computing, Taichung, Taiwan, 2011, 988-993. ACM.
55. Stojanova D, Ceci M, Appice A, Džeroski S. Network regression with predictive clustering trees. Data Min Knowl Disc 2012, 25:378-413.
56. Appice A, Malerba D. Leveraging the power of local spatial autocorrelation in geophysical interpolative clustering. Data Min Knowl Disc 2014, 28:1266-1313.
57. Levatić J, Ceci M, Kocev D, Džeroski S. Semi-supervised learning for multi-target regression. In: Proceedings of the Third International Workshop on New Frontiers In Mining Complex Patterns, Nancy, France, 2014, 110-123. Springer Verlag.
58. Aho T, Ženko B, Džeroski S. Rule ensembles for multi-target regression. In: Proceedings of the Ninth IEEE International Conference on Data Mining, Miami, FL; 2009, 21-30. IEEE Press.
59. Breiman L. Statistical modeling: the two cultures. Stat Sci 2001, 16:199-231.
60. Bache K, Lichman M. UCI Machine Learning Repository. University of California, School of Information and Computer Sciences, Irvine, CA, 2013. Available at: http://archive.ics.uci.edu/ml. (Accessed May 10, 2015).
61. Džeroski S, Demšar D, Grbović J. Predicting chemical parameters of river water quality from bioindicator data. Appl Intell 2000, 13:7-17.
62. Karalic A, Bratko I. First order regression. Mach Learn 1997, 26:147-176.
63. Stojanova D, Panov P, Gjorgjioski V, Kobler A, Džeroski S. Estimating vegetation height and canopy cover from remotely sensed data with machine learning. Ecol Inform 2010, 5:256-266.
64. Demšar D, Džeroski S, Larsen T, Struyfc J, Axelsenb J, Pedersenb MB, Kroghb PH. Using multi-objective classification to model communities of soil microarthropods. Ecol Model 2006, 191:131-143.
65. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I. Mulan: a Java library for multi-label learning. J Mach Learn Res 2011, 12:2411-2414.
66. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH. The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 2009, 11:10-18.
67. Read J, Reutemann P. MEKA: a multi-label extension to WEKA. Available at: http://meka.sourceforge.net/. (Accessed May 10, 2015).
68. Bifet A, Holmes G, Kirkby R, Pfahringer B. MOA: massive online analysis. J Mach Learn Res 2010, 11:1601-1604.
69. Reutemann P, Vanschoren J. Scientific workflow management with ADAMS. In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Bristol, UK, 2012, 833-837. Springer Verlag.
70. R Core Team. R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, 2012. Available at: http://www.R-project.org/. (Accessed May 10, 2015).
71. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010, 33:1-22.
72. De'ath G. mvpart: multivariate partitioning. Available at: http://CRAN.R-project.org/package=mvpart. (Accessed May 10, 2015).
73. Therneau T, Atkinson B, Ripley B. rpart: recursive partitioning and regression trees, R package version 4.1-9. Available at: http://CRAN.R-project.org/package=rpart. (Accessed May 10, 2015).
74. Declarative Languages and Artificial Intelligence Group (Katholieke Universiteit Leuven) and Department of Knowledge Technologies (Jožef Stefan Institute). CLUS system. Available at: https://dtai.cs.kuleuven.be/clus/. (Accessed May 10, 2015).