Many Hands Make Light Work - On Ensemble Learning Techniques for Data Fusion in Remote Sensing

IEEE Geoscience and Remote Sensing Magazine

Volumn 3, Issue 3, 2015, Pages 86-99

  Merentitis, Andreas ^a   Debes, Christian ^a  

^a AGT Group (RandD) GmbH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DATA FUSION; DECISION TREES; FORESTRY; REMOTE SENSING;

BIAS-VARIANCE TRADEOFFS; ENSEMBLE LEARNING; ENSEMBLE METHODS; LEARNING TECHNIQUES; MACHINE-LEARNING; PROPERTY; RANDOM FORESTS; REMOTE-SENSING; RESEARCH FIELDS; STATE OF THE ART;

LEARNING SYSTEMS;

ALGORITHM; DATA ASSIMILATION; DATA SET; ENSEMBLE FORECASTING; MACHINE LEARNING; REGRESSION ANALYSIS; REMOTE SENSING; RESEARCH WORK;

EID: 84943797969     PISSN: 24732397     EISSN: 21686831     Source Type: Journal    
DOI: 10.1109/MGRS.2015.2432092     Document Type: Article

References (62)

1. R. Polikar, "Ensemble based systems in decision making, " IEEE Circuits Syst. Mag., vol. 6, no. 3, pp. 21-45, 2006.
2. D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study, " J. Artif. Intell. Res., vol. 11, pp. 169-198, 1999.
3. A. Criminisi, J. Shotton, and E. Konukoglu, "Decision forests: A unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning, " Found. Trends Comput. Graph. Vis., vol. 7, nos. 2-3, pp. 81-227, Feb. 2012.
4. A. Verikas, A. Gelzinis, and M. Bacauskiene, "Mining data with random forests: A survey and results of new tests, " Pattern Recognit., vol. 44, no. 2, pp. 330-349, 2011.
5. R. M. Bell and Y. Koren, "Lessons from the Netflix Prize challenge, " ACM SIGKDD Explor. Newsletter, vol. 9, no. 2, pp. 75-79, 2007.
6. R. M. Bell, Y. Koren, and C. Volinsky, "All together now: A perspective on the Netflix Price, " Chance, vol. 23, no. 1, pp. 24-29, 2010.
7. M. Fernández-Delgado, E. Cernadas, S. Barro, and D. Amorim, "Do we need hundreds of classifiers to solve real world classification problems?" J. Mach. Learn. Res., vol. 15, pp. 3133-3181, Oct. 2014.
8. D. Wolpert, "Stacked generalization, " Neural Netw., vol. 5, no. 2, pp. 241-259, 1992.
9. A. Merentitis, C. Debes, R. Heremans, and N. Frangiadakis, "Automatic fusion and classification of hyperspectral and LiDAR data using random forests, " in Proc. IEEE Int. Geosci. Remote Sensing Symp., 2014, pp. 1245-1248.
10. L. Breiman, "Random forests, " Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
11. C. Debes, A. Merentitis, R. Heremans, J. Hahn, N. Frangiadakis, T. van Kasteren, W. Liao, R. Bellens, A. Pizurica, S. Gautama, W. Philips, S. Prasad, Q. Du, and F. Pacifici, "Hyperspectral and Li-DAR data fusion: Outcome of the 2013 GRSS Data Fusion Contest, " IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, vol. 7, pp. 2405-2418, June 2014.
12. T. Kavzoglu and I. Colkesen, "An assessment of the effectiveness of a rotation forest ensemble for land-use and land-cover mapping, " Int. J. Remote Sens., vol. 34, no. 12, pp. 4224-4241, 2013.
13. J. Xia, P. Du, X. He, and J. Chanussot, "Hyperspectral remote sensing image classification based on rotation forest, " IEEE Geosci. Remote Sensing Lett., vol. 11, no. 1, pp. 239-243, Jan. 2014.
14. A. Samat, P. Du, M. H. A. Baig, S. Chakravarty, and L. Cheng, "Ensemble learning with multiple classifiers and polarimetric features for polarized SAR image classification, " Photogramm. Eng. Remote Sens., vol. 80, no. 3, pp. 239-251, 2014.
15. J. H. Friedman, "Greedy function approximation: A gradient boosting machine, " Ann. Stat., vol. 29, no. 5, pp. 1189-1232, 2000.
16. L. Breiman, "Bagging predictors, " Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996.
17. R. T. Clemen, "Combining forecasts: A review and annotated bibliography, " Int. J. Forecasting, vol. 5, no. 4, pp. 559-583, 1989.
18. L. K. Hansen and P. Salamon, "Neural network ensembles, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 10, pp. 993-1001, 1990.
19. R. E. Schapire, "The strength of weak learnability, " Mach. Learn., vol. 5, no. 2, pp. 197-227, 1990.
20. B. Efron and B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans, vol. 38. Philadelphia, PA: SIAM, 1982.
21. A. M. Zoubir and D. R. Iskander, Bootstrap Techniques for Signal Processing. Cambridge, U. K. : Cambridge Univ. Press, 2004.
22. Y. Freund and R. E. Schapire, "A desicion-theoretic generalization of on-line learning and an application to boosting, " in Computational Learning Theory. Berlin, Germany: Springer, 1995, pp. 23-37.
23. J. Oh, M. Laubach, and A. Luczak, "Estimating neuronal variable importance with random forest, " in Proc. IEEE 29th Annu. Bioengineering Conf., Mar. 2003, pp. 33-34.
24. J. Li, H. Liu, and L. Li, "Diagnostic rules induced by an ensemble method for childhood leukemia, " in Proc. 5th IEEE Symp. Bioinformatics Bioengineering, Oct. 2005, pp. 246-249.
25. L. Vibha, G. M. Harshavardhan, K. Pranaw, P. D. Shenoy, K. R. Venugopal, and L. M. Patnaik, "Statistical classification of mammograms using random forest classifier, " in Proc. 4th Int. Conf. Intelligent Sensing Information Processing, Oct. 2006, pp. 178-183.
26. V. Svetnik, A. Liaw, C. Tong, J. Culberson, R. Sheridan, and B. Feuston, "Random forest: A classification and regression tool for compound classification and QSAR modeling, " J. Chem. Inform. Comput. Sci., vol. 43, no. 6, pp. 1947-1958, 2003.
27. A. Bosch, A. Zisserman, and X. Muoz, "Image classification using random forests and ferns, " in Proc. IEEE 11th Int. Conf. Computer Vision, Oct. 2007, pp. 1-8.
28. A. Z. Kouzani, S. Nahavandi, and K. Khoshmanesh, "Face classification by a random forest, " in Proc. TENCON IEEE Region 10 Conf., Oct. 2007, pp. 1-4.
29. S. Bernard, S. Adam, and L. Heutte, "Using random forests for handwritten digit recognition, " in Proc. 9th Int. Conf. Document Analysis Recognition, Sept. 2007, vol. 2, pp. 1043-1047.
30. W. Yan, "Application of random forest to aircraft engine fault diagnosis, " in Proc. IMACS Multiconference Computational Engineering Systems Applications, Oct. 2006, vol. 1, pp. 468-475.
31. S. Boukir, L. Guo, and N. Chehata, "Classification of remote sensing data using margin-based ensemble methods, " in Proc. 20th IEEE Int. Conf. Image Processing, Sept. 2013, pp. 2602-2606.
32. P. Zhongm and R. Wang, "A multiple conditional random fields ensemble model for urban area detection in remote sensing optical images, " IEEE Trans. Geosci. Remote Sensing, vol. 45, no. 12, pp. 3978-3988, Dec. 2007.
33. J. A. Benediktsson, J. Chanussot, and M. Fauvel, "Multiple classifier systems in remote sensing: From basics to recent developments, " in Multiple Classifier Systems (Lecture Notes in Computer Science, vol. 4472), M. Haindl, J. Kittler, and F. Roli, Eds. Berlin, Germany: Springer-Verlag, 2007, pp. 501-512.
34. M. Pal, "Random forests for land cover classification, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., July 2003, vol. 6, pp. 3510-3512.
35. S. R. Joelsson, J. A. Benediktsson, and J. R. Sveinsson, "Random forest classifiers for hyperspectral data, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., July 2005, vol. 1, p. 4.
36. J. Ham, Y. Chen, M. M. Crawford, and J. Ghosh, "Investigation of the random forest framework for classification of hyperspectral data, " IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 3, pp. 492-501, 2005.
37. P. O. Gislason, J. A. Benediktsson, and J. R. Sveinsson, "Random forest classification of multisource remote sensing and geographic data, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., Sept. 2004, vol. 2, pp. 1049-1052.
38. A. Merentitis, C. Debes, and R. Heremans, "Application of ensemble learning in hyperspectral image classification: Towards selecting favorable spots in the bias-variance plane, " IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, Apr. 2014, vol. 7, pp. 1089-1102.
39. J. Chan and D. Paelinckx, "Evaluation of random forest and adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery, " Remote Sens. Environ., vol. 112, no. 6, pp. 2999-3011, 2008.
40. J. Friedman, "On bias, variance, 0/1-loss, and the curse-of-dimensionality, " Data Mining Knowl. Discovery, vol. 1, no. 1, pp. 55-77, 1997.
41. R. Kohavi and D. H. Wolpert, "Bias plus variance decomposition for zero-one loss functions, " in Proc. 13th Int. Conf. Machine Learning, 1996, pp. 275-283.
42. L. Breiman, "Bias, variance, and arcing classifiers, " Stat. Dept., Univ. California, Oakland, CA, Tech. Rep. 460, 1996.
43. P. Domingos, "A unified bias-variance decomposition and its applications, " in Proc. 17th Int. Conf. Machine Learning, 2000, pp. 231-238.
44. G. M. James, "Variance and bias for general loss functions, " Mach. Learn., vol. 51, no. 2, pp. 115-135, 2003.
45. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Diego, CA: Academic Press, 1990.
46. P. Bartlett, R. E. Schapire, Y. Freund, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods, " in Proc. 14th Int. Conf. Machine Learning, San Francisco, CA, 1997.
47. J.-M. Poggi, R. Genuer, and C. Tuleau, "Random forests: Some methodological insights, " arXiv:0811. 3619, SSN 0249-6399, 2008.
48. Y. Freund, and R. Shapire, "Experiments with a new boosting algorithm, " in Proc. 13th Int. Conf. Machine Learning, L. Saitta, Ed., San Francisco, CA, 1996, pp. 148-156.
49. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U. K. : Cambridge Univ. Press, 2004.
50. P. Geurts, D. Ernst, and L. Wehenkel, "Extremely randomized trees, " Machine Learning, vol. 63, no. 1, pp. 3-42, 2006.
51. M. Dalponte, L. Bruzzone, and D. Gianelle, "Fusion of hyperspectral and LiDAR remote sensing data for classification of complex forest areas, " IEEE Trans. Geosci. Remote Sensing, vol. 46, no. 5, pp. 1416-1427, 2008.
52. A. F. Elakshe, "Fusion of hyperspectral images and lidar-based dems for coastal mapping, " Opt. Lasers Eng., vol. 46, no. 7, pp. 493-498, 2008.
53. A. Swatantrana, R. Dubayaha, D. Robertsb, M. Hoftona, and J. B. Blairc, "Mapping biomass and stress in the Sierra Nevada using lidar and hyperspectral data fusion, " Remote Sens. Environ., vol. 115, no. 11, pp. 2917-2930, 2011.
54. S. Kumar, J. Ghosh, and M. M. Crawford, "Hierarchical fusion of multiple classifiers for hyperspectral data analysis, " Pattern Anal. Applicat., vol. 5, no. 2, pp. 210-220, 2002.
55. S. Nourzad and A. Pradhan, "Binary and multi-class classification of fused lidar-imagery data using an ensemble method, " in Proc. Construction Research Congr., 2012, pp. 909-918.
56. G. Moser and S. B. Serpico, "Combining support vector machines and Markov random fields in an integrated framework for contextual image classification, " IEEE Trans. Geosci. Remote Sensing, vol. 51, no. 5, pp. 2734-2752, 2013.
57. A. A. Nielsen, L. Gomez-Chova, and G. Camps-Valls, "Explicit signal to noise ratio in reproducing kernel Hilbert spaces, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., 2011, pp. 3570-3573.
58. A. A. Nielsen, "Kernel maximum autocorrelation factor and minimum noise fraction transformations, " IEEE Trans. Image Processing, vol. 20, no. 3, pp. 612-624, 2011.
59. J. Besag, "On the statistical analysis of dirty pictures, " J. Roy. Stat. Soc., vol. 48, no. 3, pp. 259-302, 1986.
60. J. Preciozzi, P. Muse, A. Almansa, S. Durand, F. Cabot, Y. Kerr, A. Khazaal, and B. Rouge, "Sparsity-based restoration of SMOS images in the presence of outliers, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., July 2012, pp. 3501-3504.
61. F. Fierens and P. L. Rosin, "Filtering remote sensing data in the spatial and feature domains, " in Proc. SPIE Image Signal Processing Remote Sensing, 1994, pp. 472-482.
62. J. Ham, Y. Chen, M. M. Crawford, and J. Ghosh, "Investigation of the random forest framework for classification of hyperspectral data, " IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 3, pp. 492-501, 2005.