Ensemble learning in hyperspectral image classification: Toward selecting a favorable bias-variance tradeoff

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Volumn 7, Issue 4, 2014, Pages 1089-1102

  Merentitis, Andreas ^a   Debes, Christian ^a   Heremans, Roel ^a  

^a AGT Group (RandD) GmbH   (Germany)

Author keywords

Bagging; bias variance; classification; ensemble methods; hyperspectral image (HIS); random forest; segmentation

Indexed keywords

CHAINS; CLASSIFICATION (OF INFORMATION); DECISION TREES; FEATURE EXTRACTION; IMAGE SEGMENTATION; INDEPENDENT COMPONENT ANALYSIS; OPTIMIZATION; SPECTROSCOPY;

BAGGING; BIAS-VARIANCE; ENSEMBLE METHODS; HYPER-SPECTRAL IMAGES; RANDOM FORESTS;

IMAGE CLASSIFICATION;

ACCURACY ASSESSMENT; ALGORITHM; DATA SET; ENSEMBLE FORECASTING; IMAGE CLASSIFICATION; MULTISPECTRAL IMAGE; OPTIMIZATION; SEGMENTATION;

EID: 84899943917     PISSN: 19391404     EISSN: 21511535     Source Type: Journal    
DOI: 10.1109/JSTARS.2013.2295513     Document Type: Article

References (91)

1. C.-C. Liu, "Processing of FORMOSAT-2 daily revisit imagery for site surveillance," IEEE Trans. Geosci. Remote Sens., vol. 44, no. 11, pp. 3206-3214, Nov. 2006. (Pubitemid 44711663)
2. C. Ren and C.-I. Chang, "Automatic spectral target recognition in hyperspectral imagery," IEEE Trans. Aerosp. Electron. Syst., vol. 39, no. 4, pp. 1232-1249, Oct. 2003.
3. A. Banerjee, P. Burlina, and J. Broadwater, "Hyperspectral video for illumination-invariant tracking," in Proc. 1st Workshop Hyperspectral Image Signal Process., 2009, pp. 1-4.
4. S. Matteoli, M. Diani, and G. Corsini, "A tutorial overview of anomaly detection in hyperspectral images," IEEE Aerosp. Electron. Syst. Mag., vol. 25, no. 7, pp. 5-28, Jul. 2010.
5. P. S. Thenkabail, J. G. Lyon, and A. Huete, Eds., Hyperspectral Remote Sensing of Vegetation, Boca Raton, FL, USA: CRC Press, 2011.
6. D. Haboudane, J. R. Miller, E. Pattey, P.-J. Zarco-Tejada, and I. B. Strachan, "Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: Modeling and validation in the context of precision agriculture," Remote Sens. Environ., vol. 90, no. 3, pp. 337-352, 2004. (Pubitemid 38431251)
7. E. Hirsch and E. Agassi, "Detection of gaseous plumes in IR hyperspectral images, performance analysis," IEEE Sensors J., vol. 10, no. 3, pp. 732-736, Mar. 2010.
8. V. Farley, M. Chamberland, P. Lagueux, A. Vallires, A. Villemaire, and J. Giroux, "Chemical agent detection and identification with a hyperspectral imaging infrared sensor," in Proc. SPIE 6661 Imag. Spectrom. XII, San Diego, CA, USA, 2007.
9. R. J. Ellis and P. W. Scott, "Evaluation of hyperspectral remote sensing as a means of environmental monitoring in the St. Austell China clay (kaolin) region, Cornwall, UK," Remote Sens. Environ., vol. 93, no. 1-2, pp. 118-130, 2004. (Pubitemid 39239282)
10. S. Serranti, A. Gargiulo, and G. Bonifazi, "Dried fruits quality assessment by hyperspectral imaging," in Proc. SPIE 8369 Sens. Agri. Food Qual. Saf. IV, Baltimore, MD, USA, 2012.
11. A. A. Gowen, C. P. O'Donnell, P. J. Cullen, G. Downey, and J. M. Frias, "Hyperspectral imaging-An emerging process analytical tool for food quality and safety control," Trends Food Sci. Technol., vol. 18, no. 12, pp. 590-598, 2007. (Pubitemid 350017801)
12. S. Serranti, A. Gargiulo, and G. Bonifazi, "Characterization of postconsumer polyolefin wastes by hyperspectral imaging for quality control in recycling processes," Waste Manag., vol. 31, no. 11, pp. 2217-2227, 2011.
13. S. Serranti and G. Bonifazi, "Hyperspectral imaging based recognition procedures in particulate solid waste recycling," World Rev. Sci. Technol. Sustain. Develop., vol. 7, no. 3, pp. 271-281, 2010.
14. J. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N. Nasrabadi, and J. Chanussot, "Hyperspectral remote sensing data analysis and future challenges," IEEE Geosci. Remote Sens. Mag., vol. 1, no. 2, pp. 6-36, Jun. 2013.
15. J. Chanussot, M. M. Crawford, and B.-C. Kuo, "Special issue on hyperspectral image and signal processing," IEEE Trans. Geosci. Remote Sens., vol. 48, no. 11, pp. 3871-3876, Nov. 2010.
16. M. Fauvel, Y. Tarabalka, J. Benediktsson, J. Chanussot, and J. Tilton, "Advances in spectral-spatial classification of hyperspectral images," Proc. IEEE, vol. 101, no. 3, pp. 652-675, Mar. 2013.
17. D. G. Goodenough, H. Chen, A. Dyk, A. Richardson, and G. Hobart, "Data fusion study between polarimetric SAR, hyperspectral and Lidar data for forest information," in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2008, vol. 2, 2008, pp. II-281-II-284.
18. A. Brook, E. Ben-Dor, and R. Richter, "Fusion of hyperspectral images and LiDAR data for civil engineering structure monitoring," in Proc. Hyperspectral Workshop, Hannover, Germany, 2010.
19. M. Dalponte, L. Bruzzone, and D. Gianelle, "Fusion of hyperspectral and LiDAR remote sensing data for classification of complex forest areas," IEEE Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1416-1427, May 2008.
20. L. Liu, Y. Pang, W. Fan, Z. Li, and M. Li, "Fusion of airborne hyperspectral and LiDAR data for tree species classification in the temperate forest of northeast China," in Proc. 19th Int. Conf. Geoinf., 2011, pp. 1-5.
21. G. Chen and S.-E. Qian, "Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage," IEEE Trans. Geosci. Remote Sens., vol. 49, no. 3, pp. 973-980, Mar. 2011.
22. P. Zhong and R. Wang, "Multiple-spectral-band CRFs for denoising junk bands of hyperspectral imagery," IEEE Trans. Geosci. Remote Sens., vol. 51, no. 4, pp. 2260-2275, Apr. 2013.
23. J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, "Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches," IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 354-379, Apr. 2012.
24. I. Dopido, A. Villa, A. Plaza, and P. Gamba, "A quantitative and comparative assessment of unmixing-based feature extraction techniques for hyperspectral image classification," IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 421-435, Apr. 2012.
25. S. Samiappan, S. Prasad, and L. M. Bruce, "Automated hyperspectral imagery analysis via support vector machines based multi-classifier system with non-uniform random feature selection," in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2011, pp. 3915-3918.
26. R. Zhang, J. Ma, X. Chen, and Q. Tong, "Feature selection for hyperspectral data based on modified recursive support vector machines," in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2009, vol. 2, pp. II-847-II-850.
27. R. Fandos, C. Debes, and A. M. Zoubir, "Resampling methods for quality assessment of classifier performance and optimal number of features," Signal Process., vol. 93, no. 11, pp. 2956-2968, 2013.
28. Y. Tarabalka, J. Chanussot, and J. A. Benediktsson, "Segmentation and classification of hyperspectral images using watershed transformation," Pattern Recognit., vol. 43, no. 7, pp. 2367-2379, 2010.
29. M. Grana, I. Villaverde, J. O. Maldonado, and C. Hernandez, "Two lattice computing approaches for the unsupervised segmentation of hyperspectral images," Neurocomputing, vol. 72, no. 10-12, pp. 2111-2120, 2009.
30. W. Liao, R. Bellens, A. Pizurica, W. Philips, and Y. Pi, "Classification of hyperspectral data over urban areas using directional morphological profiles and semi-supervised feature extraction," IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 5, no. 4, pp. 1177-1190, Aug. 2012.
31. G. Matasci, D. Tuia, and M. Kanevski, "SVM-based boosting of active learning strategies for efficient domain adaptation," IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 5, no. 5, pp. 1335-1343, Oct. 2012.
32. Y. Tarabalka, J. Chanussot, and J. Benediktsson, "Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers," IEEE Trans. Syst. Man Cybern. B Cybern., vol. 40, no. 5, pp. 1267-1279, Oct. 2010.
33. X. Huang and L. Zhang, "An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery," IEEE Trans. Geosci. Remote Sens., vol. 51, no. 1, pp. 257-272, Jan. 2013.
34. J. Ham, Y. Chen, M. M. Crawford, and J. Ghosh, "Investigation of the random forest framework for classification of hyperspectral data," IEEE Trans. Geosci. Remote Sens., vol. 43, no. 3, pp. 492-501, Mar. 2005. (Pubitemid 40320271)
35. D. Tuia and G. Camps-Valls, "Urban image classification with semisupervised multiscale cluster kernels," IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens., vol. 4, no. 1, pp. 65-74, Mar. 2011.
36. M. D. Mura, A. Villa, J. A. Benediktsson, J. Chanussot, and L. Bruzzone, "Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis," IEEE Geosci. Remote Sens. Lett., vol. 8, no. 3, pp. 542-546, May 2011.
37. E. Bienenstock, S. Geman, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Comput., vol. 4, pp. 1-58, 1992.
38. J. Friedman, "On bias, variance, 0/1-loss, and the curse-of-dimensionality," Data Min. Knowl. Discov., vol. 1, pp. 55-77, 1997. (Pubitemid 127721236)
39. R. Kohavi and D. H. Wolpert, "Bias plus variance decomposition for zero-one loss functions," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 275-283.
40. L. Breiman, "Bias, variance, and arcing classifiers," Statistics Dept., Univ. California, Berkeley, CA, Tech. Rep. 460, 1996.
41. P. Domingos, "Aunified bias-variance decomposition and its applications," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 231-238.
42. G. M. James, "Variance and bias for general loss functions," Mach. Learn., vol. 51, pp. 115-135, 2003.
43. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001. (Pubitemid 32933532)
44. R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection," in Proc. 14th Int. Joint Conf. Artif. Intell. (IJCAI'95), San Francisco, CA, USA: Morgan Kaufmann (Publishers Inc.), vol. 2, 1995, pp. 1137-1143.
45. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Diego, CA, USA: Academic Press Professional, Inc., 1990.
46. H. L. Wei and S. A. Billings, "Feature subset selection and ranking for data dimensionality reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 162-166, Jan. 2007. (Pubitemid 46415953)
47. J. O'Sullivan, Integrating initialization bias and search bias in neural network learning, 1996.
48. G. Hughes, "On the mean accuracy of statistical pattern recognizers," IEEE Trans. Inf. Theory, vol. 14, no. 1, pp. 55-63, Sep. 2006.
49. L. J. P. van der Maaten, E. O. Postma, and H. J. van den Herik, "Dimensionality reduction: A comparative review," Tilburg Centre for Creative Computing, Tech. Rep. TiCC TR 2009-005, 2008.
50. P. Switzer, A. A. Green, M. Berman, and M. D. Craig, "A transformation for ordering multispectral data in terms of image quality with implications for noise removal," IEEE Trans. Geosci. Remote Sens., vol. 26, no. 1, pp. 65-74, Jan. 1988. (Pubitemid 18596008)
51. J. W. Boardman and F. A. Kruse, "Automated spectral analysis: A geological example using AVIRIS data, north grapevine mountains, Nevada," in Proc. 10th Thematic Conf. Geologic Remote Sens., San Antonio, TX, USA, 1994.
52. A. A. Nielsen, "Kernel maximum autocorrelation factor and minimum noise fraction transformations," IEEE Trans. Image Process., vol. 20, no. 3, pp. 612-624, Mar. 2011.
53. A. A. Nielsen, L. Gomez-Chova, and G. Camps-Valls, "Explicit signal to noise ratio in reproducing kernel Hilbert spaces," in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2011, pp. 3570-3573.
54. B. Boashash, P. O'Shea, M. J. Arnold, and F. T. Luk, Eds., "Algorithms for instantaneous frequency estimation: A comparative study," in Proc. Soc. Photo-Optical Instrum. Engineers (SPIE) Conf. Ser., Nov. 1990, vol. 1348, pp. 126-148.
55. C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, 3rd ed. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1995.
56. B. Luo and J. Chanussot, "Hyperspectral image classification based on spectral and geometrical features," in Proc. IEEE Int. Workshop Mach. Learn. Signal Process., 2009, pp. 1-6.
57. B. C. Kuo and D. A. Landgrebe, "Nonparametric weighted feature extraction for classification," IEEE Trans. Geosci. Remote Sens., vol. 42, no. 5, pp. 1096-1105, May 2004.
58. C. Wan-Wei, R. Hsuan, and P. Yen-Nan, "Nonparametric weighted feature extraction for noise whitening least squares," in Proc. SPIE 6756 Chemical Biol. Sensors Ind. Environ. Monit. III, 2007, vol. 67560E.
59. P. Van Der Putten and M. Van Someren, "A bias-variance analysis of a real world learning problem: The coil challenge 2000," Mach. Learn., vol. 57, pp. 177-195, 2004.
60. D. Wolpert, "Stacked generalization," Neural Netw., vol. 5, no. 2, pp. 241-259, 1992.
61. M. Prior and T. Windeatt, "An ensemble dependence measure," in Proc. 17th Int. Conf. Artif. Neural Netw. (ICANN'07), Berlin, Heidelberg: Springer-Verlag, 2007, pp. 329-338.
62. L. Kuncheva and C. Whitaker, "Measures of diversity in classifier ensembles," Mach. Learn., vol. 51, pp. 181-207, 2003.
63. B. Efron, "Bootstrap methods: Another look at the Jackknife," Ann. Stat., vol. 7, no. 1, pp. 1-26, 1979.
64. R. Harris, G. Brown, J. Wyatt, and X. Yao, "Diversity creation methods: A survey and categorisation," Inf. Fusion, vol. 6, no. 1, pp. 5-20, 2005.
65. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, pp. 123-140, 1996. (Pubitemid 126724382)
66. M. Kearns, "Thoughts on hypothesis boosting," Unpublished manuscript, 1988.
67. P. Bartlett, R. E. Schapire, Y. Freund, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," in Proc. 14th Int. Conf. Mach. Learn., San Francisco, CA, USA, 1997.
68. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998. (Pubitemid 128741345)
69. T. Dietterich, "An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization," Mach. Learn., vol. 40, pp. 1-22, 1999.
70. V. Svetnik, A. Liaw, C. Tong, J. Culberson, R. Sheridan, and B. Feuston, "Random forest: A classification and regression tool for compound classification and QSAR modeling," J. Chem. Inf. Comput. Sci., vol. 43, pp. 1947-1958, 2003.
71. R. Diaz-Uriarte and S. Alvarez de Andres, "Gene selection and classification of microarray data using random forest," BMC Bioinformat., vol. 7, no. 3, pp. 1-13, 2006.
72. J.-M. Poggi, R. Genuer, and C. Tuleau, Random forests: Some methodological insights, arXiv:0811.3619, 2008.
73. J.-M. Poggi, R. Genuer, and C. Tuleau-Malot, "Variable selection using random forests," Pattern Recognit. Lett., vol. 31, pp. 2225-2236, 2010.
74. Y. Freund and R. Shapire, "Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn., L. Saitta, Ed., San Francisco, CA, USA, 1996, pp. 14-156.
75. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis, Cambridge, U.K.: Cambridge Univ. Press, 2004.
76. L. Ceriani and P. Verme, "The origins of the Gini index: Extracts from variabilità e mutabilità (1912) by Corrado Gini," J. Econ. Inequality, vol. 10, no. 3, pp. 421-443, Sep. 2012.
77. Y. Lin and Y. Jeon, "Random forests and adaptive nearest neighbors," J. Amer. Stat. Assoc., vol. 101, pp. 578-590, 2006. (Pubitemid 43972300)
78. G. Biau and L. Devroye, "On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification," J. Multivariate Anal., vol. 101, pp. 2499-2518, 2010.
79. N. Meinshausen, "Quantile regression forests," J. Mach. Learn. Res., vol. 7, pp. 983-999, 2006. (Pubitemid 43902307)
80. G. Biau, L. Devroye, and G. Lugosi, "Consistency of random forests and other averaging classifiers," J. Mach. Learn. Res., vol. 9, pp. 2015-2033, 2008.
81. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, pp. 721-741, 1984. (Pubitemid 15453722)
82. G. Winkler, Image Analysis, Random Fields and Markov Chain Monte Carlo Methods. Berlin, Germany: Springer-Verlag, 2003.
83. J. Besag, "On the statistical analysis of dirty pictures," J. Roy. Stat. Soc., vol. 48, pp. 259-302, 1986.
84. C. Debes, J. Hahn, A. M. Zoubir, and M. G. Amin, "Target discrimination and classification in through-the-wall radar imaging," IEEE Trans. Signal Process., vol. 59, no. 10, pp. 4664-4676, 2011.
85. N. D. A. Mascarenhas and A. C. Frery, "SAR image filtering with the ICM algorithm," in Proc. IEEE Int. Geosci. Remote Sens. Symp., 1994, vol. 4, pp. 2185-2187.
86. A. N. Tikhonov, "Solution of incorrectly formulated problems and the regularization method," Soviet. Math. Dokl., vol. 5, pp. 1035-1038, 1963.
87. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems, New York, NY, USA: V. H. Winston, 1977.
88. B. Jin and J. Zou, "Augmented tikhonov regularization," Inverse Probl., vol. 25, no. 2, p. 025001, 2009.
89. K. Ito and B. Jin, "A new approach to nonlinear constrained Tikhonov regularization," Inverse Probl., vol. 27, no. 10, p. 105005, 2011.
90. Y. C. Eldar, "Minimum variance in biased estimation: Bounds and asymptotically optimal estimators," IEEE Trans. Signal Process., vol. 52, no. 7, pp. 1915-1930, Jul. 2004.
91. A. O. Hero, J. A. Fessler, and M. Usman, "Exploring estimator bias-variance tradeoffs using the uniform CR bound," IEEE Trans. Signal Process., vol. 44, no. 8, pp. 2026-2041, Aug. 1996. (Pubitemid 126778023)