Bagging for linear classifiers

Pattern Recognition

Volumn 31, Issue 7, 1998, Pages 909-930

  Skurichina, Marina ^a,b   Duin, Robert P W ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

^b VILNIUS UNIVERSITY   (Lithuania)

Author keywords

Bagging; Bias and variance; Generalization error; Instability; Linear discriminant; Regularization; Small sample size

Indexed keywords

DATA STRUCTURES; ERRORS; LINEAR SYSTEMS; PARAMETER ESTIMATION; PATTERN RECOGNITION; REGRESSION ANALYSIS; STABILITY;

BAGGING; GENERALIZATION ERROR; LINEAR CLASSIFIERS; REGULARIZATION;

MATHEMATICAL TECHNIQUES;

EID: 0032121371     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0031-3203(97)00110-6     Document Type: Article

References (29)

1. A. K. Jain and B. Chandrasekaran, Dimensionality and sample size considerations in pattern recognition practice, Handbook of Statistics, Vol. 2, P. R. Krishnaiah and L. N. Kanal, eds, pp. 835-855. North-Holland, Amsterdam (1987).
2. S. Raudys and A. K. Jain, Small sample size effects in statistical pattern recognition: recommendations for practitioners, IEEE Trans. Pattern Anal. Mach. Intell. 13(3), 252-264 (1991).
3. R. P. W. Duin, On the accuracy of Statistical Pattern Recognizers, Thesis, Delft University of Technology (1978).
4. WP. J. Rousseeuw and A. M. Leroy, Robust regression and outlier detection. Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics. Wiley, New York (1987).
5. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Chapman and Hall, New York (1993).
6. L. Breiman, Bagging predictors, Machine Learning J. 24(2), 123-140 (1996).
7. R. Tibshirani, Bias, variance and prediction error for classification rules, Technical report, University of Toronto, Canada (1996).
8. D. H. Wolpert and W. G. Macready, An efficient method to estimate bagging's generalization error, Technical Report, Santa Fe Institute, Santa Fe (1996).
9. H. Netten, I. T. Young, M. Prins, L. J. van Vliet, H. J. Tanke, J. Vrolijk, W. Sloos, Automation of Fluorescent dot counting in cell nuclei, Proc. 12th Int. Conf. on Pattern Recognition, Vol.1, Jerusalem, pp. 84-87 (1994).
10. A. Hoekstra, H. Netten and D. de Ridder, A neural network applied to spot counting, Proc. ACSI'96, the 2nd Annual Conf. of the Advanced School for Computing and Imaging, Lommel, Belgium, pp. 224-229 (1996).
11. R. A. Fisher, The use of multiple measurements in taxonomic problems, Ann. Eugenics 7(2), (1936).
12. R. A. Fisher, The precision of discriminant functions, Ann. Eugen 10(4), (1940).
13. C. R. Rao, On some problems arising of discrimination with multiple characters, Sankya 9, 343-365 (1949).
14. S. Raudys and V. Pikelis, On dimensionality, sample size, classification error and complexity of classification algorithm in pattern recognition, IEEE Trans. Pattern Anal Mach. Intell. PAMI-2(3), 242-252 (1980).
15. S. Raudys, Statistical Pattern Recognition. Small design sample problems, a manuscript, Institute of Mathematics and Cybernetics, Vilnius, pp. 1-460 (1984).
16. S. A. Aivazian, V. M. Buchstaber, I. S. Yenyukov and L. D. Mechalkin, Applied Statistics. Classification and Reduction of Dimensionality, Ref. edn, Finansy i Statistika, Moscow (1989).
17. J. M. Friedman, Regularized discriminant analysis, JASA 84, 165-175 (1989).
18. R. Peck and J. Van Ness, The use of shrinkage estimators in linear discriminant analysis, IEEE Trans. Pattern Anal. Mach. Intell. (4), 530-537 (1982).
19. Wei-Lem Loh, On linear discriminant analysis with adaptive ridge classification rules, J. Multivariate Anal. (53), 264-278 (1995).
20. D. M. Barsov, Classification error minimization using biased discriminant functions. Statistics, Probability, Economics, pp. 376-379. Nauka, Moscow (1985) (In Russian).
21. V. I. Serdobolskij, About minimal probability of misclassification in discriminant analysis, Lectures of USSR Acad. Sci. 270 (5), 1066-1070 (1983).
22. S. Raudys and M. Skurichina, Small sample properties of ridge-estimate of covariance matrix in statistical and neural net classification, New Trends in Probability and Statistics, Vol. 3, Tartu, Estonia, pp. 237-245 (1994).
23. R. P. W. Duin, Small sample size generalization, Proc. 9th Scandinavian Conf. on Image Analysis, Uppsala, Sweden (1995).
24. C. Cortes and V. Vapnik, Support-vector networks, Machine Learning 20, 273-297 (1995).
25. K. Fukunaga, Introduction to Statistical Pattern Recognition, pp. 400-407. Academic Press, New York (1990).
26. J. Kittler, M. Hatef and R. P. W. Duin, Combining classifiers, Proc. ICPR, Vienna, Austria, pp. 897-901 (1996).
27. J. S. Rao and R. Tibshirani, The out-of-bootstrap method for model averaging and selection, Report, University of Toronto, Canada (1997).
28. M. Taniguchi and V. Tresp, Averaging regularized estimators, Neural Comput. 9, 1163-1178 (1997).
29. M. Skurichina and R. P. W. Duin, Stabilizing classifiers for very small sample sizes, Proc. ICPR, Vienna, Austria, pp. 891-896 (1996).