Combinations of weak classifiers

IEEE Transactions on Neural Networks

Volumn 8, Issue 1, 1997, Pages 32-42

  Ji, Chuanyi ^a   Ma, Sheng ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

Author keywords

Combinations of classifiers; Generalization error; Space complexity; Supervised learning; Time complexity; Weak classifiers

Indexed keywords

COMPUTATIONAL COMPLEXITY; LEARNING ALGORITHMS; LEARNING SYSTEMS; MEASUREMENT ERRORS; NEURAL NETWORKS; STATE SPACE METHODS;

SPACE COMPLEXITY; SUPERVISED LEARNING; TIME COMPLEXITY; WEAK CLASSIFIERS;

PATTERN RECOGNITION SYSTEMS;

EID: 0030736561     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.554189     Document Type: Article

References (44)

1. A. Barron, "Approximation and estimation bounds for artificial neural networks," Machine Learning, vol. 14, pp. 113-143, 1994.
2. _, "Universal approximation bounds for artificial neural networks," IEEE Trans. Inform. Theory, vol. 39, pp. 930-944, 1993.
3. _, "Predicted squared error: A criterion for automatic model selection," in Self-Organizing Methods in Modeling, S. J. Farlow, Ed. New York: Marcel Dekker, 1984.
4. E. Baum and D. Haussler, "What size net gives valid generalization?" Neural Computa., vol. 1no. 1, pp. 151-160, 1989.
5. R. S. Berlind, "an alternative method of stochastic discrimination with applications to pattern recognition," Ph.D. dissertation, State Univ. New York, Buffalo, 1994.
6. A. L. Blum and R. L. Rivest, "Training a three-node neural network is NP-complete," Neural Networks, vol. 5, pp. 117-127, 1992.
7. L. Breiman, "Stacked generalization," Dep. Statist., Univ. California, Berkeley, Tech. Rep., TR-367, Aug. 1992.
8. N. Cesa-Bianchi, Y. Freud, D. P. Helmbold, D. Haussler, R. E. Shapire, and M. K. Warmuth, "How to use expert advice," in Proc. Found. Comput. Sci., pp. 382-391, 1993.
9. T. M. Cover and P. E. Hart, "Nearest neighbor pattern classification," IEEE Trans. Inform. Theory, vol. IT-13, pp. 21-27, 1967.
10. L, Devroye, "Automatic pattern recognition - A study of the probability of error," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-10, pp. 530-543, 1988.
11. H. Drucker, R. Schapire, and P. Simard, "Improving performance in neural networks using a boosting algorithm," in Proc. Neural Inform. Processing Symp., 1993, pp. 42-49.
12. H. Drucker and C. Cortes, "Boosting decision trees," in Neural Information Processing Symp., 1995.
13. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
14. F. Blayo, Y. Cheneval, J. Madrenas, M. Moreno, and J. L. Voz, "Deliverable R3-B4-P Task B4: Benchmarks," Enhanced learning for evolutive neural architecture, ESPRIT-Basic res. project 6891, June 1995 [Online]. Available FTP:/pub/neural-nets/ELENA/databases/Benchmarks.ps.Z on ftp.dice.ucl.ac.be
15. C. Aviles-Cruz, A. Guerin-Dugue, J. L. Voz, and D. Van Cappel, "Deliverable R3-B1-P Task B1: Databases," Enhanced learning for evolutive neural architecture, ESPRIT-Basic res. project 6891, June 1995. Available FTP:/pub/neural-nets/ELENA/databases/Benchmarks.ps.Z on ftp.dice.ucl.ac.be
16. W. Feller, An Introduction to Probability Theory and Its Applications, 3rd ed. New York: Wiley, vol. 1, 1968, p. 195.
17. Y. Freund, "Boosting a weak learning algorithm by majority," in Proc. 3rd Wkshp. Computa. Learning Theory. San Mateo, CA: Morgan Kaufmann, 1990, pp. 202-216.
18. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias variance dilemma," Neural Computa. vol. 4, no. 1, pp. 1-58, Jan. 1992.
19. L. K. Hansen and P. Salamon, "Neural network ensembles," IEEE Trans. Pattern Anal. Machine Intell., vol. 12, pp. 993-1001, 1990.
20. T. K. Ho, J. J. Hull, and S. N. Srihari, "Decision combination in multiple classifier systems," IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 66-75, 1994.
21. S. J. Randy and A. K. Jain, "Small sample size effects in statistical pattern recognition: recommendations for practitioners," IEEE Trans. Pattern Anal. Machine Intell., vol. 13), pp. 252-264, 1994.
22. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Computa., vol. 3, pp. 79-87, 1991.
23. C. Ji and D. Psaltis, "The VC-dimension versus the statistical capacity for two layer network with binary weights," in Proc. 4th Wkshp. Computa. Learning Theory, 1991.
24. S. Judd, Neural Network Design and The Complexity of Learning. Cambridge, MA: MIT Press, 1990.
25. A. Stwart and J. K. Ord, Kandell's Advanced Theory of Statis. Oxford, U.K.: Oxford Univ. Press, 1987.
26. E. M. Kleinberg, "Stochastic discrimination," Ann. Math. Artificial Intell., vol. 1, pp. 207-239, 1990.
27. E. M. Kleinberg and T. Ho, "Pattern recognition by stochastic modeling," in Proc. 3rd Int. Wkshp. Frontiers in Handwriting Recognition, Buffalo, NY, May 1993, pp. 175-183.
28. N. Little and M. Warmuth, "The weighted majority algorithm," in Proc. 3rd Wkshp. Computa. Learning Theory, 1989.
29. R. Meir, "Bias, variance, and the combination of estimators: The case of linear least squares," Dep. Electr. Eng., Technion, Haifa, Israel, 1994.
30. J. Moody and C. Darken, "Fast learning in networks of locally tuned processing units," Neural Computa., vol. 1, 281-294, 1989.
31. N. J. Nilsson, Learning Machines. San Mateo, CA: Morgan Kaufmann, 1990.
32. S. J. Nowlan, and G. E. Hinton, "Simplifying neural networks by soft weight sharing," Neural Computa., vol. 4, 1992, pp. 473-493.
33. M. P. Perrone and L. N. Cooper, "When networks disagree: Ensemble method for neural networks," Artificial Neural Networks for Speech and Vision, 1993, ch. 10.
34. L. Prechelt, "PROBN1-A Set of Benchmarks and benchmarking rules for neural-network training algorithms," Fakultät für Informatik, Universität Karlsuhe, D-76128 Karlsruhe, Germany, Tech. Rep. 21/94, Sept. 1994 [Online]. Anonymous FTP:/pub/papers/techreports/1994/1994-21.ps.z on ftp.ira.uka.de
35. D. Psaltis, R. R. Snap, and S. S. Venkatesh, "On the finite sample performance of nearest neighbor classifiers," IEEE Trans. Inform. Theory, vol. 40, pp. 820-837, May 1994.
36. R. E. Schapire, "The strength of weak learnability," Machine Learning, vol. 5, pp. 197-227, 1990.
37. N. Tishby, E. Levin, and S. Solla, "Consistent inference of probability in layered networks: Prediction and generalization," in Proc. Int. Joint Conf. Neural Networks, 1989.
38. K. Turner and J. Ghosh, "Theoretical foundations of linear and order statistics combiners for neural pattern classifiers." IEEE Trans. Neural Networks, 1995.
39. L. Xu, M. I. Jordan, and G. E. Hinton, "An alternative model for mixtures of experts," Advances in Neural Information Processing System 7, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. Cambridge, MA: MIT Press, 1995, pp. 633-640.
40. L. Xu, A. Krzyzak, and C. Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 418-435, 1992.
41. L. G. Valient, "A theory of learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
42. V. Vapnik, Estimation Of Dependences Based On Empirical Data. New York: Springer-Verlag, 1982.
43. S. S. Venkatesh, "Directed drift - A new linear threshold algorithm for learning binary weights online," J. Comput. Syst. vol. 46, no. 2, pp. 198-217, Apr. 1993.
44. D. Wolpert, "Staked generalization," Neural Networks, vol. 5, no. 2, pp. 241-259, 1992.