Information geometry of U-Boost and Bregman divergence

Neural Computation

Volumn 16, Issue 7, 2004, Pages 1437-1481

  Murata, Noboru ^a   Takenouchi, Takashi ^b   Kanamori, Takafumi ^c   Eguchi, Shinto ^b,d  

^a WASEDA UNIVERSITY   (Japan)

^b GRADUATE UNIVERSITY FOR ADVANCED STUDIES   (Japan)

^c TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^d INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPARATIVE STUDY; HUMAN; LEARNING; NONLINEAR SYSTEM; PHYSIOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 2942627097     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976604323057452     Document Type: Article

References (27)

1. Amari, S. (1985). Differential-geometrical methods in statistics. Berlin: Springer-Verlag.
2. Amari, S. (1995). Information geometry of the EM and EM algorithms for neural networks. Neural Networks, 8(9), 1379-1408.
3. Amari, S., & Nagaoka, H. (2000). Methods of information geometry. New York: Oxford University Press.
4. Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Information Theory, 39(3), 930-945.
5. Bishop, C. (1995). Neural networks for pattern recognition. Oxford: Clarendon Press.
6. Collins, M., Schapire, R. E., & Singer, Y. (2000). Logistic regression, Adaboost and Bregman distances. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory (pp. 158-169). San Francisco: Morgan Kaufmann.
7. Domingo, C., & Watanabe, O. (2000). MadaBoost: A modification of AdaBoost. In Proceedings of the Thirteenth Conference on Computational Learning Theory (pp. 180-189). San Francisco: Morgan Kaufmann.
8. Eguchi, S., & Copas, J. B. (2001). Recent developments in discriminant analysis from an information geometric point of view. Journal of the Korean Statistical Society, 30, 247-264.
9. Eguchi, S., & Copas, J. B. (2002). A class of logistic type discriminant functions. Biometrika, 89, 1-22.
10. Eguchi, S., & Kano, Y. (2001). Robustifing maximum likelihood estimation by psidivergence (ISM Research Memorandum 802). Tokyo: Institute of Statistical Mathematics.
11. Freund, Y. (1995). Boosting a weak learning algorithm by majority. Information and Computation, 121(2), 256-285.
12. Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference (pp. 148-156). San Francisco: Morgan Kaufmann.
13. Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139.
14. Friedman, J. H., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28, 337-407.
15. Hampel, F. R., Rousseeuw, P. J., Ronchetti, E. M., & Stahel, W. A. (1986). Robust statistics. New York: Wiley.
16. Hastie, T., Tibishirani, R., & Friedman, J. (2001). The elements of statistical learning. New York: Springer-Verlag.
17. Kearns, M., & Valiant, L. G. (1988). Learning boolean formulae or finite automata is as hard as factoring (Tech. Rep. TR-14-88). Cambridge, MA: Harvard University Aiken Computation Laboratry.
18. Kivinen, J., & Warmuth, M. K. (1999). Boosting as entropy projection. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory (pp. 134-144). New York: ACM Press.
19. Lebanon, G., & Lafferty, J. (2001). Boosting and maximum likelihood for exponential models (Tech. Rep. CMU-CS-01-144). Pittsburgh, PA: School of Computer Science, Carnegie Mellon University.
20. McLachlan, G. J. (1992). Discriminant analysis and statistical pattern recognition. New York: Wiley.
21. Minami, M., & Eguchi, S. (2002). Robust blind source separation by beta-divergence. Neural Computation, 14, 1859-1886.
22. Murata, N. (1996). An integral representation with ridge functions and approximation bounds of three-layered network. Neural Networks, 9(6), 947-956.
23. Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion -determining the number of hidden units for an artificial neural network model. IEEE Trans. Neural Networks, 5(6), 865-872.
24. Schapire, R. E. (1990). The strength of weak learnability. Machine Learning, 5, 197-227.
25. Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5), 1651-1686.
26. Takenouchi, T., & Eguchi, S. (2004). Robustifying AdaBoost by adding the naive error rate. Neural Computation, 16(4), 767-787.
27. Vapnik, V. (1995). The nature of statistical learning theory. Berlin: Springer-Verlag.