Divergence Function, Duality, and Convex Analysis

Neural Computation

Volumn 16, Issue 1, 2004, Pages 159-195

  Zhang, Jun ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85041123735     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/08997660460734047     Document Type: Article

References (36)

1. Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147-169.
2. Amari, S. (1982). Differential geometry of curved exponential families -curvatures and information loss. Annals of Statistics, 10, 357-385.
3. Amari, S. (1985). Differential geometric methods in statistics. New York: Springer-Verlag.
4. Amari, S. (1991). Dualistic geometry of the manifold higher-order neurons. Neural Networks, 4, 443-451.
5. Amari, S. (1995). Information geometry of EM and EM algorithms for neural networks. Neural Networks, 8, 1379-1408.
6. Amari, S., Kurata, K., & Nagaoka, H. (1992). Information geometry of Boltzmann machines. IEEE Transactions on Neural Networks, 3, 260-271.
7. Amari, S., Ikeda, S., & Shimokawa, H. (2001). Information geometry and mean field approximation: The α-projection approach. In M. Opper, & D. Saad (Eds), Advanced mean field methods - Theory and practice, (pp. 241-257). Cambridge, MA: MIT Press.
8. Amari, S., & Nagaoka, H. (2000). Method of information geometry. New York: Oxford University Press.
9. Bauschke, H. H., Borwein, J. M., & Combettes, P. L. (2002). Bregman monotone optimization algorithms. CECM Preprint 02:184. Available on-line: http://www.cecm.sfu.ca/preprints/2002pp.html.
10. Bauschke, H. H., & Combettes, P. L. (2002). Iterating Bregman retractions. CECM Preprint 02:186. Available on-line: http://www.cecm.sfu.ca/preprints/2002pp.html.
11. Bregman, L. M. (1967). The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Physics, 7, 200-217.
12. Chentsov, N. N. (1982). Statistical decision rules and optimal inference. Providence, RI: AMS, 1982.
13. Csiszár, I. (1967). On topical properties of f-divergence. Studia Mathematicarum Hungarica, 2, 329-339.
14. Della Pietra, S., Della Pietra, V., & Lafferty, J. (2002). Duality and auxiliary functions for Bregman distances (Tech. Rep. No. CMU-CS-01-109). Pittsburgh, PA: School of Computer Science, Carnegie Mellon University.
15. Eguchi, S. (1983). Second order efficiency of minimum contrast estimators in a curved exponential family. Annals of Statistics, 11, 793-803.
16. Eguchi, S. (1992). Geometry of minimum contrast. Hiroshima Mathematical Journal, 22, 631-647.
17. Eguchi, S. (2002). U-boosting method for classification and information geometry. Paper presented at the SRCCS International Statistical Workshop, Seoul National University, June.
18. Hardy, G., Littlewood, J. E., & Pólya, G. (1952). Inequalities. Cambridge: Cambridge University Press.
19. Ikeda, S., Amari, S., & Nakahara, H. (1999) Convergence of the wake-sleep algorithm. In M. Kearns, S. Solla, & D. Cohn (Eds.), Advances in neural information processing systems, 11 (pp. 239-245). Cambridge, MA: MIT Press.
20. Kaas, R. E., & Vos, P. W. (1997). Geometric foundation of asymptotic inference. New York: Wiley.
21. Kurose, T. (1994). On the divergences of 1-conformally flat statistical manifolds. Töhoko Mathematical Journal, 46, 427-433.
22. Lafferty, J., Della Pietra, S., & Della Pietra, V. (1997). Statistical learning algorithms based on Bregman distances. In Proceedings of 1997 Canadian Workshop on Information Theory, pp. 77-80. Toronto, Canada: Fields Institute.
23. Lauritzen, S. (1987). Statistical manifolds. In S. Amari, O. Barndorff-Nielsen, R. Kass, S. Lauritzen, and C. R. Rao (Eds.), Differential geometry in statistical inference (pp. 163-216). Hayward, CA: Institute of Mathematical Statistics.
24. Lebanon, G., & Lafferty, J. (2002). Boosting and maximum likelihood for exponential models. In T. G. Dietterich, S. Becker, & Z. Ghahramani (eds.) Advances in neural information processing systems, 14 (pp. 447-454). Cambridge, MA: MIT Press.
25. Matsuzoe, H. (1998). On realization of conformally-projectively flat statistical manifolds and the divergences. Hokkaido Mathematical Journal, 27, 409-421.
26. Matsuzoe, H. (1999). Geometry of contrast functions and conformal geometry. Hiroshima Mathematical Journal, 29, 175-191.
27. 3-functions taking the minimum at the diagonal of the product manifold. Hiroshima Mathematical Journal, 23, 327-332.
28. Mihoko, M., & Eguchi, S. (2002). Robust blink source separation by beta-divergence. Neural Computation, 14, 1859-1886.
29. Rao, C. R. (1987). Differential metrics in probability spaces. In S. Amari, O. Barndorff-Nielsen, R. Kass, S. Lauritzen, & C. R. Rao (Eds.), Differential geometry in statistical inference, (pp. 217-240). Hayward, CA: Institute of Mathematical Statistics.
30. Rockafellar, R. T. (1970). Convex analysis. Princeton, NJ: Princeton University Press.
31. Shima, H. (1978). Compact locally Hessian manifolds. Osaka Journal of Mathematics, 15, 509-513.
32. Shima, H., & Yagi, K. (1997). Geometry of Hessian manifolds. Differential Geometry and Its Applications, 7, 277-290.
33. Takeuchi, J., & Amari, S. (submitted). α-Parallel prior and its properties. IEEE Transaction on Information Theory. Manuscript under review.
34. Uohashi, K., Ohara, A., & Fujii, T. (2000). 1-Conformally flat statistical submanifolds. Osaka Journal of Mathematics, 37, 501-507.
35. Zhu, H. Y., & Rohwer, R. (1995). Bayesian invariant measurements of generalization. Neural Processing Letter, 2, 28-31.
36. Zhu, H. Y., & Rohwer, R. (1997) Measurements of generalisation based on information geometry. In S. W. Ellacott, J. C. Mason, & I. J. Anderson (Eds.), Mathematics of neural networks: Model algorithms and applications (pp 394-398). Norwell, MA: Kluwer.