Integration of stochastic models by minimizing α-Divergence

Neural Computation

Volumn 19, Issue 10, 2007, Pages 2780-2796

  Amari, Shun Ichi ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; PROBABILITY; STATISTICS; THEORETICAL MODEL;

MODELS, THEORETICAL; PROBABILITY; STOCHASTIC PROCESSES;

EID: 35549004167     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2007.19.10.2780     Document Type: Article

References (29)

1. Amari, S., Ikeda, S., & Shimokawa, H. (2001). Information geometry of α-projection in mean field approximation. In M. Opper & D. Saad (Eds.), Advanced mean field methods: Theory and practice (pp. 241-257). Cambridge, MA: MIT Press.
2. Amari, S., & Nagaoka, H. (2000). Methods of information geometry. Providence, RI: American Mathematical Society, and New York: Oxford University Press.
3. Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on a sum of observations. Annals of Mathematical Statistics, 23, 493-507.
4. Corcuera, J. M., & Giummolè, F. (1999). A generalized Bayes rule for prediction. Scandinavian Journal of Statistics, 26, 265-279.
5. Csiszar, I. (1975). I-divergence geometry of probability distributions and minimization problem. Annals of Probability, 3, 146-158.
6. Dayan, P., Hinton, G., Neal, R., & Zemel, R. (1995). The Helmholtz machine. Neural Computation, 7, 889-904.
7. Eguchi, S. (1983). Second order efficiency of minimum contrast estimators in a curved exponential family. Annals of Statistics, 11, 793-803.
8. Falmagne, J.-C. (1985). Elements of psychophysical theory. New York: Clarendon Press.
9. Hardy, G. H., Littlewood, J. E., & Polya, G. (1952). Inequalities (2nd ed.). Cambridge: Cambridge University Press.
10. Heller, J. (2006). Illumination-invariance of Plateau's midgray. Journal of Mathematical Psychology, 50, 263-270.
11. Hida, E., Ohno, H., Hashimoto, N., Amari, S., & Saito, H. (2006). Neural representation for perception of wide-field visual flow in MST: Bidirectional transparent motion and its illusory aftereffect. Manuscript submitted for publication.
12. Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14, 1771-1800.
13. Ikeda, S., Tanaka, T., & Amari, S. (2004). Stochastic reasoning, free energy, and information geometry. Neural Computation, 16, 1779-1810.
14. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-81.
15. Jordan, M. I., & Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181-214.
16. Komaki, F. (1996). On asymptotic properties of predictive distributions. Biometrika, 83, 299-313.
17. Marriott, P. (2002). On the local geometry of mixture models. Biometrika, 89, 79-93.
18. Matsuyama, Y. (2003). The alpha-EM algorithm: Surrogate likelihood maximization using alpha-logarithmic information measures. IEEE Trans. on Information Theory, 49, 692-706.
19. Minami, M., & Eguchi, S. (2004). Robust blind source separation by beta-divergence. Neural Computation, 14, 1859-1886.
20. Minka, T. (2005). Divergence measures and message passing (MSR-TR-2005-173). Redmond, WA: Microsoft.
21. Murata, N., Takenouchi, T., Kanamori, T., & Eguchi, S. (2004). Information geometry of U-boost and Bregman divergence. Neural Computation, 16, 1437-1481.
22. Petz, D., & Temesi, R. (2005). Means of positive numbers and matrices. SIAM Journal on Matrix Analysis and Applications, 27, 712-720.
23. Renyi, H. (1961). On measures of entropy and information. In Proc. 4th Berkeley Symp. on Mathematical Statistics and Probability. Berkeley: University of California Press.
24. Toyoizumi, T., & Aihara, K. (2006). Generalization of the mean-field method for power-law distributions. International Journal of Bifurcation and Chaos, 16, 129-135.
25. Tsallis, C. (1988). Possible generalization of Boltzmann-Gibbs statistics. Journal of Statistical Physics, 52, 479-487.
26. Wolpert, D., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11, 1317-1329.
27. Xu, L. (2004). Advances on BYY harmony learning: Information theoretic perspective, generalized projection geometry, and independent factor auto-determination. IEEE Trans on Neural Networks, 15(4), 885-902.
28. Zhang, J. (2004). Divergence function, duality and convex analysis. Neural Computation, 16, 159-195.
29. Zhu, H., & Rohwer, R. (1995). Bayesian invariant measurements of generalization. Unpublished manuscript. Available online at http://www.santafe.edu/research/publications/wpabstract/199806045.