Deterministic annealing EM algorithm

Neural Networks

Volumn 11, Issue 2, 1998, Pages 271-282

  Ueda, Naonori ^a   Nakano, Ryohei ^a  

^a S   (Japan)

Author keywords

Deterministic annealing; EM algorithm; Maximum entropy principle; Maximum likelihood estimation; Probabilistic neural networks

Indexed keywords

MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; NEURAL NETWORKS; PERFORMANCE; PROBABILITY; SIMULATED ANNEALING; VECTOR QUANTIZATION;

DETERMINISTIC ANNEALING; MAXIMUM ENTROPY PRINCIPLE; PROBABILISTIC NEURAL NETWORK;

ALGORITHMS;

ALGORITHM; ARTICLE; DERIVATIZATION; ELECTRON MICROSCOPY; NERVE CELL NETWORK; NONHUMAN; PRIORITY JOURNAL; PROBABILITY; SIMULATION; TECHNIQUE; THERMODYNAMICS;

EID: 0032029288     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(97)00133-0     Document Type: Article

References (22)

1. Buhmann J., Kuhnel H. Complexity optimized data clustering by competitive neural networks. Neural Computation. 5:1993;75-88.
2. Byrne W. Alternating minimization and Boltzmann machine learning. IEEE Trans. Neural Networks. 3:1992;612-620.
3. Dempster A.P., Laird N.M., Rubin D.B. Maximum-likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. Ser. B (methodological). 39:1977;1-38.
4. Feder M. Maximum entropy as a special case of the minimum description length criterion. IEEE Trans. Inf. Theory. 32(6):1986;847-849.
5. Geman S., Geman D. Stochastic relaxation, Gibbs distribution and the Bayesian restoration in images. IEEE Trans. Pattern Anal. Machine Intell. 6(6):1984;721-741.
6. Ghahramani, Z. & Jordan, M.I. (1994). Supervised learning from incomplete data via an EM approach. In J. Cowan et al. (Eds.), Advances in NIPS 6 (pp. 120-127). Morgan Kaufmann.
7. Hinton G.E., Dayan P., Frey B.J., Neal R.M. The 'wake-sleep' algorithm for unsupervised neural networks. Science. 268(26):1995;1158-1161.
8. Huang, X.D., Ariki, Y. and Jack, M.A. (1990). Hidden Markov Models for Speech Recognition. Edinburgh University Press.
9. Jordan M.I., Jacob R.A. Hierarchical mixtures of experts and the EM algorithm. Neural Computation. 6:1994;181-214.
10. McLachlan, G. & Basford, K. (1988). Mixture Models: Inference and Application to Clustering. New York and Basel: Marcel Dekker.
11. Neal, R.M. and Hinton, G.E. (1993). A new view of the EM algorithm that justifies incremental and other variants. Biometrika (submitted).
12. Oda M., Akamatsu S., Fukamachi H. Adaptability of K-L expansion technique for ambiguous face image retrieval. Electron. Commun. Jpn. 79(10):1996;35-46.
13. Rissanen J. Modeling by shortest data description. Automatica. 14:1978;465-471.
14. Rose K., Gurewitz E., Fox G.C. Vector quantization by deterministic annealing. IEEE Trans. Inf. Theory. 38(4):1992;1249-1257.
15. Specht D.F. Probabilistic neural networks. Neural Networks. 3:1990;109-118.
16. Streit R.L., Luginbuhl T.E. Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Networks. 5(5):1994;764-783.
17. Traven H.G.C. A neural network approach to statistical pattern classification by semiparametric estimation of probability density functions. IEEE Trans. Neural Networks. 2(3):1991;366-377.
18. Ueda, N. & Nakano, R. (1994). Mixture density estimation via EM algorithm with deterministic annealing. In Proceedings of the IEEE Neural Networks for Signal Processing (pp. 69-77). Greece. New York: IEEE.
19. Ueda, N. & Nakano, R. (1995a). A new maximum likelihood training and application to probabilistic neural networks. In Proceedings of the International Conference on Artificial Neural Networks (ICANN95) (pp. 497-502). Paris: EC2&Cie.
20. Ueda, N. & Nakano, R. (1995b). Deterministic annealing variant of the EM algorithm. In G. Tesauro et al. (Eds.), Advances in NIPS 7 (pp. 545-552). Cambridge, MA: MIT Press.
21. Wong Y. Clustering data by melting. Neural Computation. 5:1993;89-104.
22. Yuille A.L., Stolorz P., Utans J. Statistical physics, mixtures of distributions and the EM algorithm. Neural Computation. 6:1994;334-340.