About the maximum information and maximum likelihood principles in neural networks

Kybernetika

Volumn 34, Issue 4, 1998, Pages 485-494

  Vajda, Igor ^a   Grim, Jǐrí ^a  

^a INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0347662716     PISSN: 00235954     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (33)

1. J.J. Atick and A.N. Redlich: Towards a theory of early visual processing. Neural Computation 2 (1990), 308-320.
2. F. Attneave: Some informational aspects of visual perception. Psychological Review 61 (1954), 183-193.
3. S. Becker and G.E. Hinton: A self-organizing neural network that discovers surfaces in random-dot stereograms. Nature (London) 355 (1992), 161-163.
4. D.S. Bromhead and D. Lowe: Multivariate functional interpolation and adaptive networks. Complex Systems 2 (1988), 321-355.
5. M. Casdagli: Nonlinear prediction of chaotic time-series. Physica 35D (1989), 335-356.
6. T.M. Cover and J.B. Thomas: Elements of Information Theory. Wiley, New York 1991.
7. A.P. Dempster, N.M. Laird and D.B. Rubin: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39 (1977), 1-38.
8. 1 View. Wiley, New York 1985.
9. L. Devroye, L. Gyorfi and G. Lugosi: A Probabilistic Theory of Pattern Recognition. Springer, New York 1996.
10. J. Grim: Design of multilayer neural networks by information preserving transforms. In: Proceedings of the Third European Congress on System Science (E. Pessa, M.P. Penna and A. Montesanto, eds.), Edizzioni Kappa, Roma 1996, pp. 977-982,
11. S. Haykin: Neural Networks: A Comprehensive Foundation. MacMillan, New York 1994.
12. J. Hertz, A. Krogh and R.G. Palmer: Introduction to the Theory of Neural Computation. Addison-Wesley, New York-Menlo Park - Amsterdam 1991.
13. R.A. Jacobs, M.I. Jordan: A competitive modular connectionist architecture. In: Advances in Neural Information Processing Systems (R.P. Lippmann, J.E. Moody and D.J. Touretzky, eds.), Morgan Kaufman, San Mateo CA 1991, Vol. 3. pp. 767-773.
14. J. Kay: Feature discovery under contextual supervision using mutual information. In: International Joint Conference on Neural Networks, Baltimore MD 1992, Vol. 4, pp. 79-84.
15. F. Liese and I. Vajda: Convex Statistical Distances. Teubner Verlag, Leipzig 1987.
16. R. Linsker: Self-organization in perceptual network. Computer 21 (1988), 105-117.
17. R. Linsker: Perceptual neural organization: Some approaches based on network models and information theory. Annual Review of Neuroscience 13 (1990), 257-281.
18. D. Lowe: Adaptive radial basis function nonlinearities, and the problem of generalization. In: First IEE International Conference on Artificial Neural Networks, 1989, pp. 95-99.
19. J. Moody and C. Darken: Fast learning in locally-tuned processing units. Neural Computation 1 (1989), 281-294.
20. H. CH. Palm: A new method for generating statistical classifiers assuming linear mixtures of Gaussiian densities. In: Proc. 12th IAPR Internat. Conference on Pattern Recognition, IEEE Computer Society Press Jerusalem 1994, Vol.II., pp. 483-486.
21. M.D. Plumbley: A Hebbian/anti-Hebbian network which optimizes information capacity by orthonormalizing the principle subspace. In: IEE Artificial Neural Networks Conference, ANN-93, Brighton 1992, pp. 86-90.
22. M.D. Plumbley and F. Fallside: An information-theoretic approach to unsupervised connectionist models. In: Proceedings of the 1988 Connectionist Models Summer School, (D. Touretzky, G. Hinton and T. Sejnowski, eds.), Morgan Kaufmann, San Mateo 1988, pp. 239-245.
23. T. Poggio and F. Girosi: Regularization algorithms for learning that are equivalent to multilayer networks. Science 247 (1990), 978-982.
24. J. Rissanen: Stochastic Complexity in Statistical Inquiry. World Scientific, NJ 1989.
25. D.F. Specht: Probabilistic neural networks for classification, mapping or associative memory. In: Proc. of the IEEE Internat. Conference on Neural Networks, 1988, Vol. I., pp. 525-532.
26. C.E. Shannon: A mathematical theory of communication. Bell System Tech. J. 27 (1948), 379-423, 623-656.
27. L.R. Streit and T.E. Luginbuhl: Maximum likelihood training of probabilistic neural networks. IEEE Trans. Neural Networks 5 (1994), 5, 764-783.
28. I. Vajda and J. Grim: Bayesian optimality of decisions is achievable by RBF neural networks. IEEE Trans. Neural Networks, submitted.
29. A. Ukrainec and S. Haykin: A modular neural network for unhancement of errors-polar radar targets. Neural Networks 9 (1996), 141-168.
30. A.M. Uttley: The transmission of information and the effect of local feedback in theoretical and neural networks. Brain Research 102 (1966), 23-35.
31. S. Watanabe and K. Fukumizu: Probabilistic design of layered neural networks based on their unified framework. IEEE Trans. Neural Networks 6 (1995), 3, 691-702.
32. L. Xu and M.I. Jordan: EM learning on a generalized finite mixture model for combining multiple classifiers. In: World Congress on Neural Networks, 1993, Vol. 4, pp. 227-230.
33. L. Xu, A. Krzyzak and E. Oja: Rival penalized competitive learning for clustering analysis, RBF net and curve detection. IEEE Trans. Neural Networks 4 (1993), 636-649.