The IM algorithm : A variational approach to information maximization

Advances in Neural Information Processing Systems

Volumn , Issue , 2004, Pages

  Barber, David ^a   Agakov, Felix ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EM ALGORITHMS; INFORMATION MAXIMIZATION; INFORMATION TRANSMISSION; LINEAR COMPRESSION; MUTUAL INFORMATIONS; NOISY CHANNEL; VARIATIONAL APPROACHES; VARIATIONAL APPROXIMATION;

ALGORITHMS;

EID: 38149077590     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (13)

1. D. Barber, Tractable Approximate Belief Propagation, Advanced Mean Field Methods Theory and Practice (D. Saad and M. Opper, eds.), MIT Press, 2001.
2. H. Barlow, Unsupervised Learning, Neural Computation 1 (1989), 295-311.
3. S Becker, An Information-theoretic unsupervised learning algorithm for neural networks, Ph.D. thesis, University of Toronto, 1992.
4. A.J. Bell and T.J. Sejnowski, An information-maximisation approach to blind sepa- ration and blind deconvolution, Neural Computation 7 (1995), no. 6, 1004-1034.
5. N. Brunel and J.-P. Nadal, Mutual Information, Fisher Information and Population Coding, Neural Computation 10 (1998), 1731-1757.
6. T. Jaakkola and M. Jordan., Improving the mean field approximation via the use of mixture distributions, Proceedings of the NATO ASI on Learning in Graphical Models, Kluwer, 1997.
7. R. Linsker, Deriving Receptive Fields Using an Optimal Encoding Criterion, Advances in Neural Information Processing Systems (Lee Giles (eds) Steven Hanson, Jack Cowan, ed.), vol. 5, Morgan-Kaufmann, 1993.
8. S. Mika, B. Schoelkopf, A.J. Smola, K-R. Muller, M. Scholz, and Gunnar Ratsch, Kernel PCA and De-Noising in Feature Spaces, Advances in Neural Information Processing Systems 11 (1999).
9. R. M. Neal and G. E. Hinton, A View of the EM Algorithm That Justifies Incremental, Sparse, and Other Variants, Learning in Graphical Models (M.J. Jordan, ed.), MIT Press, 1998.
10. D. Saad and M. Opper, Advanced Mean Field Methods Theory and Practice, MIT Press, 2001.
11. T. Tanaka, Analysis of Bit Error Probability of Direct-Sequence CDMA Multiuser Demodulators, Advances in Neural Information Processing Systems (T. K. Leen et al. (eds.), ed.), vol. 13, MIT Press, 2001, pp. 315-321.
12. K. Torkkola and W. M. Campbell, Mutual Information in Learning Feature Transformations, Proc. 17th International Conf. on Machine Learning (2000).
13. M. Wainwright, T. Jaakkola, and A. Willsky, A new class of upper bounds on the log partition function, Uncertainty in Artificial Intelligence, 2002.