Generative binary codes

Pattern Analysis and Applications

Volumn 6, Issue 4, 2004, Pages 269-284

  Haft, M ^a   Hofman, R ^a   Tresp, V ^a  

^a ImageDiagnost GmbH   (Germany)

Author keywords

Bayesian networks; Distributed binary coding; EM learning; Generative models; Image coding and denoising; Mean field inference

Indexed keywords

EID: 1642317045     PISSN: 14337541     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10044-003-0194-x     Document Type: Article

References (82)

1. Pearl J (1988) Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann
2. Horn B (1986) Robot Vision. MIT Press
3. Everitt BS (1984) An Introduction to Latent Variable Models. Chapman & Hall, London
4. Heckermann D (1997) Bayesian networks for data mining. Data Mining and Knowledge Discovery 1: 79-119
5. Jordan MI, Jacobs RA (1994) Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6: 181-214
6. Tresp V, Haft M, Hofmann R (1999) Mixture approximations to Bayesian Networks. In: Laskey KB, Prade H (eds) Uncertainty in Artificial Intelligence, Proc Fifteenth Conf 693-646
7. Neal RM, Dayan P (1997) Factor analysis using delta-rule wake-sleep learning. Neural Computation 9: 1781-1803
8. Frey BJ, Hinton GE (1997) A simple algorithm that discovers efficient perceptual codes. In: Harris L, Jenkin M (eds) Computational and Biological Mechanisms of Visual Coding. Cambridge University Press
9. Attias H, Schreiner CE (1998) Blind source separation and deconvolution: The dynamic component analysis algorithm. Neural Computation 10: 1373-1424
10. Lee TW, Girolami M, Sejnowski TJ (1999) Independent component analysis using an extended infomax algorithm for mixed subgausian sources. Neural Computation 11: 417-441
11. Hyvärinen A (1999) Survey on independent component analysis. Neural Comput Surv 2: 94-128. http://www.icsi.berkeley.edu/jagota/NCS
12. Obradovic D, Deco G (1998) Information maximization and independent component analysis: Is there a difference? Neural Computation 10(8): 2085-2101
13. Oja E (1992) Principal components, minor components, and linear neural networks. Neural Networks 5: 927-935
14. Tipping ME, Bishop CM (1997) Probabilistic principal component analysis. Technical Report (NCRG/97/010), Aston University, Neural Computing Research Group, http://www.ncrg.aston.ac.uk/
15. Revow M, Williams KIW, Hinton GE (1996) Using generative models for handwritten digit recognition. IEEE Trans Patt Anal Machine Intell 18(6): 592-606
16. Dayan P, Hinton GE, Neal RM (1995) The Helmholtz Machine. Neural Computation 7: 889-904
17. Hinton GE, Dayan P, Frey BJ, Neal RM (1995) The wake-sleep algorithm for unsupervised neural networks. Science 268: 1158-1160
18. Bengio Y (1998) Markovian models for sequential data. Neural Comput Surv 2: 129-162. http://www.icsi.berkeley.edu/jagota/NCS
19. Hollmén J, Tresp V (1999) Call-based fraud detection in mobile communication networks using a hierarchical regime-switching. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in Neural Information Processing Systems 11. MIT Press
20. Cadez I, Heckerman D, Meek C, Smyth P, White S (2000) Visualization of navigation patterns on a web site using model based clustering. MSR-TR-00-18, Microsoft Research
21. Ghahramani Z, Jordan MI (1998) Factorial hidden markov models. In: Touretzky D, Moser M, Hasselmo M (eds) Advances in Neural Information Processing Systems 8, 472-478. MIT Press
22. Dunmur AP, Titterington DM (1999) On a modification to the mean field EM algorithm in factorial learning. In: Touretzky D, Moser M, Hasselmo M (eds) Advances in Neural Information Processing Systems 8 431-437. MIT Press
23. Bishop CM, Svensén M, Williams CKI (1998) GTM: The generative topographic mapping. Neural Computation 10(1): 215-235
24. Bishop CM, Svensén M. Williams CKI (1999) Developments of the generative topographic mapping. Neurocomputing 21: 203-224
25. Girolami M (2001) The topographic organization and visualization of binary data using multivariate-bemoulli latent variable models. IEEE Trans Neural Networks 12(6): 1367-74
26. Tipping ME, Bishop CM (1999) Mixtures of principal component analysers. Neural Computation 11: 443-482
27. Attias H (1999) Independent factor analysis. Neural Computation 11(4): 803-851
28. Fine S, Singer Y, Tishby N (1998) The hierarchical hidden markov model: Analysis and applications. Machine Learning 32(1): 41-63
29. Bishop CM, Tipping ME (1998) A hierarchical latent variable model for data visualization. IEEE Trans Patt Anal Machine Intell 20(3): 281-293
30. Romberg JK, Choi H, Baraniuk RG (1999) Bayesian wavelet-domain image modeling using hidden markov trees. Proceedings International Conference on Image Processing, 158-62. IEEE
31. Romberg JK, Choi H, Baraniuk RG (2001) Bayesian tree-structured image modeling using wavelet-domain hidden markov models. IEEE Trans Image Process 10(7): 1056-68
32. Williams CKI, Adams NJ (1999) Dynamic trees. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in Neural Information Processing Systems 11. MIT Press
33. Adams MJ (2001) Dynamic trees: A hierarchical probabilistic approach to image modelling. PhD Thesis, http://www.anc.ed.ac.uk/code/adams/
34. Frey BJ, Hinton GE, Dayan P (1995) Does the wake-sleep algorithm produce good density estimators? In: Touretzky D, Moser M, Hasselmo M (eds) Advances in Neural Information Processing Systems 8, 661-7. MIT Press
35. Bishop CM (1998) Variational learning in graphical models and neural networks. In: Niklasson L, Boden M, Ziemke T (eds) Proceedings 8th International Conference on Artificial Neural Networks. 13-22. Springer-Verlag
36. Haft H, Hofmann R, Tresp V (1999) Model-independent mean-field theory as a local method for approximate propagation of information. Network 10(1): 93-105
37. Jordan M, Ghahramani Z, Jaakkola T, Saul L (1999) An introduction to variational methods for graphical models. Machine Learning 37(2): 183-205
38. Ghahramani Z, Hinton GE (2000) Variational learning for switching state-space models. Neural Computation 12(4): 831-64
39. Olshausen BA, Field DJ (1996) Natural image statistics and efficient coding. Network 7: 333-339
40. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607-607
41. Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1. Vision Res 37(23): 3311-3325
42. Lewicki SM, Olshausen BA (1999) Probabilistic framework for the adaption and comparison of image codes. J Opt Soc Am A 16(7): 1587-1601
43. Hyvärinen A, Hoyer P (1999) Emergence of topography and complex cell properties from natural images using extensions of ICA. In: Solla SA, Leen TK, Müller K-R (eds) Advances in Neural Information Processing Systems 12, 827-833. MIT Press
44. Hyvärinen A (1999) Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Computation 11(7): 1739-1768
45. Hateren JH, Van Der Schaaf A (1998) Independent component filters for natural images compared with simple cells in primary visual cortex. Proc Roy Soc Lond B 265(1394): 359-366
46. Lewicki M, Sejnowski TJ (2000) Learning overcomplete representations. Neural Computation 12(2): 337-365
47. Simoncelli EP, Adelson EH (1996) Noise removal via Bayesian wavelet coring. Proc Int Conf Image Processing, 379-382. IEEE
48. Buccigrossi RW, Simoncelli EP (1999) Image compression via joint statistical characterization in the wavelet domain. IEEE Trans Image Process 8(12): 1688-1701
49. Haft M (1998) Robust 'topological' codes by keeping control of internal redundancy. Phys Rev Lett 81(18): 4016-4019
50. Cooper G (1990) Computational complexity of probabilistic inference using bayesian belief networks. Artif Intell 42: 393-405
51. van Hemmen JL, Kühn R (1986) Nonlinear neural networks. Phys Rev Lett 57(7): 913-916
52. Parisi G (1988) Statistical Field Theory. Addison-Wesley
53. Binney JJ, Dowrick NJ, Fisher AJ, Newman MEJ (1995) The Theory of Critical Phenomena. Oxford University Press
54. Peterson C, Anderson JR (1987) A mean field theory learning algorithm for neural networks. Complex Syst 1: 995-1019
55. Peterson C, Hartman E (1989) Explorations of the mean field theory learning algorithm. Neural Networks 2: 475-494
56. Saul LK, Jaakkola T, Jordan MI (1996) Mean field theory for sigmoide belief networks. J Artif Intell Res 4: 61-76
57. Jaakkola TS, Jordan MI (1998) Improving the mean field approximation via the use of mixture distributions. In: Jordan MI (ed) Learning in Graphical Models, 163-173. Kulwer
58. Frey BJ, Hinton GE (1999) Variational learning in non-linear Gaussian belief networks. Neural Computation 11(1): 193-213. Available at http://www.cs.utoronto.ca/frey
59. Cover TM, Thomas JA (1991) Elements of Information Theory. Wiley
60. Dempster AP, Laird N, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Roy Stat Soc B 39: 1-38
61. Tanner MA (1996) Tools for Statistical Inference. Springer-Verlag
62. Dowling JE (1987) The Retina, an Approachable Part of the Brain. Harvard University Press
63. Wandell BA (1995) Foundations of Vision. Sinauer, Sunderland, MA
64. Haft M, van Hemmen JA (1998) Theory and implementation of infomax filters for the retina. Network 9: 39-71
65. Atick JJ, Redlich N (1992) What does the retina know about natural scenes? Neural Computation 4: 196-210
66. Bialek W, Ruderman DL (1994) Statistics of natural images: Scaling in the woods. Phys Rev Lett 73(6): 814-817
67. Ruderman DL (1994) The statistics of natural images. Network 5: 517-548
68. Olshausen BA, Field DJ (1996) Learning efficient linear codes for natural images: The role of sparseness, overcompleteness, and statistical independence. Proc SPIE 2657: 132-138
69. Földiak P (1990) Forming sparse representations by local anti-Hebbian learning. Biol Cybern 64: 165-170
70. Fyfe C, Baddeley R (1995) Finding compact and sparse-distributed representations of visual images. Network 6: 333-344
71. Lauritzen SL, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their application to expert systems. J Roy Stat Soc B 50: 154-227
72. Jensen VJ (1996) An Introduction to Bayesian Networks. UCL Press
73. Bell AJ, Sejnowski TK (1997) The 'independent components' of natural scenes are edge filters. Vision Res 37(23): 3327-38
74. Hateren JH, Rudermann DL (1998) Independent component analysis for natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proc Roy Soc Lond B 265(1412): 2315-2320
75. Jain AK (1989) Fundamentals of Digitsl Image Processing. Prentice Hall
76. Tresp V (2000) Committee Machines. In: Hu YH, Hwang J-N (eds) Handbook on Neural Network Signal Processing. CRC Press
77. Jordan MI (ed) (1998) Learning in Graphical Models. The MIT Press
78. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol 160: 106-154
79. Hubel DH, Wiesel TN (1977) Functional architecture of macaque visual cortex. Proc Roy Soc Lond B 198: 1-59
80. Hubel DH, Wiesel TN (1979) Brain mechanism of vision. Sci Am 241: 150-160
81. Nicholls JG, Martin AR, Wallace BG (1992) From Neuron to Brain. Sinauer, Sunderland, MA
82. Singer W (1984) Learning to see: Mechanisms in experience-dependent development. In: Marler P, Terrace HS (eds) The Biology of Learning. Springer-Verlag