Nonlinear spike-And-Slab sparse coding for interpretable image encoding

PLoS ONE

Volumn 10, Issue 5, 2015, Pages

  Shelton, Jacquelyn A ^a   Sheikh, Abdul Saboor ^a,d   Bornschein, Jörg ^b   Sterne, Philip ^c   Lücke, Jörg ^d  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

^c FRANKFURT INSTITUTE FOR ADVANCED STUDIES   (Germany)

^d UNIVERSITY OF OLDENBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CODING; CONTROLLED STUDY; IMAGE RECONSTRUCTION; MATHEMATICAL PARAMETERS; MAXIMUM A POSTERIORI; NONLINEAR COMBINATION OF COMPONENT; NONLINEAR SYSTEM; NORMAL DISTRIBUTION; PROCESS OPTIMIZATION; SPIKE AND SLAB SPARSE CODING; ALGORITHM; BOOK; COMPUTER ASSISTED DIAGNOSIS; DATA BASE; IMAGE PROCESSING; LEARNING;

ALGORITHMS; DATABASES AS TOPIC; DICTIONARIES AS TOPIC; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGE PROCESSING, COMPUTER-ASSISTED; LEARNING; NONLINEAR DYNAMICS;

EID: 84957588085     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0124088     Document Type: Article

References (53)

1. Mallat S. A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way. 3rd ed. Academic Press; 2008.
2. Eldar Y, Kutyniok G. Compressed Sensing: Theory and Applications; 2012.
3. Hubel DH, Wiesel TN. Receptive fields of single neurones in the cat's striate cortex. The Journal of Physiology. 1959; 148(3):574-591. doi: 10.1113/jphysiol.1959.sp006308 PMID: 14403679.
4. Olshausen B, Field D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature. 1996; 381:607-9. doi: 10.1038/381607a0 PMID: 8637596.
5. Goodfellow IJ, Courville A, Bengio Y. Scaling up spike-and-slab models for unsupervised feature learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2013; 35(8):1902-1914. doi: 10. 1109/TPAMI.2012.273 PMID: 23787343.
6. Lee H, Battle A, Raina R, Ng A. Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. vol. 20; 2007. p. 801-808.
7. Chen SS, Donoho DL, Michael, Saunders A. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing. 1998; 20:33-61. doi: 10.1137/S1064827596304010.
8. Tibshirani R. Regression shrinkage and selection via the lasso. J Royal Statistical Society Series B. 1996; 58(1):267-288.
9. Murphy KP. Machine Learning: A Probabilistic Perspective. The MIT Press; 2012.
10. Opper M, Winther O. Expectation Consistent Approximate Inference. Journal of Machine Learning Research. 2005;p. 2177-2204.
11. Seeger M. Bayesian Inference and Optimal Design for the Sparse Linear Model. Journal of Machine Learning Research. 2008 June; 9:759-813.
12. Mohamed S, Heller K, Ghahramani Z. Evaluating Bayesian and L1 Approaches for Sparse Unsu-pervised Learning. In: ICML. vol. 29; 2012.
13. Lázaro-gredilla M, Titsias MK. Spike and Slab Variational Inference for Multi-Task and Multiple Kernel Learning. In: Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ, editors. Advances in Neural Information Processing Systems. vol. 24. Curran Associates, Inc.; 2011. p. 2339-2347.
14. Goodfellow I, Courville A, Bengio Y. Large-Scale Feature Learning With Spike-and-Slab Sparse Coding. In: ICML. vol. 29; 2012.
15. Sheikh AS, Shelton J, Lücke J. A Truncated Variational EM Approach for Spike-and-Slab Sparse Coding. Journal of Machine Learning Research. 2014; 15(1):2653-2687.
16. Lücke J, Eggert J. Expectation Truncation And the Benefits of Preselection in Training Generative Models. Journal of Machine Learning Research. 2010; 11:2855-2900.
17. Shelton J, Bornschein J, Sheikh AS, Berkes P, Lücke J. Select and Sample - A Model of Efficient Neural Inference and Learning. Advances in Neural Information Processing Systems. 2011; 24:2618-2626.
18. Ringach D. Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex. Journal of Neurophysiology. 2002; 88:455-63. PMID: 12091567.
19. Bornschein J, Henniges M, Lücke J. Are V1 Simple Cells Optimized for Visual Occlusions? A Comparative Study. PLoS Computational Biology. 2013 06; 9(6):e1003062.
20. Lücke J, Sahani M. Maximal Causes for Non-linear Component Extraction. Journal of Machine Learning Research. 2008; 9:1227-67.
21. Puertas G, Bornschein J, Lücke J. The Maximal Causes of Natural Scenes are Edge Filters. In: Advances in Neural Information Processing Systems. vol. 23; 2010. p. 1939-47.
22. Shelton J, Sterne P, Bornschein J, Sheikh AS, Lücke J. Why MCA? Nonlinear sparse coding with spike-and-slab prior for neurally plausible image encoding. Advances in Neural Information Processing Systems. 2012; 25:2285-2293.
23. Goodfellow I, Courville A, Bengio Y. Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery. In: NIPS Workshop on Challenges in Learning Hierarchical Models; 2011.
24. Donoho DL. Compressed sensing. IEEE Trans Inform Theory. 2006; 52:1289-1306. doi: 10.1109/TIT. 2006.871582.
25. Dayan P, Zemel RS. Competition and Multiple Cause Models. Neural Computation. 1995; 7(3):565- 579. doi: 10.1162/neco.1995.7.3.565.
26. Saund E. A Multiple Cause Mixture Model for Unsupervised Learning. Neural Computation. 1995; 7 (1):51-71. doi: 10.1162/neco.1995.7.1.51.
27. Singliar T, Hauskrecht M. Noisy-OR Component Analysis and its Application to Link Analysis. Journal of Machine Learning Research. 2006; p. 2189-2213.
28. Wood F, Griffiths TL, Ghahramani Z. A Non-Parametric Bayesian Method for Inferring Hidden Causes. In: Uncertainty in Artificial Intelligence. AUAI Press; 2006.
29. Jernite Y, Halpern Y, Sontag D. Discovering Hidden Variables in Noisy-Or Networks using Quartet Tests. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ, editors. Advances in Neural Information Processing Systems 26; 2013. p. 2355-2363.
30. Frolov AA, Husek D, Polyakov PY. Two Expectation-Maximization algorithms for Boolean Factor Analysis. Neurocomputing. 2014; 130:83-97. doi: 10.1016/j.neucom.2012.02.055.
31. Dai Z, Exarchakis G, Lücke J. What Are the Invariant Occlusive Components of Image Patches? A Probabilistic Generative Approach. In: Advances in Neural Information Processing Systems; 2013. p. 243-251.
32. Roweis ST. Factorial models and refiltering for speech separation and denoising. In: Proc. Eu-rospeech. vol. 7; 2003. p. 1009-1012.
33. Valpola H, Oja E, Ilin A, Honkela A, Karhunen J. Nonlinear Blind Source Separation by Variational Bayesian Learning; 1999.
34. Neal RM. Connectionist Learning of Belief Networks. Artificial Intelligence. 1992 Jul; 56(1):71-113. doi: 10.1016/0004-3702(92)90065-6.
35. Neal R, Hinton G. A View of the EM Algorithm that Justifies Incremental, Sparse, and other Variants. In: Jordan MI, editor. Learning in Graphical Models. Kluwer; 1998.
36. Olshausen B, Millman K. Learning sparse codes with a mixture-of-Gaussians prior. Advances in Neural Information Processing Systems. 2000; 12:841-847.
37. Tan X, Li J, Stoica P. Efficient sparse Bayesian learning via Gibbs sampling. In: ICASSP; 2010. p. 3634-3637.
38. Bornschein J, Dai Z, Lücke J. Approximate EM Learning on Large Computer Clusters. In: NIPS Workshop: Learning on Cores, Clusters and Clouds; 2010.
39. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A. Non-Local Sparse Models for Image Restoration. International Conference on Computer Vision. 2009; 25:2272-2279.
40. Jernite Y, Halpern Y, Sontag D. Discovering Hidden Variables in Noisy-Or Networks using Quartet Tests. In: Advances in Neural Information Processing Systems 26. MIT Press; 2013.
41. Olshausen B, Field D. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research. 1997 Dec; 37(23):3311-3325. doi: 10.1016/S0042-6989(97)00169-7 PMID: 9425546.
42. van Hateren JH, van der Schaaf A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London B. 1998; 265:359-66. doi: 10.1098/rspb.1998.0577.
43. Usrey WM, Sceniak MP, Chapman B. Receptive Fields and Response Properties of Neurons in Layer 4 of Ferret Visual Cortex. Journal of Neurophysiology. 2003; 89:1003-1015. doi: 10.1152/jn.00749. 2002 PMID: 12574476.
44. Niell C, Stryker M. Highly Selective Receptive Fields in Mouse Visual Cortex. The Journal of Neuroscience. 2008; 28(30):7520-7536. doi: 10.1523/JNEUROSCI.0623-08.2008 PMID: 18650330.
45. Berkes P, Orban G, Lengyel M, Fiser J. Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment. Science. 2011 Jan; 331(6013):83-87. doi: 10.1126/science. 1195870 PMID: 21212356.
46. Haft M, Hofman R, Tresp V. Generative binary codes. Pattern Anal Appl. 2004; 6:269-84. doi: 10.1007/s10044-003-0194-x.
47. Henniges M, Puertas G, Bornschein J, Eggert J, Lücke J. Binary Sparse Coding. In: Proceedings LVA/ICA. LNCS 6365. Springer; 2010. p. 450-57.
48. Henniges M, Turner RE, Sahani M, Eggert J, Lücke J. Efficient Occlusive Components Analysis. Journal of Machine Learning Research. 2014; 15:2689-2722.
49. Zoran D, Weiss Y. "Natural Images, Gaussian Mixtures and Dead Leaves". In: Bartlett PL, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ, editors. NIPS; 2012. p. 1745-1753.
50. Olshausen B. Highly over complete sparse coding. In: Proc. SPIE, 8651, Human Vision and Electronic Imaging XVIII; 2013.
51. Spratling M. Learning Image Components for Object Recognition. Journal of Machine Learning Research. 2006; 7:793-815.
52. Hoyer PO. Non-negative Matrix Factorization with Sparseness Constraints. Journal of Machine Learning Research. 2004; 5:1457-69.
53. Guan N, Tao D, Luo Z, Shawe-Taylor J. MahNMF: Manhattan Non-negative Matrix Factorization. CoRR. 2012;abs/1207.3438.