A common network architecture efficiently implements a variety of sparsity-based inference problems

Neural Computation

Volumn 24, Issue 12, 2012, Pages 3317-3339

  Charles, Adam S ^a   Garrigues, Pierre ^b   Rozell, Christopher J ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b IQ Engines Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMAN; PHYSIOLOGY; SOMATOSENSORY CORTEX;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; SOMATOSENSORY CORTEX;

EID: 84871006492     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00372     Document Type: Article

References (50)

1. Antoniadis, A., & Fan, J. (2001). Regularization of wavelet approximations. Journal of the American Statistical Association, 96, 939-967.
2. Balavoine, A., Romberg, J., & Rozell, C. (2012). Convergence and rate analysis of neural networks for sparse approximation. IEEE Transactions on Neural Networks and Learning Systems, 23, 1377-1389.
3. Battaglia, P., Jacobs, R.,& Aslin, R. (2003). Bayesian integration of visual and auditory signals for spatial localization. JOSA A, 20, 1391-1397.
4. Baum, E., Moody, J., & Wilczek, F. (1988). Internal representations for associative memory. Biological Cybernetics, 59, 217-228.
5. Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
6. Cadieu, C. F., & Olshausen, B. A. (2012). Learning intermediate-level representations of form and motion from natural movies. Neural Computation, 24, 827-866.
7. Candès, E., Wakin, M., & Boyd, S. (2008). Enhancing sparsity by reweighted ℓ1 minimization. Journal of Fourier Analysis and Applications, 14, 877-905.
8. Charles, A. S., Yap, H. L., & Rozell, C. J. (2012). Short-term memory capacity in recurrent networks via compressed sensing. Cosyne Abstracts 2012, Salt Lake City, UT.
9. Chartrand, R., & Yin, W. (2008). Iteratively reweighted algorithms for compressive sensing. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 3869-3872). Piscataway, NJ: IEEE.
10. Coen-Cagli, R., Dayan, P., & Schwartz, O. (2012). Cortical surround interactions and perceptual salience via natural scene statistics. PLoS Comput. Biol., 8, e1002405.
11. Doya, K. (2007). Bayesian brain: Probabilistic approaches to neural coding. Cambridge, MA: MIT Press.
12. Elad, M., Figueiredo, M., & Ma, Y. (2010). On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 98, 972-982.
13. Elad, M., Matalon, B., & Zibulevsky, M. (2007). Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization. Applied and Computational Harmonic Analysis, 23, 346-367.
14. Eldar, Y. C., Kuppinger, P., & Bolcskei, H. (2010). Block-sparse signals: Uncertainty relationships and efficient recovery. IEEE Transactions on Signal Processing, 58, 3042-3054.
15. Fan, J. (1997). Comments on & quot;Wavelets in statistics: A review& quot; by A. Antoniadis. Statistical Methods and Applications, 6, 131-138.
16. Figueiredo, M.A.T., & Nowak, R. D. (2001). Wavelet-based image estimation: An empirical Bayes approach using Jeffrey's noninformative prior. IEEE Transactions on Image Processing, 10, 1322-1331.
17. Figueiredo, M.A.T., Nowak, R. D., & Wright, S. J. (2007). Gradient projection for sparse reconstruction:Application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 1, 586-597.
18. Gao, H. (2001).Wavelet shrinkage denoising using the non-negative Garrote. Journal of Computational and Graphical Statistics, 7, 469-488.
19. Garrigues, P., & Olshausen, B. (2010). Group sparse coding with a Laplacian scale mixture prior. In J. Lafferty, C.K.J. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.), Advances in neural information processing systems, 23 (pp. 1-9). Red Hook, NY: Curran.
20. Haider, B., Krause, M., Duque, A., Yu, Y., Touryan, J., Mazer, J., et al. (2010). Synaptic and network mechanisms of sparse and reliable visual cortical activity during nonclassical receptive field stimulation. Neuron, 65, 107-121.
21. Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554-2558.
22. Hu, T., Genkin, A., & Chklovskii, D. B. (2012). A network of spiking neurons for computing sparse representations in an energy efficientway. Neural Computation, 24, 2852-2872.
23. Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Annals of Statistics, 1, 799-821.
24. Hürlimann, F., Kiper, D., & Carandini, M. (2002). Testing the Bayesian model of perceived speed. Vision Research, 42, 2253-2257.
25. Karklin, Y., & Lewicki, M. (2008). Emergence of complex cell properties by learning to generalize in natural scenes. Nature, 457, 83-86.
26. Khajehnejad, M., Xu,W., Avestimehr, S., & Hassibi, B. (2010). Improved sparse recovery thresholds with two-step reweighted ℓ1 minimization. arXiv:1004.0402.
27. Lennie, P. (2003). The cost of cortical computation. Current Biology, 13, 493-497.
28. Nikolova, M. (2000). Local strong homogeneity of a regularized estimator. SIAM Journal on Applied Mathematics, 61, 633-658.
29. Olshausen, B., & Field, D. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607-609.
30. Olshausen, B. A., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311-3325.
31. Olshausen, B., & Field, D. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14, 481-487.
32. Perrinet, L., Samuelides, M., & Thorpe, S. (2004). Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit. Neurocomputing, 57, 125-134.
33. Rao, R., & Ballard, D. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2, 79-87.
34. Rehn, M., & Sommer, F. T. (2007). A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. Journal of Computational Neuroscience, 22, 135-146.
35. Rozell, C., & Garrigues, P. (2010). Analog sparse approximation for compressed sensing recovery. In Proceedings of the 2010 ASILOMAR Conference on Signals, Systems and Computers (pp. 822-826). Piscataway, NJ: IEEE.
36. Rozell, C. J., Johnson,D.H., Baraniuk, R.G.,& Olshausen, B. A. (2010). Sparse coding via thresholding and local competition in neural circuits. Neural Computation, 20, 2526-2563.
37. Saab, R., Chartrand, R., & Yilmaz, O. (2008). Stable sparse approximations via nonconvex optimization. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal (pp. 3885-3888). Piscataway, NJ: IEEE.
38. Schwartz, O., & Simoncelli, E. (2001). Natural signal statistics and sensory gain control. Nature Neuroscience, 4, 819-825.
39. Seriès, P., Lorenceau, J., & Frégnac, Y. (2003). The silent surround of V1 receptive fields: Theory and experiments. Journal of Physiology-Paris, 97, 453-474.
40. Shapero, S., Rozell, C. J., & Hasler, P. (2012). Configurable hardware integrate and fire neurons for sparse approximation. Manuscript submitted for publication.
41. Shapero, S., Charles, A., Rozell, C. J., & Hasler, P. (in press). Low power sparse approximation on reconfigurable analog hardware. IEEE Journal on Emerging and Selected Topics in Circuits and Systems.
42. Spratling, M. (2011). A single functional model accounts for the distinct properties of suppression in cortical area V1. Vision Research, 51, 563-576.
43. Tikhonov,A. (1963). Regularization of incorrectly posed problems. Soviet Math. Dokl, 4, 1624-1627.
44. Twigg, C., & Hasler, P. (2009). Configurable analog signal processing. Digital Signal Processing, 19, 904-922.
45. Vinje,W., & Gallant, J. (2002). Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1. Journal of Neuroscience, 22, 2904-2915.
46. Wipf, D., & Nagarajan, S. (2010). Iterative reweighted ℓ1 and ℓ2 methods for finding sparse solutions. IEEE Journal of Selected Topics in Signal Processing, 4, 317-329.
47. Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., & Yan, S. (2010). Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 98, 1031-1044.
48. Zhu, M., & Rozell, C. J. (2012). Population characteristics and interpretations of NCRF effects emerging from sparse coding. Cosyne Abstracts 2012, Salt Lake City, UT.
49. Zou, H. (2006). The adaptive LASSO and its oracle properties. Journal of the American Statistical Association, 101, 1418-1429.
50. Zylberberg, J., Murphy, J. T., & DeWeese, M. R. (2011). A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields. PLoS Comput. Biol., 7, e1002250.