Improved sparse coding under the influence of perceptual attention

Neural Computation

Volumn 26, Issue 2, 2014, Pages 377-420

  Amiri, Ashkan ^a   Haykin, Simon ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ATTENTION; BAYES THEOREM; HUMAN; LETTER; PERCEPTION; PHYSIOLOGY;

ALGORITHMS; ATTENTION; BAYES THEOREM; HUMANS; PERCEPTION;

EID: 84893020672     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00546     Document Type: Letter

References (70)

1. Abdallah, S., & Plumbley, M. (2006). Unsupervised analysis of polyphonic music by sparse coding. IEEE Transactions on Neural Networks, 17(1), 179-196.
2. Ablowitz, M., & Fokas, A. (2003). Complex variables: Introduction and applications. Cambridge: Cambridge University Press.
3. Adali, T., & Haykin, S. (2010). Adaptive signal processing: Next generation solutions. Hoboken, NJ: Wiley-IEEE Press.
4. Anastassopoulos, V., Lampropoulos, G. A., Drosopoulos, A., & Rey, M. (1999). High resolution radar clutter statistics. IEEE Transactions on Aerospace and Electronic Systems, 35(1), 43-60.
5. Arbib, M. A. (1981). Perceptual structures and distributed motor control. In V. B. Brooks (Ed.), Handbook of Physiology-The Nervous System. II: Motor Control (pp. 1449-1480). Bethesda, MD: American Physiological Society.
6. Au, W. W. (1993). The sonar of dolphins. New York: Springer.
7. Bar-Shalom, Y., Li, X. R., & Kirubarajan, T. (2001). Estimation with applications to tracking and navigation: Theory, algorithms and software. Hoboken, NJ: Wiley- Interscience.
8. Barlow, H. (1972). Single units and sensation: A neuron doctrine for perceptual psychology. Perception, 1(4), 371-394.
9. Berkes, P., & Wiskott, L. (2005). Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6), 579-602.
10. Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
11. Buchler, E. (1976). The use of echolocation by the wandering shrew (Sorex vagrans). Animal Behaviour, 24(4), 858-873.
12. Cadieu, C. F., & Olshausen, B. A. (2012). Learning intermediate-level representations of form and motion from natural movies. Neural Computation, 24(4), 827-866.
13. Chang, C.-C., & Lin, C.-J. (2011). Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.
14. Chikkerur, S., Serre, T., Tan, C., & Poggio, T. (2010). What & where: A Bayesian inference theory of attention. Vision Research, 50(22), 2233-2247.
15. Conte, E., De Maio, A., & Galdi, C. (2004). Statistical analysis of real clutter at different range resolutions. IEEE Transactions on Aerospace and Electronic Systems, 40(3), 903-918.
16. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193-222.
17. 1 minimization. Proceedings of the National Academy of Sciences, 100(5), 2197.
18. Donoho, D., Elad, M.,&Temlyakov, V. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory, 52(1), 6-18.
19. Einhäuser, W., Kayser, C., König, P., & Körding, K. (2002). Learning the invariance properties of complex cells from their responses to natural stimuli. European Journal of Neuroscience, 15(3), 475-486.
20. Field, D. (1994). What is the goal of sensory coding? Neural Computation, 6(4), 559-601.
21. Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3(2), 194-200.
22. Fraser, D. C. (1967). A new technique for the optimal smoothing of data. Unpublished doctoral dissertation, MIT.
23. Fuster, J. (1989). The prefrontal cortex (2nd ed.) New York: Raven Press.
24. Fuster, J. (2003). Cortex and mind: Unifying cognition. New York: Oxford University Press.
25. Fuster, J. (2008). The prefrontal cortex (4th ed.) Orlando, FL: Academic Press.
26. Gao, J., Hu, J., Tung, W.-W., Cao, Y., Sarshar, N., & Roychowdhury, V. P. (2006). Assessment of long-range correlation in time series: How to avoid pitfalls. Phys. Rev. E, 73, 016117.
27. Griffin, D. R. (1944). Echolocation by blind men, bats and radar. Science, 100(2609), 589-590.
28. Hadamard, J. (1902). Sur les problémes aux dérivées partielles et leur signification physique. Princeton University Bulletin, 13 (49-52), 28.
29. Haykin, S. (2009). Neural networks and learning machines. Upper Saddle River, NJ: Prentice Hall.
30. Haykin, S. (2012). Cognitive dynamic systems. Cambridge: Cambridge University Press.
31. Haykin, S. (2013). Adaptive filter theory (5th ed.). Upper Saddle River, NJ: Pearson.
32. Haykin, S., Xue, Y., & Setoodeh, P. (2012). Cognitive radar: Step toward bridging the gap between neuroscience and engineering. Proceedings of the IEEE, 100(11), 3102-3130.
33. Hu, J., Tung, W., & Gao, J. (2006). Detection of low observable targets within sea clutter by structure function based multifractal analysis. IEEE Transactions on Antennas and Propagation, 54(1), 136-143.
34. Hyvärinen, A. (1999). Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Computation, 11(7), 1739-1768.
35. Hyvärinen, A., & Hoyer, P. (2000). Emergence of phase-and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12(7), 1705-1720.
36. Hyvärinen, A., Hurri, J., & Väyrynen, J. (2003). Bubbles: A unifying framework for low-level statistical properties of natural image sequences. JOSA A, 20(7), 1237-1252.
37. Karklin, Y., & Lewicki, M. (2005). A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals. Neural Computation, 17(2), 397-423.
38. Knill, D. C., & Richards, W. (1996). Perception as Bayesian inference. Cambridge: Cambridge University Press.
39. Körding, K., &Wolpert, D. (2006). Bayesian decision theory in sensorimotor control. Trends in Cognitive Sciences, 10(7), 319-326.
40. Kreutz-Delgado, K. (2009). The complex gradient operator and the CR-calculus. arXiv preprint arXiv:0906.4835.
41. LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks. Cambridge, MA: MIT Press.
42. Lee, T. S.,&Mumford, D. (2003). Hierarchical Bayesian inference in the visual cortex. JOSA A, 20(7), 1434-1448.
43. Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182-1195.
44. Mehra, R. (1970). On the identification of variances and adaptive Kalman filtering. IEEE Transactions on Automatic Control, 15(2), 175-184.
45. Miller, E. K.,&Cohen, J. D. (2001).An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24(1), 167-202.
46. Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. New York: Freeman.
47. Neuweiler, G. (1984). Foraging, echolocation and audition in bats. Naturwissenschaften, 71(9), 446-455.
48. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583), 607-609.
49. Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by VI?. Vision Research, 37(23), 3311-3326.
50. Picinbono, B. (1996). Second-order complex random vectors and normal distributions. IEEE Transactions on Signal Processing, 44(10), 2637-2640.
51. Posner, M. I. (2011). Cognitive neuroscience of attention (2nd ed.). New York: Guilford Press.
52. Raina, R., Battle, A., Lee, H., Packer, B., & Ng, A. Y. (2007). Self-taught learning: Transfer learning from unlabeled data. In Proceedings of the 24th International conference on Machine Learning (pp. 759-766). New York: ACM.
53. Ranzato, M., Huang, F. J., Boureau, Y. L., & LeCun, Y. (2007). Unsupervised learning of invariant feature hierarchies with applications to object recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2007 (pp. 1-8). Piscataway, NJ: IEEE.
54. Rao, R. P. (2005). Bayesian inference and attentional modulation in the visual cortex. Neuroreport, 16(16), 1843-1848.
55. Rao, R., Olshausen, B. A., & Lewicki, M. (2002). Probabilistic models of the brain: Perception and neural function. Cambridge, MA: MIT Press.
56. Remmert, R. (1991). Theory of complex functions. NewYork: Springer.
57. Reynolds, J. H., & Chelazzi, L. (2004). Attentional modulation of visual processing. Annual Review of Neuroscience, 27, 611-647.
58. Rojas, J. A. M., Hermosilla, J. A., Montero, R. S., & Espi, P. L. L. (2009). Physical analysis of several organic signals for human echolocation: Oral vacuum pulses. Acta Acustica United with Acustica, 95(2), 325-330.
59. Sangston, K., Gini, F., & Greco, M. (2010). New results on coherent radar target detection in heavy-tailed compound-gaussian clutter. In Proceedings of the Radar Conference, 2010 (pp. 779-784). Piscataway, NJ: IEEE.
60. Tikhonov, A. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl., 5, 1035.
61. Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006). Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychological Review, 113(4), 766-786.
62. Tropp, J. (2006). Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Transactions on Information Theory, 52(3), 1030-1051.
63. Tropp, J., & Wright, S. (2010). Computational methods for sparse solution of linear inverse problems. Proceedings of the IEEE, 98(6), 948-958.
64. Van Trees, H. (1968). Detection, estimation, and modulation theory, Part I. New York: Wiley.
65. von Weizsäcker, V. (1950). Der Gestaltkreis. Stuttgart: Thieme.
66. Wiener, N. (1948). Cybernetics: Or the control and communication in the animal and the machine. NewYork: Wiley.
67. Wirtinger, W. (1927). Zur formalen theorie der funktionen von mehr komplexen veränderlichen. Mathematische Annalen, 97(1), 357-375.
68. Wiskott, L., & Sejnowski, T. (2002). Slow feature analysis: Unsupervised learning of invariances. Neural Computation, 14(4), 715-770.
69. Zetzsche, C. (1990). Sparse coding: The link between low level vision and associative memory. In G. Hartmann, R. Eckmiller, & G. Hauske (Eds.), Parallel processing in neural systems and computers (pp. 273-276). Amsterdam: North-Holland.
70. Zetzsche, C. (1990). Sparse coding: The link between low level vision and associative memory. In G. Hartmann, R. Eckmiller, & G. Hauske (Eds.), Parallel processing in neural systems and computers (pp. 273-276). Amsterdam: North-Holland.