Unsupervised learning of digit recognition using spike-timing-dependent plasticity

Frontiers in Computational Neuroscience

Volumn 9, Issue AUGUST, 2015, Pages

  Diehl, Peter U ^a   Cook, Matthew ^a  

^a Institute of Neuroinformatics Zurich   (Switzerland)

Author keywords

Classification; Digit recognition; Spiking neural network; STDP; Unsupervised learning

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPLEX NETWORKS; INTELLIGENT AGENTS; MAMMALS; NEURAL NETWORKS; NEURONS; PATTERN RECOGNITION; UNSUPERVISED LEARNING;

BIOLOGICAL NEURAL NETWORKS; DIGIT RECOGNITION; DOMAIN-SPECIFIC KNOWLEDGE; SPIKE TIMING DEPENDENT PLASTICITIES; SPIKING NEURAL NETWORK(SNN); SPIKING NEURAL NETWORKS; STDP; TIME-DEPENDENT WEIGHTS;

NETWORK ARCHITECTURE;

ARTICLE; CONTROLLED STUDY; DIGIT RECOGNITION; INTUITION; MATHEMATICAL COMPUTING; MENTAL TASK; NERVE CELL NETWORK; NERVE CELL PLASTICITY; PATTERN RECOGNITION; RECOGNITION; SIGNAL DETECTION; SPIKE; SPIKING NEURAL NETWORK; TASK PERFORMANCE; WEIGHT CHANGE;

EID: 84940184638     PISSN: None     EISSN: 16625188     Source Type: Journal    
DOI: 10.3389/fncom.2015.00099     Document Type: Article

References (49)

1. Abbott, L., and Song, S. (1999). Temporally asymmetric hebbian learning, spike timing and neuronal response variability. Adv. Neural Inform. Process. Syst. 11, 69-75.
2. Azghadi, M. R., Iannella, N., Al-Sarawi, S., and Abbott, D. (2014). Tunable low energy, compact and high performance neuromorphic circuit for spike-based synaptic plasticity. PLoS ONE 9:e88326. doi: 10.1371/journal.pone.0088326
3. Barroso, L. A. (2005). The price of performance. Queue 3, 48-53. doi: 10.1145/1095408.1095420
4. Benjamin, B. V., Gao, P., McQuinn, E., Choudhary, S., Chandrasekaran, A. R., Bussat, J., et al. (2014). Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations. Proc. IEEE 102, 699-716. doi: 10.1109/JPROC.2014.2313565
5. Beyeler, M., Dutt, N. D., and Krichmar, J. L. (2013). Categorization and decision-making in a neurobiologically plausible spiking network using a stdp-like learning rule. Neural Netw. 48, 109-124. doi: 10.1016/j.neunet.2013.07.012
6. Bi, G.-Q., and Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464-10472.
7. Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J.-P., and Gamrat, C. (2012). Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity. Neural Netw. 32, 339-348. doi: 10.1016/j.neunet.2012.02.022
8. Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881-2912. doi: 10.1162/neco.2007.19.11.2881
9. Coates, A., and Ng, A. Y. (2012). "Learning feature representations with k-means," in Neural Networks: Tricks of the Trade, Vol. 7700, eds G. Montavon, G. B. Orr, and K. R. Müller (Berlin; Heidelberg: Springer), 561-580. doi: 10.1007/978-3-642-35289-8_30
10. Diehl, P. U., and Cook, M. (2014). "Efficient implementation of stdp rules on spinnaker neuromorphic hardware," in Neural Networks (IJCNN), 2014 International Joint Conference on (Beijing: IEEE), 4288-4295.
11. Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., and Pfeiffer, M. (2015). "Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing," in Neural Networks (IJCNN), 2015 International Joint Conference on(Killarney: IEEE).
12. Fritzke, B. (1995). A growing neural gas network learns topologies. Adv. Neural Inform. Process. Syst. 7, 625-632.
13. Galluppi, F., Lagorce, X., Stromatias, E., Pfeiffer, M., Plana, L. A., Furber, S. B., et al. (2014). A framework for plasticity implementation on the spinnaker neural architecture. Front. Neurosci. 8:429. doi: 10.3389/fnins.2014.00429
14. Goodhill, G. J., and Barrow, H. G. (1994). The role of weight normalization in competitive learning. Neural Comput. 6, 255-269. doi: 10.1162/neco.1994.6.2.255
15. Goodman, D., and Brette, R. (2008). Brian: a simulator for spiking neural networks in python. Front. Neuroinform. 2:5. doi: 10.3389/neuro.11.005.2008
16. Habenschuss, S., Bill, J., and Nessler, B. (2012). "Homeostatic plasticity in bayesian spiking networks as expectation maximization with posterior constraints," in Advances in Neural Information Processing Systems, eds P. Bartlett, F. C. N. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Lake Tahoe, CA: Nips Foundation), 773-781.
17. Hinton, G. E., and Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science 313, 504-507. doi: 10.1126/science.1127647
18. Hussain, S., Liu, S.-C., and Basu, A. (2014). "Improved margin multi-class classification using dendritic neurons with morphological learning," in Circuits and Systems (ISCAS), 2014 IEEE International Symposium on (Melbourne, VIC: IEEE), 2640-2643.
19. Indiveri, G., Chicca, E., and Douglas, R. (2006). A vlsi array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. Neural Netw. IEEE Trans. 17, 211-221. doi: 10.1109/TNN.2005.860850
20. Javed, F., He, Q., Davidson, L. E., Thornton, J. C., Albu, J., Boxt, L., et al. (2010). Brain and high metabolic rate organ mass: contributions to resting energy expenditure beyond fat-free mass. Am. J. Clin. Nutr. 91, 907-912. doi: 10.3945/ajcn.2009.28512
21. Jug, F. (2012). On Competition and Learning in Cortical Structures. Ph.D. thesis, Diss., Eidgenössische Technische Hochschule ETH Zürich, Nr. 20194, 2012.
22. Khan, M., Lester, D., Plana, L., Rast, A., Jin, X., Painkras, E., et al. (2008). "Spinnaker: mapping neural networks onto a massively-parallel chip multiprocessor," in Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on (Hong Kong: IEEE), 2849-2856.
23. Kheradpisheh, S. R., Ganjtabesh, M., and Masquelier, T. (2015). Bio-inspired unsupervised learning of visual features leads to robust invariant object recognition. arXiv 1504.03871, 1-19.
24. Kohonen, T. (1990). The self-organizing map. Proc. IEEE 78, 1464-1480.
25. Larochelle, H., Bengio, Y., Louradour, J., and Lamblin, P. (2009). Exploring strategies for training deep neural networks. J. Mach. Learn. Res. 10, 1-40.
26. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE86, 2278-2324. doi: 10.1109/5.726791
27. Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2010). "A signed spatial contrast event spike retina chip," in Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on (Paris: IEEE), 2438-2441.
28. Lichtsteiner, P., Posch, C., and Delbruck, T. (2008). A 128 × 128 120 db 15 μs latency asynchronous temporal contrast vision sensor. Solid State Circ. IEEE J. 43, 566-576. doi: 10.1109/JSSC.2007.914337
29. Masquelier, T., and Thorpe, S. J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity.PLoS Comput. Biol. 3:e31. doi: 10.1371/journal.pcbi.0030031
30. Mayr, C., Partzsch, J., Noack, M., Hanzsche, S., Scholze, S., Hoppner, S., et al. (2015). A biological-realtime neuromorphic system in 28 nm CMOS using low-leakage switched capacitor circuits. IEEE Trans. Biomed. Circuits Syst. doi: 10.1109/TBCAS.2014.2379294. [Epub ahead of print].
31. McClelland, J. L., Rumelhart, D. E., Asanuma, C., Kawamoto, A. H., Smolensky, P., Crick, F. H. C., et al. (1986). Parallel distributed processing. Explor. Microstruct. Cogn. 2, 1-567.
32. Merolla, P., Arthur, J., Akopyan, F., Imam, N., Manohar, R., and Modha, D. S. (2011). "A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm," in Custom Integrated Circuits Conference (CICC), 2011 IEEE (San Jose, CA: IEEE), 1-4.
33. Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., et al. (2014). A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668-673. doi: 10.1126/science.1254642
34. Morrison, A., Aertsen, A., and Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Comput. 19, 1437-1467. doi: 10.1162/neco.2007.19.6.1437
35. Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K., and Cauwenberghs, G. (2013). Event-driven contrastive divergence for spiking neuromorphic systems. Front. Neurosci. 7:272. doi: 10.3389/fnins.2013.00272
36. Neil, D., and Liu, S.-C. (2014). Minitaur, an event-driven fpga-based spiking network accelerator. Very Large Scale Int. Syst. IEEE Trans. 22, 2621-2628. doi: 10.1109/TVLSI.2013.2294916
37. Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013). Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS Comput. Biol. 9:e1003037. doi: 10.1371/journal.pcbi.1003037
38. O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., and Pfeiffer, M. (2013). Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7:178. doi: 10.3389/fnins.2013.00178
39. O'Reilly, R. C., and Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. MITpress.
40. Park, J., Ha, S., Yu, T., Neftci, E., and Cauwenberghs, G. (2014). "A 65k-neuron 73-mevents/s 22-pj/event asynchronous micro-pipelined integrate-and-fire array transceiver," in Biomedical Circuits and Systems Conference (BioCAS) (Lausanne: IEEE).
41. Pfister, J.-P., and Gerstner, W. (2006). Triplets of spikes in a model of spike timing-dependent plasticity. J. Neurosci. 26, 9673-9682. doi: 10.1523/JNEUROSCI.1425-06.2006
42. Posch, C., Matolin, D., and Wohlgenannt, R. (2010). "High-dr frame-free pwm imaging with asynchronous aer intensity encoding and focal-plane temporal redundancy suppression," in Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on (Paris: IEEE), 2430-2433.
43. Querlioz, D., Bichler, O., Dollfus, P., and Gamrat, C. (2013). Immunity to device variations in a spiking neural network with memristive nanodevices. Nanotechnol. IEEE Trans. 12, 288-295. doi: 10.1109/TNANO.2013.2250995
44. Querlioz, D., Bichler, O., and Gamrat, C. (2011a). "Simulation of a memristor-based spiking neural network immune to device variations," in Neural Networks (IJCNN), The 2011 International Joint Conference on (San Jose, CA: IEEE), 1775-1781.
45. Querlioz, D., Dollfus, P., Bichler, O., and Gamrat, C. (2011b). "Learning with memristive devices: How should we model their behavior?," in Nanoscale Architectures (NANOARCH), 2011 IEEE/ACM International Symposium on (San Diego, CA: IEEE), 150-156.
46. Rahimi Azghadi, M., Iannella, N., Al-Sarawi, S. F., Indiveri, G., and Abbott, D. (2014). Spike-based synaptic plasticity in silicon: Design, implementation, application, and challenges. Proc. IEEE 102, 717-737. doi: 10.1109/JPROC.2014.2314454
47. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1985). Learning internal representations by error propagation. Technical report, DTIC Document.
48. Zhang, W., and Linden, D. J. (2003). The other side of the engram: experience-driven changes in neuronal intrinsic excitability.Nat. Rev. Neurosci. 4, 885-900. doi: 10.1038/nrn1248
49. Zhao, B., Ding, R., Chen, S., Linares-Barranco, B., and Tang, H. (2014). Feedforward categorization on AER motion events using cortex-like features in a spiking neural network. IEEE Trans. Neural Netw. Learn. Sys. 54, 981-993. doi: 10.1109/TNNLS.2014.2362542