A compound memristive synapse model for statistical learning through STDP in spiking neural networks

Frontiers in Neuroscience

Volumn 8, Issue DEC, 2014, Pages

  Bill, Johannes ^a   Legenstein, Robert ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Bayesian inference; Memristor; Neuromorphic; STDP; Synapse; Synaptic plasticity; Unsupervised learning; WTA

Indexed keywords

ARTICLE; COMPUTER SIMULATION; ELECTRIC CONDUCTIVITY; LEARNING; MEMRISTIVE SYNAPSE; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE ENDING; NOISE; POSTSYNAPTIC POTENTIAL; SPIKE; SPIKE TIMING DEPENDENT PLASTICITY; STANDARD; SYNAPSE; SYNAPTIC EFFICACY;

EID: 84920586979     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2014.00412     Document Type: Article

References (47)

1. Bi, G.-Q., and Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464-10472.
2. Caporale, N., and Dan, Y. (2008). Spike timing-dependent plasticity: a hebbian learning rule. Annu. Rev. Neurosci. 31, 25-46. doi: 10.1146/annurev.neuro.31.060407.125639
3. Chicca, E., Stefanini, F., Bartolozzi, C., and Indiveri, G. (2014). Neuromorphic electronic circuits for building autonomous cognitive systems. Proc. IEEE 102, 1367-1388. doi: 10.1109/JPROC.2014.2313954
4. Choi, H., Jung, H., Lee, J., Yoon, J., Park, J., Seong, D.-J., et al. (2009). An electrically modifiable synapse array of resistive switching memory. Nanotechnology 20:345201. doi: 10.1088/0957-4484/20/34/345201
5. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. J. R. Stat. Soc. B 39, 1-38.
6. Douglas, R. J., and Martin, K. A. (2004). Neuronal circuits of the neocortex. Annu. Rev. Neurosci. 27, 419-451. doi: 10.1146/annurev.neuro.27.070203.144152
7. Fang, Z., Yu, H., Li, X., Singh, N., Lo, G., and Kwong, D. (2011). Hfox/tiox/hfox/tiox multilayer-based forming-free rram devices with excellent uniformity. IEEE Elexctron Device Lett. 32, 566-568. doi: 10.1109/LED.2011.2109033
8. Fusi, S. (2002). Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern. 87, 459-470. doi: 10.1007/s00422-002-0356-8
9. Gaba, S., Sheridan, P., Zhou, J., Choi, S., and Lu, W. (2013). Stochastic memristive devices for computing and neuromorphic applications. Nanoscale 5, 5872-5878. doi: 10.1039/c3nr01176c
10. Gerstner, W., and Kistler, W. M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511815706
11. Graça, J. V., Ganchev, K., and Taskar, B. (2007). "Expectation maximization and posterior constraints," in Advances in Neural Information Processing Systems (Vancouver).
12. Habenschuss, S., Bill, J., and Nessler, B. (2012). "Homeostatic plasticity in bayesian spiking networks as expectation maximization with posterior constraints," in Advances in Neural Information Processing Systems (Lake Tahoe), 773-781.
13. Habenschuss, S., Puhr, H., and Maass, W. (2013). Emergence of optimal decoding of population codes through stdp. Neural Comput. 25, 1371-1407. doi: 10.1162/NECO-a-00446
14. Indiveri, G. (2000). Modeling selective attention using a neuromorphic analog vlsi device. Neural Comput. 12, 2857-2880. doi: 10.1162/089976600300014755
15. Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, G., and Prodromakis, T. (2013). Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24:384010. doi: 10.1088/0957-4484/24/38/384010
16. Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297-1301. doi: 10.1021/nl904092h
17. Jo, S. H., Kim, K.-H., and Lu, W. (2009a). High-density crossbar arrays based on a si memristive system. Nano Lett. 9, 870-874. doi: 10.1021/nl8037689
18. Jo, S. H., Kim, K.-H., and Lu, W. (2009b). Programmable resistance switching in nanoscale two-terminal devices. Nano Lett. 9, 496-500. doi: 10.1021/nl803669s
19. Jolivet, R., Rauch, A., Lüscher, H.-R., and Gerstner, W. (2006). Predicting spike timing of neocortical pyramidal neurons by simple threshold models. J. Comput. Neurosci. 21, 35-49. doi: 10.1007/s10827-006-7074-5
20. Kappel, D., Nessler, B., and Maass, W. (2014). Stdp installs in winner-take-all circuits an online approximation to hidden markov model learning. PLoS Comput. Biol. 10:e1003511. doi: 10.1371/journal.pcbi.1003511
21. Keck, C., Savin, C., and Lücke, J. (2012). Feedforward inhibition and synaptic scaling-two sides of the same coin? PLoS Comput. Biol. 8:e1002432. doi: 10.1371/journal.pcbi.1002432
22. Kuzum, D., Jeyasingh, R. G., Lee, B., and Wong, H.-S. P. (2011). Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Lett. 12, 2179-2186. doi: 10.1371/journal.pcbi.1002432
23. Lansner, A. (2009). Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations. Trends Neurosci. 32, 178-186. doi: 10.1016/j.tins.2008.12.002
24. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278-2324. doi: 10.1109/5.726791
25. Lee, D., Seong, D.-J., Jung Choi, H., Jo, I., Dong, R., Xiang, W., et al. (2006a). "Excellent uniformity and reproducible resistance switching characteristics of doped binary metal oxides for non-volatile resistance memory applications," in Electron Devices Meeting, 2006. IEDM'06. International (San Francisco: IEEE), 1-4.
26. Lee, J. H., Ma, X., and Likharev, K. (2006b). Cmol crossnets: possible neuromorphic nanoelectronic circuits. Adv. Neural Inf. Process. Syst. 18:755.
27. Malenka, R. C., and Bear, M. F. (2004). Ltp and ltd: an embarrassment of riches. Neuron 44, 5-21. doi: 10.1016/j.neuron.2004.09.012
28. Markram, H., Gerstner, W., and Sjöström, P. J. (2012). Spike-timing-dependent plasticity: a comprehensive overview. Front. Synaptic Neurosci. 4:2. doi: 10.3389/fnsyn.2012.00002
29. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 275, 213-215. doi: 10.1126/science.275.5297.213
30. Mayr, C., Stärke, P., Partzsch, J., Cederstroem, L., Schüffny, R., Shuai, Y., et al. (2012). "Waveform driven plasticity in bifeo3 memristive devices: model and implementation," in Advances in Neural Information Processing Systems (Lake Tahoe), 1700-1708.
31. Mead, C., and Ismail, M. (1989). Analog VLSI Implementation of Neural Systems. Norwell, MA: Springer. doi: 10.1007/978-1-4613-1639-8
32. Morrison, A., Aertsen, A., and Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Comput. 19, 1437-1467. doi: 10.1162/neco.2007.19.6.1437
33. Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013). Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS Comput. Biol. 9:e1003037. doi: 10.1371/journal.pcbi.1003037
34. Nessler, B., Pfeiffer, M., and Maass, W. (2009). "Stdp enables spiking neurons to detect hidden causes of their inputs," in Advances in Neural Information Processing Systems (Vancouver), 1357-1365.
35. Petrovici, M. A., Bill, J., Bytschok, I., Schemmel, J., and Meier, K. (2013). Stochastic inference with deterministic spiking neurons. arXiv preprint arXiv:1311.3211.
36. Querlioz, D., Bichler, O., and Gamrat, C. (2011). "Simulation of a memristor-based spiking neural network immune to device variations," in Neural Networks (IJCNN), The 2011 International Joint Conference on (IEEE), 1775-1781.
37. Schemmel, J., Fieres, J., and Meier, K. (2008). "Wafer-scale integration of analog neural networks," in Neural Networks, 2008. IJCNN 2008 (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on (Hong Kong: IEEE), 431-438.
38. Serrano-Gotarredona, T., Masquelier, T., Prodromakis, T., Indiveri, G., and Linares-Barranco, B. (2013). Stdp and stdp variations with memristors for spiking neuromorphic learning systems. Front. Neurosci. 7:2. doi: 10.3389/fnins.2013.00002
39. Simoncelli, E. P., Paninski, L., Pillow, J., and Schwartz, O. (2004). Characterization of neural responses with stochastic stimuli. Cogn. Neurosci. 3, 327-338.
40. Snider, G. S. (2008). "Spike-timing-dependent learning in memristive nanodevices," in Nanoscale Architectures, 2008. NANOARCH 2008. IEEE International Symposium (Anaheim, CA: IEEE), 85-92.
41. Suri, M., Querlioz, D., Bichler, O., Palma, G., Vianello, E., Vuillaume, D., et al. (2013). Bio-inspired stochastic computing using binary cbram synapses. Elect. Devices IEEE Trans. 60, 2402-2409. doi: 10.1109/TED.2013.2263000
42. Van Rossum, M. C., Bi, G. Q., and Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812-8821.
43. Vincent, A., Larroque, J., Zhao, W., Romdhane, N. B., Bichler, O., Gamrat, C., et al. (2014). "Spin-transfer torque magnetic memory as a stochastic memristive synapse," in Circuits and Systems (ISCAS), 2014 IEEE International Symposium (Melbourne: IEEE), 1074-1077.
44. Wu, S.-Y., Liaw, J., Lin, C., Chiang, M., Yang, C., Cheng, J., et al. (2009). "A highly manufacturable 28nm cmos low power platform technology with fully functional 64mb sram using dual/tripe gate oxide process," in VLSI Technology, 2009 Symposium on (IEEE), 210-211.
45. Yang, J. J., Strukov, D. B., and Stewart, D. R. (2013). Memristive devices for computing. Nat. Nanotechnol. 8, 13-24. doi: 10.1038/nnano.2012.240
46. Yu, S., Gao, B., Fang, Z., Yu, H., Kang, J., and Wong, H.-S. P. (2013). Stochastic learning in oxide binary synaptic device for neuromorphic computing. Front. Neurosci. 7:186. doi: 10.3389/fnins.2013.00186
47. Zamarreño-Ramos, C., Camuñas-Mesa, L. A., Pérez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5:26. doi: 10.3389/fnins.2011.00026