{"type":"ScholarlyArticle","title":"Brain-inspired Balanced Tuning for Spiking Neural Networks","authors":[{"name":"Tielin Zhang","slug":"tielin-zhang","given":"Tielin","family":"Zhang","affiliation":"Institute of Automation , Chinese Academy of Sciences (CAS) , China; Research Center for Brain-inspired Intelligence , Institute of Automation , CAS , China"},{"name":"Yi Zeng","slug":"yi-zeng","given":"Yi","family":"Zeng","affiliation":"Institute of Automation , Chinese Academy of Sciences (CAS) , China; University of Chinese Academy of Sciences , China; Research Center for Brain-inspired Intelligence , Institute of Automation , CAS , China; National Laboratory of Pattern Recognition , Institute of Automation , CAS , China; Center for Excellence in Brain Science and Intelligence Technology , CAS , China"},{"name":"Dongcheng Zhao","slug":"dongcheng-zhao","given":"Dongcheng","family":"Zhao","affiliation":"Institute of Automation , Chinese Academy of Sciences (CAS) , China; University of Chinese Academy of Sciences , China; Research Center for Brain-inspired Intelligence , Institute of Automation , CAS , China"},{"name":"Bo Xu","slug":"bo-xu","given":"Bo","family":"Xu","affiliation":"Institute of Automation , Chinese Academy of Sciences (CAS) , China; University of Chinese Academy of Sciences , China; Research Center for Brain-inspired Intelligence , Institute of Automation , CAS , China; Center for Excellence in Brain Science and Intelligence Technology , CAS , China"}],"abstract":"Due to the nature of Spiking Neural Networks (SNNs), it is challenging to be trained by biologically plausible learning principles. The multi-layered SNNs are with non-differential neurons, temporary-centric synapses, which make them nearly impossible to be directly tuned by back propagation. Here we propose an alternative biological inspired balanced tuning approach to train SNNs. The approach contains three main inspirations from the brain: Firstly, the biological network will usually be trained towards the state where the temporal update of variables are equilibrium (e.g. membrane potential); Secondly, specific proportions of excitatory and inhibitory neurons usually contribute to stable representations; Thirdly, the short-term plasticity (STP) is a general principle to keep the input and output of synapses balanced towards a better learning convergence. With these inspirations, we train SNNs with three steps: Firstly, the SNN model is trained with three brain-inspired principles; then weakly supervised learning is used to tune the membrane potential in the final layer for network classification; finally the learned information is consolidated from membrane potential into the weights of synapses by Spike-Timing Dependent Plasticity (STDP). The proposed approach is verified on the MNIST hand-written digit recognition dataset and the performance (the accuracy of 98.64%) indicates that the ideas of balancing state could indeed improve the learning ability of SNNs, which shows the power of proposed brain-inspired approach on the tuning of biological plausible SNNs.","doi":"10.24963/ijcai.2018/229","url":"https://ijcai.org/proceedings/2018/229","pdf":"https://ijcai.org/proceedings/2018/0229.pdf","markdown":"https://ijcai.org/proceedings/2018/229.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/229","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Humans and AI","track":"Main track","number":229,"pages":{"first":1653,"last":1659},"keywords":["Humans and AI: Brain Sciences","Humans and AI: Cognitive Modeling"],"videos":[],"references":[{"position":1,"raw":"[Abbott et al., 2016] LF Abbott, Brian DePasquale, and Raoul-Martin Memmesheimer. Building functional net- works of spiking model neurons. Nature neuroscience, 19(3):350-355, 2016.","title":"Building functional networks of spiking model neurons","authors":"Abbott","year":"2016","venue":"Nature neuroscience","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abraham and Bear, 1996] Wickliffe C Abraham and Mark F Bear. Metaplasticity: the plasticity of synaptic plasticity. Trends in neurosciences, 19(4):126-130, 1996.","title":"Metaplasticity: the plasticity of synaptic plasticity","authors":"C Wickliffe, Mark F Abraham, Bear","year":"1996","venue":"Trends in neurosciences","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Alemi et al., 2018] Alireza Alemi, Christian Machens, So- phie Denève, and Jean-Jacques Slotine. Learning nonlin- ear dynamics in efficient, balanced spiking networks using local plasticity rules. In The 32th AAAI Conference on Ar- tificial Intelligence (AAAI-2018), 2018.","title":"Learning nonlinear dynamics in efficient, balanced spiking networks using local plasticity rules","authors":"Alireza Alemi, Christian Machens, Sophie Denève, Jean-Jacques Slotine","year":"2018","venue":"The 32th AAAI Conference on Artificial Intelligence (AAAI-2018)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bengio et al., 2015] Yoshua Bengio, Thomas Mesnard, As- ja Fischer, Saizheng Zhang, and Yuhuai Wu. Stdp as presy- naptic activity times rate of change of postsynaptic activi- ty. arXiv preprint arXiv:1509.05936, 2015.","title":"Stdp as presynaptic activity times rate of change of postsynaptic activity","authors":"Yoshua Bengio, Thomas Mesnard, Asja Fischer, Saizheng Zhang, Yuhuai Wu","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bi and Poo, 2001] Guo-qiang Bi and Mu-ming Poo. Synap- tic modification by correlated activity: Hebb's postulate revisited. Annual review of neuroscience, 24(1):139-166, 2001.","title":"Synaptic modification by correlated activity: Hebb's postulate revisited","authors":"Guo-Qiang Bi, Mu-Ming Poo","year":"2001","venue":"Annual review of neuroscience","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bohte et al., 2002] Sander M Bohte, Joost N Kok, and Han La Poutre. Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing, 48(1):17- 37, 2002.","title":"Error-backpropagation in temporally encoded networks of spiking neurons","authors":"Joost N Sander M Bohte, Han Kok, Poutre La","year":"2002","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Diehl and Cook, 2015] Peter U Diehl and Matthew Cook. Unsupervised learning of digit recognition using spike- timing-dependent plasticity. Frontiers in Computational Neuroscience, 9, 2015.","title":"Unsupervised learning of digit recognition using spiketiming-dependent plasticity","authors":"U Peter, Matthew Diehl, Cook","year":"2015","venue":"Frontiers in Computational Neuroscience","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Diehl et al., 2015] Peter U Diehl, Daniel Neil, Jonathan Bi- nas, Matthew Cook, Shih-Chii Liu, and Michael Pfeif- fer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In The 2015 In- ternational Joint Conference on Neural Networks (IJCNN- 2015), pages 1-8. IEEE, 2015.","title":"Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing","authors":"Daniel Peter U Diehl, Jonathan Neil, Matthew Binas, Shih-Chii Cook, Michael Liu, Pfeiffer","year":"2015","venue":"The 2015 International Joint Conference on Neural Networks (IJCNN-2015)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hassabis et al., 2017] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience-inspired artificial intelligence. Neuron, 95(2):245-258, 2017.","title":"Neuroscience-inspired artificial intelligence","authors":"Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, Matthew Botvinick","year":"2017","venue":"Neuron","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Häusser and Mel, 2003] Michael Häusser and Bartlett Mel. Dendrites: bug or feature? Current opinion in neurobiolo- gy, 13(3):372-383, 2003.","title":"Dendrites: bug or feature?","authors":"Mel ; Häusser, Michael Häusser, Bartlett Mel","year":"2003","venue":"Current opinion in neurobiology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. Nature, 521(7553):436- 444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[LeCun, 1998] Yann LeCun. The mnist database of hand- written digits. http://yann. lecun. com/exdb/mnist/, 1998.","title":"The mnist database of handwritten digits","authors":"Yann Lecun","year":null,"venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist/,1998","internal_url":null},{"position":13,"raw":"[Lee et al., 2016] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural network- s using backpropagation. Frontiers in Neuroscience, 10, 2016.","title":"Training deep spiking neural networks using backpropagation","authors":"Lee","year":"2016","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: the third generation of neural network models. Neu- ral Networks, 10(9):1659-1671, 1997.","title":"Networks of spiking neurons: the third generation of neural network models","authors":"Wolfgang Maass, Maass","year":"1997","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Okun and Lampl, 2009] Michael Okun and Ilan Lampl. Balance of excitation and inhibition. Scholarpedia, 4(8):7467, 2009.","title":"Balance of excitation and inhibition","authors":"Lampl Okun, Michael Okun, Ilan Lampl","year":"2009","venue":"Scholarpedia","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rueckert et al., 2016] Elmar Rueckert, David Kappel, Daniel Tanneberg, Dejan Pecevski, and Jan Peters. Re- current spiking networks solve planning tasks. Scientific reports, 6:21142, 2016.","title":"Recurrent spiking networks solve planning tasks","authors":"Elmar Rueckert, David Kappel, Daniel Tanneberg, Dejan Pecevski, Jan Peters","year":"2016","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Soltani and Wang, 2010] Alireza Soltani and Xiao-Jing Wang. Synaptic computation underlying probabilistic in- ference. Nature neuroscience, 13(1):112-119, 2010.","title":"Synaptic computation underlying probabilistic inference","authors":"Alireza Soltani, Xiao-Jing Wang","year":"2010","venue":"Nature neuroscience","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sporea and Grüning, 2013] Ioana Sporea and André Grüning. Supervised learning in multilayer spiking neural networks. Neural computation, 25(2):473-509, 2013.","title":"Supervised learning in multilayer spiking neural networks","authors":"Ioana Sporea, André Grüning","year":"2013","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zeng et al., 2017] Yi Zeng, Tielin Zhang, and Bo Xu. Im- proving multi-layer spiking neural networks by incorpo- rating brain-inspired rules. Science China Information Sci- ences, 60(5):052201, 2017.","title":"Improving multi-layer spiking neural networks by incorporating brain-inspired rules","authors":"Yi Zeng, Tielin Zhang, Bo Xu","year":"2017","venue":"Science China Information Sciences","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zenke et al., 2015] Friedemann Zenke, Everton J Agnes, and Wulfram Gerstner. Diverse synaptic plasticity mecha- nisms orchestrated to form and retrieve memories in spik- ing neural networks. Nature Communications, 6:6922, 2015.","title":"Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks","authors":"Friedemann Zenke, Everton J Agnes, Wulfram Gerstner","year":"2015","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2018] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Mengting Shi. A plasticity-centric approach to train the non-differential spiking neural networks. In The 32th AAAI Conference on Artificial Intelligence (AAAI- 2018), 2018.","title":"A plasticity-centric approach to train the non-differential spiking neural networks","authors":"Zhang","year":"2018","venue":"The 32th AAAI Conference on Artificial Intelligence (AAAI-2018)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zucker and Regehr, 2002] Robert S Zucker and Wade G Regehr. Short-term synaptic plasticity. Annual review of physiology, 64(1):355-405, 2002.","title":"Short-term synaptic plasticity","authors":"S Robert, Wade G Zucker, Regehr","year":"2002","venue":"Annual review of physiology","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}