{"type":"ScholarlyArticle","title":"Jointly Learning Network Connections and Link Weights in Spiking Neural Networks","authors":[{"name":"Yu Qi","slug":"yu-qi","given":"Yu","family":"Qi","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Jiangrong Shen","slug":"jiangrong-shen","given":"Jiangrong","family":"Shen","affiliation":"College of Computer Science and Technology , Zhejiang University; Qiushi Academy for Advanced Studies , Zhejiang University"},{"name":"Yueming Wang","slug":"yueming-wang","given":"Yueming","family":"Wang","affiliation":"Qiushi Academy for Advanced Studies , Zhejiang University"},{"name":"Huajin Tang","slug":"huajin-tang","given":"Huajin","family":"Tang","affiliation":"College of Computer Science , Sichuan University"},{"name":"Hang Yu","slug":"hang-yu","given":"Hang","family":"Yu","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Zhaohui Wu","slug":"zhaohui-wu","given":"Zhaohui","family":"Wu","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Gang Pan","slug":"gang-pan","given":"Gang","family":"Pan","affiliation":"College of Computer Science and Technology , Zhejiang University"}],"abstract":"Spiking neural networks (SNNs) are considered to be biologically plausible and power-efficient on neuromorphic hardware. However, unlike the brain mechanisms, most existing SNN algorithms have fixed network topologies and connection relationships. This paper proposes a method to jointly learn network connections and link weights simultaneously. The connection structures are optimized by the spike-timing-dependent plasticity (STDP) rule with timing information, and the link weights are optimized by a supervised algorithm. The connection structures and the weights are learned alternately until a termination condition is satisfied. Experiments are carried out using four benchmark datasets. Our approach outperforms classical learning methods such as STDP, Tempotron, SpikeProp, and a state-of-the-art supervised algorithm. In addition, the learned structures effectively reduce the number of connections by about 24%, thus facilitate the computational efficiency of the network.","doi":"10.24963/ijcai.2018/221","url":"https://ijcai.org/proceedings/2018/221","pdf":"https://ijcai.org/proceedings/2018/0221.pdf","markdown":"https://ijcai.org/proceedings/2018/221.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/221","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Humans and AI","track":"Main track","number":221,"pages":{"first":1597,"last":1603},"keywords":["Humans and AI: Cognitive Systems","Machine Learning: Neural Networks","Humans and AI: Brain Sciences","Humans and AI: Cognitive Modeling","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bi and Poo, 2001] Guoqiang Bi and Muming Poo. Synaptic modification by correlated activity: Hebb's postulate re- visited. Annual Review of Neuroscience, 24(1):139-166, March 2001.","title":"Synaptic modification by correlated activity: Hebb's postulate revisited","authors":"Poo Bi, Guoqiang Bi, Muming Poo","year":"2001","venue":"Annual Review of Neuroscience","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bohte et al., 2002] Sander M. Bohte, Joost N. Kok, and Han La Poutré. Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing, 48(1):17- 37, October 2002.","title":"Error-backpropagation in temporally encoded networks of spiking neurons","authors":"M Sander, Bohte, N Joost, Han Kok, Poutré La","year":"2002","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dheeru and Karra Taniskidou, 2017] Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017.","title":"Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository","authors":"Karra Dheeru, Taniskidou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Diehl and Cook, 2015] Peter U. Diehl and Matthew Cook. Unsupervised learning of digit recognition using Spike- Timing-Dependent Plasticity. Frontiers in Computational Neuroscience, 9:99, August 2015.","title":"Unsupervised learning of digit recognition using Spike-Timing-Dependent Plasticity","authors":"U Peter, Matthew Diehl, Cook","year":"2015","venue":"Frontiers in Computational Neuroscience","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Eurich and Wilke, 2000] C Eurich and S Wilke. Multidi- mensional encoding strategy of spiking neurons. Neural Computation, 12(7):1519-1529, July 2000.","title":"Multidimensional encoding strategy of spiking neurons","authors":"C Eurich, Wilke","year":"2000","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Filip, 2005] Ponulak Filip. ReSuMe new supervised learn- ing method for Spiking Neural Networks. Institute of Con- trol and Information Engineering, Poznan University of Technology., 2005.","title":"Ponulak Filip. ReSuMe new supervised learning method for Spiking Neural Networks","authors":"Filip","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gerstner and Kistler, 2002] Wulfram Gerstner and Werner M. Kistler. Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, Cambridge, 2002.","title":"Spiking neuron models: single neurons, populations, plasticity","authors":"Wulfram Gerstner, Werner M Kistler","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ghosh-Dastidar and Adeli, 2009] S. Ghosh-Dastidar and H. Adeli. Spiking neural networks. International Jour- nal of Neural Systems, 19(04):295-308, August 2009.","title":"Spiking neural networks","authors":"Ghosh-Dastidar Adeli, ; ] S Ghosh-Dastidar, H Adeli","year":"2009","venue":"International Journal of Neural Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gütig and Sompolinsky, 2006] Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature Neuroscience, 9(3):420- 428, March 2006.","title":"Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions","authors":"Sompolinsky Gütig","year":"2006","venue":"Nature Neuroscience","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hebb, 1949] D. O. Hebb. In the organization of behavior: a neuropsychological theory. John Wiley , Chapman and Hall, 1949.","title":"the organization of behavior: a neuropsychological theory","authors":"D O Hebb","year":"1949","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hinton and Salakhutdinov, 2006] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with Neural Networks. Science, 313(5786):504-507, July 2006.","title":"Reducing the dimensionality of data with Neural Networks","authors":"G E Hinton, R R Salakhutdinov","year":"2006","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hodgkin and Huxley, 1952] A. L. Hodgkin and A. F. Hux- ley. A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of Physiology, 117(4):500-544, August 1952.","title":"A quantitative description of membrane current and its application to conduction and excitation in nerve","authors":"A L Hodgkin, A F Huxley","year":"1952","venue":"The Journal of Physiology","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: The third generation of neural network models. Neu- ral Networks, 10(9):1659-1671, December 1997.","title":"Networks of spiking neurons: The third generation of neural network models","authors":"Wolfgang Maass, Maass","year":"1997","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Masquelier et al., 2009] Timothée Masquelier, Rudy Guy- onneau, and Simon J. Thorpe. Competitive STDP-based spike pattern learning. Neural computation, 21(5):1259- 1276, May 2009.","title":"Competitive STDP-based spike pattern learning","authors":"Timothée Masquelier, Rudy Guyonneau, Simon J Thorpe","year":"2009","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ranhel, 2012] J. Ranhel. Neural assembly computing. IEEE Transactions on Neural Networks and Learning Systems, 23(6):916-927, June 2012.","title":"Neural assembly computing","authors":"J Ranhel","year":"2012","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shatz, 1992] C. J. Shatz. The developing brain. Scientific American, 267(3):60-67, September 1992.","title":"The developing brain","authors":"C J Shatz","year":"1992","venue":"Scientific American","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tsukada and Pan, 2005] M. Tsukada and X. Pan. The spatiotemporal learning rule and its efficiency in sepa- rating spatiotemporal patterns. Biological Cybernetics, 92(2):139-146, February 2005.","title":"The spatiotemporal learning rule and its efficiency in separating spatiotemporal patterns","authors":"M Tsukada, X Pan","year":"2005","venue":"Biological Cybernetics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xie et al., 2017] X. Xie, H. Qu, Z. Yi, and J. Kurths. Effi- cient training of supervised Spiking Neural Network via accurate synaptic-efficiency adjustment method. IEEE Transactions on Neural Networks and Learning Systems, 28(6):1411-1424, June 2017.","title":"Efficient training of supervised Spiking Neural Network via accurate synaptic-efficiency adjustment method","authors":"Xie","year":"2017","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yu et al., 2013a] Q. Yu, H. Tang, K.C. Tan, and H. Li. Precise-Spike-Driven synaptic plasticity: learning Hetero- Association of spatiotemporal spike patterns. PLOS ONE, 8(11):e78318, November 2013.","title":"Precise-Spike-Driven synaptic plasticity: learning Hetero-Association of spatiotemporal spike patterns","authors":"Yu","year":"2013","venue":"PLOS ONE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yu et al., 2013b] Q. Yu, H. Tang, K.C. Tan, and H. Li. Rapid feedforward computation by temporal encoding and learning with spiking neurons. IEEE Transactions on Neu- ral Networks and Learning Systems, 24(10):1539-1552, October 2013.","title":"Rapid feedforward computation by temporal encoding and learning with spiking neurons","authors":"Yu","year":"2013","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}