{"type":"ScholarlyArticle","title":"Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network","authors":[{"name":"Haowen Fang","slug":"haowen-fang","given":"Haowen","family":"Fang","affiliation":"Syracuse University"},{"name":"Amar Shrestha","slug":"amar-shrestha","given":"Amar","family":"Shrestha","affiliation":"Syracuse University"},{"name":"Ziyi Zhao","slug":"ziyi-zhao","given":"Ziyi","family":"Zhao","affiliation":"Syracuse University"},{"name":"Qinru Qiu","slug":"qinru-qiu","given":"Qinru","family":"Qiu","affiliation":"Syracuse University"}],"abstract":"The recently discovered spatial-temporal information processing capability of bio-inspired Spiking neural networks (SNN) has enabled some interesting models and applications. However designing large-scale and high-performance model is yet a challenge due to the lack of robust training algorithms. A bio-plausible SNN model with spatial-temporal property is a complex dynamic system. Synapses and neurons behave as filters capable of preserving temporal information. As such neuron dynamics and filter effects are ignored in existing training algorithms, the SNN downgrades into a memoryless system and loses the ability of temporal signal processing. Furthermore, spike timing plays an important role in information representation, but conventional rate-based spike coding models only consider spike trains statistically, and discard information carried by its temporal structures. To address the above issues, and exploit the temporal dynamics of SNNs, we formulate SNN as a network of infinite impulse response (IIR) filters with neuron nonlinearity. We proposed a training algorithm that is capable to learn spatial-temporal patterns by searching for the optimal synapse filter kernels and weights. The proposed model and training algorithm are applied to construct associative memories and classifiers for synthetic and public datasets including MNIST, NMNIST, DVS 128 etc. Their accuracy outperforms state-of-the-art approaches.","doi":"10.24963/ijcai.2020/388","url":"https://ijcai.org/proceedings/2020/388","pdf":"https://ijcai.org/proceedings/2020/0388.pdf","markdown":"https://ijcai.org/proceedings/2020/388.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/388","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":388,"pages":{"first":2799,"last":2806},"keywords":["Machine Learning: Deep-learning Theory","Machine Learning: Deep Learning","Machine Learning: Other"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24943/file_24943.m3u8","length_sec":261,"provider":"hls"}],"references":[{"position":1,"raw":"[Abdollahi and Liu, 2011] Mohammad Abdollahi and Shih- Chii Liu. Speaker-independent isolated digit recognition using an aer silicon cochlea. In 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS), pages 269- 272. IEEE, 2011.","title":"Speaker-independent isolated digit recognition using an aer silicon cochlea","authors":"Mohammad Abdollahi, Shih-Chii Liu","year":"2011","venue":"IEEE Biomedical Circuits and Systems Conference (BioCAS)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amir et al., 2017] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Gar- reau, Marcela Mendoza, et al. A low power, fully event- based gesture recognition system. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 7243-7252, 2017.","title":"A low power, fully eventbased gesture recognition system","authors":"Amir","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Back and Tsoi, 1991] Andrew D Back and Ah Chung Tsoi. Fir and iir synapses, a new neural network architecture for time series modeling. Neural computation, 3(3):375-385, 1991.","title":"Fir and iir synapses, a new neural network architecture for time series modeling","authors":"D Andrew, Ah Back, Tsoi Chung","year":"1991","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brette et al., 2007] Romain Brette, Michelle Rudolph, Ted Carnevale, Michael Hines, David Beeman, James M Bower, Markus Diesmann, Abigail Morrison, Philip H Goodman, Frederick C Harris, et al. Simulation of net- works of spiking neurons: a review of tools and strate- gies. Journal of computational neuroscience, 23(3):349- 398, 2007.","title":"Simulation of networks of spiking neurons: a review of tools and strategies","authors":"Romain Brette, Michelle Rudolph, Ted Carnevale, Michael Hines, David Beeman, James M Bower, Markus Diesmann, Abigail Morrison, Frederick C Philip H Goodman, Harris","year":"2007","venue":"Journal of computational neuroscience","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Campolucci et al., 1999] Paolo Campolucci, Aurelio Uncini, Francesco Piazza, and Bhaskar D Rao. On-line learning algorithms for locally recurrent neural networks. IEEE transactions on neural networks, 10(2):253-271, 1999.","title":"On-line learning algorithms for locally recurrent neural networks","authors":"Paolo Campolucci, Aurelio Uncini, Francesco Piazza, Bhaskar D Rao","year":"1999","venue":"IEEE transactions on neural networks","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Diehl et al., 2015] Peter U Diehl, Daniel Neil, Jonathan Bi- nas, Matthew Cook, Shih-Chii Liu, and Michael Pfeif- fer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 Inter- national Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2015.","title":"Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing","authors":"Daniel Peter U Diehl, Jonathan Neil, Matthew Binas, Shih-Chii Cook, Michael Liu, Pfeiffer","year":"2015","venue":"2015 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Esser et al., 2015] Steve K Esser, Rathinakumar Ap- puswamy, Paul Merolla, John V Arthur, and Dharmen- dra S Modha. Backpropagation for energy-efficient neuromorphic computing. In Advances in Neural Information Processing Systems, pages 1117-1125, 2015.","title":"Backpropagation for energy-efficient neuromorphic computing","authors":"Steve K Esser, Rathinakumar Appuswamy, Paul Merolla, John V Arthur, Dharmendra S Modha","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gerstner et al., 2014] Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.","title":"Neuronal dynamics: From single neurons to networks and models of cognition","authors":"Wulfram Gerstner, M Werner, Richard Kistler, Liam Naud, Paninski","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2019] Pengjie Gu, Rong Xiao, Gang Pan, and Huajin Tang. Stca: spatio-temporal credit assignment with delayed feedback in deep spiking neural networks. In Pro- ceedings of the 28th International Joint Conference on Ar- tificial Intelligence, pages 1366-1372. AAAI Press, 2019.","title":"Stca: spatio-temporal credit assignment with delayed feedback in deep spiking neural networks","authors":"Gu","year":"2019","venue":"Proceedings of the 28th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/189"},{"position":10,"raw":"[Gütig and Sompolinsky, 2006] Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature neuroscience, 9(3):420, 2006.","title":"Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions","authors":"Sompolinsky Gütig","year":"2006","venue":"Nature neuroscience","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gütig, 2016] Robert Gütig. Spiking neurons can discover predictive features by aggregate-label learning. Science, 351(6277):aab4113, 2016.","title":"Spiking neurons can discover predictive features by aggregate-label learning","authors":"Robert Gütig","year":"2016","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hennig, 2013] Matthias H Hennig. Theoretical models of synaptic short term plasticity. Frontiers in computational neuroscience, 7:45, 2013.","title":"Theoretical models of synaptic short term plasticity","authors":"H Matthias, Hennig","year":"2013","venue":"Frontiers in computational neuroscience","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jin et al., 2018] Yingyezhe Jin, Wenrui Zhang, and Peng Li. Hybrid macro/micro level backpropagation for train- ing deep spiking neural networks. In Advances in Neural Information Processing Systems, pages 7005-7015, 2018.","title":"Hybrid macro/micro level backpropagation for train-ing deep spiking neural networks","authors":"Jin","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kadous and others, 2002] Mohammed Waleed Kadous et al. Temporal classification: Extending the classification paradigm to multivariate time series. University of New South Wales Kensington, 2002.","title":"Temporal classification: Extending the classification paradigm to multivariate time series","authors":"Mohammed Waleed, Kadous","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kaiser et al., 2018] Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep con- tinuous local learning. arXiv preprint arXiv:1811.10766, 2018.","title":"Synaptic plasticity dynamics for deep continuous local learning","authors":"Jacques Kaiser, Hesham Mostafa, Emre Neftci","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kaiser et al., 2019] Jacques Kaiser, Alexander Friedrich, J Tieck, Daniel Reichard, Arne Roennau, Emre Neftci, and Rüdiger Dillmann. Embodied event-driven random back- propagation. arXiv preprint arXiv:1904.04805, 2019.","title":"Embodied event-driven random backpropagation","authors":"Jacques Kaiser, Alexander Friedrich, J Tieck, Daniel Reichard, Arne Roennau, Emre Neftci, Rüdiger Dillmann","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Karim et al., 2019] Fazle Karim, Somshubra Majumdar, Houshang Darabi, and Samuel Harford. Multivariate lstm-fcns for time series classification. Neural Networks, 116:237-245, 2019.","title":"Multivariate lstm-fcns for time series classification","authors":"Fazle Karim, Somshubra Majumdar, Houshang Darabi, Samuel Harford","year":"2019","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2016] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in neuroscience, 10:508, 2016.","title":"Training deep spiking neural networks using backpropagation","authors":"Lee","year":"2016","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mohemmed et al., 2012] Ammar Mohemmed, Stefan Schliebs, Satoshi Matsuda, and Nikola Kasabov. Span: Spike pattern association neuron for learning spatio- temporal spike patterns. International journal of neural systems, 22(04):1250012, 2012.","title":"Span: Spike pattern association neuron for learning spatiotemporal spike patterns","authors":"Ammar Mohemmed, Stefan Schliebs, Satoshi Matsuda, Nikola Kasabov","year":"2012","venue":"International journal of neural systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Neftci et al., 2019] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks. arXiv preprint arXiv:1901.09948, 2019.","title":"Surrogate gradient learning in spiking neural networks","authors":"Hesham Emre O Neftci, Friedemann Mostafa, Zenke","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pan et al., 2019] Zihan Pan, Jibin Wu, Malu Zhang, Haizhou Li, and Yansong Chua. Neural population coding for effective temporal classification. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2019.","title":"Neural population coding for effective temporal classification","authors":"Zihan Pan, Jibin Wu, Malu Zhang, Haizhou Li, Yansong Chua","year":"2019","venue":"2019 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shrestha and Orchard, 2018] Sumit Bam Shrestha and Gar- rick Orchard. Slayer: Spike layer error reassignment in time. In Advances in Neural Information Processing Sys- tems, pages 1412-1421, 2018.","title":"Slayer: Spike layer error reassignment in time","authors":"Sumit Bam, Shrestha, Garrick Orchard","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shrestha et al., 2019] Amar Shrestha, Haowen Fang, Qing Wu, and Qinru Qiu. Approximating back-propagation for a biologically plausible local learning rule in spiking neu- ral networks. In Proceedings of the International Confer- ence on Neuromorphic Systems, page 10. ACM, 2019.","title":"Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks","authors":"Amar Shrestha, Haowen Fang, Qing Wu, Qinru Qiu","year":"2019","venue":"Proceedings of the International Conference on Neuromorphic Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Stevens and Zador, 1996] Charles F Stevens and An- thony M Zador. When is an integrate-and-fire neuron like a poisson neuron? In Advances in neural information processing systems, pages 103-109, 1996.","title":"When is an integrate-and-fire neuron like a poisson neuron?","authors":"Zador ; Stevens, F Charles, Anthony M Stevens, Zador","year":"1996","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tavanaei and Maida, 2019] Amirhossein Tavanaei and An- thony Maida. Bp-stdp: Approximating backpropagation using spike timing dependent plasticity. Neurocomputing, 330:39-47, 2019.","title":"Amirhossein Tavanaei and Anthony Maida. Bp-stdp: Approximating backpropagation using spike timing dependent plasticity","authors":"Maida Tavanaei","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2018a] Jibin Wu, Yansong Chua, and Haizhou Li. A biologically plausible speech recognition framework based on spiking neural networks. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2018.","title":"A biologically plausible speech recognition framework based on spiking neural networks","authors":"Wu","year":"2018","venue":"2018 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2018b] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Fron- tiers in neuroscience, 12, 2018.","title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","authors":"Wu","year":"2018","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2019] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1311-1318, 2019.","title":"Direct training for spiking neural networks: Faster, larger, better","authors":"Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, Luping Shi","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zenke and Ganguli, 2018] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multi- layer spiking neural networks. Neural computation, 30(6):1514-1541, 2018.","title":"Superspike: Supervised learning in multilayer spiking neural networks","authors":"Ganguli Zenke, Friedemann Zenke, Surya Ganguli","year":"2018","venue":"Neural computation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}