{"type":"ScholarlyArticle","title":"Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes","authors":[{"name":"Yang Li","slug":"yang-li","given":"Yang","family":"Li","affiliation":"Research Center for Brain-inspired Intelligence , Institute of Automation , Chinese Academy of Sciences; School of Artificial Intelligence , University of Chinese Academy of Sciences"},{"name":"Yi Zeng","slug":"yi-zeng","given":"Yi","family":"Zeng","affiliation":"Research Center for Brain-inspired Intelligence , Institute of Automation , Chinese Academy of Sciences; School of Artificial Intelligence , University of Chinese Academy of Sciences; Center for Excellence in Brain Science and Intelligence Technology , Chinese Academy of Sciences; National Laboratory of Pattern Recognition , Institute of Automation , Chinese Academy of Sciences"}],"abstract":"Spiking neural network (SNN), as a brain-inspired energy-efficient neural network, has attracted the interest of researchers. While the training of spiking neural networks is still an open problem. One effective way is to map the weight of trained ANN to SNN to achieve high reasoning ability. However, the converted spiking neural network often suffers from performance degradation and a considerable time delay. To speed up the inference process and obtain higher accuracy, we theoretically analyze the errors in the conversion process from three perspectives: the differences between IF and ReLU, time dimension, and pooling operation. We propose a neuron model for releasing burst spikes, a cheap but highly efficient method to solve residual information. In addition, Lateral Inhibition Pooling (LIPooling) is proposed to solve the inaccuracy problem caused by MaxPooling in the conversion process. Experimental results on CIFAR and ImageNet demonstrate that our algorithm is efficient and accurate. For example, our method can ensure nearly lossless conversion of SNN and only use about 1/10 (less than 100) simulation time under 0.693x energy consumption of the typical method. Our code is available at https://github.com/Brain-Inspired-Cognitive-Engine/Conversion_Burst.","doi":"10.24963/ijcai.2022/345","url":"https://ijcai.org/proceedings/2022/345","pdf":"https://ijcai.org/proceedings/2022/0345.pdf","markdown":"https://ijcai.org/proceedings/2022/345.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/345","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Humans and AI","track":"Main Track","number":345,"pages":{"first":2485,"last":2491},"keywords":["Humans and AI: Cognitive Modeling"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986892","length_sec":78,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985035","length_sec":274,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Akopyan et al., 2015] Filipp Akopyan, Jun Sawada, An- drew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi- Joon Nam, et al. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE TCAD, 34(10):1537-1557, 2015.","title":"Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip","authors":"Jun Filipp Akopyan, Andrew Sawada, Rodrigo Cassidy, John Alvarez-Icaza, Paul Arthur, Nabil Merolla, Yutaka Imam, Pallab Nakamura, Gi-Joon Datta, Nam","year":"2015","venue":"IEEE TCAD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2015] Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113(1):54-66, 2015.","title":"Spiking deep convolutional neural networks for energy-efficient object recognition","authors":"Yongqiang Cao, Yang Chen, Deepak Khosla","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Davies et al., 2018] Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82-99, 2018.","title":"Loihi: A neuromorphic manycore processor with on-chip learning","authors":"Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain","year":"2018","venue":"IEEE Micro","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng and Gu, 2021] Shikuang Deng and Shi Gu. Op- timal conversion of conventional artificial neural net- works to spiking neural networks. arXiv preprint arXiv:2103.00476, 2021.","title":"Optimal conversion of conventional artificial neural networks to spiking neural networks","authors":"Gu Deng, Shikuang Deng, Shi Gu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Diehl et al., 2015] Peter U Diehl, Daniel Neil, Jonathan Bi- nas, Matthew Cook, Shih-Chii Liu, and Michael Pfeif- fer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In IJCNN, pages 1-8. ieee, 2015.","title":"Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing","authors":"Daniel Peter U Diehl, Jonathan Neil, Matthew Binas, Shih-Chii Cook, Michael Liu, Pfeiffer","year":"2015","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding et al., 2021] Jianhao Ding, Zhaofei Yu, Yonghong Tian, and Tiejun Huang. Optimal ann-snn conversion for fast and accurate inference in deep spiking neural net- works. arXiv preprint arXiv:2105.11654, 2021.","title":"Optimal ann-snn conversion for fast and accurate inference in deep spiking neural networks","authors":"Ding","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2021/321"},{"position":7,"raw":"[Han and Roy, 2020] Bing Han and Kaushik Roy. Deep spik- ing neural network: Energy efficiency through time based coding. In ECCV, pages 388-404. Springer, 2020.","title":"Deep spiking neural network: Energy efficiency through time based coding","authors":"Roy ; Han, Bing Han, Kaushik Roy","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2020] Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In CVPR, pages 13558-13567, 2020.","title":"Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network","authors":"Han","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2018] Jaehyun Kim, Heesu Kim, Subin Huh, Jinho Lee, and Kiyoung Choi. Deep neural networks with weighted spikes. Neurocomputing, 311:373-386, 2018.","title":"Deep neural networks with weighted spikes","authors":"Kim","year":"2018","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2020] Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: Spiking neural net- work for energy-efficient object detection. In AAAI, vol- ume 34, pages 11270-11277, 2020.","title":"Spiking-yolo: Spiking neural network for energy-efficient object detection","authors":"Kim","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koyama and Pujala, 2018] Minoru Koyama and Avinash Pujala. Mutual inhibition of lateral inhibition: a network motif for an elementary computation in the brain. Current opinion in neurobiology, 49:69-74, 2018.","title":"Mutual inhibition of lateral inhibition: a network motif for an elementary computation in the brain","authors":"Minoru Koyama, Avinash Pujala","year":"2018","venue":"Current opinion in neurobiology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2021a] Yang Li, Yi Zeng, and Dongcheng Zhao. Bsnn: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons. arXiv preprint arXiv:2105.12917, 2021.","title":"Bsnn: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons","authors":"Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2021b] Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, and Shi Gu. A free lunch from ann: To- wards efficient, accurate spiking neural networks calibra- tion. arXiv preprint arXiv:2106.06984, 2021.","title":"A free lunch from ann: Towards efficient, accurate spiking neural networks calibration","authors":"Li","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luo et al., 2020] Yihao Luo, Min Xu, Caihong Yuan, Xi- ang Cao, Liangqi Zhang, Yan Xu, Tianjiang Wang, and Qi Feng. Siamsnn: Siamese spiking neural net- works for energy-efficient object tracking. arXiv preprint arXiv:2003.07584, 2020.","title":"Siamsnn: Siamese spiking neural networks for energy-efficient object tracking","authors":"Yihao Luo, Min Xu, Caihong Yuan, Xiang Cao, Liangqi Zhang, Yan Xu, Tianjiang Wang, Qi Feng","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: the third generation of neural network models. Neu- ral networks, 10(9):1659-1671, 1997.","title":"Networks of spiking neurons: the third generation of neural network models","authors":"Wolfgang Maass, Maass","year":"1997","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rathi and Roy, 2021] Nitin Rathi and Kaushik Roy. Diet- snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE TNNLS, 2021.","title":"Dietsnn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization","authors":"Nitin Rathi, Kaushik Roy","year":"2021","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rathi et al., 2020] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.01807, 2020.","title":"Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation","authors":"Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, Kaushik Roy","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Roy et al., 2019] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607-617, 2019.","title":"Towards spike-based machine intelligence with neuromorphic computing","authors":"Kaushik Roy, Akhilesh Jaiswal, Priyadarshini Panda","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rueckauer et al., 2017] Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to effi- cient event-driven networks for image classification. Fron- tiers in neuroscience, 11:682, 2017.","title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","authors":"Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, Shih-Chii Liu","year":"2017","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sengupta et al., 2019] Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019.","title":"Going deeper in spiking neural networks: Vgg and residual architectures","authors":"Sengupta","year":"2019","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tan et al., 2020] Weihao Tan, Devdhar Patel, and Robert Kozma. Strategy and benchmark for converting deep q- networks to event-driven spiking neural networks. arXiv preprint arXiv:2009.14456, 2020.","title":"Strategy and benchmark for converting deep qnetworks to event-driven spiking neural networks","authors":"Tan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2018] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Fron- tiers in neuroscience, 12:331, 2018.","title":"Spatio-temporal backpropagation for training high-performance spiking neural networks","authors":"Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi","year":"2018","venue":"Frontiers in neuroscience","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zeng et al., 2017] Yi Zeng, Tielin Zhang, and Bo Xu. Im- proving multi-layer spiking neural networks by incorpo- rating brain-inspired rules. Science China Information Sci- ences, 60(5):1-11, 2017.","title":"Improving multi-layer spiking neural networks by incorporating brain-inspired rules","authors":"Yi Zeng, Tielin Zhang, Bo Xu","year":"2017","venue":"Science China Information Sciences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2019] Lei Zhang, Shengyuan Zhou, Tian Zhi, Zidong Du, and Yunji Chen. Tdsnn: From deep neural networks to deep spike neural networks with temporal- coding. In AAAI, volume 33, pages 1319-1326, 2019.","title":"Tdsnn: From deep neural networks to deep spike neural networks with temporalcoding","authors":"Lei Zhang, Shengyuan Zhou, Tian Zhi, Zidong Du, Yunji Chen","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}