{"type":"ScholarlyArticle","title":"One-step Spiking Transformer with a Linear Complexity","authors":[{"name":"Xiaotian Song","slug":"xiaotian-song","given":"Xiaotian","family":"Song","affiliation":"College of Computer Science , Sichuan University"},{"name":"Andy Song","slug":"andy-song","given":"Andy","family":"Song","affiliation":"School of Computing Technologies , RMIT University"},{"name":"Rong Xiao","slug":"rong-xiao","given":"Rong","family":"Xiao","affiliation":"College of Computer Science , Sichuan University"},{"name":"Yanan Sun","slug":"yanan-sun","given":"Yanan","family":"Sun","affiliation":"College of Computer Science , Sichuan University"}],"abstract":"Spiking transformers have recently emerged as a robust alternative in deep learning.  One focus of this field is the reduction of energy consumption, given that spiking transformers require lengthy simulation timesteps and complex floating-point attention mechanisms. In this paper, we propose a one-step approach that requires only one timestep and is of linear complexity.  The proposed One-step Spiking Transformer (OST) incorporates a Time Domain Compression and Compensation (TDCC) component, which can significantly mitigate the spatio-temporal overhead of spiking transformers. Another novel component in OST is the Spiking Linear Transformation (SLT), designed to greatly reduce the number of floating-point multiply-and-accumulate operations. Experiments on both static and neuromorphic images show that OST can perform as well as or better than SOTA methods with just one timestep, even for more difficult tasks.  For instance, comparing with Spikeformer, OST gains 1.59% in accuracy on ImageNet, yet 40.27% more efficient, and gains 0.7% on DVS128 Gesture. The supplementary materials and source code are available at https://github.com/songxt3/OST.","doi":"10.24963/ijcai.2024/348","url":"https://ijcai.org/proceedings/2024/348","pdf":"https://ijcai.org/proceedings/2024/0348.pdf","markdown":"https://ijcai.org/proceedings/2024/348.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/348","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Humans and AI","track":"Main Track","number":348,"pages":{"first":3142,"last":3150},"keywords":["Humans and AI: HAI: Cognitive modeling","Humans and AI: HAI: Applications","Machine Learning: General"],"videos":[],"references":[{"position":1,"raw":"[Amir et al., 2017] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Gar- reau, Marcela Mendoza, et al. A low power, fully event- based gesture recognition system. In Proc. of CVPR, 2017.","title":"A low power, fully eventbased gesture recognition system","authors":"Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey Mckinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza","year":"2017","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Proc. of NeurIPS, 2020.","title":"Language models are few-shot learners","authors":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Proc. of ECCV, 2020.","title":"End-to-end object detection with transformers","authors":"Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko","year":"2020","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Child et al., 2019] Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509, 2019.","title":"Rewon Child","authors":"Child","year":"2019","venue":"Generating long sequences with sparse transformers","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng and Gu, 2021] Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. In Proc. of ICLR, 2021.","title":"Optimal conversion of conventional artificial neural networks to spiking neural networks","authors":"Gu ; ] Shikuang Deng, Shi Deng, Gu","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2022] Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spik- ing neural network via gradient re-weighting. In Proc. of ICLR, 2022.","title":"Temporal efficient training of spiking neural network via gradient re-weighting","authors":"Deng","year":"2022","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In Proc. of ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2021a] Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Proc. of NeurIPS, 2021.","title":"Deep residual learning in spiking neural networks","authors":"Fang","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fang et al., 2021b] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to en- hance learning of spiking neural networks. In Proc. of ICCV, 2021.","title":"Incorporating learnable membrane time constant to enhance learning of spiking neural networks","authors":"Fang","year":"2021","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fang et al., 2023] Wei Fang, Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, and Yonghong Tian. Spik- ingjelly: An open-source machine learning infrastructure platform for spike-based intelligence. Science Advances, 9(40):eadi1480, 2023.","title":"Spikingjelly: An open-source machine learning infrastructure platform for spike-based intelligence","authors":"Wei Fang, Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, Yonghong Tian","year":"2023","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fedus et al., 2022] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parame- ter models with simple and efficient sparsity. The Journal of Machine Learning Research, 23(1):5232-5270, 2022.","title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity","authors":"William Fedus, Barret Zoph, Noam Shazeer","year":"2022","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gerstner et al., 2014] Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.","title":"Neuronal dynamics: From single neurons to networks and models of cognition","authors":"Wulfram Gerstner, M Werner, Richard Kistler, Liam Naud, Paninski","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hao et al., 2020] Yunzhe Hao, Xuhui Huang, Meng Dong, and Bo Xu. A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Networks, 121:387-395, 2020.","title":"A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule","authors":"Hao","year":"2020","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hu et al., 2021] Yangfan Hu, Huajin Tang, and Gang Pan. Spiking deep residual networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.","title":"Spiking deep residual networks","authors":"Hu","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hua et al., 2022] Weizhe Hua, Zihang Dai, Hanxiao Liu, and Quoc Le. Transformer quality in linear time. In Proc. of ICML, 2022.","title":"Transformer quality in linear time","authors":"Hua","year":"2022","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kaiser et al., 2020] Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep contin- uous local learning (decolle). Frontiers in Neuroscience, 14:424, 2020.","title":"Synaptic plasticity dynamics for deep continuous local learning (decolle)","authors":"Jacques Kaiser, Hesham Mostafa, Emre Neftci","year":"2020","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Katharopoulos et al., 2020] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franc ¸ois Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In Proc. of ICML, 2020.","title":"Transformers are rnns: Fast autoregressive transformers with linear attention","authors":"Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Franc ¸ois, Fleuret","year":"2020","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Khan et al., 2022] Salman Khan, Muzammal Naseer, Mu- nawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. ACM Computing Surveys (CSUR), 54(10s):1-41, 2022.","title":"Transformers in vision: A survey","authors":"Salman Khan, Muzammal Naseer, Munawar Hayat, Fahad Syed Waqas Zamir, Mubarak Shahbaz Khan, Shah","year":"2022","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kim and Panda, 2021] Youngeun Kim and Priyadarshini Panda. Optimizing deeper spiking neural networks for dynamic vision sensing. Neural Networks, 144:686-698, 2021.","title":"Youngeun Kim and Priyadarshini Panda. Optimizing deeper spiking neural networks for dynamic vision sensing","authors":"Panda Kim","year":"2021","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kim et al., 2020] Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: spiking neural net- work for energy-efficient object detection. In Proc. of AAAI, 2020.","title":"Spiking-yolo: spiking neural network for energy-efficient object detection","authors":"Kim","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kim et al., 2022] Youngeun Kim, Hyoungseob Park, Ab- hishek Moitra, Abhiroop Bhattacharjee, Yeshwanth Venkatesha, and Priyadarshini Panda. Rate coding or di- rect coding: Which one is better for accurate, robust, and energy-efficient spiking neural networks? In Proc. of ICASSP. IEEE, 2022.","title":"Rate coding or direct coding: Which one is better for accurate, robust, and energy-efficient spiking neural networks?","authors":"Kim","year":"2022","venue":"Proc. of ICASSP. IEEE","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kugele et al., 2020] Alexander Kugele, Thomas Pfeil, Michael Pfeiffer, and Elisabetta Chicca. Efficient process- ing of spatio-temporal data streams with spiking neural networks. Frontiers in Neuroscience, 14:439, 2020.","title":"Efficient processing of spatio-temporal data streams with spiking neural networks","authors":"Alexander Kugele, Thomas Pfeil, Michael Pfeiffer, Elisabetta Chicca","year":"2020","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2017] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in Neuroscience, 11:309, 2017.","title":"Cifar10-dvs: an event-stream dataset for object classification","authors":"Li","year":"2017","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2021] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differen- tiable spike: Rethinking gradient-descent for training spik- ing neural networks. Proc. of NeurIPS, 2021.","title":"Differentiable spike: Rethinking gradient-descent for training spiking neural networks","authors":"Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, Shi Gu","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2022] Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda. Neuromor- phic data augmentation for training spiking neural net- works. In Proc. of ECCV, 2022.","title":"Neuromorphic data augmentation for training spiking neural networks","authors":"Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, Priyadarshini Panda","year":"2022","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Liu et al., 2021] Hanxiao Liu, Zihang Dai, David So, and Quoc V Le. Pay attention to mlps. Proc. of NeurIPS, 2021.","title":"Pay attention to mlps","authors":"Liu","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: the third generation of neural network models. Neu- ral Networks, 10(9):1659-1671, 1997.","title":"Networks of spiking neurons: the third generation of neural network models","authors":"Wolfgang Maass, Maass","year":"1997","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Meng et al., 2022] Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Training high-performance low-latency spiking neural net- works by differentiation on spike representation. In Proc. of CVPR, 2022.","title":"Training high-performance low-latency spiking neural networks by differentiation on spike representation","authors":"Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, Zhi-Quan Luo","year":"2022","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Merolla et al., 2014] Paul A Merolla, John V Arthur, Ro- drigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Fil- ipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, et al. A million spiking-neuron inte- grated circuit with a scalable communication network and interface. Science, 345(6197):668-673, 2014.","title":"A million spiking-neuron integrated circuit with a scalable communication network and interface","authors":"John V Paul A Merolla, Rodrigo Arthur, Andrew S Alvarez-Icaza, Jun Cassidy, Filipp Sawada, Akopyan, Nabil Bryan L Jackson, Chen Imam, Yutaka Guo, Nakamura","year":"2014","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Mueller et al., 2021] Etienne Mueller, Viktor Studenyak, Daniel Auge, and Alois Knoll. Spiking transformer net- works: A rate coded approach for processing sequential data. In Proc. of ICSAI, 2021.","title":"Spiking transformer networks: A rate coded approach for processing sequential data","authors":"Etienne Mueller, Viktor Studenyak, Daniel Auge, Alois Knoll","year":"2021","venue":"Proc. of ICSAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Neftci et al., 2019] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spik- ing neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Pro- cessing Magazine, 36(6):51-63, 2019.","title":"Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks","authors":"Hesham Emre O Neftci, Friedemann Mostafa, Zenke","year":"2019","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Patterson et al., 2021] David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Car- bon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021.","title":"Carbon emissions and large neural network training","authors":"David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, Jeff Dean","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Rao and Ballard, 1999] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional in- terpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79-87, 1999.","title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","authors":"P N Rajesh, Dana H Rao, Ballard","year":"1999","venue":"Nature Neuroscience","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Rathi and Roy, 2021] Nitin Rathi and Kaushik Roy. Diet- snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Transactions on Neural Networks and Learning Sys- tems, 2021.","title":"Dietsnn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization","authors":"Nitin Rathi, Kaushik Roy","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Rathi et al., 2020] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. In Proc. of ICLR, 2020.","title":"Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation","authors":"Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, Kaushik Roy","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Roy et al., 2019] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607-617, 2019.","title":"Towards spike-based machine intelligence with neuromorphic computing","authors":"Kaushik Roy, Akhilesh Jaiswal, Priyadarshini Panda","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Rueckauer et al., 2017] Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to effi- cient event-driven networks for image classification. Fron- tiers in Neuroscience, 11:682, 2017.","title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","authors":"Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, Shih-Chii Liu","year":"2017","venue":"Frontiers in Neuroscience","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Tay et al., 2021] Yi Tay, Dara Bahri, Donald Metzler, Da- Cheng Juan, Zhe Zhao, and Che Zheng. Synthesizer: Re- thinking self-attention for transformer models. In Proc. of ICML, 2021.","title":"Synthesizer: Rethinking self-attention for transformer models","authors":"Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng","year":"2021","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Proc. of NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wang et al., 2023] Ziqing Wang, Yuetong Fang, Jiahang Cao, Qiang Zhang, Zhongrui Wang, and Renjing Xu. Masked spiking transformer. In Proc. of ICCV, 2023.","title":"Masked spiking transformer","authors":"Wang","year":"2023","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wolff et al., 2022] Annemarie Wolff, Nareg Berberian, Mehrshad Golesorkhi, Javier Gomez-Pilar, Federico Zilio, and Georg Northoff. Intrinsic neural timescales: temporal integration and segregation. Trends in Cognitive Sciences, 26(2):159-173, 2022.","title":"Intrinsic neural timescales: temporal integration and segregation","authors":"Annemarie Wolff, Nareg Berberian, Mehrshad Golesorkhi, Javier Gomez-Pilar, Federico Zilio, Georg Northoff","year":"2022","venue":"Trends in Cognitive Sciences","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wu et al., 2019] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neu- ral networks: Faster, larger, better. In Proc. of AAAI, 2019.","title":"Direct training for spiking neural networks: Faster, larger, better","authors":"Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, Luping Shi","year":"2019","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wu et al., 2021] Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions on Neural Networks and Learning Systems, 33(11):6249-6262, 2021.","title":"Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing","authors":"Wu","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Yao et al., 2021] Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang, Yihan Lin, Zhaoxu Yang, and Guoqi Li. Temporal-wise attention spiking neural net- works for event streams classification. In Proc. of ICCV, 2021.","title":"Temporal-wise attention spiking neural networks for event streams classification","authors":"Yao","year":"2021","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yao et al., 2023a] Man Yao, JiaKui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo XU, and Guoqi Li. Spike- driven transformer. In Proc. of NeurIPS, 2023.","title":"Spikedriven transformer","authors":"Yao","year":"2023","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yao et al., 2023b] Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2023.","title":"Attention spiking neural networks","authors":"Yao","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhai et al., 2021] Shuangfei Zhai, Walter Talbott, Nitish Srivastava, Chen Huang, Hanlin Goh, Ruixiang Zhang, and Josh Susskind. An attention free transformer. arXiv preprint arXiv:2105.14103, 2021.","title":"An attention free transformer","authors":"Zhai","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang and Linden, 2003] Wei Zhang and David J Linden. The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nature Reviews Neuro- science, 4(11):885-900, 2003.","title":"The other side of the engram: experience-driven changes in neuronal intrinsic excitability","authors":"Wei Zhang, David J Linden","year":"2003","venue":"Nature Reviews Neuroscience","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zheng et al., 2021] Hanle Zheng, Yujie Wu, Lei Deng, Yi- fan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proc. of AAAI, 2021.","title":"Going deeper with directly-trained larger spiking neural networks","authors":"Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, Guoqi Li","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhou et al., 2023] Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng YAN, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. In Proc. of ICLR, 2023.","title":"Spikformer: When spiking neural network meets transformer","authors":"Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Yan Shuicheng, Yonghong Tian, Li Yuan","year":"2023","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}