{"type":"ScholarlyArticle","title":"A Lightweight U-like Network Utilizing Neural Memory Ordinary Differential Equations for Slimming the Decoder","authors":[{"name":"Quansong He","slug":"quansong-he","given":"Quansong","family":"He","affiliation":"College of Computer Science , Sichuan University , Chengdu , China"},{"name":"Xiaojun Yao","slug":"xiaojun-yao","given":"Xiaojun","family":"Yao","affiliation":"Department of Thoracic Surgery , Public Health Clinical Center of Chengdu , China"},{"name":"Jun Wu","slug":"jun-wu","given":"Jun","family":"Wu","affiliation":"College of Computer Science , Sichuan University , Chengdu , China"},{"name":"Zhang Yi","slug":"zhang-yi","given":"Zhang","family":"Yi","affiliation":"College of Computer Science , Sichuan University , Chengdu , China"},{"name":"Tao He","slug":"tao-he","given":"Tao","family":"He","affiliation":"College of Computer Science , Sichuan University , Chengdu , China"}],"abstract":"In recent years, advanced U-like networks have demonstrated remarkable performance in medical image segmentation tasks. However, their drawbacks, including excessive parameters, high computational complexity, and slow inference speed, pose challenges for practical implementation in scenarios with limited computational resources. Existing lightweight U-like networks have alleviated some problems, but they often have pre-designed structures and consist of non-detachable modules, limiting their application scenarios. In this paper, we propose three plug-and-play decoders by employing different discretization methods of the neural memory Ordinary Differential Equation (nmODE). These decoders integrate features at various levels of abstraction by processing information from skip connections and performing numerical operations on upward paths. Through experiments on the PH2, ISIC2017, and ISIC2018 datasets, we embed these decoders into different U-like networks, demonstrating their effectiveness in significantly reducing the number of parameters and computation while maintaining performance. In summary, the proposed discretized nmODE decoder is capable of reducing the number of parameters by about 20% ~ 50% and computation by up to 74%, while being adaptive to all U-like networks. Our code is available at https://github.com/nayutayuki/Lightweight-nmODE-Decoders-For-U-like-networks.","doi":"10.24963/ijcai.2024/91","url":"https://ijcai.org/proceedings/2024/91","pdf":"https://ijcai.org/proceedings/2024/0091.pdf","markdown":"https://ijcai.org/proceedings/2024/91.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/91","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":91,"pages":{"first":821,"last":829},"keywords":["Computer Vision: CV: Biomedical image analysis","Machine Learning: ML: Convolutional networks","Computer Vision: CV: Segmentation","Computer Vision: CV: Machine learning for vision"],"videos":[],"references":[{"position":1,"raw":"[Bashforth and Adams, 1883] Francis Bashforth and John Couch Adams. An attempt to test the theories of capillary action by comparing the theoretical and measured forms of drops of fluid. University Press, 1883.","title":"An attempt to test the theories of capillary action by comparing the theoretical and measured forms of drops of fluid","authors":"Adams ; Bashforth, Francis Bashforth, John Couch, Adams","year":"1883","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2022] Cao, Hu, Yueyue Wang, Joy Chen, Dong- sheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. Swin-unet: Unet-like pure transformer for medical image segmentation. In European conference on computer vision, pages 205-218, 2022.","title":"Swin-unet: Unet-like pure transformer for medical image segmentation","authors":"Cao, Yueyue Hu, Joy Wang, Dongsheng Chen, Xiaopeng Jiang, Qi Zhang, Manning Tian, Wang","year":"2022","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary dif- ferential equations. Advances in neural information pro- cessing systems, 31, 2018.","title":"Neural ordinary differential equations","authors":"Yulia Ricky Tq Chen, Jesse Rubanova, David K Bettencourt, Duvenaud","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2023] Junlong Cheng, Chengrui Gao, Fengjie Wang, and Min Zhu. Segnetr: Rethinking the local-global interactions and skip connections in u-shaped networks, 2023.","title":"Segnetr: Rethinking the local-global interactions and skip connections in u-shaped networks","authors":"Junlong Cheng, Chengrui Gao, Fengjie Wang, Min Zhu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2023] Peizhi Dong, Hao Niu, Zhang Yi, and Xiuyuan Xu. nmpls-net: Segmenting pulmonary lobes us- ing nmode. Mathematics, 11(22), 2023.","title":"nmpls-net: Segmenting pulmonary lobes using nmode","authors":"Dong","year":"2023","venue":"Mathematics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy et al., 2020] A Dosovitskiy, L Beyer, A Kolesnikov, D Weissenborn, X Zhai, and T Un- terthiner. Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"Transformers for image recognition at scale","authors":"Dosovitskiy, Beyer, D Kolesnikov, X Weissenborn, Zhai, Unterthiner","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dupont et al., 2019] Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes, 2019.","title":"Augmented neural odes","authors":"Emilien Dupont, Arnaud Doucet, Yee Whye Teh","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Euler, 1845] Leonhard Euler. Institutionum calculi inte- gralis, volume 4. impensis Academiae imperialis scien- tiarum, 1845.","title":"Leonhard Euler. Institutionum calculi integralis","authors":"Euler","year":"1845","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2022] Yunhe Gao, Mu Zhou, Ding Liu, and Dim- itris N. Metaxas. A multi-scale transformer for medical image segmentation: Architectures, model efficiency, and benchmarks. ArXiv, abs/2203.00131, 2022.","title":"A multi-scale transformer for medical image segmentation: Architectures, model efficiency, and benchmarks","authors":"Yunhe Gao, Mu Zhou, Ding Liu, Dimitris N Metaxas","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition, 2015.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2023] Quansong He, Tao He, and Zhang Yi. Med- ical image segmentation using discretized nmode. In 2023 CSIS-IAC, pages 497-503, 2023.","title":"Medical image segmentation using discretized nmode","authors":"He","year":"2023","venue":"CSIS-IAC","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Heun, 1900] Karl Heun. Neue methoden zur approxi- mativen integration der differentialgleichungen einer un- abhängigen veränderlichen. Z. Math. Phys, 45:23-38, 1900.","title":"Neue methoden zur approximativen integration der differentialgleichungen einer unabhängigen veränderlichen","authors":"Karl Heun, Heun","year":"1900","venue":"Z. Math. Phys","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hu et al., 2023] Junjie Hu, Chengrong Yu, Zhang Yi, and Haixian Zhang. Enhancing robustness of medical image segmentation model with neural memory ordinary differ- ential equation. International Journal of Neural Systems, 33(12):2350060, 2023. PMID: 37743765.","title":"Enhancing robustness of medical image segmentation model with neural memory ordinary differential equation","authors":"Junjie Hu, Chengrong Yu, Zhang Yi, Haixian Zhang","year":"2023","venue":"International Journal of Neural Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF in- ternational conference on computer vision, pages 10012- 10022, 2021.","title":"Swin transformer: Hierarchical vision transformer using shifted windows","authors":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo","year":"2021","venue":"Proceedings of the IEEE/CVF international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Loshchilov and Hutter, ] I Loshchilov and F SGDR Hutter. Stochastic gradient descent with warm restarts. 2016.","title":"Stochastic gradient descent with warm restarts","authors":"Loshchilov, ] I Hutter, F Sgdr Loshchilov, Hutter","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Loshchilov and Hutter, 2017] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mehta and Rastegari, 2022] Sachin Mehta and Mohammad Rastegari. Separable self-attention for mobile vision trans- formers, 2022.","title":"Separable self-attention for mobile vision transformers","authors":"Sachin Mehta, Mohammad Rastegari","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oktay et al., 2018] Ozan Oktay, Jo Schlemper, Loic Le Fol- goc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, Ben Glocker, and Daniel Rueckert. At- tention u-net: Learning where to look for the pancreas, 2018. [Poucet and Save, 2005] Bruno Poucet and Etienne Save. Attractors in memory. Science, 308(5723):799-800, 2005.","title":"Attention u-net: Learning where to look for the pancreas, 2018. [Poucet and Save, 2005] Bruno Poucet and Etienne Save. Attractors in memory","authors":"Ozan Oktay, Jo Schlemper, Le Loic, Matthew Folgoc, Mattias Lee, Kazunari Heinrich, Kensaku Misawa, Steven Mori, Nils Y Mcdonagh, Bernhard Hammerla, Ben Kainz, Daniel Glocker, Rueckert","year":"2005","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pages 234-241, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ruan et al., 2022] Jiacheng Ruan, Suncheng Xiang, Mingye Xie, Ting Liu, and Yuzhuo Fu. Malunet: A multi-attention and light-weight unet for skin lesion segmentation. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1150-1156, 2022.","title":"Malunet: A multi-attention and light-weight unet for skin lesion segmentation","authors":"Ruan","year":"2022","venue":"IEEE International Conference on Bioinformatics and Biomedicine (BIBM)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ruan et al., 2023] Jiacheng Ruan, Mingye Xie, Jingsheng Gao, Ting Liu, and Yuzhuo Fu. Ege-unet: an efficient group enhanced unet for skin lesion segmentation. arXiv preprint arXiv:2307.08473, 2023.","title":"Ege-unet: an efficient group enhanced unet for skin lesion segmentation","authors":"Ruan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tolstikhin et al., 2021] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural infor- mation processing systems, 34:24261-24272, 2021.","title":"Mlp-mixer: An all-mlp architecture for vision","authors":"Neil Ilya O Tolstikhin, Alexander Houlsby, Lucas Kolesnikov, Xiaohua Beyer, Thomas Zhai, Jessica Unterthiner, Andreas Yung, Daniel Steiner, Jakob Keysers, Uszkoreit","year":"2021","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Valanarasu and Patel, 2022] Jeya Maria Jose Valanarasu and Vishal M Patel. Unext: Mlp-based rapid medical im- age segmentation network. In International Conference on Medical Image Computing and Computer-Assisted In- tervention, pages 23-33, 2022.","title":"Unext: Mlp-based rapid medical image segmentation network","authors":"Jeya Maria, Jose Valanarasu, M Vishal, Patel","year":"2022","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2024] Shubin Wang, Yuanyuan Chen, and Zhang Yi. nmode-unet: A novel network for semantic segmentation of medical images. Applied Sciences, 14(1), 2024.","title":"nmode-unet: A novel network for semantic segmentation of medical images","authors":"Shubin Wang, Yuanyuan Chen, Zhang Yi","year":"2024","venue":"Applied Sciences","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2021] Jun Wei, Yiwen Hu, Ruimao Zhang, Zhen Li, S. Kevin Zhou, and Shuguang Cui. Shallow attention network for polyp segmentation, 2021.","title":"Shallow attention network for polyp segmentation","authors":"Jun Wei, Yiwen Hu, Ruimao Zhang, Zhen Li, S Kevin Zhou, Shuguang Cui","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wills et al., 2005] Tom J. Wills, Colin Lever, Francesca Cacucci, Neil Burgess, and John O'Keefe. Attractor dy- namics in the hippocampal representation of the local en- vironment. Science, 308(5723):873-876, 2005.","title":"Attractor dynamics in the hippocampal representation of the local environment","authors":"Tom J Wills, Colin Lever, Francesca Cacucci, Neil Burgess, John O' Keefe","year":"2005","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2022] Huisi Wu, Shihuai Chen, Guilian Chen, Wei Wang, Baiying Lei, and Zhenkun Wen. Fat-net: Fea- ture adaptive transformers for automated skin lesion seg- mentation. Medical Image Analysis, 76:102327, 2022.","title":"Fat-net: Feature adaptive transformers for automated skin lesion segmentation","authors":"Huisi Wu, Shihuai Chen, Guilian Chen, Wei Wang, Baiying Lei, Zhenkun Wen","year":"2022","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yi, 2023] Zhang Yi. nmode: neural memory ordinary differ- ential equation. Artificial Intelligence Review, pages 1-36, 2023.","title":"nmode: neural memory ordinary differential equation","authors":"Zhang Yi","year":"2023","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018] Zhengxin Zhang, Qingjie Liu, and Yun- hong Wang. Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters, 15(5):749-753, 2018.","title":"Road extraction by deep residual u-net","authors":"Zhang","year":"2018","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2021] Yundong Zhang, Huiye Liu, and Qiang Hu. Transfuse: Fusing transformers and cnns for med- ical image segmentation. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part I 24, pages 14-24, 2021.","title":"Transfuse: Fusing transformers and cnns for medical image segmentation","authors":"Zhang","year":"2021","venue":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2018] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: A nested u-net architecture for medical image segmenta- tion, 2018.","title":"Unet++: A nested u-net architecture for medical image segmentation","authors":"Zhou","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}