{"type":"ScholarlyArticle","title":"TSESNet: Temporal-Spatial Enhanced Breast Tumor Segmentation in DCE-MRI Using Feature Perception and Separability","authors":[{"name":"Jiezhou He","slug":"jiezhou-he","given":"Jiezhou","family":"He","affiliation":"Institute of Artificial Intelligence , Xiamen University , China."},{"name":"Xue Zhao","slug":"xue-zhao","given":"Xue","family":"Zhao","affiliation":"National Institute for Data Science in Health and Medicine , Xiamen University , China.; Fujian Key Laboratory of Precision Diagnosis and Treatment in Breast Cancer , Xiang'an Hospital of Xiamen University , School of Medicine , Xiamen University , China."},{"name":"Zhiming Luo","slug":"zhiming-luo","given":"Zhiming","family":"Luo","affiliation":"Department of Artificial Intelligence , Xiamen University , China."},{"name":"Songzhi Su","slug":"songzhi-su","given":"Songzhi","family":"Su","affiliation":"Department of Artificial Intelligence , Xiamen University , China.; The Third Affiliated Hospital of Kunming Medical University &; Peking University Cancer Hospital Yunnan , China."},{"name":"Shaozi Li","slug":"shaozi-li","given":"Shaozi","family":"Li","affiliation":"Department of Artificial Intelligence , Xiamen University , China."},{"name":"Guojun Zhang","slug":"guojun-zhang","given":"Guojun","family":"Zhang","affiliation":"Fujian Key Laboratory of Precision Diagnosis and Treatment in Breast Cancer , Xiang'an Hospital of Xiamen University , School of Medicine , Xiamen University , China.; Yunnan Cancer Hospital &"}],"abstract":"Accurate segmentation of breast tumors in dynamic contrast-enhanced magnetic resonance images (DCE-MRI) is critical for early diagnosis of breast cancer. However, this task remains challenging due to the wide range of tumor sizes, shapes, and appearances. Additionally, the complexity is further compounded by the high dimensionality and ill-posed artifacts present in DCE-MRI data. Furthermore, accurately modeling features in DCE-MRI sequences presents a challenge that hinders the effective representation of essential tumor characteristics. Therefore, this paper introduces a novel Temporal-Spatial Enhanced Network (TSESNet) for breast tumor segmentation in DCE-MRI. TSESNet leverages the spatial and temporal dependencies of DCE-MRI to provide a comprehensive representation of tumor features. To address sequence modeling challenges, we propose a Temporal-Spatial Contrastive Loss (TSCLoss) that maximizes the distance between different classes and minimizes the distance within the same class, thereby improving the separation between tumors and the background. Moreover, we design a novel Temporal Series Feature Fusion (TSFF) module that effectively integrates temporal MRI features from multiple time points, enhancing the model's ability to handle temporal sequences and improving overall performance. Finally, we introduce a simple and effective Tumor-Aware (TA) module that enriches feature representation to accommodate tumors of various sizes. We conducted comprehensive experiments to validate the proposed method and demonstrate its superior performance compared to recent state-of-the-art segmentation methods on two breast cancer DCE-MRI datasets.","doi":"10.24963/ijcai.2024/89","url":"https://ijcai.org/proceedings/2024/89","pdf":"https://ijcai.org/proceedings/2024/0089.pdf","markdown":"https://ijcai.org/proceedings/2024/89.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/89","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Computer Vision","track":"Main Track","number":89,"pages":{"first":803,"last":811},"keywords":["Computer Vision: CV: Biomedical image analysis","Computer Vision: CV: Segmentation","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Benjelloun et al., 2018] Mohammed Benjelloun, Mo- hammed El Adoui, Mohamed Amine Larhmam, and Sidi Ahmed Mahmoudi. Automated breast tumor seg- mentation in dce-mri using deep learning. In 2018 4th International Conference on Cloud Computing Technolo- gies and Applications (Cloudtech), pages 1-6. IEEE, 2018.","title":"Automated breast tumor segmentation in dce-mri using deep learning","authors":"Benjelloun","year":"2018","venue":"2018 4th International Conference on Cloud Computing Technologies and Applications (Cloudtech)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caliskan, 2023] Aylin Caliskan. Artificial intelligence, bias, and ethics. In Proceedings of the Thirty-Second Interna- tional Joint Conference on Artificial Intelligence (IJCAI), 2023.","title":"Artificial intelligence, bias, and ethics","authors":"Aylin Caliskan","year":"2023","venue":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2023/799"},{"position":3,"raw":"[Chen et al., 2017] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. Sca- cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 5659-5667, 2017.","title":"Scacnn: Spatial and channel-wise attention in convolutional networks for image captioning","authors":"Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, Tat-Seng Chua","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Haq et al., 2022] Imran Ul Haq, Haider Ali, Hong Yu Wang, Lei Cui, and Jun Feng. Bts-gan: computer-aided segmen- tation system for breast tumor using mri and conditional adversarial networks. Engineering Science and Technol- ogy, an International Journal, 36:101154, 2022.","title":"Bts-gan: computer-aided segmentation system for breast tumor using mri and conditional adversarial networks","authors":"Imran Ul Haq, Haider Ali, Hong Yu Wang, Lei Cui, Jun Feng","year":"2022","venue":"Engineering Science and Technology, an International Journal","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Lee et al., 2023] Ho Hin Lee, Yucheng Tang, Qi Yang, Xin Yu, Leon Y Cai, Lucas W Remedios, Shunxing Bao, Ben- nett A Landman, and Yuankai Huo. Semantic-aware con- trastive learning for multi-object medical image segmenta- tion. IEEE journal of biomedical and health informatics, 2023.","title":"Semantic-aware contrastive learning for multi-object medical image segmentation","authors":"Yucheng Ho Hin Lee, Qi Tang, Xin Yang, Leon Y Yu, Lucas W Cai, Shunxing Remedios, Bao, Yuankai Bennett A Landman, Huo","year":"2023","venue":"IEEE journal of biomedical and health informatics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Lv and Pan, 2021] Tianxu Lv and Xiang Pan. Temporal- spatial graph attention networks for dce-mri breast tumor segmentation. 2021.","title":"Tianxu Lv and Xiang Pan. Temporalspatial graph attention networks for dce-mri breast tumor segmentation","authors":"Pan Lv","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lv et al., 2022] Tianxu Lv, Youqing Wu, Yihang Wang, Yuan Liu, Lihua Li, Chuxia Deng, and Xiang Pan. A hybrid hemodynamic knowledge-powered and fea- ture reconstruction-guided scheme for breast cancer seg- mentation based on dce-mri. Medical Image Analysis, 82:102572, 2022.","title":"A hybrid hemodynamic knowledge-powered and feature reconstruction-guided scheme for breast cancer segmentation based on dce-mri","authors":"Lv","year":"2022","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ma et al., 2021] Jun Ma, Jianan Chen, Matthew Ng, Rui Huang, Yu Li, Chen Li, Xiaoping Yang, and Anne L Mar- tel. Loss odyssey in medical image segmentation. Medical Image Analysis, 71:102035, 2021.","title":"Loss odyssey in medical image segmentation","authors":"Jun Ma, Jianan Chen, Matthew Ng, Rui Huang, Yu Li, Chen Li, Xiaoping Yang, Anne L Martel","year":"2021","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Milletari et al., 2016] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565-571. Ieee, 2016.","title":"V-net: Fully convolutional neural networks for volumetric medical image segmentation","authors":"Fausto Milletari, Nassir Navab, Seyed-Ahmad Ahmadi","year":"2016","venue":"fourth international conference on 3D vision (3DV)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Peng et al., 2022] Chengtao Peng, Yue Zhang, You Meng, Yang Yang, Bensheng Qiu, Yuzhu Cao, and Jian Zheng. Lma-net: A lesion morphology aware network for medical image segmentation towards breast tumors. Computers in Biology and Medicine, 147:105685, 2022.","title":"Lma-net: A lesion morphology aware network for medical image segmentation towards breast tumors","authors":"Peng","year":"2022","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Piantadosi et al., 2018] Gabriele Piantadosi, Mario San- sone, and Carlo Sansone. Breast segmentation in mri via u-net deep convolutional neural networks. In 2018 24th international conference on pattern recognition (ICPR), pages 3917-3922. IEEE, 2018.","title":"Breast segmentation in mri via u-net deep convolutional neural networks","authors":"Gabriele Piantadosi, Mario Sansone, Carlo Sansone","year":"2018","venue":"24th international conference on pattern recognition (ICPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Qi et al., 2023] Wenbo Qi, HC Wu, and SC Chan. Mdf-net: A multi-scale dynamic fusion network for breast tumor segmentation of ultrasound images. IEEE Transactions on Image Processing, 2023.","title":"Mdf-net: A multi-scale dynamic fusion network for breast tumor segmentation of ultrasound images","authors":"Wenbo Qi, Wu, Chan","year":"2023","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Qin et al., 2022] ChuanBo Qin, JingYin Lin, JunYing Zeng, YiKui Zhai, LianFang Tian, ShuTing Peng, and Fang Li. Joint dense residual and recurrent attention network for dce-mri breast tumor segmentation. Computational Intel- ligence and Neuroscience, 2022, 2022.","title":"Joint dense residual and recurrent attention network for dce-mri breast tumor segmentation","authors":"Qin","year":"2022","venue":"Computational Intelligence and Neuroscience","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ru et al., 2023] Jintao Ru, Beichen Lu, Buran Chen, Jialin Shi, Gaoxiang Chen, Meihao Wang, Zhifang Pan, Yezhi Lin, Zhihong Gao, Jiejie Zhou, et al. Attention guided neu- ral ode network for breast tumor segmentation in medical images. Computers in Biology and Medicine, 159:106884, 2023.","title":"Attention guided neural ode network for breast tumor segmentation in medical images","authors":"Jintao Ru, Beichen Lu, Buran Chen, Jialin Shi, Gaoxiang Chen, Meihao Wang, Zhifang Pan, Yezhi Lin, Zhihong Gao, Jiejie Zhou","year":"2023","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Vidal et al., 2022] Joel Vidal, Joan C Vilanova, Robert Martí, et al. A u-net ensemble for breast lesion segmen- tation in dce mri. Computers in Biology and Medicine, 140:105093, 2022.","title":"A u-net ensemble for breast lesion segmentation in dce mri","authors":"Joel Vidal, Joan C Vilanova, Robert Martí","year":"2022","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2021] Shuai Wang, Kun Sun, Li Wang, Liangqiong Qu, Fuhua Yan, Qian Wang, and Dinggang Shen. Breast tumor segmentation in dce-mri with tumor sensitive synthesis. IEEE Transactions on Neural Net- works and Learning Systems, 2021.","title":"Breast tumor segmentation in dce-mri with tumor sensitive synthesis","authors":"Shuai Wang, Kun Sun, Li Wang, Liangqiong Qu, Fuhua Yan, Qian Wang, Dinggang Shen","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2023] Yu Wang, Bo Liu, and Fugen Zhou. Pcmask: A dual-branch self-supervised medical image segmentation method using pixel-level contrastive learn- ing and masked image modeling. In Image and Vision Computing: 37th International Conference, IVCNZ 2022, Auckland, New Zealand, November 24-25, 2022, Revised Selected Papers, pages 501-510. Springer, 2023.","title":"Pcmask: A dual-branch self-supervised medical image segmentation method using pixel-level contrastive learning and masked image modeling","authors":"Yu Wang, Bo Liu, Fugen Zhou","year":"2022","venue":"Image and Vision Computing: 37th International Conference, IVCNZ 2022","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yeung et al., 2022] Michael Yeung, Evis Sala, Carola- Bibiane Schönlieb, and Leonardo Rundo. Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95:102026, 2022.","title":"Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation","authors":"Michael Yeung, Evis Sala, Carola-Bibiane Schönlieb, Leonardo Rundo","year":"2022","venue":"Computerized Medical Imaging and Graphics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[You et al., 2022] Chenyu You, Yuan Zhou, Ruihan Zhao, Lawrence Staib, and James S Duncan. Simcvd: Simple contrastive voxel-wise representation distillation for semi- supervised medical image segmentation. IEEE Transac- tions on Medical Imaging, 41(9):2228-2237, 2022.","title":"Simcvd: Simple contrastive voxel-wise representation distillation for semisupervised medical image segmentation","authors":"Chenyu You, Yuan Zhou, Ruihan Zhao, Lawrence Staib, James S Duncan","year":"2022","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zeng et al., 2021] Dewen Zeng, Yawen Wu, Xinrong Hu, Xiaowei Xu, Haiyun Yuan, Meiping Huang, Jian Zhuang, Jingtong Hu, and Yiyu Shi. Positional contrastive learning for volumetric medical image segmentation. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2021: 24th International Conference, Stras- bourg, France, September 27-October 1, 2021, Proceed- ings, Part II 24, pages 221-230. Springer, 2021.","title":"Positional contrastive learning for volumetric medical image segmentation","authors":"Dewen Zeng, Yawen Wu, Xinrong Hu, Xiaowei Xu, Haiyun Yuan, Meiping Huang, Jian Zhuang, Jingtong Hu, Yiyu Shi","year":"2021","venue":"Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2019] Lei Zhang, Zhimeng Luo, Ruimei Chai, Dooman Arefan, Jules Sumkin, and Shandong Wu. Deep- learning method for tumor segmentation in breast dce- mri. In Medical Imaging 2019: Imaging Informatics for Healthcare, Research, and Applications, volume 10954, pages 97-102. SPIE, 2019.","title":"Deeplearning method for tumor segmentation in breast dcemri","authors":"Lei Zhang, Zhimeng Luo, Ruimei Chai, Dooman Arefan, Jules Sumkin, Shandong Wu","year":"2019","venue":"Imaging Informatics for Healthcare, Research, and Applications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2022] Jiyang Zhang, Jianxiao Zou, Zhiheng Su, Jianxiong Tang, Yuhao Kang, Hongbing Xu, Zhil- iang Liu, and Shicai Fan. A class-aware supervised con- trastive learning framework for imbalanced fault diagno- sis. Knowledge-Based Systems, 252:109437, 2022.","title":"A class-aware supervised contrastive learning framework for imbalanced fault diagnosis","authors":"Jiyang Zhang, Jianxiao Zou, Zhiheng Su, Jianxiong Tang, Yuhao Kang, Hongbing Xu, Zhiliang Liu, Shicai Fan","year":"2022","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2023] Jiadong Zhang, Zhiming Cui, Zhenwei Shi, Yingjia Jiang, Zhiliang Zhang, Xiaoting Dai, Zhenlu Yang, Yuning Gu, Lei Zhou, Chu Han, et al. A robust and efficient ai assistant for breast tumor segmentation from dce-mri via a spatial-temporal framework. Patterns, 4(9), 2023.","title":"A robust and efficient ai assistant for breast tumor segmentation from dce-mri via a spatial-temporal framework","authors":"Jiadong Zhang, Zhiming Cui, Zhenwei Shi, Yingjia Jiang, Zhiliang Zhang, Xiaoting Dai, Zhenlu Yang, Yuning Gu, Lei Zhou, Chu Han","year":"2023","venue":"Patterns","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2023a] Xiaoming Zhao, Yuehui Liao, Jiahao Xie, Xiaxia He, Shiqing Zhang, Guoyu Wang, Jiangxiong Fang, Hongsheng Lu, and Jun Yu. Breastdm: A dce-mri dataset for breast tumor image segmentation and classifi- cation. Computers in Biology and Medicine, 164:107255, 2023.","title":"Breastdm: A dce-mri dataset for breast tumor image segmentation and classification","authors":"Zhao","year":"2023","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2023b] Xue Zhao, Jing-Wen Bai, Qiu Guo, Ke Ren, and Guo-Jun Zhang. Clinical applications of deep learning in breast mri. Biochimica et Biophysica Acta (BBA)-Reviews on Cancer, page 188864, 2023.","title":"Clinical applications of deep learning in breast mri","authors":"Zhao","year":"2023","venue":"Biochimica et Biophysica Acta (BBA)-Reviews on Cancer","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhao et al., 2024] Zhihe Zhao, Siyao Du, Zeyan Xu, Zhi Yin, Xiaomei Huang, Xin Huang, Chinting Wong, Yant- ing Liang, Jing Shen, Jianlin Wu, et al. Swinhr: Hemodynamic-powered hierarchical vision transformer for breast tumor segmentation. Computers in Biology and Medicine, page 107939, 2024.","title":"Swinhr: Hemodynamic-powered hierarchical vision transformer for breast tumor segmentation","authors":"Zhao","year":"2024","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2022] Lei Zhou, Shuai Wang, Kun Sun, Tao Zhou, Fuhua Yan, and Dinggang Shen. Three-dimensional affinity learning based multi-branch ensemble network for breast tumor segmentation in mri. Pattern Recognition, 129:108723, 2022.","title":"Three-dimensional affinity learning based multi-branch ensemble network for breast tumor segmentation in mri","authors":"Lei Zhou, Shuai Wang, Kun Sun, Tao Zhou, Fuhua Yan, Dinggang Shen","year":"2022","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}