{"type":"ScholarlyArticle","title":"Unsupervised Domain Adaptation with Dual-Scheme Fusion Network for Medical Image Segmentation","authors":[{"name":"Danbing Zou","slug":"danbing-zou","given":"Danbing","family":"Zou","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"},{"name":"Qikui Zhu","slug":"qikui-zhu","given":"Qikui","family":"Zhu","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"},{"name":"Pingkun Yan","slug":"pingkun-yan","given":"Pingkun","family":"Yan","affiliation":"Rensselaer Polytechnic Institute , Department of Biomedical Engineering , Troy , NY , USA"}],"abstract":"Domain adaptation aims to alleviate the problem of retraining a pre-trained model when applying it to a different domain, which requires large amount of additional training data of the target domain. Such an objective is usually achieved by establishing connections between the source domain labels and target domain data. However, this imbalanced source-to-target one way pass may not eliminate the domain gap, which limits the performance of the pre-trained model. In this paper, we propose an innovative Dual-Scheme Fusion Network (DSFN) for unsupervised domain adaptation. By building both source-to-target and target-to-source connections, this balanced joint information flow helps reduce the domain gap to further improve the network performance. The mechanism is further applied to the inference stage, where both the original input target image and the generated source images are segmented with the proposed joint network. The results are fused to obtain more robust segmentation. Extensive experiments of unsupervised cross-modality medical image segmentation are conducted on two tasks -- brain tumor segmentation and cardiac structures segmentation. The experimental results show that our method achieved significant performance improvement over other state-of-the-art domain adaptation methods.","doi":"10.24963/ijcai.2020/455","url":"https://ijcai.org/proceedings/2020/455","pdf":"https://ijcai.org/proceedings/2020/0455.pdf","markdown":"https://ijcai.org/proceedings/2020/455.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/455","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":455,"pages":{"first":3291,"last":3298},"keywords":["Machine Learning: Deep-learning Theory","Machine Learning: Deep Learning","Machine Learning: Deep Learning: Convolutional networks"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24272/file_24272.m3u8","length_sec":304,"provider":"hls"}],"references":[{"position":1,"raw":"[Bousmalis et al., 2017] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Kr- ishnan. Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 3722-3731, 2017.","title":"Unsupervised pixel-level domain adaptation with generative adversarial networks","authors":"Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, Dilip Krishnan","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Cheng Chen, Qi Dou, Hao Chen, and Pheng-Ann Heng. Semantic-aware generative adversar- ial nets for unsupervised domain adaptation in chest x- ray segmentation. In International Workshop on Machine Learning in Medical Imaging, pages 143-151. Springer, 2018.","title":"Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest xray segmentation","authors":"Cheng Chen, Qi Dou, Chen Hao, Pheng-Ann Heng","year":"2018","venue":"International Workshop on Machine Learning in Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Cheng Chen, Qi Dou, Hao Chen, Jing Qin, and Pheng-Ann Heng. Synergistic image and fea- ture adaptation: Towards cross-modality domain adap- tation for medical image segmentation. arXiv preprint arXiv:1901.08211, 2019.","title":"Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation","authors":"Cheng Chen, Qi Dou, Hao Chen, Jing Qin, Pheng-Ann Heng","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dou et al., 2018] Qi Dou, Cheng Ouyang, Cheng Chen, Hao Chen, and Pheng-Ann Heng. Unsupervised cross- modality domain adaptation of convnets for biomedical image segmentations with adversarial loss. arXiv preprint arXiv:1804.10916, 2018.","title":"Unsupervised crossmodality domain adaptation of convnets for biomedical image segmentations with adversarial loss","authors":"Qi Dou, Cheng Ouyang, Cheng Chen, Hao Chen, Pheng-Ann Heng","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fang and Yan, 2020] Xi Fang and Pingkun Yan. Multi- organ segmentation over partially labeled datasets with multi-scale feature abstraction. arXiv preprint arXiv:2001.00208, 2020.","title":"Multiorgan segmentation over partially labeled datasets with multi-scale feature abstraction","authors":"Yan ; Fang, Xi Fang, Pingkun Yan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoffman et al., 2017] Judy Hoffman, Eric Tzeng, Tae- sung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. Cycada: Cycle- consistent adversarial domain adaptation. arXiv preprint arXiv:1711.03213, 2017.","title":"Cycada: Cycleconsistent adversarial domain adaptation","authors":"Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, Trevor Darrell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with con- ditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125-1134, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kamnitsas et al., 2017] Konstantinos Kamnitsas, Christian Baumgartner, Christian Ledig, Virginia Newcombe, Joanna Simpson, Andrew Kane, David Menon, Aditya Nori, Antonio Criminisi, Daniel Rueckert, et al. Unsuper- vised domain adaptation in brain lesion segmentation with adversarial networks. In International conference on in- formation processing in medical imaging, pages 597-609. Springer, 2017.","title":"Unsupervised domain adaptation in brain lesion segmentation with adversarial networks","authors":"Konstantinos Kamnitsas, Christian Baumgartner, Christian Ledig, Virginia Newcombe, Joanna Simpson, Andrew Kane, David Menon, Aditya Nori, Antonio Criminisi, Daniel Rueckert","year":"2017","venue":"International conference on information processing in medical imaging","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable fea- tures with deep adaptation networks. arXiv preprint arXiv:1502.02791, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael I Jordan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mao et al., 2017] Xudong Mao, Qing Li, Haoran Xie, Ray- mond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In ICCV, pages 2794-2802, 2017.","title":"Least squares generative adversarial networks","authors":"Xudong Mao, Qing Li, Haoran Xie, Raymond Yk Lau, Zhen Wang, Stephen Paul Smolley","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Menze et al., 2014] Bjoern H Menze, Andras Jakab, Ste- fan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slot- boom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging, 34(10):1993-2024, 2014.","title":"The multimodal brain tumor image segmentation benchmark (brats)","authors":"Andras Bjoern H Menze, Stefan Jakab, Jayashree Bauer, Keyvan Kalpathy-Cramer, Justin Farahani, Yuliya Kirby, Nicole Burren, Johannes Porz, Roland Slotboom, Wiest","year":"2014","venue":"IEEE transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2017] Dong Nie, Roger Trullo, Jun Lian, Caro- line Petitjean, Su Ruan, Qian Wang, and Dinggang Shen. Medical image synthesis with context-aware generative adversarial networks. In International Conference on Medical Image Computing and Computer-Assisted Inter- vention, pages 417-425. Springer, 2017.","title":"Medical image synthesis with context-aware generative adversarial networks","authors":"Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, Dinggang Shen","year":"2017","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tsai et al., 2018] Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang, and Manmo- han Chandraker. Learning to adapt structured output space for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7472-7481, 2018.","title":"Learning to adapt structured output space for semantic segmentation","authors":"Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang, Manmohan Chandraker","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tzeng et al., 2014] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confu- sion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014.","title":"Deep domain confusion: Maximizing for domain invariance","authors":"Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wolterink et al., 2017] Jelmer M Wolterink, Anna M Din- kla, Mark HF Savenije, Peter R Seevinck, Cornelis AT van den Berg, and Ivana Išgum. Deep mr to ct synthesis using unpaired data. In International Workshop on Sim- ulation and Synthesis in Medical Imaging, pages 14-23. Springer, 2017.","title":"Deep mr to ct synthesis using unpaired data","authors":"Anna M Jelmer M Wolterink, Dinkla, H F Mark, Savenije, Peter R Seevinck, A T Cornelis, Ivana Van Den Berg, Išgum","year":"2017","venue":"International Workshop on Simulation and Synthesis in Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhao et al., 2018] He Zhao, Huiqi Li, Sebastian Maurer- Stroh, Yuhong Guo, Qiuju Deng, and Li Cheng. Su- pervised segmentation of un-annotated retinal fundus im- ages by synthesis. IEEE transactions on medical imaging, 38(1):46-56, 2018.","title":"Supervised segmentation of un-annotated retinal fundus images by synthesis","authors":"Zhao","year":"2018","venue":"IEEE transactions on medical imaging","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. In ICCV, pages 2223-2232, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhu et al., 2020] Qikui Zhu, Bo Du, and Pingkun Yan. Boundary-Weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation. IEEE Transactions on Medical Imaging, 39(3):753-763, March 2020.","title":"Boundary-Weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation","authors":"Zhu","year":"2020","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhuang and Shen, 2016] Xiahai Zhuang and Juan Shen. Multi-scale patch and multi-modality atlases for whole heart segmentation of mri. Medical image analysis, 31:77- 87, 2016.","title":"Xiahai Zhuang and Juan Shen. Multi-scale patch and multi-modality atlases for whole heart segmentation of mri","authors":"Shen Zhuang","year":"2016","venue":"Medical image analysis","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}