{"type":"ScholarlyArticle","title":"TCCNet: Temporally Consistent Context-Free Network for Semi-supervised Video Polyp Segmentation","authors":[{"name":"Xiaotong Li","slug":"xiaotong-li","given":"Xiaotong","family":"Li","affiliation":"School of Computer Science , Shanghai Key Laboratory of Intelligent Information Processing , Shanghai Collaborative Innovation Center of Intelligent Visual Computing , Fudan University"},{"name":"Jilan Xu","slug":"jilan-xu","given":"Jilan","family":"Xu","affiliation":"School of Computer Science , Shanghai Key Laboratory of Intelligent Information Processing , Shanghai Collaborative Innovation Center of Intelligent Visual Computing , Fudan University"},{"name":"Yuejie Zhang","slug":"yuejie-zhang","given":"Yuejie","family":"Zhang","affiliation":"School of Computer Science , Shanghai Key Laboratory of Intelligent Information Processing , Shanghai Collaborative Innovation Center of Intelligent Visual Computing , Fudan University"},{"name":"Rui Feng","slug":"rui-feng","given":"Rui","family":"Feng","affiliation":"School of Computer Science , Shanghai Key Laboratory of Intelligent Information Processing , Shanghai Collaborative Innovation Center of Intelligent Visual Computing , Fudan University"},{"name":"Rui-Wei Zhao","slug":"rui-wei-zhao","given":"Rui-Wei","family":"Zhao","affiliation":"Academy for Engineering and Technology , Fudan University"},{"name":"Tao Zhang","slug":"tao-zhang","given":"Tao","family":"Zhang","affiliation":"School of Information Management and Engineering , Shanghai Key Laboratory of Financial Information Technology , Shanghai University of Finance and Economics"},{"name":"Xuequan Lu","slug":"xuequan-lu","given":"Xuequan","family":"Lu","affiliation":"School of Information Technology , Deakin University"},{"name":"Shang Gao","slug":"shang-gao","given":"Shang","family":"Gao","affiliation":"School of Information Technology , Deakin University"}],"abstract":"Automatic video polyp segmentation (VPS) is highly valued for the early diagnosis of colorectal cancer. However, existing methods are limited in three respects: 1) most of them work on static images, while ignoring the temporal information in consecutive video frames; 2) all of them are fully supervised and easily overfit in presence of limited annotations; 3) the context of polyp (i.e., lumen, specularity and mucosa tissue) varies in an endoscopic clip, which may affect the predictions of adjacent frames. To resolve these challenges, we propose a novel Temporally Consistent Context-Free Network (TCCNet) for semi-supervised VPS. It contains a segmentation branch and a propagation branch with a co-training scheme to supervise the predictions of unlabeled image. To maintain the temporal consistency of predictions, we design a Sequence-Corrected Reverse Attention module and a Propagation-Corrected Reverse Attention module. A Context-Free Loss is also proposed to mitigate the impact of varying contexts. Extensive experiments show that even trained under 1/15 label ratio, TCCNet is comparable to the state-of-the-art fully supervised methods for VPS. Also, TCCNet surpasses existing semi-supervised methods for natural image and other medical image segmentation tasks.","doi":"10.24963/ijcai.2022/155","url":"https://ijcai.org/proceedings/2022/155","pdf":"https://ijcai.org/proceedings/2022/0155.pdf","markdown":"https://ijcai.org/proceedings/2022/155.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/155","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":155,"pages":{"first":1109,"last":1115},"keywords":["Computer Vision: Biomedical Image Analysis"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987029","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985172","length_sec":546,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bernal et al., 2015] Jorge Bernal, Francisco Javier Sánchez, Gloria Fernández-Esparrach, Debora Gil, Cristina Rodríguez de Miguel, and Fernando Vilariño. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physi- cians. Comput. Medical Imaging Graph., pages 99-111, 2015.","title":"WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians","authors":"Bernal","year":"2015","venue":"Comput. Medical Imaging Graph","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229, 2020.","title":"End-to-end object detection with transformers","authors":"Carion","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Shuhan Chen, Xiuli Tan, Ben Wang, and Xuelong Hu. Reverse attention for salient object detection. In ECCV, pages 234-250, 2018.","title":"Reverse attention for salient object detection","authors":"Shuhan Chen, Xiuli Tan, Ben Wang, Xuelong Hu","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic seg- mentation with cross pseudo supervision. In CVPR, pages 2613-2622, 2021.","title":"Semi-supervised semantic segmentation with cross pseudo supervision","authors":"Xiaokang Chen, Yuhui Yuan, Gang Zeng, Jingdong Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2020] Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao. Pranet: Parallel reverse attention network for polyp seg- mentation. In MICCAI, pages 263-273, 2020.","title":"Pranet: Parallel reverse attention network for polyp segmentation","authors":"Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, Ling Shao","year":"2020","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2021] Li Hu, Peng Zhang, Bang Zhang, Pan Pan, Yinghui Xu, and Rong Jin. Learning position and target consistency for memory-based video object segmentation. In CVPR, pages 4144-4154, 2021.","title":"Learning position and target consistency for memory-based video object segmentation","authors":"Li Hu, Peng Zhang, Bang Zhang, Pan Pan, Yinghui Xu, Rong Jin","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jha et al., 2019] Debesh Jha, Pia H Smedsrud, Michael A Riegler, Dag Johansen, Thomas De Lange, Pål Halvorsen, and Håvard D Johansen. Resunet++: An advanced archi- tecture for medical image segmentation. In ISM, pages 225-2255, 2019.","title":"Resunet++: An advanced architecture for medical image segmentation","authors":"Debesh Jha, Pia H Smedsrud, Michael A Riegler, Dag Johansen, Thomas De Lange, Pål Halvorsen, Håvard D Johansen","year":"2019","venue":"ISM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ji et al., 2021] Ge-Peng Ji, Yu-Cheng Chou, Deng-Ping Fan, Geng Chen, Huazhu Fu, Debesh Jha, and Ling Shao. Progressively normalized self-attention network for video polyp segmentation. In MICCAI, pages 142-152, 2021.","title":"Progressively normalized self-attention network for video polyp segmentation","authors":"Ge-Peng Ji, Yu-Cheng Chou, Deng-Ping Fan, Geng Chen, Huazhu Fu, Debesh Jha, Ling Shao","year":"2021","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2021] Taehun Kim, Hyemin Lee, and Daijin Kim. Uacanet: Uncertainty augmented context attention for polyp segmentation. In ACM Multimedia, pages 2167- 2175, 2021.","title":"Uacanet: Uncertainty augmented context attention for polyp segmentation","authors":"Kim","year":"2021","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Luo et al., 2021] Xiangde Luo, Wenjun Liao, Jieneng Chen, Tao Song, Yinan Chen, Shichuan Zhang, Nianyong Chen, Guotai Wang, and Shaoting Zhang. Efficient semi- supervised gross target volume of nasopharyngeal carci- noma segmentation via uncertainty rectified pyramid con- sistency. In MICCAI, pages 318-329, 2021.","title":"Efficient semisupervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency","authors":"Luo","year":"2021","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Oh et al., 2019] Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim. Video object segmentation using space-time memory networks. In ICCV, pages 9226-9235, 2019.","title":"Video object segmentation using space-time memory networks","authors":"Oh","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ouali et al., 2020] Yassine Ouali, Céline Hudelot, and Myr- iam Tami. Semi-supervised semantic segmentation with cross-consistency training. In CVPR, pages 12674-12684, 2020.","title":"Semi-supervised semantic segmentation with cross-consistency training","authors":"Yassine Ouali, Céline Hudelot, Myriam Tami","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Silva et al., 2014] Juan Silva, Aymeric Histace, Olivier Ro- main, Xavier Dray, and Bertrand Granado. Toward embed- ded detection of polyps in wce images for early diagnosis of colorectal cancer. Comput. Assist. Radiol. Surg., pages 283-293, 2014.","title":"Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer","authors":"Silva","year":"2014","venue":"Comput. Assist. Radiol. Surg","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tajbakhsh et al., 2016] Nima Tajbakhsh, Suryakanth R. Gu- rudu, and Jianming Liang. Automated polyp detection in colonoscopy videos using shape and context information. TMI, pages 630-644, 2016.","title":"Automated polyp detection in colonoscopy videos using shape and context information","authors":"Nima Tajbakhsh, R Suryakanth, Jianming Gurudu, Liang","year":"2016","venue":"TMI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. In NIPS, pages 1195-1204, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Verma et al., 2019] Vikas Verma, Alex Lamb, Juho Kan- nala, Yoshua Bengio, and David Lopez-Paz. Interpolation consistency training for semi-supervised learning. In IJ- CAI, pages 3635-3641, 2019.","title":"Interpolation consistency training for semi-supervised learning","authors":"Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, David Lopez-Paz","year":"2019","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2019/504"},{"position":17,"raw":"[Yu et al., 2019] Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng. Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. In MICCAI, pages 605-613, 2019.","title":"Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation","authors":"Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, Pheng-Ann Heng","year":"2019","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang et al., 2017] Yizhe Zhang, Lin Yang, Jianxu Chen, Maridel Fredericksen, David P Hughes, and Danny Z Chen. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In IJCAI, pages 408-416, 2017.","title":"Deep adversarial networks for biomedical image segmentation utilizing unannotated images","authors":"Zhang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhou et al., 2018] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: A nested u-net architecture for medical image segmenta- tion. In MICCAI, pages 3-11, 2018.","title":"Unet++: A nested u-net architecture for medical image segmentation","authors":"Zhou","year":"2018","venue":"MICCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}