{"type":"ScholarlyArticle","title":"Biological Instance Segmentation with a Superpixel-Guided Graph","authors":[{"name":"Xiaoyu Liu","slug":"xiaoyu-liu","given":"Xiaoyu","family":"Liu","affiliation":"University of Science and Technology of China"},{"name":"Wei Huang","slug":"wei-huang","given":"Wei","family":"Huang","affiliation":"University of Science and Technology of China"},{"name":"Yueyi Zhang","slug":"yueyi-zhang","given":"Yueyi","family":"Zhang","affiliation":"University of Science and Technology of China; Institute of Artificial Intelligence , Hefei Comprehensive National Science Center"},{"name":"Zhiwei Xiong","slug":"zhiwei-xiong","given":"Zhiwei","family":"Xiong","affiliation":"University of Science and Technology of China; Institute of Artificial Intelligence , Hefei Comprehensive National Science Center"}],"abstract":"Recent advanced proposal-free instance segmentation methods have made significant progress in biological images. However, existing methods are vulnerable to local imaging artifacts and similar object appearances, resulting in over-merge and over-segmentation. To reduce these two kinds of errors, we propose a new biological instance segmentation framework based on a superpixel-guided graph, which consists of two stages, i.e., superpixel-guided graph construction and superpixel agglomeration. Specifically, the first stage generates enough superpixels as graph nodes to avoid over-merge, and extracts node and edge features to construct an initialized graph. The second stage agglomerates superpixels into instances based on the relationship of graph nodes predicted by a graph neural network (GNN). To solve over-segmentation and prevent introducing additional over-merge, we specially design two loss functions to supervise the GNN, i.e., a repulsion-attraction (RA) loss to better distinguish the relationship of nodes in the feature space, and a maximin agglomeration score (MAS) loss to pay more attention to crucial edge classification. Extensive experiments on three representative biological datasets demonstrate the superiority of our method over existing state-of-the-art methods. Code is available at https://github.com/liuxy1103/BISSG.","doi":"10.24963/ijcai.2022/169","url":"https://ijcai.org/proceedings/2022/169","pdf":"https://ijcai.org/proceedings/2022/0169.pdf","markdown":"https://ijcai.org/proceedings/2022/169.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/169","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":169,"pages":{"first":1209,"last":1215},"keywords":["Computer Vision: Biomedical Image Analysis","Computer Vision: Segmentation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986490","length_sec":81,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984633","length_sec":345,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Araslanov et al., 2019] Nikita Araslanov, Constantin A Rothkopf, and Stefan Roth. Actor-critic instance segmen- tation. In CVPR, 2019.","title":"Actor-critic instance segmentation","authors":"Nikita Araslanov, Constantin A Rothkopf, Stefan Roth","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beier et al., 2017] Thorsten Beier, Constantin Pape, Nasim Rahaman, et al. Multicut brings automated neurite seg- mentation closer to human performance. Nature Methods, 14(2):101-102, 2017.","title":"Multicut brings automated neurite segmentation closer to human performance","authors":"Thorsten Beier, Constantin Pape, Nasim Rahaman","year":"2017","venue":"Nature Methods","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[De Brabandere et al., 2017] Bert De Brabandere, Davy Neven, and Luc Van Gool. Semantic instance segmen- tation with a discriminative loss function. arXiv preprint arXiv:1708.02551, 2017.","title":"Semantic instance segmentation with a discriminative loss function","authors":"De Brabandere","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[De Geus et al., 2018] Daan De Geus, Panagiotis Meletis, and Gijs Dubbelman. Panoptic segmentation with a joint semantic and instance segmentation network. arXiv preprint arXiv:1809.02110, 2018.","title":"Panoptic segmentation with a joint semantic and instance segmentation network","authors":"Daan De Geus, Panagiotis Meletis, Gijs Dubbelman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fukunaga and Hostetler, 1975] Keinosuke Fukunaga and Larry Hostetler. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory, 21(1):32-40, 1975.","title":"Keinosuke Fukunaga and Larry Hostetler. The estimation of the gradient of a density function, with applications in pattern recognition","authors":"Hostetler Fukunaga","year":"1975","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Funke et al., 2018] Jan Funke, Fabian Tschopp, William Grisaitis, et al. Large scale image segmentation with struc- tured loss based deep learning for connectome reconstruc- tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(7):1669-1680, 2018.","title":"Large scale image segmentation with structured loss based deep learning for connectome reconstruction","authors":"Funke","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2022] Wei Huang, Shiyu Deng, Chang Chen, Xueyang Fu, and Zhiwei Xiong. Learning to model pixel- embedded affinity for homogeneous instance segmenta- tion. In AAAI, 2022.","title":"Learning to model pixelembedded affinity for homogeneous instance segmentation","authors":"Wei Huang, Shiyu Deng, Chang Chen, Xueyang Fu, Zhiwei Xiong","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kasthuri et al., 2015] Narayanan Kasthuri, Kenneth Jeffrey Hayworth, et al. Saturated reconstruction of a volume of neocortex. Cell, 162(3):648-661, 2015.","title":"Saturated reconstruction of a volume of neocortex","authors":"Narayanan Kasthuri, Kenneth Jeffrey Hayworth","year":"2015","venue":"Cell","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2019] Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo. Edge-labeling graph neural net- work for few-shot learning. In CVPR, 2019.","title":"Edge-labeling graph neural network for few-shot learning","authors":"Jongmin Kim, Taesup Kim, Sungwoong Kim, Chang D Yoo","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kirillov et al., 2019] Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár. Panoptic feature pyramid networks. In CVPR, 2019.","title":"Panoptic feature pyramid networks","authors":"Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kulikov and Lempitsky, 2020] Victor Kulikov and Victor Lempitsky. Instance segmentation of biological images us- ing harmonic embeddings. In CVPR, 2020.","title":"Victor Kulikov and Victor Lempitsky. Instance segmentation of biological images using harmonic embeddings","authors":"Lempitsky Kulikov","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kuznichov et al., 2019] Dmitry Kuznichov, Alon Zvirin, Yaron Honen, and Ron Kimmel. Data augmentation for leaf segmentation and counting tasks in rosette plants. In CVPRW, 2019.","title":"Data augmentation for leaf segmentation and counting tasks in rosette plants","authors":"Dmitry Kuznichov, Alon Zvirin, Yaron Honen, Ron Kimmel","year":"2019","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019] Yanwei Li, Xinze Chen, Zheng Zhu, et al. Attention-guided unified network for panoptic segmenta- tion. In CVPR, 2019.","title":"Attention-guided unified network for panoptic segmentation","authors":"Yanwei Li, Xinze Chen, Zheng Zhu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2018] Yiding Liu, Siyu Yang, Bin Li, et al. Affin- ity derivation and graph merge for instance segmentation. In ECCV, 2018.","title":"Affinity derivation and graph merge for instance segmentation","authors":"Yiding Liu, Siyu Yang, Bin Li","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2019a] Dongnan Liu, Donghao Zhang, Yang Song, Chaoyi Zhang, Fan Zhang, Lauren O'Donnell, and Weidong Cai. Nuclei segmentation via a deep panoptic model with semantic feature fusion. In IJCAI, 2019.","title":"Nuclei segmentation via a deep panoptic model with semantic feature fusion","authors":"Liu","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/121"},{"position":17,"raw":"[Liu et al., 2019b] Huanyu Liu, Chao Peng, Changqian Yu, Jingbo Wang, Xu Liu, Gang Yu, and Wei Jiang. An end- to-end network for panoptic segmentation. In CVPR, 2019.","title":"An endto-end network for panoptic segmentation","authors":"Liu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2021] Dongnan Liu, Donghao Zhang, Yang Song, Heng Huang, and Weidong Cai. Panoptic feature fusion net: A novel instance segmentation paradigm for biomedical and biological images. IEEE Transactions on Image Processing, 30:2045-2059, 2021.","title":"Panoptic feature fusion net: A novel instance segmentation paradigm for biomedical and biological images","authors":"Liu","year":"2021","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ljosa et al., 2012] Vebjorn Ljosa, Katherine L Sokolnicki, and Anne E Carpenter. Annotated high-throughput mi- croscopy image sets for validation. Nature Methods, 9(7):637-637, 2012.","title":"Annotated high-throughput microscopy image sets for validation","authors":"Ljosa","year":"2012","venue":"Nature Methods","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Minervini et al., 2016] Massimo Minervini, Andreas Fis- chbach, Hanno Scharr, and Sotirios A Tsaftaris. Finely- grained annotated datasets for image-based plant pheno- typing. Pattern recognition letters, 81:80-89, 2016.","title":"Finelygrained annotated datasets for image-based plant phenotyping","authors":"Massimo Minervini, Andreas Fischbach, Hanno Scharr, Sotirios A Tsaftaris","year":"2016","venue":"Pattern recognition letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pape and Klukas, 2014] Jean-Michel Pape and Christian Klukas. 3-d histogram-based segmentation and leaf de- tection for rosette plants. In ECCV, 2014.","title":"3-d histogram-based segmentation and leaf detection for rosette plants","authors":"Klukas ; Jean-Michel Pape, Christian Pape, Klukas","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ren and Zemel, 2017] Mengye Ren and Richard S Zemel. End-to-end instance segmentation with recurrent attention. In CVPR, 2017.","title":"End-to-end instance segmentation with recurrent attention","authors":"Zemel Ren, Mengye Ren, Richard S Zemel","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Scharr et al., 2014] Hanno Scharr, Massimo Minervini, An- dreas Fischbach, and Sotirios A Tsaftaris. Annotated im- age datasets of rosette plants. In ECCV, 2014.","title":"Annotated image datasets of rosette plants","authors":"Hanno Scharr, Massimo Minervini, Andreas Fischbach, Sotirios A Tsaftaris","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Scharr et al., 2016] Hanno Scharr, Massimo Minervini, An- drew P French, et al. Leaf segmentation in plant phenotyp- ing: a collation study. Machine Vision and Applications, 27(4):585-606, 2016.","title":"Leaf segmentation in plant phenotyping: a collation study","authors":"Hanno Scharr, Massimo Minervini, ; P French","year":"2016","venue":"Machine Vision and Applications","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Takemura et al., 2015] Shin-ya Takemura, C Shan Xu, Zhiyuan Lu, et al. Synaptic circuits and their varia- tions within different columns in the visual system of drosophila. Proceedings of the National Academy of Sci- ences, 112(44):13711-13716, 2015.","title":"Synaptic circuits and their variations within different columns in the visual system of drosophila","authors":"Shin-Ya Takemura, Shan Xu, Zhiyuan Lu","year":"2015","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Turaga et al., 2009] Srinivas C Turaga, Kevin L Briggman, Moritz Helmstaedter, Winfried Denk, and H Sebastian Se- ung. Maximin affinity learning of image segmentation. arXiv preprint arXiv:0911.5372, 2009.","title":"Maximin affinity learning of image segmentation","authors":"Kevin L Srinivas C Turaga, Moritz Briggman, Winfried Helmstaedter, H Denk, Seung Sebastian","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ward et al., 2018] Daniel Ward, Peyman Moghadam, and Nicolas Hudson. Deep leaf segmentation using synthetic data. arXiv preprint arXiv:1807.10931, 2018.","title":"Deep leaf segmentation using synthetic data","authors":"Daniel Ward, Peyman Moghadam, Nicolas Hudson","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xiong et al., 2019] Yuwen Xiong, Renjie Liao, Heng- shuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun. Upsnet: A unified panoptic segmentation net- work. In CVPR, 2019.","title":"Upsnet: A unified panoptic segmentation network","authors":"Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018] Donghao Zhang, Yang Song, Dongnan Liu, Haozhe Jia, Siqi Liu, Yong Xia, Heng Huang, and Weidong Cai. Panoptic segmentation with an end-to-end cell r-cnn for pathology image analysis. In MICCAI, 2018.","title":"Panoptic segmentation with an end-to-end cell r-cnn for pathology image analysis","authors":"Donghao Zhang, Yang Song, Dongnan Liu, Haozhe Jia, Siqi Liu, Yong Xia, Heng Huang, Weidong Cai","year":"2018","venue":"MICCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}