{"type":"ScholarlyArticle","title":"I2CNet: An Intra- and Inter-Class Context Information Fusion Network for Blastocyst Segmentation","authors":[{"name":"Hua Wang","slug":"hua-wang","given":"Hua","family":"Wang","affiliation":"School of Biological Science and Medical Engineering , Beihang University , Beijing 100191 , China; Hefei Innovation Research Institute , Beihang University , Hefei 230012 , China"},{"name":"Linwei Qiu","slug":"linwei-qiu","given":"Linwei","family":"Qiu","affiliation":"School of Biological Science and Medical Engineering , Beihang University , Beijing 100191 , China; Hefei Innovation Research Institute , Beihang University , Hefei 230012 , China"},{"name":"Jingfei Hu","slug":"jingfei-hu","given":"Jingfei","family":"Hu","affiliation":"School of Biological Science and Medical Engineering , Beihang University , Beijing 100191 , China; Hefei Innovation Research Institute , Beihang University , Hefei 230012 , China"},{"name":"Jicong Zhang","slug":"jicong-zhang","given":"Jicong","family":"Zhang","affiliation":"School of Biological Science and Medical Engineering , Beihang University , Beijing 100191 , China; Hefei Innovation Research Institute , Beihang University , Hefei 230012 , China"}],"abstract":"The quality of a blastocyst directly determines the embryo's implantation potential, thus making it essential to objectively and accurately identify the blastocyst morphology. In this work, we propose an automatic framework named I2CNet to perform the blastocyst segmentation task in human embryo images. The I2CNet contains two components: IntrA-Class Context Module (IACCM) and InteR-Class Context Module (IRCCM). The IACCM aggregates the representations of specific areas sharing the same category for each pixel, where the categorized regions are learned under the supervision of the groundtruth. This aggregation decomposes a K-category recognition task into K recognition tasks of two labels while maintaining the ability of garnering intra-class features. In addition, the IRCCM is designed based on the blastocyst morphology to compensate for inter-class information which is gradually gathered from inside out. Meanwhile, a weighted mapping function is applied to facilitate edges of the inter classes and stimulate some hard samples. Eventually, the learned intra- and inter-class cues are integrated from coarse to fine, rendering sufficient information interaction and fusion between multi-scale features. Quantitative and qualitative experiments demonstrate that the superiority of our model compared with other representative methods. The I2CNet achieves accuracy of 94.14% and Jaccard of 85.25% on blastocyst public dataset.","doi":"10.24963/ijcai.2022/197","url":"https://ijcai.org/proceedings/2022/197","pdf":"https://ijcai.org/proceedings/2022/0197.pdf","markdown":"https://ijcai.org/proceedings/2022/197.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/197","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":197,"pages":{"first":1415,"last":1422},"keywords":["Computer Vision: Segmentation","Computer Vision: Biomedical Image Analysis","Machine Learning: Convolutional Networks"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38984883","length_sec":344,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38986740","length_sec":346,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Cao et al., 2021] Hu Cao, Yueyue Wang, Joy Chen, Dong- sheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. Swin-unet: Unet-like pure transformer for medi- cal image segmentation. arXiv preprint arXiv:2105.05537, 2021.","title":"Swin-unet: Unet-like pure transformer for medical image segmentation","authors":"Yueyue Hu Cao, Joy Wang, Dongsheng Chen, Xiaopeng Jiang, Qi Zhang, Manning Tian, Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Liang Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the Euro- pean conference on computer vision (ECCV), pages 801- 818, 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Yukun Liang Chieh Chen, George Zhu, Florian Papandreou, Hartwig Schroff, Adam","year":"2018","venue":"Proceedings of the European conference on computer vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.","title":"Transunet: Transformers make strong encoders for medical image segmentation","authors":"Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Csurka et al., 2004] Gabriela Csurka, Diane Larlus, Florent Perronnin, and France Meylan. What is a good evaluation measure for semantic segmentation? IEEE PAMI, 26(1), 2004.","title":"What is a good evaluation measure for semantic segmentation","authors":"Gabriela Csurka, Diane Larlus, Florent Perronnin, France Meylan","year":"2004","venue":"IEEE PAMI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2020] Deng Ping Fan, Ge Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao. Pranet: Parallel reverse attention network for polyp seg- mentation. In International conference on medical im- age computing and computer-assisted intervention, pages 263-273. Springer, 2020.","title":"Pranet: Parallel reverse attention network for polyp segmentation","authors":"Ping Deng, Ge Fan, Ji Peng, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, Ling Shao","year":"2020","venue":"International conference on medical image computing and computer-assisted intervention","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Filho et al., 2010] Efraim Santos Filho, Julia Alison Noble, and Darren Wells. A review on automatic analysis of hu- man embryo microscope images. The open biomedical en- gineering journal, 4:170, 2010.","title":"A review on automatic analysis of human embryo microscope images","authors":"Filho","year":"2010","venue":"The open biomedical engineering journal","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Filho et al., 2012] Efraim Santos Filho, Julia Alison Noble, Maurizio Poli, Tracey Griffiths, Gerri Emerson, and Dar- ren Wells. A method for semi-automatic grading of hu- man blastocyst microscope images. Human Reproduction, 27(9):2641-2648, 2012.","title":"A method for semi-automatic grading of human blastocyst microscope images","authors":"Efraim Santos Filho, Julia, Alison Noble, Maurizio Poli, Tracey Griffiths, Gerri Emerson, Darren Wells","year":"2012","venue":"Human Reproduction","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2020] Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xi- anhua Han, Yen Wei Chen, and Jian Wu. Unet 3+: A full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1055-1059. IEEE, 2020.","title":"Unet 3+: A full-scale connected unet for medical image segmentation","authors":"Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen Wei Chen, Jian Wu","year":"2020","venue":"ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Isensee et al., 2018] Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, et al. nnu-net: Self-adapting framework for u- net-based medical image segmentation. arXiv preprint arXiv:1809.10486, 2018.","title":"nnu-net: Self-adapting framework for unet-based medical image segmentation","authors":"Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khan et al., 2016] Aisha Khan, Stephen Gould, and Math- ieu Salzmann. Segmentation of developing human embryo in time-lapse microscopy. In 2016 IEEE 13th international symposium on biomedical imaging (ISBI), pages 930-934. IEEE, 2016.","title":"Segmentation of developing human embryo in time-lapse microscopy","authors":"Aisha Khan, Stephen Gould, Mathieu Salzmann","year":"2016","venue":"IEEE 13th international symposium on biomedical imaging (ISBI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kheradmand et al., 2016] Shakiba Kheradmand, Parvaneh Saeedi, and Ivan Bajic. Human blastocyst segmentation using neural network. In 2016 IEEE Canadian conference on electrical and computer engineering (CCECE), pages 1-4. IEEE, 2016.","title":"Human blastocyst segmentation using neural network","authors":"Kheradmand","year":"2016","venue":"IEEE Canadian conference on electrical and computer engineering (CCECE)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kheradmand et al., 2017] Shakiba Kheradmand, Amarjot Singh, Parvaneh Saeedi, Jason Au, and Jon Havelock. In- ner cell mass segmentation in human hmc embryo images using fully convolutional network. In 2017 IEEE Inter- national Conference on Image Processing (ICIP), pages 1752-1756. IEEE, 2017.","title":"Inner cell mass segmentation in human hmc embryo images using fully convolutional network","authors":"Kheradmand","year":"2017","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Leahy et al., 2020] Brian D Leahy, Won Dong Jang, He- len Y Yang, Robbert Struyven, Donglai Wei, Zhe Sun, Kylie R Lee, Charlotte Royston, Liz Cam, Yael Kalma, et al. Automated measurements of key morphological features of human embryos for ivf. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 25-35. Springer, 2020.","title":"Automated measurements of key morphological features of human embryos for ivf","authors":"Brian D Leahy, Won, Dong Jang, Helen Y Yang, Robbert Struyven, Donglai Wei, Zhe Sun, Kylie R Lee, Charlotte Royston, Liz Cam, Yael Kalma","year":"2020","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Oktay et al., 2018] Ozan Oktay, Jo Schlemper, Loic Le Fol- goc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.","title":"Attention u-net: Learning where to look for the pancreas","authors":"Ozan Oktay, Jo Schlemper, Le Loic, Matthew Folgoc, Mattias Lee, Kazunari Heinrich, Kensaku Misawa, Steven Mori, Nils Y Mcdonagh, Bernhard Hammerla, Kainz","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rad et al., 2017] Reza Moradi Rad, Parvaneh Saeedi, Jason Au, and Jon Havelock. Coarse-to-fine texture analysis for inner cell mass identification in human blastocyst mi- croscopic images. In 2017 Seventh International Confer- ence on Image Processing Theory, Tools and Applications (IPTA), pages 1-5. IEEE, 2017.","title":"Coarse-to-fine texture analysis for inner cell mass identification in human blastocyst microscopic images","authors":"Reza Moradi, Rad, Parvaneh Saeedi, Jason Au, Jon Havelock","year":"2017","venue":"Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rad et al., 2018] Reza Moradi Rad, Parvaneh Saeedi, Ja- son Au, and Jon Havelock. Multi-resolutional ensemble of stacked dilated u-net for inner cell mass segmentation in human embryonic images. In 2018 25th IEEE inter- national conference on image processing (ICIP), pages 3518-3522. IEEE, 2018.","title":"Multi-resolutional ensemble of stacked dilated u-net for inner cell mass segmentation in human embryonic images","authors":"Reza Moradi, Rad, Parvaneh Saeedi, Jason Au, Jon Havelock","year":"2018","venue":"25th IEEE international conference on image processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rad et al., 2019] Reza Moradi Rad, Parvaneh Saeedi, Jason Au, and Jon Havelock. Blast-net: Semantic segmentation of human blastocyst components via cascaded atrous pyra- mid and dense progressive upsampling. In 2019 IEEE In- ternational Conference on Image Processing (ICIP), pages 1865-1869. IEEE, 2019.","title":"Blast-net: Semantic segmentation of human blastocyst components via cascaded atrous pyramid and dense progressive upsampling","authors":"Reza Moradi, Rad, Parvaneh Saeedi, Jason Au, Jon Havelock","year":"2019","venue":"IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Saeedi et al., 2017] Parvaneh Saeedi, Dianna Yee, Jason Au, and Jon Havelock. Automatic identification of human blastocyst components via texture. IEEE Transactions on Biomedical Engineering, 64(12):2968-2978, 2017.","title":"Automatic identification of human blastocyst components via texture","authors":"Parvaneh Saeedi, Dianna Yee, Jason Au, Jon Havelock","year":"2017","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Singh et al., 2014] Amarjot Singh, Jason Au, Parvaneh Saeedi, and Jon Havelock. Automatic segmentation of trophectoderm in microscopic images of human blasto- cysts. IEEE Transactions on Biomedical Engineering, 62(1):382-393, 2014.","title":"Automatic segmentation of trophectoderm in microscopic images of human blastocysts","authors":"Singh","year":"2014","venue":"IEEE Transactions on Biomedical Engineering","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Taha and Hanbury, 2015] Abdel Aziz Taha and Allan Han- bury. Metrics for evaluating 3d medical image segmenta- tion: analysis, selection, and tool. BMC medical imaging, 15(1):1-28, 2015.","title":"Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool","authors":"Hanbury ; Taha, Aziz Abdel, Allan Taha, Hanbury","year":"2015","venue":"BMC medical imaging","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tamrakar and Bastakoti, 2019] Suman Raj Tamrakar and Rashmi Bastakoti. Determinants of infertility in couples. Journal of Nepal Health Research Council, 17(1):85-89, 2019.","title":"Determinants of infertility in couples","authors":"Raj Suman, Rashmi Tamrakar, Bastakoti","year":"2019","venue":"Journal of Nepal Health Research Council","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thoma, 2016] Martin Thoma. A survey of semantic seg- mentation. arXiv preprint arXiv:1602.06541, 2016.","title":"A survey of semantic segmentation","authors":"Martin Thoma","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Valanarasu et al., 2021] Jeya Maria Jose Valanarasu, Poo- jan Oza, Ilker Hacihaliloglu, and Vishal M Patel. Med- ical transformer: Gated axial-attention for medical im- age segmentation. In International Conference on Med- ical Image Computing and Computer-Assisted Interven- tion, pages 36-46. Springer, 2021.","title":"Medical transformer: Gated axial-attention for medical image segmentation","authors":"Jeya Maria, Jose Valanarasu, Poojan Oza, Ilker Hacihaliloglu, M Vishal, Patel","year":"2021","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu and Koltun, 2015] Fisher Yu and Vladlen Koltun. Multi- scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015.","title":"Multiscale context aggregation by dilated convolutions","authors":"Koltun ; Fisher Yu, Vladlen Yu, Koltun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zegers Hochschild et al., 2009] Fernando Zegers Hochschild, Geoffrey David Adamson, Jacques de Mouzon, Osamu Ishihara, Ragaa Mansour, Karl Nygren, Elisabeth Sullivan, and Sher van der Poel. The international committee for monitoring assisted reproduc- tive technology (icmart) and the world health organization (who) revised glossary on art terminology, 2009. Human reproduction, 24(11):2683-2687, 2009.","title":"The international committee for monitoring assisted reproductive technology (icmart) and the world health organization (who) revised glossary on art terminology","authors":"Fernando Zegers Hochschild, Geoffrey David Adamson, Jacques De Mouzon, Osamu Ishihara, Ragaa Mansour, Karl Nygren, Elisabeth Sullivan, Sher Van Der Poel","year":"2009","venue":"Human reproduction","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zheng et al., 2021] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmentation from a sequence-to- sequence perspective with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pages 6881-6890, 2021.","title":"Rethinking semantic segmentation from a sequence-tosequence perspective with transformers","authors":"Zheng","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2018] Zongwei Zhou, Md Mahfuzur Rah- man Siddiquee, Nima Tajbakhsh, and Jianming Liang. Unet++: A nested u-net architecture for medical image segmentation. In Deep learning in medical image analy- sis and multimodal learning for clinical decision support, pages 3-11. Springer, 2018.","title":"Unet++: A nested u-net architecture for medical image segmentation","authors":"Zhou","year":"2018","venue":"Deep learning in medical image analysis and multimodal learning for clinical decision support","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu et al., 2016] Hongyuan Zhu, Fanman Meng, Jianfei Cai, and Shijian Lu. Beyond pixels: A comprehensive sur- vey from bottom-up to semantic image segmentation and cosegmentation. Journal of Visual Communication and Image Representation, 34:12-27, 2016.","title":"Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation","authors":"Hongyuan Zhu, Fanman Meng, Jianfei Cai, Shijian Lu","year":"2016","venue":"Journal of Visual Communication and Image Representation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}