{"type":"ScholarlyArticle","title":"Two-Stage Generative Models of Simulating Training Data at The Voxel Level for Large-Scale Microscopy Bioimage Segmentation","authors":[{"name":"Deli Wang","slug":"deli-wang","given":"Deli","family":"Wang","affiliation":"Qiushi Academy for Advanced Studies , Zhejiang University; School of Computer Science , Zhejiang University"},{"name":"Ting Zhao","slug":"ting-zhao","given":"Ting","family":"Zhao","affiliation":"Janelia Research Campus , Howard Hughes Medical Institute"},{"name":"Nenggan Zheng","slug":"nenggan-zheng","given":"Nenggan","family":"Zheng","affiliation":"Qiushi Academy for Advanced Studies , Zhejiang University; School of Computer Science , Zhejiang University"},{"name":"Zhefeng Gong","slug":"zhefeng-gong","given":"Zhefeng","family":"Gong","affiliation":"School of Basic Medical Sciences , Zhejiang University"}],"abstract":"Bioimage Informatics is a growing area that aims to extract biological knowledge from microscope images of biomedical samples automatically. Its mission is vastly challenging, however, due to the complexity of diverse imaging modalities and big scales of multi-dimensional images. One major challenge is automatic image segmentation, an essential step towards high-level modeling and analysis. While progresses in deep learning have brought the goal of automation much closer to reality, creating training data for producing powerful neural networks is often laborious. To provide a shortcut for this costly step, we propose a novel two-stage generative model for simulating voxel level training data based on a specially designed objective function of preserving foreground labels. Using segmenting neurons from LM (Light Microscopy) image stacks as a testing example, we showed that segmentation networks trained by our synthetic data were able to produce satisfactory results. Unlike other simulation methods available in the field, our method can be easily extended to many other applications because it does not involve sophisticated cell models and imaging mechanisms.","doi":"10.24963/ijcai.2019/664","url":"https://ijcai.org/proceedings/2019/664","pdf":"https://ijcai.org/proceedings/2019/0664.pdf","markdown":"https://ijcai.org/proceedings/2019/664.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/664","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":664,"pages":{"first":4781,"last":4787},"keywords":["Multidisciplinary Topics and Applications: AI for life science","Machine Learning Applications: Bio;Medicine","Multidisciplinary Topics and Applications: Biology and Medicine"],"videos":[],"references":[{"position":1,"raw":"[Feng et al., 2015] Linqing Feng, Ting Zhao, and Jinhyun Kim. neutube 1.0: A new design for efficient neuron re- construction software based on the swc format. Eneuro, 2(1), 2015.","title":"neutube 1.0: A new design for efficient neuron reconstruction software based on the swc format","authors":"Linqing Feng, Ting Zhao, Jinhyun Kim","year":"2015","venue":"Eneuro","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Xu Bing, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In International Conference on Neural Information Processing Systems, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Xu Bing, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gu and Cheng, 2015] Lin Gu and Li Cheng. Learning to boost filamentary structure segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pages 639-647, 2015.","title":"Learning to boost filamentary structure segmentation","authors":"Lin Gu, Li Cheng","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Sun Jian. Deep residual learning for image recog- nition. In IEEE Conference on Computer Vision & Pattern Recognition, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Sun Jian","year":"2016","venue":"IEEE Conference on Computer Vision & Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Johnson et al., 2017] Gregory R Johnson, Rory M Donovan-Maiye, and Mary M Maleckar. Generative modeling with conditional autoencoders: Building an integrated cell. arXiv preprint arXiv:1705.00092, 2017.","title":"Generative modeling with conditional autoencoders: Building an integrated cell","authors":"Rory M Gregory R Johnson, Mary M Donovan-Maiye, Maleckar","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2017] Rongjian Li, Tao Zeng, Hanchuan Peng, and Shuiwang Ji. Deep learning segmentation of opti- cal microscopy images improves 3-d neuron reconstruc- tion. IEEE Transactions on Medical Imaging, 36(7):1533- 1541, 2017.","title":"Deep learning segmentation of optical microscopy images improves 3-d neuron reconstruction","authors":"Li","year":"2017","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2017] Siqi Liu, Donghao Zhang, Yang Song, Hanchuan Peng, and Weidong Cai. Triple-crossing 2.5 d convolutional neural network for detecting neuronal ar- bours in 3d microscopic images. In International Work- shop on Machine Learning in Medical Imaging, pages 185-193. Springer, 2017.","title":"Triple-crossing 2.5 d convolutional neural network for detecting neuronal arbours in 3d microscopic images","authors":"Siqi Liu, Donghao Zhang, Yang Song, Hanchuan Peng, Weidong Cai","year":"2017","venue":"International Workshop on Machine Learning in Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2018] Min Liu, Huiqiong Luo, Yinghui Tan, Xueping Wang, and Weixun Chen. Improved v-net based image segmentation for 3d neuron reconstruction. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 443-448. IEEE, 2018.","title":"Improved v-net based image segmentation for 3d neuron reconstruction","authors":"Liu","year":"2018","venue":"IEEE International Conference on Bioinformatics and Biomedicine (BIBM)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Long et al., 2014] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 39(4):640-651, 2014.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2014","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Meijering et al., 2016] E Meijering, A. E. Carpenter, H. Peng, F. A. Hamprecht, and J. C. Olivo-Marin. Imagin- ing the future of bioimage analysis. Nature Biotechnology, 34(12):1250, 2016.","title":"Imagining the future of bioimage analysis","authors":"A E Meijering, H Carpenter, F A Peng, J C Hamprecht, Olivo-Marin","year":"2016","venue":"Nature Biotechnology","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Meijering, 2012] Erik Meijering. Cell segmentation: 50 years down the road [life sciences]. IEEE Signal Process- ing Magazine, 29(5):140-145, 2012.","title":"Cell segmentation: 50 years down the road [life sciences","authors":"Erik Meijering, Meijering","year":"2012","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Peng and Murphy, 2011] T. Peng and R. F. Murphy. Image- derived, three-dimensional generative models of cellular organization. Cytometry Part A the Journal of the Interna- tional Society for Analytical Cytology, 79A(5):383-391, 2011.","title":"Imagederived, three-dimensional generative models of cellular organization","authors":"T Peng, R F Murphy","year":"2011","venue":"Cytometry Part A the Journal of the International Society for Analytical Cytology","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Peng et al., 2010] H. Peng, Z Ruan, F. Long, J. H. Simpson, and E. W. Myers. V3d enables real-time 3d visualization and quantitative analysis of large-scale biological image data sets. Nature Biotechnology, 28(4):348-353, 2010.","title":"V3d enables real-time 3d visualization and quantitative analysis of large-scale biological image data sets","authors":"H Peng, F Ruan, J H Long, E W Simpson, Myers","year":"2010","venue":"Nature Biotechnology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Peng et al., 2015] Hanchuan Peng, Erik Meijering, and Giorgio A Ascoli. From diadem to bigneuron, 2015.","title":"From diadem to bigneuron","authors":"Peng","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Peng, 2008] Hanchuan Peng. Bioimage informatics: a new area of engineering biology. Bioinformatics, 24(17):1827- 36, 2008.","title":"Bioimage informatics: a new area of engineering biology","authors":"Hanchuan Peng, Peng","year":"2008","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing & Computer- assisted Intervention, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical Image Computing & Computerassisted Intervention","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Takemura et al., 2015] Shin-ya Takemura, C Shan Xu, Zhiyuan Lu, Patricia K Rivlin, Toufiq Parag, Donald J Olbris, Stephen Plaza, Ting Zhao, William T Katz, Low- ell Umayam, et al. Synaptic circuits and their varia- tions within different columns in the visual system of drosophila. Proceedings of the National Academy of Sci- ences, 112(44):13711-13716, 2015.","title":"Synaptic circuits and their variations within different columns in the visual system of drosophila","authors":"Shin-Ya Takemura, Shan Xu, Zhiyuan Lu, Patricia K Rivlin, Toufiq Parag, Donald J Olbris, Stephen Plaza, Ting Zhao, William T Katz, Lowell Umayam","year":"2015","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Takemura et al., 2017] Shin-ya Takemura, Yoshinori Aso, Toshihide Hige, Allan Wong, Zhiyuan Lu, C Shan Xu, Patricia K Rivlin, Harald Hess, Ting Zhao, Toufiq Parag, et al. A connectome of a learning and memory center in the adult drosophila brain. Elife, 6:e26975, 2017.","title":"A connectome of a learning and memory center in the adult drosophila brain","authors":"Shin-Ya Takemura, Yoshinori Aso, Toshihide Hige, Allan Wong, Zhiyuan Lu, Shan Xu, Patricia K Rivlin, Harald Hess, Ting Zhao, Toufiq Parag","year":"2017","venue":"Elife","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Weigert et al., 2018] Martin Weigert, Kaushikaram Subra- manian, Sebastian T Bundschuh, Eugene W Myers, and Moritz Kreysing. Biobeam-multiplexed wave-optical simulations of light-sheet microscopy. PLoS Computa- tional Biology, 14(4):e1006079, 2018.","title":"Biobeam-multiplexed wave-optical simulations of light-sheet microscopy","authors":"Martin Weigert, Kaushikaram Subramanian, Sebastian T Bundschuh, Eugene W Myers, Moritz Kreysing","year":"2018","venue":"PLoS Computational Biology","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wiesmann et al., 2017] Veit Wiesmann, Matthias Bergler, Ralf Palmisano, Martin Prinzen, Daniela Franz, and Thomas Wittenberg. Using simulated fluorescence cell micrographs for the evaluation of cell image segmentation algorithms. BMC Bioinformatics, 18(1):176, 2017.","title":"Using simulated fluorescence cell micrographs for the evaluation of cell image segmentation algorithms","authors":"Matthias Veit Wiesmann, Ralf Bergler, Martin Palmisano, Daniela Prinzen, Thomas Franz, Wittenberg","year":"2017","venue":"BMC Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yuan et al., 2017] Zehuan Yuan, Lu Tong, Yirui Wu, Ze- huan Yuan, Lu Tong, Yirui Wu, Zehuan Yuan, Lu Tong, and Yirui Wu. Deep-dense conditional random fields for object co-segmentation. In Twenty-sixth International Joint Conference on Artificial Intelligence, 2017.","title":"Deep-dense conditional random fields for object co-segmentation","authors":"Zehuan Yuan, Lu Tong, Yirui Wu, Zehuan Yuan, Lu Tong, Yirui Wu, Zehuan Yuan, Lu Tong, Yirui Wu","year":"2017","venue":"Twenty-sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/471"},{"position":22,"raw":"[Yuan et al., 2018] Hao Yuan, Lei Cai, Zhengyang Wang, Xia Hu, Shaoting Zhang, and Shuiwang Ji. Computational modeling of cellular structures using conditional deep gen- erative networks. Bioinformatics, 2018.","title":"Computational modeling of cellular structures using conditional deep generative networks","authors":"Lei Hao Yuan, Zhengyang Cai, Xia Wang, Shaoting Hu, Shuiwang Zhang, Ji","year":"2018","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhao and Murphy, 2007] Ting Zhao and Robert F Murphy. Automated learning of generative models for subcellular location: building blocks for systems biology. Cytometry Part A, 71(12):978-990, 2007.","title":"Automated learning of generative models for subcellular location: building blocks for systems biology","authors":"Ting Zhao, Robert F Murphy","year":"2007","venue":"Cytometry Part A","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zheng and Hong, 2016] Zhihao Zheng and Pengyu Hong. Incorporate deep-transfer-learning into automatic 3d neu- ron tracing. In The First International Conference on Neu- roscience and Cognitive Brain Information, 2016.","title":"Incorporate deep-transfer-learning into automatic 3d neuron tracing","authors":"Zhihao Zheng, Pengyu Hong","year":"2016","venue":"The First International Conference on Neuroscience and Cognitive Brain Information","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}