{"type":"ScholarlyArticle","title":"Dense Transformer Networks for Brain Electron Microscopy Image Segmentation","authors":[{"name":"Jun Li","slug":"jun-li","given":"Jun","family":"Li","affiliation":"Washington State University"},{"name":"Yongjun Chen","slug":"yongjun-chen","given":"Yongjun","family":"Chen","affiliation":"Washington State University"},{"name":"Lei Cai","slug":"lei-cai","given":"Lei","family":"Cai","affiliation":"Washington State University"},{"name":"Ian Davidson","slug":"ian-davidson","given":"Ian","family":"Davidson","affiliation":"University of California , Davis"},{"name":"Shuiwang Ji","slug":"shuiwang-ji","given":"Shuiwang","family":"Ji","affiliation":"Texas A&M University"}],"abstract":"The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by network architecture instead of learned from data. In this work, we propose the dense transformer networks, which can learn the shapes and sizes of patches from data. The dense transformer networks employ an encoder-decoder architecture, and a pair of dense transformer modules are inserted into each of the encoder and decoder paths. The novelty of this work is that we provide technical solutions for learning the shapes and sizes of patches from data and efficiently restoring the spatial correspondence required for dense prediction. The proposed dense transformer modules are differentiable, thus the entire network can be trained. We apply the proposed networks on biological image segmentation tasks and show superior performance is achieved in comparison to baseline methods.","doi":"10.24963/ijcai.2019/401","url":"https://ijcai.org/proceedings/2019/401","pdf":"https://ijcai.org/proceedings/2019/0401.pdf","markdown":"https://ijcai.org/proceedings/2019/401.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/401","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":401,"pages":{"first":2894,"last":2900},"keywords":["Machine Learning: Deep Learning","Computer Vision: Biomedical Image Understanding"],"videos":[],"references":[{"position":1,"raw":"[Badrinarayanan et al., 2015] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolu- tional encoder-decoder architecture for image segmenta- tion. arXiv preprint arXiv:1511.00561, 2015.","title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","authors":"Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bookstein, 1989] Fred L. Bookstein. Principal warps: Thin- plate splines and the decomposition of deformations. IEEE Transactions on pattern analysis and machine intelli- gence, 11(6):567-585, 1989.","title":"Principal warps: Thinplate splines and the decomposition of deformations","authors":"Fred L Bookstein","year":"1989","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2015] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Se- mantic image segmentation with deep convolutional nets and fully connected CRFs. In Proceedings of the Interna- tional Conference on Learning Representations, 2015.","title":"Semantic image segmentation with deep convolutional nets and fully connected CRFs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2015","venue":"Proceedings of the International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ciresan et al., 2012] Dan Ciresan, Alessandro Giusti, Luca M Gambardella, and Jürgen Schmidhuber. Deep neural networks segment neuronal membranes in electron microscopy images. In Advances in neural information processing systems, pages 2843-2851, 2012.","title":"Deep neural networks segment neuronal membranes in electron microscopy images","authors":"Dan Ciresan, Alessandro Giusti, M Luca, Jürgen Gambardella, Schmidhuber","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cohen and Welling, 2016] Taco Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of The 33rd International Conference on Machine Learn- ing, pages 2990-2999, 2016.","title":"Group equivariant convolutional networks","authors":"Welling Cohen, Taco Cohen, Max Welling","year":"2016","venue":"Proceedings of The 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. arXiv preprint arXiv:1703.06211, 2017.","title":"Deformable convolutional networks","authors":"Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dieleman et al., 2016] Sander Dieleman, Jeffrey De Fauw, and Koray Kavukcuoglu. Exploiting cyclic symmetry in convolutional neural networks. In Proceedings of The 33rd International Conference on Machine Learning, pages 1889-1898, 2016.","title":"Exploiting cyclic symmetry in convolutional neural networks","authors":"Jeffrey Sander Dieleman, Koray De Fauw, Kavukcuoglu","year":"2016","venue":"Proceedings of The 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dong et al., 2016] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295-307, 2016.","title":"Image super-resolution using deep convolutional networks","authors":"Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Eigen and Fergus, 2015] David Eigen and Rob Fergus. Pre- dicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In Pro- ceedings of the IEEE International Conference on Com- puter Vision, pages 2650-2658, 2015.","title":"Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture","authors":"David Eigen, Rob Fergus","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Farabet et al., 2013] Clement Farabet, Camille Couprie, Laurent Najman, and Yann LeCun. Learning hierarchi- cal features for scene labeling. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 35(8):1915-1929, 2013.","title":"Learning hierarchical features for scene labeling","authors":"Clement Farabet, Camille Couprie, Laurent Najman, Yann Lecun","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Henriques and Vedaldi, 2016] João F Henriques and An- drea Vedaldi. Warped convolutions: Efficient invariance to spatial transformations. arXiv preprint arXiv:1609.04382, 2016.","title":"Warped convolutions: Efficient invariance to spatial transformations","authors":"F João, Andrea Henriques, Vedaldi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with con- ditional adversarial networks. arXiv preprint, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jaderberg et al., 2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In Advances in neural information processing systems, pages 2017-2025, 2015.","title":"Spatial transformer networks","authors":"Jaderberg","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2016] Xu Jia, Bert De Brabandere, Tinne Tuyte- laars, and Luc V Gool. Dynamic filter networks. In Ad- vances in Neural Information Processing Systems, pages 667-675, 2016.","title":"Dynamic filter networks","authors":"Xu Jia, Bert De Brabandere, Tinne Tuytelaars, Luc V Gool","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Laina et al., 2016] Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. Deeper depth prediction with fully convolutional residual net- works. In 3D Vision (3DV), 2016 Fourth International Conference on, pages 239-248. IEEE, 2016.","title":"Deeper depth prediction with fully convolutional residual networks","authors":"Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, Nassir Navab","year":"2016","venue":"Fourth International Conference on","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[LeCun et al., 1998] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2015] Kisuk Lee, Aleksandar Zlateski, Vish- wanathan Ashwin, and H Sebastian Seung. Recursive training of 2D-3D convolutional networks for neuronal boundary prediction. In Advances in Neural Information Processing Systems, pages 3573-3581, 2015.","title":"Recursive training of 2D-3D convolutional networks for neuronal boundary prediction","authors":"Lee","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431- 3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Noh et al., 2015] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for se- mantic segmentation. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 1520-1528, 2015.","title":"Learning deconvolution network for semantic segmentation","authors":"Noh","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical Image Computing and Computer-Assisted Intervention","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sermanet et al., 2014] Pierre Sermanet, David Eigen, Xi- ang Zhang, Michael Mathieu, Rob Fergus, and Yann Le- Cun. OverFeat: Integrated recognition, localization and detection using convolutional networks. In Proceedings of the International Conference on Learning Representa- tions, April 2014.","title":"OverFeat: Integrated recognition, localization and detection using convolutional networks","authors":"Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, Yann Le-Cun","year":"2014","venue":"Proceedings of the International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of the Inter- national Conference on Learning Representations, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"Proceedings of the International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zeiler et al., 2010] Matthew D Zeiler, Dilip Krishnan, Gra- ham W Taylor, and Rob Fergus. Deconvolutional net- works. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 2528-2535. IEEE, 2010.","title":"Deconvolutional networks","authors":"Dilip Matthew D Zeiler, Graham W Krishnan, Rob Taylor, Fergus","year":"2010","venue":"Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}