DeepFacade: A Deep Learning Approach to Facade Parsing

DeepFacade: A Deep Learning Approach to Facade Parsing

Hantang Liu, Jialiang Zhang, Jianke Zhu, Steven C. H. Hoi

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 2301-2307. https://doi.org/10.24963/ijcai.2017/320

The parsing of building facades is a key component to the problem of 3D street scenes reconstruction, which is long desired in computer vision. In this paper, we propose a deep learning based method for segmenting a facade into semantic categories. Man-made structures often present the characteristic of symmetry. Based on this observation, we propose a symmetric regularizer for training the neural network. Our proposed method can make use of both the power of deep neural networks and the structure of man-made architectures. We also propose a method to refine the segmentation results using bounding boxes generated by the Region Proposal Network. We test our method by training a FCN-8s network with the novel loss function. Experimental results show that our method has outperformed previous state-of-the-art methods significantly on both the ECP dataset and the eTRIMS dataset. As far as we know, we are the first to employ end-to-end deep convolutional neural network on full image scale in the task of building facades parsing.
Keywords:
Machine Learning: Deep Learning
Robotics and Vision: Vision and Perception