DeepFacade: A Deep Learning Approach to Facade Parsing

Hantang Liu, Jialiang Zhang, Jianke Zhu, Steven C. H. Hoi

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

Main track. Pages 2301-2307.

Abstract

The parsing of building facades is a key component to the problem of 3D street

scenes reconstruction, which is long desired in computer vision. In this

paper, we propose a deep learning based method for segmenting a facade into

semantic categories. Man-made structures often present the characteristic of

symmetry. Based on this observation, we propose a symmetric regularizer for

training the neural network. Our proposed method can make use of both the

power of deep neural networks and the structure of man-made architectures. We

also propose a method to refine the segmentation results using bounding boxes

generated by the Region Proposal Network. We test our method by training a

FCN-8s network with the novel loss function. Experimental results show that

our method has outperformed previous state-of-the-art methods significantly on

both the ECP dataset and the eTRIMS dataset. As far as we know, we are the

first to employ end-to-end deep convolutional neural network on full image

scale in the task of building facades parsing.