---
title: "A Stochastic Image Grammar for Fine-Grained 3D Scene Reconstruction"
authors:
  - "Xiaobai Liu"
  - "Yadong Mu"
  - "Liang Lin"
doi: ""
url: "https://ijcai.org/proceedings/2016/484"
pdf: "https://ijcai.org/proceedings/2016/0484.pdf"
venue: "Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence"
year: 2016
pages: "3425-3430"
track: ""
keywords:
---

# A Stochastic Image Grammar for Fine-Grained 3D Scene Reconstruction

Xiaobai Liu, Yadong Mu, Liang Lin

*Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence*, pages 3425-3430.

## Abstract

This paper presents a stochastic grammar for fine-grained 3D scene reconstruction from a single image. At the heart of our approach is a small number of grammar rules that can describe the most common geometric structures, e.g., two straights lines being co-linear or orthogonal, or that a line lying on a planar region etc. With these grammar rules, we re-frame single-view 3D reconstruction problem as jointly solving two coupled sub-tasks: i) segmenting of image entities, e.g. planar regions, straight edge segments, and ii) optimizing pixel-wise 3D scene model through the application of grammar rules over image entities. To reconstruct a new image, we design an efficient hybrid Monte Carlo (HMC) algorithm to simulate Markov Chain walking towards a posterior distribution. Our algorithm utilizes two iterative dynamics: i) Hamiltonian Dynamics that makes proposals along the gradient direction to search the continuous pixel-wise 3D scene model; and ii) Cluster Dynamics, that flip the colors of clusters of pixels to form planar region partition. Following the Metropolis-hasting principle, these dynamics not only make distant proposals but also guarantee detail-balance and fast convergence. Results with comparisons on public image dataset show that our method clearly outperforms the alternate state-of-the-art single-view reconstruction methods.