Context-Aware Image Inpainting with Learned Semantic Priors

Zhang, Wendong; Zhu, Junwei; Tai, Ying; Wang, Yunbo; Chu, Wenqing; Ni, Bingbing; Wang, Chengjie; Yang, Xiaokang

doi:10.24963/ijcai.2021/183

Context-Aware Image Inpainting with Learned Semantic Priors

Wendong Zhang, Junwei Zhu, Ying Tai, Yunbo Wang, Wenqing Chu, Bingbing Ni, Chengjie Wang, Xiaokang Yang

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Main Track. Pages 1323-1329.

https://doi.org/10.24963/ijcai.2021/183

Keywords

Computer Vision: 2D and 3D Computer Vision
Machine Learning: Deep Learning

Abstract

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets

Wendong Zhang, Junwei Zhu, Ying Tai, Yunbo Wang, Wenqing Chu, Bingbing Ni, Chengjie Wang, and Xiaokang Yang. 2021. Context-Aware Image Inpainting with Learned Semantic Priors. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 1323–1329. https://doi.org/10.24963/ijcai.2021/183

@inproceedings{ijcai2021p183,
  title     = {Context-Aware Image Inpainting with Learned Semantic Priors},
  author    = {Zhang, Wendong and Zhu, Junwei and Tai, Ying and Wang, Yunbo and Chu, Wenqing and Ni, Bingbing and Wang, Chengjie and Yang, Xiaokang},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Prof. Zhi-Hua Zhou},
  pages     = {1323--1329},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/183},
  url       = {https://doi.org/10.24963/ijcai.2021/183},
}

TY  - CPAPER
TI  - Context-Aware Image Inpainting with Learned Semantic Priors
AU  - Zhang, Wendong
AU  - Zhu, Junwei
AU  - Tai, Ying
AU  - Wang, Yunbo
AU  - Chu, Wenqing
AU  - Ni, Bingbing
AU  - Wang, Chengjie
AU  - Yang, Xiaokang
T2  - Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2021
SP  - 1323
EP  - 1329
DO  - 10.24963/ijcai.2021/183
UR  - https://doi.org/10.24963/ijcai.2021/183
ER  -