Instance-Aware Coherent Video Style Transfer for Chinese Ink Wash Painting

Instance-Aware Coherent Video Style Transfer for Chinese Ink Wash Painting

Hao Liang, Shuai Yang, Wenjing Wang, Jiaying Liu

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 823-829. https://doi.org/10.24963/ijcai.2021/114

Recent researches have made remarkable achievements in fast video style transfer based on western paintings. However, due to the inherent different drawing techniques and aesthetic expressions of Chinese ink wash painting, existing methods either achieve poor temporal consistency or fail to transfer the key freehand brushstroke characteristics of Chinese ink wash painting. In this paper, we present a novel video style transfer framework for Chinese ink wash paintings. The two key ideas are a multi-frame fusion for temporal coherence and an instance-aware style transfer. The frame reordering and stylization based on reference frame fusion are proposed to improve temporal consistency. Meanwhile, the proposed method is able to adaptively leave the white spaces in the background and to select proper scales to extract features and depict the foreground subject by leveraging instance segmentation. Experimental results demonstrate the superiority of the proposed method over state-of-the-art style transfer methods in terms of both temporal coherence and visual quality. Our project website is available at https://oblivioussy.github.io/InkVideo/.
Keywords:
Computer Vision: 2D and 3D Computer Vision
Machine Learning: Adversarial Machine Learning
Multidisciplinary Topics and Applications: Art and Music