---
title: "A Sketch-Transformer Network for Face Photo-Sketch Synthesis"
authors:
  - "Mingrui Zhu"
  - "Changcheng Liang"
  - "Nannan Wang"
  - "Xiaoyu Wang"
  - "Zhifeng Li"
  - "Xinbo Gao"
doi: "10.24963/ijcai.2021/187"
url: "https://ijcai.org/proceedings/2021/187"
pdf: "https://ijcai.org/proceedings/2021/0187.pdf"
venue: "Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"
year: 2021
pages: "1352-1358"
track: "Main Track"
keywords:
  - "Computer Vision: Biometrics, Face and Gesture Recognition"
  - "Computer Vision: 2D and 3D Computer Vision"
---

# A Sketch-Transformer Network for Face Photo-Sketch Synthesis

Mingrui Zhu, Changcheng Liang, Nannan Wang, Xiaoyu Wang, Zhifeng Li, Xinbo Gao

*Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence*, pages 1352-1358. DOI: [10.24963/ijcai.2021/187](https://doi.org/10.24963/ijcai.2021/187)

## Abstract

We present a face photo-sketch synthesis model, which converts a face photo into an artistic face sketch or recover a photo-realistic facial image from a sketch portrait. Recent progress has been made by convolutional neural networks (CNNs) and generative adversarial networks (GANs), so that promising results can be obtained through real-time end-to-end architectures. However, convolutional architectures tend to focus on local information and neglect long-range spatial dependency, which limits the ability of existing approaches in keeping global structural information. In this paper, we propose a Sketch-Transformer network for face photo-sketch synthesis, which consists of three closely-related modules, including a multi-scale feature and position encoder for patch-level feature and position embedding, a self-attention module for capturing long-range spatial dependency, and a multi-scale spatially-adaptive de-normalization decoder for image reconstruction. Such a design enables the model to generate reasonable detail texture while maintaining global structural information. Extensive experiments show that the proposed method achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations.