{"type":"ScholarlyArticle","title":"A Sketch-Transformer Network for Face Photo-Sketch Synthesis","authors":[{"name":"Mingrui Zhu","slug":"mingrui-zhu","given":"Mingrui","family":"Zhu","affiliation":""},{"name":"Changcheng Liang","slug":"changcheng-liang","given":"Changcheng","family":"Liang","affiliation":""},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":""},{"name":"Xiaoyu Wang","slug":"xiaoyu-wang","given":"Xiaoyu","family":"Wang","affiliation":""},{"name":"Zhifeng Li","slug":"zhifeng-li","given":"Zhifeng","family":"Li","affiliation":""},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":""}],"abstract":"We present a face photo-sketch synthesis model, which converts a face photo into an artistic face sketch or recover a photo-realistic facial image from a sketch portrait. Recent progress has been made by convolutional neural networks (CNNs) and generative adversarial networks (GANs), so that promising results can be obtained through real-time end-to-end architectures. However, convolutional architectures tend to focus on local information and neglect long-range spatial dependency, which limits the ability of existing approaches in keeping global structural information. In this paper, we propose a Sketch-Transformer network for face photo-sketch synthesis, which consists of three closely-related modules, including a multi-scale feature and position encoder for patch-level feature and position embedding, a self-attention module for capturing long-range spatial dependency, and a multi-scale spatially-adaptive de-normalization decoder for image reconstruction. Such a design enables the model to generate reasonable detail texture while maintaining global structural information. Extensive experiments show that the proposed method achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations.","doi":"10.24963/ijcai.2021/187","url":"https://ijcai.org/proceedings/2021/187","pdf":"https://ijcai.org/proceedings/2021/0187.pdf","markdown":"https://ijcai.org/proceedings/2021/187.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/187","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Computer Vision II","track":"Main Track","number":187,"pages":{"first":1352,"last":1358},"keywords":["Computer Vision: Biometrics, Face and Gesture Recognition","Computer Vision: 2D and 3D Computer Vision"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}