{"type":"ScholarlyArticle","title":"Robust Face Sketch Synthesis via Generative Adversarial Fusion of Priors and Parametric Sigmoid","authors":[{"name":"Shengchuan Zhang","slug":"shengchuan-zhang","given":"Shengchuan","family":"Zhang","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , Xiamen University; School of Information Science and Engineering , Xiamen University"},{"name":"Rongrong Ji","slug":"rongrong-ji","given":"Rongrong","family":"Ji","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , Xiamen University; School of Information Science and Engineering , Xiamen University"},{"name":"Jie Hu","slug":"jie-hu","given":"Jie","family":"Hu","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , Xiamen University; School of Information Science and Engineering , Xiamen University"},{"name":"Yue Gao","slug":"yue-gao","given":"Yue","family":"Gao","affiliation":"School of Software , Tsinghua University"},{"name":"Chia-Wen Lin","slug":"chia-wen-lin","given":"Chia-Wen","family":"Lin","affiliation":"Department of Electrical Engineering , National Tsing Hua University"}],"abstract":"Despite the extensive progress in face sketch synthesis, existing methods are mostly workable under constrained conditions, such as fixed illumination, pose, background and ethnic origin that are hardly to control in real-world scenarios. The key issue lies in the difficulty to use data under fixed conditions to train a model against imaging variations. In this paper, we propose a novel generative adversarial network termed pGAN, which can generate face sketches efficiently using training data under fixed conditions and handle the aforementioned uncontrolled conditions. In pGAN, we embed key photo priors into the process of synthesis and design a parametric sigmoid activation function for compensating illumination variations. Compared to the existing methods, we quantitatively demonstrate that the proposed method can work well on face photos in the wild.","doi":"10.24963/ijcai.2018/162","url":"https://ijcai.org/proceedings/2018/162","pdf":"https://ijcai.org/proceedings/2018/0162.pdf","markdown":"https://ijcai.org/proceedings/2018/162.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/162","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":162,"pages":{"first":1163,"last":1169},"keywords":["Computer Vision: Computer Vision","Computer Vision: Biometrics, Face and Gesture Recognition"],"videos":[],"references":[{"position":1,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448-456, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Isola et al., 2016] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004, 2016.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Liu et al., 2005] Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma. A nonlinear approach for face sketch synthesis and recognition. In Computer Vision and Pattern Recognition, pages 1005-1010, 2005.","title":"A nonlinear approach for face sketch synthesis and recognition","authors":"Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, Songde Ma","year":"2005","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Liu et al., 2015] Sifei Liu, Jimei Yang, Chang Huang, and Ming-Hsuan Yang. Multi-objective convolutional learning for face labeling. In Computer Vision and Pattern Recog- nition, pages 3451-3459, 2015.","title":"Multi-objective convolutional learning for face labeling","authors":"Liu","year":"2015","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Martinez, 1998] Aleix M. Martinez. The ar face database. CVC technical report, 1998.","title":"The ar face database","authors":"M Aleix, Martinez","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Messer et al., 1999] Kieron Messer, Jiri Matas, Josef Kit- tler, Juergen Luettin, and Gilbert Maitre. Xm2vtsdb: The extended m2vts database. In Second international confer- ence on audio and video-based biometric person authenti- cation, pages 965-966, 1999.","title":"Xm2vtsdb: The extended m2vts database","authors":"Kieron Messer, Jiri Matas, Josef Kittler, Juergen Luettin, Gilbert Maitre","year":"1999","venue":"Second international conference on audio and video-based biometric person authentication","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Peng et al., 2016] Chunlei Peng, Xinbo Gao, Nannan Wang, Dacheng Tao, Xuelong Li, and Jie Li. Multiple representations-based face sketch-photo synthesis. IEEE transactions on neural networks and learning systems, 27(11):2201-2215, 2016.","title":"Multiple representations-based face sketch-photo synthesis","authors":"Chunlei Peng, Xinbo Gao, Nannan Wang, Dacheng Tao, Xuelong Li, Jie Li","year":"2016","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Phillips et al., 2000] P. Jonathon Phillips, Hyeonjoon Moon, Syed /A Rizvi, and Patrick J Rauss. The feret evaluation methodology for face recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10):1090-1104, 2000.","title":"The feret evaluation methodology for face recognition algorithms","authors":"P, Jonathon Phillips, Hyeonjoon Moon, /A Syed, Patrick J Rizvi, Rauss","year":"2000","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Pizer et al., 1987] Stephen M. Pizer, E. Philip Amburn, John D. Austin, Robert Cromartie, Ari Geselowitz, Trey Greer, Bart ter Haar Romeny, John B. Zimmerman, and Karel Zuiderveld. Adaptive histogram equalization and its variations. Computer vision, graphics, and image process- ing, 39(3):355-368, 1987.","title":"Adaptive histogram equalization and its variations","authors":"Stephen M Pizer, E Philip Amburn, John D Austin, Robert Cromartie, Ari Geselowitz, Trey Greer, Bart Ter Haar, John B Romeny, Karel Zimmerman, Zuiderveld","year":"1987","venue":"Computer vision, graphics, and image processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Shen et al., 2016] Xiaoyong Shen, Aaron Hertzmann, Jiaya Jia, Sylvain Paris, Brian Price, Eli Shechtman, and Ian Sachs. Automatic portrait segmentation for image styl- ization. Computer Graphics Forum, 35(2):93-102, 2016.","title":"Automatic portrait segmentation for image stylization","authors":"Xiaoyong Shen, Aaron Hertzmann, Jiaya Jia, Sylvain Paris, Brian Price, Eli Shechtman, Ian Sachs","year":"2016","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Song et al., 2014] Yibing Song, Linchao Bao, Qingxiong Yang, and Ming-Hsuan Yang. Real-time exemplar-based face sketch synthesis. In European Conference on Com- puter Vision, pages 800-813, 2014.","title":"Real-time exemplar-based face sketch synthesis","authors":"Yibing Song, Linchao Bao, Qingxiong Yang, Ming-Hsuan Yang","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Song et al., 2017] Yibing Song, Jiawei Zhang, Linchao Bao, and Qingxiong Yang. Fast preprocessing for robust face sketch synthesis. In International Joint Conferences on Artificial Intelligence, pages 4530-4536, 2017.","title":"Fast preprocessing for robust face sketch synthesis","authors":"Yibing Song, Jiawei Zhang, Linchao Bao, Qingxiong Yang","year":"2017","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/632"},{"position":13,"raw":"[Wang and Tang, 2009] Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 31(11):1955-1967, 2009.","title":"Face photo-sketch synthesis and recognition","authors":"Wang, Tang, Xiaogang Wang, Xiaoou Tang","year":"2009","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2013] Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, and Jie Li. Transductive face sketch- photo synthesis. IEEE transactions on neural networks and learning systems, 24(9):1364-1376, 2013.","title":"Transductive face sketchphoto synthesis","authors":"Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, Jie Li","year":"2013","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2014] Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, and Jie Li. A comprehensive survey to face hallucination. International journal of computer vision, 106(1):9-30, 2014.","title":"A comprehensive survey to face hallucination","authors":"Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, Jie Li","year":"2014","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2017] Nannan Wang, Mingrui Zhu, Jie Li, Bin Song, and Zan Li. Data-driven vs. model-driven: Fast face sketch synthesis. Neurocomputing, pages 1-8, 2017.","title":"Data-driven vs. model-driven: Fast face sketch synthesis","authors":"Nannan Wang, Mingrui Zhu, Jie Li, Bin Song, Zan Li","year":"2017","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang, 2014] Nannan Wang. Heterogeneous facial image synthesis and its applications. Xidian University, 2014.","title":"Heterogeneous facial image synthesis and its applications","authors":"Nannan Wang","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang et al., 2010] Wei Zhang, Xiaogang Wang, and Xi- aoou Tang. Lighting and pose robust face sketch synthesis. In European Conference on Computer Vision, pages 420- 433, 2010.","title":"Lighting and pose robust face sketch synthesis","authors":"Wei Zhang, Xiaogang Wang, Xiaoou Tang","year":"2010","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2011] Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. Fsim: A feature similarity index for image quality assessment. IEEE transactions on image processing, 20(8):2378-2386, 2011.","title":"Fsim: A feature similarity index for image quality assessment","authors":"Lin Zhang, Lei Zhang, Xuanqin Mou, David Zhang","year":"2011","venue":"IEEE transactions on image processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2015a] Liliang Zhang, Liang Lin, Xian Wu, Shengyong Ding, and Lei Zhang. End-to-end photo-sketch generation via fully convolutional representation learning. In Proceedings of the 5th ACM on International Confer- ence on Multimedia Retrieval, pages 627-634, 2015.","title":"End-to-end photo-sketch generation via fully convolutional representation learning","authors":"Zhang","year":"2015","venue":"Proceedings of the 5th ACM on International Conference on Multimedia Retrieval","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2015b] Shengchuan Zhang, Xinbo Gao, Nan- nan Wang, Jie Li, and Mingjin Zhang. Face sketch synthesis via sparse representation-based greedy search. IEEE transactions on image processing, 24(8):2466-2477, 2015.","title":"Face sketch synthesis via sparse representation-based greedy search","authors":"Zhang","year":"2015","venue":"IEEE transactions on image processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2017] Dongyu Zhang, Liang Lin, Tianshui Chen, Xian Wu, Wenwei Tan, and Ebroul Izquierdo. Content-adaptive sketch portrait generation by decompo- sitional representation learning. IEEE transactions on im- age processing, 26(11):328-339, 2017.","title":"Content-adaptive sketch portrait generation by decompositional representation learning","authors":"Dongyu Zhang, Liang Lin, Tianshui Chen, Xian Wu, Wenwei Tan, Ebroul Izquierdo","year":"2017","venue":"IEEE transactions on image processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2012] Hao Zhou, Zhanghui Kuang, and Kwan- Yee K. Wong. Markov weight fields for face sketch syn- thesis. In Computer Vision and Pattern Recognition, pages 1091-1097, 2012.","title":"Markov weight fields for face sketch synthesis","authors":"Hao Zhou, Zhanghui Kuang, K Kwan-Yee, Wong","year":"2012","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu et al., 2017a] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2017b] Mingrui Zhu, Nannan Wang, Xinbo Gao, and Jie Li. Deep graphical feature learning for face sketch synthesis. In International Joint Conferences on Artificial Intelligence, pages 3574-3580, 2017.","title":"Deep graphical feature learning for face sketch synthesis","authors":"Zhu","year":"2017","venue":"International Joint Conferences on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/500"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}