{"type":"ScholarlyArticle","title":"Markov Random Neural Fields for Face Sketch Synthesis","authors":[{"name":"Mingjin Zhang","slug":"mingjin-zhang","given":"Mingjin","family":"Zhang","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Telecommunications Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Telecommunications Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":"‡ State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Yunsong Li","slug":"yunsong-li","given":"Yunsong","family":"Li","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Telecommunications Engineering , Xidian University , Xi'an 710071 , China"}],"abstract":"Synthesizing face sketches with both common and specific information from photos has been recently attracting considerable attentions in digital entertainment. However, the existing approaches either make the strict similarity assumption on face sketches and photos, leading to lose some identity-specific information, or learn the direct mapping relationship from face photos to sketches by the simple neural network, resulting in the lack of some common information. In this paper, we propose a novel face sketch synthesis based on the Markov random neural fields including two structures. In the first structure, we utilize the neural network to learn the non-linear photo-sketch relationship and obtain the identity-specific information of the test photo, such as glasses, hairpins and hairstyles. In the second structure, we choose the nearest neighbors of the test photo patch and the sketch pixel synthesized in the first structure from the training data which ensure the common information of Miss or Mr Average. Experimental results on the Chinese University of Hong Kong face sketch database illustrate that our proposed framework can preserve the common structure and capture the characteristic features. Compared with the state-of-the-art methods, our method achieves better results in terms of both quantitative and qualitative experimental evaluations.","doi":"10.24963/ijcai.2018/159","url":"https://ijcai.org/proceedings/2018/159","pdf":"https://ijcai.org/proceedings/2018/0159.pdf","markdown":"https://ijcai.org/proceedings/2018/159.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/159","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":159,"pages":{"first":1142,"last":1148},"keywords":["Machine Learning: Learning Graphical Models","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Baltrusaitis et al., 2014] T. Baltrusaitis, P. Robinson, and Morency L. Continuous conditional neural fields for struc- tured regression. In Proc. Eur. Conf. Comput. Vis., pages 593-608, 2014.","title":"Continuous conditional neural fields for structured regression","authors":"T Baltrusaitis, P Robinson, Morency L","year":"2014","venue":"Proc. Eur. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2010] M. Chang, L. Zhou, Y. Han, and X. Deng. Face sketch synthesis via sparse representation. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 2146-2149, 2010.","title":"Face sketch synthesis via sparse representation","authors":"M Chang, L Zhou, Y Han, X Deng","year":"2010","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gao et al., 2012] X. Gao, N. Wang, D. Tao, and X. Li. Face sketch-photo synthesis and retrieval using sparse rep- resentation. IEEE Trans. Circuits Syst. Video Technol., 22(8):1213-1226, 2012.","title":"Face sketch-photo synthesis and retrieval using sparse representation","authors":"Gao","year":"2012","venue":"IEEE Trans. Circuits Syst. Video Technol","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gao et al., 2015] F. Gao, D. Tao, and X. Li. Learning to rank for blind image quality assessment. IEEE Trans. Neu- ral Netw. Learn. Syst., 26(10):2275-2290, 2015.","title":"Learning to rank for blind image quality assessment","authors":"F Gao, D Tao, X Li","year":"2015","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gatys et al., 2017] L. Gatys, A. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv Preprint:1508.06576, 2017.","title":"A neural algorithm of artistic style","authors":"L Gatys, A Ecker, M Bethge","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] I.","title":null,"authors":"Goodfellow","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Proc. Int. Conf. Neural Information Proc. Syst., pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"J Goodfellow, M Pouget-Abadie, B Mirza, D Xu, S Warde-Farley, A Ozair, Y Courville, Bengio","year":"2014","venue":"Proc. Int. Conf. Neural Information Proc. Syst","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2005] Q. Liu, X. Tang, H. Jin, H. Lu, and S. Ma. A nonlinear approach for face sketch synthesis and recog- nition. In Proc. IEEE Conf. Comput. Vis. Pattern Recog- nit., pages 1005-1010, 2005.","title":"A nonlinear approach for face sketch synthesis and recognition","authors":"Q Liu, X Tang, H Jin, H Lu, S Ma","year":"2005","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2007] W. Liu, X. Tang, and J. Liu. Bayesian ten- sor inference for sketch-based face photo hallucination. In Proc. IEEE Conf. Artif. Intell., pages 2141-2146, 2007.","title":"Bayesian tensor inference for sketch-based face photo hallucination","authors":"W Liu, X Tang, J Liu","year":"2007","venue":"Proc. IEEE Conf. Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Martinez and Benavente, 1998] A. Martinez and R. Be- navente. The AR face database. CVC Technical Report, 24, 1998.","title":"The AR face database","authors":"A Martinez, R Benavente","year":"1998","venue":"CVC Technical Report","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Messer et al., 1999] K. Messer, J. Matas, J. Kittler, and G. Luettin, J.and Maitre. XM2VTSDB: the extended M2VTS database. In Proc. Int. Conf. Audio and Video- Based Biometric Person Authentication, pages 72-77, 1999.","title":"XM2VTSDB: the extended M2VTS database","authors":"K Messer, J Matas, J Kittler, G Luettin, J Maitre","year":"1999","venue":"Proc. Int. Conf. Audio and Video-Based Biometric Person Authentication","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Peng et al., 2015] C. Peng, X. Gao, N. Wang, and J. Li. Superpixel-based face sketch-photo synthesis. IEEE Trans. Circuits Syst. Video Technol., pages 1-12, 2015.","title":"Superpixel-based face sketch-photo synthesis","authors":"C Peng, X Gao, N Wang, J Li","year":"2015","venue":"IEEE Trans. Circuits Syst. Video Technol","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Peng et al., 2016] C. Peng, X. Gao, N. Wang, D. Tao, X. Li, and J. Li. Multiple representations based face sketch-photo synthesis. IEEE Trans. Neural Netw. Learn. Syst., pages 1-15, 2016.","title":"Multiple representations based face sketch-photo synthesis","authors":"C Peng, X Gao, N Wang, D Tao, X Li, J Li","year":"2016","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sheikh and Bovik, 2006] H. Sheikh and A. Bovik. Image information and visual quality. IEEE Trans. Image Pro- cess., 15(2):430-444, 2006.","title":"Image information and visual quality","authors":"H Sheikh, A Bovik","year":"2006","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2014] Y. Song, L. Bao, Q. Yang, and M. H. Yang. Real-time exemplar-based face sketch synthesis. In Proc. Eur. Conf. Comput. Vis., pages 800-813, 2014.","title":"Real-time exemplar-based face sketch synthesis","authors":"Y Song, L Bao, Q Yang, M H Yang","year":"2014","venue":"Proc. Eur. Conf. Comput. Vis","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tang and Wang, 2002] X. Tang and X. Wang. Face photo recognition using sketch. In Proc. IEEE Int. Conf. Image Process., pages 257-260, 2002.","title":"Face photo recognition using sketch","authors":"Wang ; Tang, X Tang, X Wang","year":"2002","venue":"Proc. IEEE Int. Conf. Image Process","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tang and Wang, 2003] X. Tang and X. Wang. Face sketch synthesis and recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 687-694, 2003.","title":"Face sketch synthesis and recognition","authors":"Wang ; Tang, X Tang, X Wang","year":"2003","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tang and Wang, 2004] X. Tang and X. Wang. Face sketch recognition. IEEE Trans. Circuits Syst. Video Technol., 14(1):1-7, 2004.","title":"Face sketch recognition","authors":"Wang ; Tang, X Tang, X Wang","year":"2004","venue":"IEEE Trans. Circuits Syst. Video Technol","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang and Tang, 2009] X. Wang and X. Tang. Face photo- sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(11):1955-1967, 2009.","title":"Face photosketch synthesis and recognition","authors":"Wang, Tang, X Wang, X Tang","year":"2009","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2004] Z. Wang, A. Bovik, H. Sheikh, and E. Si- moncelli. Image quality assessment: from error visibil- ity to structural similarity. IEEE Trans. Image Process., 13(4):600-612, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"Z Wang, A Bovik, H Sheikh, E","year":"2004","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2011] N. Wang, X. Gao, D. Tao, and X. Li. Face sketch-photo synthesis under multi-dictionary sparse representation framework. In Proc. 6th Int. Conf. Image Graph., pages 82-87, 2011.","title":"Face sketch-photo synthesis under multi-dictionary sparse representation framework","authors":"N Wang, X Gao, D Tao, X Li","year":"2011","venue":"Proc. 6th Int. Conf. Image Graph","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2012] S. Wang, L. Zhang, Y. Liang, and Q. Pan. Semi-coupled dictionary learning with applications to im- age super-resolution and photo-sketch synthesis. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 2216- 2223, 2012.","title":"Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis","authors":"S Wang, L Zhang, Y Liang, Q Pan","year":"2012","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2013] N. Wang, D. Tao, X. Gao, X. Li, and J. Li. Transductive face photo-sketch synthesis. IEEE Trans. Neural Netw. Learn. Syst., 24(9):1364-1376, 2013.","title":"Transductive face photo-sketch synthesis","authors":"N Wang, D Tao, X Gao, X Li, J Li","year":"2013","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] N. Wang, X. Gao, Sun L., and Li J. Bayesian face sketch synthesis. IEEE Trans. Image Pro- cess., 26(3):1264-1274, 2017.","title":"Bayesian face sketch synthesis","authors":"N Wang, X Gao, Sun L, Li J","year":"2017","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2015] S. Zhang, X. Gao, N. Wang, J. Li, and M. Zhang. Face sketch synthesis via sparse representation- base greedy search. IEEE Trans. Image Process., 24(8):2466-2477, 2015.","title":"Face sketch synthesis via sparse representationbase greedy search","authors":"S Zhang, X Gao, N Wang, J Li, M Zhang","year":"2015","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2017] L. Zhang, L. Lin, X. Wu, S. Ding, and L. Zhang. End-to end photo-sketch generation via fully convolutional representation learning. arXiv Preprint:1508.06576, 2017.","title":"End-to end photo-sketch generation via fully convolutional representation learning","authors":"L Zhang, L Lin, X Wu, S Ding, L Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2012] H. Zhou, Z. Kuang, and K. Wong. Markov weight fields for face sketch synthesis. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pages 1091- 1097, 2012.","title":"Markov weight fields for face sketch synthesis","authors":"H Zhou, Z Kuang, K Wong","year":"2012","venue":"Proc. IEEE Conf. Comput. Vis. Pattern Recognit","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}