{"type":"ScholarlyArticle","title":"Deep Graphical Feature Learning for Face Sketch Synthesis","authors":[{"name":"Mingrui Zhu","slug":"mingrui-zhu","given":"Mingrui","family":"Zhu","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":"‡ State Key Laboratory of Integrated Services Networks , School of Telecommunications , Xidian University , Xi'an 710071 , China"},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"},{"name":"Jie Li","slug":"jie-li","given":"Jie","family":"Li","affiliation":"† State Key Laboratory of Integrated Services Networks , School of Electronic Engineering , Xidian University , Xi'an 710071 , China"}],"abstract":"The exemplar-based face sketch synthesis method generally contains two steps: neighbor\r\nselection and reconstruction weight representation. Pixel intensities are widely used\r\nas features by most of the existing exemplar-based methods, which lacks of representation\r\nability and robustness to light variations and clutter backgrounds. We present a novel\r\nface sketch synthesis method combining generative exemplar-based method and discriminatively\r\ntrained deep convolutional neural networks (dCNNs) via a deep graphical feature learning\r\nframework. Our method works in both two steps by using deep discriminative representations\r\nderived from dCNNs. Instead of using it directly, we boost its representation capability\r\nby a deep graphical feature learning framework. Finally, the optimal weights of deep\r\nrepresentations and optimal reconstruction weights for face sketch synthesis can be\r\nobtained simultaneously. With the optimal reconstruction weights, we can synthesize\r\nhigh quality sketches which is robust against light variations and clutter backgrounds.\r\nExtensive experiments on public face sketch databases show that our method outperforms\r\nstate-of-the-art methods, in terms of both synthesis quality and recognition ability.","doi":"10.24963/ijcai.2017/500","url":"https://ijcai.org/proceedings/2017/500","pdf":"https://ijcai.org/proceedings/2017/0500.pdf","markdown":"https://ijcai.org/proceedings/2017/500.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/500","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":500,"pages":{"first":3574,"last":3580},"keywords":["Machine Learning: Learning Graphical Models","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Beis and Lowe, 1997] Jeffrey S. Beis and David G. Lowe. Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1000-1006, 1997.","title":"Shape indexing using approximate nearest-neighbour search in high-dimensional spaces","authors":"Jeffrey S Beis, David G Lowe","year":"1997","venue":"Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2000] Li-Fen Chen, Hong-Yuan Mark Liao, Ming-Tat Ko, Ja-Chen Lin, and Gwo-Jong Yu. A new lda- based face recognition system which can solve the small sample size problem. Pattern Recognition, 33(10):1713- 1726, 2000.","title":"A new ldabased face recognition system which can solve the small sample size problem","authors":"Li-Fen Chen, Hong-Yuan Mark Liao, Ming-Tat Ko, Ja-Chen Lin, Gwo-Jong Yu","year":"2000","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gao et al., 2012] Xinbo Gao, Nannan Wang, Dacheng Tao, and Xuelong Li. Face sketch-photo synthesis and retrieval using sparse representation. IEEE Transactions on Cir- cuits and Systems for Video Technology, 22(8):1213-1226, 2012.","title":"Face sketch-photo synthesis and retrieval using sparse representation","authors":"Xinbo Gao, Nannan Wang, Dacheng Tao, Xuelong Li","year":"2012","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gatys et al., 2016] Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2414- 2423, 2016.","title":"Image style transfer using convolutional neural networks","authors":"Leon A Gatys, Alexander S Ecker, Matthias Bethge","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances In Neural Informa- tion Processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances In Neural Information Processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[K. Messer and J. Luettin, 1999] J. Kittler K. Messer, J. Matas and G. Maitre J. Luettin. Xm2vtsdb: the extended m2vts database. In Proceedings of the Interna- tional Conference on Audio-and Video-Based Biometric Person Authentication, pages 72-77, 1999.","title":"Xm2vtsdb: the extended m2vts database","authors":"K Messer, J Luettin, ] J Kittler, K Messer, J Matas, G Maitre, J Luettin","year":"1999","venue":"Proceedings of the International Conference on Audio-and Video-Based Biometric Person Authentication","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Klare et al., 2011] Brendan Klare, Zhifeng Li, and Anil K. Jain. On matching forensic sketches to mugshot photos. IEEE Transactions on Pattern Analysis and Machine In- telligence, 33(3):639-646, 2011.","title":"On matching forensic sketches to mugshot photos","authors":"Brendan Klare, Zhifeng Li, Anil K Jain","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lei et al., 2014] Zhen Lei, Matti Pietikainen, and Stan Z. Li. Learning discriminant face descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2):289- 302, 2014.","title":"Learning discriminant face descriptor","authors":"Zhen Lei, Matti Pietikainen, Stan Z Li","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liang Chang et al., 2010] Mingquan Zhou Liang Chang, Yanjun Han, and Xiaoming Deng. Face sketch synthe- sis via sparse representation. In Proc. 20th Int. Conf. Pat- tern Recognit, pages 2146-2149, Istanbul, Turkey, August 2010.","title":"Face sketch synthesis via sparse representation","authors":"Liang Chang","year":"2010","venue":"Proc. 20th Int. Conf. Pattern Recognit","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2005] Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma. A nonlinear approach for face sketch synthesis and recognition. In Proceed- ings of IEEE Conference on Computer Vision and Pattern Recognition, pages 1005-1010, 2005.","title":"A nonlinear approach for face sketch synthesis and recognition","authors":"Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, Songde Ma","year":"2005","venue":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2007] Wei Liu, Xiaoou Tang, and Jianzhuang Liu. Bayesian tensor inference for sketch-based facial photo hallucination. In Proceedings of International Joint Conference on Artificial Intelligence, pages 2141-2146, 2007.","title":"Bayesian tensor inference for sketch-based facial photo hallucination","authors":"Wei Liu, Xiaoou Tang, Jianzhuang Liu","year":"2007","venue":"Proceedings of International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2007/345"},{"position":13,"raw":"[Martinez and Benavente, 1998] Aleix Martinez and Robert Benavente. The ar face database. In CVC, Barcelona, Spain, 1998.","title":"The ar face database","authors":"Benavente Martinez, Aleix Martinez, Robert Benavente","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2014] Yibing Song, Linchao Bao, Qingxiong Yang, and Ming-Hsuan Yang. Real-time exemplar-based face sketch synthesis. In Proceedings of Eureopean Con- ference on Computer Vision, pages 800-813, 2014.","title":"Real-time exemplar-based face sketch synthesis","authors":"Yibing Song, Linchao Bao, Qingxiong Yang, Ming-Hsuan Yang","year":"2014","venue":"Proceedings of Eureopean Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tang and Wang, 2002] Xiaogang Tang and Xiaoou Wang. Face photo recognition using sketch. In Proceedings of IEEE International Conference on Image Processing, pages 257-260, 2002.","title":"Face photo recognition using sketch","authors":"Wang ; Tang, Xiaogang Tang, Xiaoou Wang","year":"2002","venue":"Proceedings of IEEE International Conference on Image Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tang and Wang, 2003] Xiaoou Tang and Xiaogang Wang. Face sketch synthesis and recognition. In Proceedings of IEEE International Conference on Computer Vision, pages 687-694, 2003.","title":"Face sketch synthesis and recognition","authors":"Wang ; Tang, Xiaoou Tang, Xiaogang Wang","year":"2003","venue":"Proceedings of IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tang and Wang, 2009] Xiaoou Tang and Xiaogang Wang. Face photo-sketch synthesis and recognition. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 31(11):1955-1967, November 2009.","title":"Face photo-sketch synthesis and recognition","authors":"Wang ; Tang, Xiaoou Tang, Xiaogang Wang","year":"2009","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2004] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"A C Zhou Wang, H R Bovik, E P Sheikh, Simoncelli","year":"2004","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2014] Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, and Jie Li. A comprehensive survey to face hallucination. International Journal of Computer Vision, 106(1):9-30, January 2014.","title":"A comprehensive survey to face hallucination","authors":"Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, Jie Li","year":"2014","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Nannan Wang, Xinbo Gao, Leiyu Sun, and Jie Li. Bayesian face sketch synthesis. IEEE Transac- tions on Image Processing, PP:1-11, January 2017.","title":"Bayesian face sketch synthesis","authors":"Nannan Wang, Xinbo Gao, Leiyu Sun, Jie Li","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2015] Liliang Zhang, Liang Lin, Xian Wu, Shengyong Ding, and Lei Zhang. End-to-end photo-sketch generation via fully convolutional representation learning. In Proceedings of the 5th ACM on International Confer- ence on Multimedia Retrieval, pages 627-634, 2015.","title":"End-to-end photo-sketch generation via fully convolutional representation learning","authors":"Liliang Zhang, Liang Lin, Xian Wu, Shengyong Ding, Lei Zhang","year":"2015","venue":"Proceedings of the 5th ACM on International Conference on Multimedia Retrieval","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2012] Hao Zhou, Zhanghui Kuang, and Kwan- Yee K. Wong. Markov weight fields for face sketch syn- thesis. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 1091-1097, 2012.","title":"Markov weight fields for face sketch synthesis","authors":"Hao Zhou, Zhanghui Kuang, K Kwan-Yee, Wong","year":"2012","venue":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu and Wang, 2016] Mingrui Zhu and Nannan Wang. A simple and fast method for face sketch synthesis. In In- ternational Conference on Internet Multimedia Computing and Service, pages 168-171, 2016.","title":"A simple and fast method for face sketch synthesis","authors":"Wang ; Zhu, Mingrui Zhu, Nannan Wang","year":"2016","venue":"ternational Conference on Internet Multimedia Computing and Service","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}