{"type":"ScholarlyArticle","title":"Learning to Hallucinate Face Images via Component Generation and Enhancement","authors":[{"name":"Yibing Song","slug":"yibing-song","given":"Yibing","family":"Song","affiliation":"City University of Hong Kong"},{"name":"Jiawei Zhang","slug":"jiawei-zhang","given":"Jiawei","family":"Zhang","affiliation":"City University of Hong Kong"},{"name":"Shengfeng He","slug":"shengfeng-he","given":"Shengfeng","family":"He","affiliation":"South China University of Technology"},{"name":"Linchao Bao","slug":"linchao-bao","given":"Linchao","family":"Bao","affiliation":"Tencent AI Lab"},{"name":"Qingxiong Yang","slug":"qingxiong-yang","given":"Qingxiong","family":"Yang","affiliation":"University of Science and Technology of China"}],"abstract":"We propose a two-stage method for face hallucination. First, we generate facial components of the input image using CNNs. These components represent the basic facial structures. Second, we synthesize fine-grained facial structures from high resolution training images. The details of these structures are transferred into facial components for enhancement. Therefore, we generate facial components to approximate ground truth global appearance in the first stage and enhance them through recovering details in the second stage. The experiments demonstrate that our method performs favorably against state-of-the-art methods.","doi":"10.24963/ijcai.2017/633","url":"https://ijcai.org/proceedings/2017/633","pdf":"https://ijcai.org/proceedings/2017/0633.pdf","markdown":"https://ijcai.org/proceedings/2017/633.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/633","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Robotics and Vision","track":"Main track","number":633,"pages":{"first":4537,"last":4543},"keywords":["Robotics and Vision: Manipulation","Robotics and Vision: Robotics and Vision","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Dong et al., 2015] Chao Dong, Chen Change Loy, Kaim- ing He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2015.","title":"Image super-resolution using deep convolutional networks","authors":"Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Eisemann and Durand, 2004] Elmar Eisemann and Frédo Durand. Flash photography enhancement via intrinsic re- lighting. ACM Transactions on Graphics (SIGGRAPH), 2004.","title":"Flash photography enhancement via intrinsic relighting","authors":"Elmar Eisemann, Frédo Durand","year":"2004","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In IEEE conference on computer vision and pattern recogni- tion, 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gross et al., 2010] Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, and Simon Baker. Multi-pie. Im- age and Vision Computing, 2010.","title":"Multi-pie. Image and Vision Computing","authors":"Ralph Gross, Iain Matthews, Jeffrey Cohn, Takeo Kanade, Simon Baker","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gunturk et al., 2003] Bahadir K Gunturk, Aziz U Batur, Yucel Altunbasak, Monson H Hayes, and Russell M Mersereau. Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing, 2003.","title":"Eigenface-domain super-resolution for face recognition","authors":"Aziz U Bahadir K Gunturk, Yucel Batur, Altunbasak, H Monson, Russell M Hayes, Mersereau","year":"2003","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2013] Kaiming He, Jian Sun, and Xiaoou Tang. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.","title":"Guided image filtering","authors":"Kaiming He, Jian Sun, Xiaoou Tang","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hertzmann et al., 2001] Aaron Hertzmann, Charles E Ja- cobs, Nuria Oliver, Brian Curless, and David H Salesin. Image analogies. ACM Transactions on Graphics (SIG- GRAPH), 2001.","title":"Image analogies","authors":"Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, David H Salesin","year":"2001","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jia and Gong, 2008] Kui Jia and Shaogang Gong. General- ized face super-resolution. IEEE Transactions on Image Processing, 2008.","title":"Generalized face super-resolution","authors":"Kui Jia, Shaogang Gong","year":"2008","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2016] Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Deeply-recursive convolutional network for im- age super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.","title":"Deeply-recursive convolutional network for image super-resolution","authors":"Jiwon Kim, Jung Kwon Lee, Kyoung Mu, Lee","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kumar et al., 2009] Neeraj Kumar, Alexander C Berg, Pe- ter N Belhumeur, and Shree K Nayar. Attribute and sim- ile classifiers for face verification. In IEEE International Conference on Computer Vision, 2009.","title":"Attribute and simile classifiers for face verification","authors":"Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, Shree K Nayar","year":"2009","venue":"IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ledig et al., 2017] Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, and Zehan Wang. Photo-realistic single image super- resolution using a generative adversarial network. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.","title":"Photo-realistic single image superresolution using a generative adversarial network","authors":"Ledig","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2005] Wei Liu, Dahua Lin, and Xiaoou Tang. Hallucinating faces: Tensorpatch super-resolution and coupled residue compensation. In IEEE Conference on Computer Vision and Pattern Recognition, 2005.","title":"Hallucinating faces: Tensorpatch super-resolution and coupled residue compensation","authors":"Wei Liu, Dahua Lin, Xiaoou Tang","year":"2005","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2007] Ce Liu, Heung-Yeung Shum, and William T Freeman. Face hallucination: Theory and practice. International Journal of Computer Vision, 2007.","title":"Face hallucination: Theory and practice","authors":"Ce Liu, Heung-Yeung Shum, William T Freeman","year":"2007","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2011] Ce Liu, Jenny Yuen, and Antonio Torralba. Sift flow: Dense correspondence across scenes and its ap- plications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011.","title":"Sift flow: Dense correspondence across scenes and its applications","authors":"Liu","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lowe, 2004] David G Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004.","title":"Distinctive image features from scale-invariant keypoints","authors":"David G Lowe","year":"2004","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2010] Xiang Ma, Junping Zhang, and Chun Qi. Hallucinating face by position-patch. Pattern Recognition, 2010.","title":"Hallucinating face by position-patch","authors":"Xiang Ma, Junping Zhang, Chun Qi","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Petschnigg et al., 2004] Georg Petschnigg, Maneesh Agrawala, Hugues Hoppe, Richard Szeliski, Michael Cohen, and Kentaro Toyama. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics (SIGGRAPH), 2004.","title":"Digital photography with flash and no-flash image pairs","authors":"Georg Petschnigg, Maneesh Agrawala, Hugues Hoppe, Richard Szeliski, Michael Cohen, Kentaro Toyama","year":"2004","venue":"ACM Transactions on Graphics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2014] Yibing Song, Linchao Bao, Qingxiong Yang, and Ming-Hsuan Yang. Real-time exemplar-based face sketch synthesis. In European Conference on Com- puter Vision, 2014.","title":"Real-time exemplar-based face sketch synthesis","authors":"Yibing Song, Linchao Bao, Qingxiong Yang, Ming-Hsuan Yang","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tappen and Liu, 2012] Marshall Tappen and Ce Liu. A bayesian approach to alignment-based image hallucina- tion. In European Conference on Computer Vision, 2012.","title":"A bayesian approach to alignment-based image hallucination","authors":"Liu Tappen, Marshall Tappen, Ce Liu","year":"2012","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang and Tang, 2005] Xiaogang Wang and Xiaoou Tang. Hallucinating face by eigentransformation. IEEE Trans- actions on Systems, Man, and Cybernetics, Part C: Appli- cations and Reviews, 2005.","title":"Hallucinating face by eigentransformation","authors":"Wang, Tang, Xiaogang Wang, Xiaoou Tang","year":"2005","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2004] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, Eero P Simoncelli","year":"2004","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2014] Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, and Jie Li. A comprehensive survey to face hallucination. International Journal of Computer Vision, 2014.","title":"A comprehensive survey to face hallucination","authors":"Nannan Wang, Dacheng Tao, Xinbo Gao, Xuelong Li, Jie Li","year":"2014","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017] Nannan Wang, Xinbo Gao, Leiyu Sun, and Jie Li. Bayesian face sketch synthesis. IEEE Transac- tions on Image Processing, 2017.","title":"Bayesian face sketch synthesis","authors":"Nannan Wang, Xinbo Gao, Leiyu Sun, Jie Li","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2010] Jianchao Yang, John Wright, Thomas Huang, and Yi Ma. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010.","title":"Image super-resolution via sparse representation","authors":"Jianchao Yang, John Wright, Thomas Huang, Yi Ma","year":"2010","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2013] Chih-Yuan Yang, Sifei Liu, and Ming- Hsuan Yang. Structured face hallucination. In IEEE Conference on Computer Vision and Pattern Recognition, 2013.","title":"Structured face hallucination","authors":"Chih-Yuan Yang, Sifei Liu, Ming-Hsuan Yang","year":"2013","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu and Porikli, 2016] Xin Yu and Fatih Porikli. Ultra- resolving face images by discriminative generative net- works. In European Conference on Computer Vision, 2016.","title":"Ultraresolving face images by discriminative generative networks","authors":"Xin Yu, Fatih Porikli","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2015] Erjin Zhou, Haoqiang Fan, Zhimin Cao, Yuning Jiang, and Qi Yin. Learning face hallucination in the wild. In AAAI Conference on Artificial Intelligence, 2015.","title":"Learning face hallucination in the wild","authors":"Zhou","year":"2015","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}