{"type":"ScholarlyArticle","title":"FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces","authors":[{"name":"Run Wang","slug":"run-wang","given":"Run","family":"Wang","affiliation":"Nanyang Technological University , Singapore"},{"name":"Felix Juefei-Xu","slug":"felix-juefei-xu","given":"Felix","family":"Juefei-Xu","affiliation":"AMAP , Alibaba Group , USA"},{"name":"Lei Ma","slug":"lei-ma","given":"Lei","family":"Ma","affiliation":"Kyushu University , Japan"},{"name":"Xiaofei Xie","slug":"xiaofei-xie","given":"Xiaofei","family":"Xie","affiliation":"Nanyang Technological University , Singapore"},{"name":"Yihao Huang","slug":"yihao-huang","given":"Yihao","family":"Huang","affiliation":"East China Normal University , China"},{"name":"Jian Wang","slug":"jian-wang","given":"Jian","family":"Wang","affiliation":"Nanyang Technological University , Singapore"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"Nanyang Technological University , Singapore; Institute of Computing Innovation , Zhejiang University , China"}],"abstract":"In recent years, generative adversarial networks (GANs) and its variants have achieved unprecedented success in image synthesis. They are widely adopted in synthesizing facial images which brings potential security concerns to humans as the fakes spread and fuel the misinformation. However, robust detectors of these AI-synthesized fake faces are still in their infancy and are not ready to fully tackle this emerging challenge. In this work, we propose a novel approach, named FakeSpotter, based on monitoring neuron behaviors to spot AI-synthesized fake faces. The studies on neuron coverage and interactions have successfully shown that they can be served as testing criteria for deep learning systems, especially under the settings of being exposed to adversarial attacks. Here, we conjecture that monitoring neuron behavior can also serve as an asset in detecting fake faces since layer-by-layer neuron activation patterns may capture more subtle features that are important for the fake detector. Experimental results on detecting four types of fake faces synthesized with the state-of-the-art GANs and evading four perturbation attacks show the effectiveness and robustness of our approach.","doi":"10.24963/ijcai.2020/476","url":"https://ijcai.org/proceedings/2020/476","pdf":"https://ijcai.org/proceedings/2020/0476.pdf","markdown":"https://ijcai.org/proceedings/2020/476.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/476","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":476,"pages":{"first":3444,"last":3451},"keywords":["Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24212/file_24212.m3u8","length_sec":277,"provider":"hls"}],"references":[{"position":1,"raw":"[Agarwal et al., 2019] Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. Protecting world lead- ers against deep fakes. In CVPR Workshops, pages 38-45, 2019.","title":"Protecting world leaders against deep fakes","authors":"Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, Hao Li","year":"2019","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Böhme and Kirchner, 2013] Rainer Böhme and Matthias Kirchner. Counter-forensics: Attacking image forensics. In Digital Image Forensics, pages 327-366. Springer, 2013.","title":"Counter-forensics: Attacking image forensics","authors":"Kirchner Böhme, Rainer Böhme, Matthias Kirchner","year":"2013","venue":"Digital Image Forensics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Buchana et al., 2016] P. Buchana, I. Cazan, M. Diaz-Granados, F. Juefei-Xu, and M.Savvides. Simultaneous Forgery Identifica- tion and Localization in Paintings Using Advanced Correlation Filters. In ICIP, 2016.","title":"Simultaneous Forgery Identification and Localization in Paintings Using Advanced Correlation Filters","authors":"P Buchana, I Cazan, M Diaz-Granados, F Juefei-Xu, M Savvides","year":"2016","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Choi et al., 2018] Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. StarGAN: Uni- fied generative adversarial networks for multi-domain image-to- image translation. In CVPR, pages 8789-8797, 2018.","title":"StarGAN: Unified generative adversarial networks for multi-domain image-toimage translation","authors":"Choi","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cole, 2018] Samantha Cole. We Are Truly F-ed: Everyone Is Making AI-Generated Fake Porn Now. https://www.vice.com/ en us/article/bjye8a/reddit-fake-porn-app-daisy-ridley/, 2018. (Jan 25 2018).","title":"We Are Truly F-ed: Everyone Is Making AI-Generated Fake Porn Now","authors":"Samantha Cole","year":"2018","venue":null,"doi":null,"url":"https://www.vice.com/enus/article/bjye8a/reddit-fake-porn-app-daisy-ridley/,2018","internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2015] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial exam- ples. In ICLR, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo et al., 2020] Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Wei Feng, and Yang Liu. ABBA: Saliency- Regularized Motion-Based Adversarial Blur Attack. arXiv preprint arXiv:2002.03500, 2020.","title":"ABBA: Saliency-Regularized Motion-Based Adversarial Blur Attack","authors":"Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Wei Feng, Yang Liu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2020] Yihao Huang, Felix Juefei-Xu, Run Wang, Xi- aofei Xie, Lei Ma, Jianwen Li, Weikai Miao, Yang Liu, and Geguang Pu. FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles. arXiv preprint arXiv:2001.09598, 2020.","title":"FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles","authors":"Yihao Huang, Felix Juefei-Xu, Run Wang, Xiaofei Xie, Lei Ma, Jianwen Li, Weikai Miao, Yang Liu, Geguang Pu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Karras et al., 2018] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. ICLR, 2018.","title":"Progressive growing of GANs for improved quality, stability, and variation","authors":"Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Karras et al., 2019a] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial net- works. In CVPR, pages 4401-4410, 2019.","title":"A style-based generator architecture for generative adversarial networks","authors":"Karras","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karras et al., 2019b] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958, 2019.","title":"Analyzing and improving the image quality of stylegan","authors":"Karras","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Korshunov and Marcel, 2018] Pavel Korshunov and Sébastien Marcel. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685, 2018.","title":"Deepfakes: a new threat to face recognition? assessment and detection","authors":"Marcel ; Korshunov, Pavel Korshunov, Sébastien Marcel","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Lyu, 2019] Yuezun Li and Siwei Lyu. Exposing deepfake videos by detecting face warping artifacts. CVPRW, 2, 2019.","title":"Exposing deepfake videos by detecting face warping artifacts","authors":"Lyu ; Li, Yuezun Li, Siwei Lyu","year":"2019","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2019] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-DF: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962, 2019.","title":"Celeb-DF: A new dataset for deepfake forensics","authors":"Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, Siwei Lyu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang W., and X. Tang. Deep learning face attributes in the wild. In ICCV, 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang W, X Tang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2019] Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Er- rui Ding, Wangmeng Zuo, and Shilei Wen. STGAN: A unified selective transfer network for arbitrary image attribute editing. In CVPR, pages 3673-3682, 2019.","title":"STGAN: A unified selective transfer network for arbitrary image attribute editing","authors":"Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, Shilei Wen","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ma et al., 2018a] Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, et al. DeepGauge: Multi-granularity testing criteria for deep learning systems. In ASE, pages 120-131, 2018.","title":"DeepGauge: Multi-granularity testing criteria for deep learning systems","authors":"Ma","year":"2018","venue":"ASE","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ma et al., 2018b] Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jian- jun Zhao, et al. Deepmutation: Mutation testing of deep learning systems. In ISSRE, 2018.","title":"Deepmutation: Mutation testing of deep learning systems","authors":"Ma","year":"2018","venue":"ISSRE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ma et al., 2019a] Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, and Jianjun Zhao. Deepct: Tomographic combi- natorial testing for deep learning systems. In SANER, 2019.","title":"Deepct: Tomographic combinatorial testing for deep learning systems","authors":"Ma","year":"2019","venue":"SANER","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ma et al., 2019b] Shiqing Ma, Yingqi Liu, Guanhong Tao, Wen- Chuan Lee, and Xiangyu Zhang. NIC: Detecting adversarial sam- ples with neural network invariant checking. In NDSS, 2019.","title":"NIC: Detecting adversarial samples with neural network invariant checking","authors":"Ma","year":"2019","venue":"NDSS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mahendran and Vedaldi, 2015] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In CVPR, June 2015.","title":"Understanding deep image representations by inverting them","authors":"Vedaldi Mahendran, Aravindh Mahendran, Andrea Vedaldi","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[McCloskey and Albright, 2018] Scott McCloskey and Michael Al- bright. Detecting GAN-generated imagery using color cues. arXiv preprint arXiv:1812.08247, 2018.","title":"Detecting GAN-generated imagery using color cues","authors":"Scott Mccloskey, Michael Albright","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nataraj et al., 2019] Lakshmanan Nataraj, Tajuddin Manhar Mo- hammed, BS Manjunath, Shivkumar Chandrasekaran, Arjuna Flenner, Jawadul H Bappy, and Amit K Roy-Chowdhury. De- tecting GAN generated fake images using co-occurrence matri- ces. arXiv preprint arXiv:1903.06836, 2019.","title":"Detecting GAN generated fake images using co-occurrence matrices","authors":"Lakshmanan Nataraj, Manhar Tajuddin, B S Mohammed, Shivkumar Manjunath, Arjuna Chandrasekaran, Flenner, H Jawadul, Amit K Roy-Chowdhury Bappy","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pei et al., 2017] Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. DeepXplore: Automated whitebox testing of deep learning systems. In SOSP, 2017.","title":"DeepXplore: Automated whitebox testing of deep learning systems","authors":"Kexin Pei, Yinzhi Cao, Junfeng Yang, Suman Jana","year":"2017","venue":"SOSP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rössler et al., 2019] Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. FaceForensics++: Learning to detect manipulated facial images. In ICCV, 2019.","title":"FaceForensics++: Learning to detect manipulated facial images","authors":"Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Stehouwer et al., 2020] Joel Stehouwer, Hao Dang, Feng Liu, Xi- aoming Liu, and Anil Jain. On the detection of digital face ma- nipulation. CVPR, 2020.","title":"On the detection of digital face manipulation","authors":"Joel Stehouwer, Hao Dang, Feng Liu, Xiaoming Liu, Anil Jain","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2019] Run Wang, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Yihao Huang, and Yang Liu. Amora: Black-box Adversarial Morphing Attack. arXiv preprint arXiv:1912.03829, 2019.","title":"Amora: Black-box Adversarial Morphing Attack","authors":"Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2020] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. CNN-generated images are surprisingly easy to spot... for now. CVPR, 2020.","title":"CNN-generated images are surprisingly easy to spot","authors":"Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, Alexei A Efros","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2019] Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, and Simon See. DeepHunter: a coverage-guided fuzz test- ing framework for deep neural networks. In ISSTA, 2019.","title":"DeepHunter: a coverage-guided fuzz testing framework for deep neural networks","authors":"Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, Simon See","year":"2019","venue":"ISSTA","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2019] Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In ICASSP, 2019.","title":"Exposing deep fakes using inconsistent head poses","authors":"Xin Yang, Yuezun Li, Siwei Lyu","year":"2019","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yi et al., 2017] Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. Dualgan: Unsupervised dual learning for image-to-image trans- lation. In ICCV, pages 2849-2857, 2017.","title":"Dualgan: Unsupervised dual learning for image-to-image translation","authors":"Zili Yi, Hao Zhang, Ping Tan, Minglun Gong","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yu et al., 2019] Ning Yu, Larry S Davis, and Mario Fritz. Attribut- ing fake images to gans: Learning and analyzing gan fingerprints. In ICCV, pages 7556-7566, 2019.","title":"Attributing fake images to gans: Learning and analyzing gan fingerprints","authors":"Ning Yu, Larry S Davis, Mario Fritz","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zakharov et al., 2019] Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. Few-shot adversarial learn- ing of realistic neural talking head models. arXiv preprint arXiv:1905.08233, 2019.","title":"Few-shot adversarial learning of realistic neural talking head models","authors":"Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, Victor Lempitsky","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2019a] Jie M Zhang, Mark Harman, Lei Ma, and Yang Liu. Machine learning testing: Survey, landscapes and hori- zons. arXiv preprint arXiv:1906.10742, 2019.","title":"Machine learning testing: Survey, landscapes and horizons","authors":"Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2019b] Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detecting and simulating artifacts in gan fake images. arXiv preprint arXiv:1907.06515, 2019.","title":"Detecting and simulating artifacts in gan fake images","authors":"Zhang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}