Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis

1 Introduction

In the past years, image generation and tampering techniques have been evolving quickly, benefiting from the continuous breakthroughs in generative adversarial networks (GANs) [Goodfellow et al., 2014] and its variations [Radford et al., 2016; Gulrajani et al., 2017; Karras et al., 2018; Karras et al., 2019; Karras et al., 2020; Yu et al., 2021a]. The fidelity and diversity of generated images have improved to a level that is arguably already photorealistic. Although fostering the techniques for numerous novel applications [Reed et al., 2016; Thies et al., 2016; Zhu et al., 2017; Choi et al., 2018; Yu et al., 2019a; Yu et al., 2020; Wang et al., 2021], this development, on the other hand, poses new risks as recent results are challenging to be distinguished from real images by human eyes.

Malicious individuals may rely on the above techniques to alter or create the media, and spread misleading information, which will cause unpredictable results. Therefore, several precautions of misuse of the techniques are developed, including targeting the source of forgery [Zhang et al., 2020; Yu et al., 2019b; Yu et al., 2021b; Yu et al., 2021c] and detection of forgery, which provides people warning messages to trust the media suitably.

In particular, detecting fake images has received tremendous attention, because images are the ubiquitous media and appear widely in various platforms, such as social networks, advertisement, etc. Besides, detecting fake images is the backbone of many alarm systems working in more sophisticated cases, such as videos. Recent fake detection techniques mainly rely on local regions artifacts [Yu et al., 2019b; Masi et al., 2020; Chai et al., 2020], global textures [Liu et al., 2020] or rely on a mismatch in the generated frequency distribution [Durall et al., 2019; Zhang et al., 2019; Frank et al., 2020]. A CNN classifier is typically trained on the extracted features to perform binary classification for fake detection. However, recent work has shown that such low-level features are rather easy to recognize [Wang et al., 2020] and can be effectively concealed [Durall et al., 2020; Jung and Keuper, 2021]. Due to the steady improvement of generative models and the constantly narrowing gap between real and fake images, this appears not to yield in a reliable and sustainable approach to distinguish real and fake images. Therefore, it motivates us to seek different, diverse, or at least complementary approaches for robust generated images detection targeting potential unknown configurations.

In order to overcome the issue of excessive reliance on simplistic frequency and low-level artifacts, we propose a novel feature representation for generated images detection. We achieve this by processing both real and fake images with a generator that in turn induces similar frequency artifacts to both images while distinct residuals, as sketched in Figure 1. The generator is trained with real images to perform several synthesis tasks. We aim to complete information for real images from some sketched information, such as colorization, denoising and super-resolution. The frequency artifacts become non-discriminative features and will - as we show - not be used for the detection. Instead, the proposed detection mechanism leverages the features of multi-stage reconstruction errors w.r.t. the re-synthesis model. It turns out to be remarkably effective to distinguish real and fake images - which in addition is more generalized across different GAN techniques and more robust against a variety of perturbations that try to conceal fakes.

We highlight the contributions and novelty of our work as follows:(1) We present a novel feature representation for fake image detection based on re-synthesis, which is based on a super-resolution pipeline that learns a detector agnostic to the previously-used simplistic frequency features. (2) We validate the improvements of our method in terms of detection accuracy, generalization, and robustness, compared to prior work in a diverse range of settings.

2 RelatedWork

2.1 Generative Adversarial Networks (GANs)

GANs [Goodfellow et al., 2014] have achieved tremendous success in modeling data distributions. The breakthroughs mainly come from the improvements of training strategies [Karras et al., 2018] or model architectures [Karras et al., 2019; Karras et al., 2020; Yu et al., 2021a]. The current state-of-the-art GANs are capable of producing highresolution images with realistic details, which make it rather challenging for human eyes to distinguish generated and real images apart. Specifically, we study the problem of detecting deepfakes with the state of the art GANs: ProGAN [Karras et al., 2018], StarGAN2 [Choi et al., 2020], StyleGAN [Karras et al., 2019], StyleGAN2 [Karras et al., 2020].

2.2 Low-Level Artifacts Produced by GANs

GANs have been significantly improved in recent years and are able to synthesize high fidelity images fooling human eyes. However, there persist some problems in GANs revealing the differences between generated distributions and real ones because of commonly-used up-convolution (or deconvolution) operation [Durall et al., 2020], which maps lowresolution tensors to high-resolution ones. Yet these problems are never long-lasting compared to the steady improvement of GANs. For example, spectral regularization is proposed [Durall et al., 2020] to close the gap in the spectral domain. Recently, [Jung and Keuper, 2021] learn an additional discriminator with spectrum inputs with adversarial training, and the frequency gap of fake images is reduced further. Hence, it is not sustainable to establish fake detection mechanisms based on the known problems of GANs - these problems are also known to malicious individuals and can be sidestepped along with the steady development of improved GANs. That motivates us to propose a novel mechanism for detecting fake images, which is not reliant on such low-level artifacts.

2.3 Fake Detection with Spatial Analysis

Because of the emerging risks of fake information explosion, fake image detection has become an increasingly crucial and prevalent topic [Marra et al., 2018; Nataraj et al., 2019; R¨ossler et al., 2019; Wang et al., 2020; Marra et al., 2019b]. Recent work analyzes different low-level visual pattern representations so as to attribute images into real or fake [Marra et al., 2019a; Yu et al., 2019b; Liu et al., 2020; Chai et al., 2020]. First, [Yu et al., 2019b; Marra et al., 2019a] validate that GAN training naturally leaves a unique fingerprint for each model, which serves as a visual cue for fake detection. Furthermore, [Liu et al., 2020] design a network to induce texture representations using a Gram matrix, and validate that global textures at different levels of a CNN are effective cues for fake detection. Also, Laplacian of Gaussian (LoG) is augmented along with images to foster fake image and video detection [Masi et al., 2020]. Last, according to a patch-level prediction from different stages of a CNN, it has been shown that hair and background are the most informative areas for detecting fake facial images [Chai et al., 2020], which may help detection across various data distribution. In this paper, we compare to GAN fingerprint techniques [Marra et al., 2019a; Yu et al., 2019b] as representatives of this approach, and show improved performance.

2.4 Fake Detection with Frequency Analysis

Frequency analysis has a long history and broad applications in image processing. Several recent methods based on analyzing frequency patterns of images are adapted to fake detection. A simple yet effective method based on azimuthallyaveraged spectrum magnitude and SVM is proposed [Durall et al., 2019]. The 2d-FFT magnitudes serve as input features for CNN binary classification [Zhang et al., 2019; Wang et al., 2020]. In a similar spirit, 2d-DCT is also studied as CNN input features [Frank et al., 2020] and demonstrates improved detection results compared to image-based method [Yu et al., 2019b]. To the best of our knowledge, the most recent state-of-the-art detector leverages global and local 2D DCT features [Qian et al., 2020] and further validates the effectiveness of frequency analysis in terms of detecting fake images. In this paper, we compare to azimuthallyaveraged spectrum, 2d-FFT, and 2d-DCT as representatives of this approach, and show improved performance.

3 Detection by Re-Synthesis

The key goal of our work is to propose a more robust feature presentation for fake image detection, which should be generalized enough across different fake sources, robust enough against image perturbations, and more importantly, not reliant on low-level artifacts naturally induced by generative models.

To achieve this, we propose a novel method that is explicitly designed to re-synthesize testing images according to several auxiliary tasks to model the distribution for real images, for the purpose of robust fake image detection. Our model has two components, a re-synthesizer S and a classifier C, and they are jointly trained end to end. The re-synthesizer aims to distinct real/fake images that it is only learned with real images to provide different residuals from fake ones. Besides, the re-synthesizer allows us to incorporate a variety of noisy patterns to avoid overfitting on the representations from it for the robust detection in unknown scenarios. As an instance, our method incorporates a super-resolution model, which aims to predict the high-frequency information from low-resolution inputs. Owing to the differences w.r.t highfrequency information between real and fake images, this allows us to build a fake image detector based on the residual errors (i.e., reconstruction errors) formalized by this superresolution model. Further, we take different forms of inputs for the super-resolution model to capture richer and more robust visual representations from real images for improving the robustness of detection.

3.1 Re-Synthesis Residuals as Structural Artifacts

We train a re-synthesizer S on real images, which takes a downsampled version to reconstruct the original image, and regard the structural reconstruction error maps as features for classification. Particularly, S is trained on real images only, and then we are able to show different residual distributions for real and fake images.

Mathematically, given a dataset D+ representing real images, we first train a super-resolution model Φ on D+, which is formulated as a regression task with the loss function

where Ω could be an image degeneration operation, X ∈ D+ and X↓ is a downsampled version of X. As a result, our resynthesizer S(·) = Φ(Ω(·)). In this work, Ω = {I,G,N}, referring to the indentity, grayscaling and noising operations respectively.

After training the super-resolution model, we apply the structural artifact with downsampled images |X - Φ(X↓)| as the feature for fake detection. We collect another dataset of fake images from a generator, denoted as D-. A fake image detector is then trained on D+ ∪ D-. We formulate the detector as a neural network classifier C(·) trained with softmax loss. In testing, given a query image X∗ , the detection decision is formulated as

3.2 Hierarchical Artifacts via Perceptual Loss

Despite the success of low-level artifacts in fake detection [Yu et al., 2019b; Masi et al., 2020], high-level information remains unexplored and should be equally effective, such as semantic parts, global coherence, etc. Perceptual loss [Johnson et al., 2016], on the other hand, evaluates the difference between two examples w.r.t CNN activations at various layers, which correspond to hierarchical representations of visual information. Consequently, in order to boost our detection pipeline with high-level information, we take advantage of perceptual loss from a pretrained network to train our super-resolution model, instead of only considering the reconstruction errors in pixels.

Let Θ be a pretrained network, and Θi(·) be the operation to extract features in the i-th stage of a total of n stages. The loss function with perceptions and image degeneration is formulated as

where α0, α1, ..., αn are loss weights to control the importance of corresponding feature stages during training. After training the super-resolution model, we leverage the ℓ1 residual map at each stage, as the feature for fake detection. Notably, we can detect fake images by a single classifier or by a combination of classifiers at different stages. Let {C0(·), C1(·), ..., Cn(·)} be the set of classifiers, where we define C0(·) is the pixel-level classifier, and others are the classifiers trained on artifacts at different stages from the perceptual loss. The final decision for input image X∗ is computed with weights β0, β1, ..., βn and formulated as

In Figure 3, we show examples from CelebA-HQ [Karras et al., 2018] and their structural artifacts in the pixel level and in stage5 of the perceptual network. We observe: (1) The magnitudes of artifacts of fake images are larger than those of real images, which lay the foundation to distinguish fake from real. In particular, there are more severe artifacts on hairs, eyes, or mouths, which is consistent with the recent study of generalization of fake detection [Chai et al., 2020]. (2) The artifact structures are distinct between real and fake, where they look more randomly distributed in real images while with stronger patterns in fake images. (3) The stage5 perceptual artifacts are more discriminative than pixel artifacts to attribute fake, even between ProGAN and StyleGAN trained on the same dataset. This results in the potential for cross-GAN fake detection. For quantitative demonstration, in Figure 4 we plot the histograms of spatially-averaged amounts of artifacts in the pixel level and in stage5, and show the clear margins between distributions of real and fake.

4 Experiments

We present the experimental setup and configuration details from section 4.1 to 4.5. Accordingly, we provide the results and discussion in the rest of this section.

4.1 Datasets

CelebA-HQ is a 1024×1024 facial image dataset released by [Karras et al., 2018]. We choose to necessarily conduct experiments on faces because of the prevalent applications of identity recognition and biometric feature recognition. From this dataset, we prepare 25k/25k real and fake images as the training set, and 2.5k/2.5k images as the testing set.

FFHQ [Karras et al., 2019] provides another 70k 1024×1024 facial images. Compared to CelebA-HQ, it covers vastly more variation on age, ethnicity, eyeglasses, hats, background, etc. To perform cross-domain detection, we test on this dataset all the detectors trained on CelebA-HQ.

LSUN is large-scale scene dataset [Yu et al., 2015]. We select bedroom and church outdoor classes to perform fake detection beyond human faces. We prepare 50k/50k real and fake images for training, and 10k/10k images for testing.

4.2 Deep Fake DetectionMethods

We compare our method to the recently-proposed methods, including PRNU [Marra et al., 2019a], plain image [Yu et al., 2019b], 1d spectrum [Durall et al., 2019], FFT-2d magnitude [Wang et al., 2020; Zhang et al., 2019], DCT-2d [Frank et al., 2020] and GramNet [Liu et al., 2020]. For our models, we report the performance based on the pixel- and stage5level artifacts using ResNet-50 [He et al., 2016], which capture low- and high-level spatial errors respectively. In addition, we also report the performance of averaging between the two artifacts. In particular, we compare three re-synthesizers in our study: (1) A super-resolution model is used which predicts high-resolution images from downsampled images, namely SR for short; (2) A super-resolution model which takes partially gray-scaled images to predict high-resolution color images , referred as SR+C. 50% images are performed grayscaling operation. For those images, 10% ∼ 25% pixels are randomly set to gray-scale versions. (3) A superresolution model which takes noisy images to predict highresolution clean images, referred as SR+D. 50% images are performed noising operation and Gaussian noises with standard deviation 4/255 are applied.

4.3 Robustness against Image Perturbations

Image perturbations are able to alter image details or distributions while preserving the contents of them (e.g., denoising, JPEG compression, etc), which may be used to process fake images and challenge the detectors. In order to compare the robustness of different methods, we test on perturbed fake images that are not seen in training phases. First, we calibrate the frequency distributions of generated images using the equalization operation [Durall et al., 2020]. We regard this operation as post-processing to reduce the frequency distribution gap between real and fake images. Second, we follow the protocol in the previous work [Yu et al., 2019b] to perturb testing images with an ordered combination of JPEG compression, blurring, cropping, and adding Gaussian noise.

4.4 Generators

We generate fake images from ProGAN [Karras et al., 2018], StarGAN2 [Choi et al., 2020], StyleGAN [Karras et al., 2019], and StyleGAN2 [Karras et al., 2020]. To overcome the known frequency artifacts of up-convolution operations in the generators and provide unknown detection scenarios, we apply the spectral regularization (+R) [Durall et al., 2020] and spectral-aware adversarial training (+A) [Jung and Keuper, 2021] to finetune released GANs and Figure 5 shows some examples. Besides, we also apply spectrum equalization (+E), which is similar to the regularization, but we process the fake images as a postprocessing.

4.5 Implementation Details

We train the super-resolution model on the real images from CelebA-HQ or LSUN. We build a 4× super-resolution model using [Zhang et al., 2018] supervised by the ell1 pixel loss plus VGG-based perceptual loss [Johnson et al., 2016; Wang et al., 2018]. We set the loss weight for each feature from pixel to stage 5 as [1, 1/32, 1/16, 1/8, 1/4, 1]. Second, we train the detectors on pixel artifacts and stage5 artifacts from the VGG network. For each detector, we train a ResNet-50 [He et al., 2016] from scratch for 20 epochs using the SGD optimizer with momentum. The initial learning rate is 0.01 and we reduce it to 0.001 at the 10th epoch.

4.6 Results on CelebA-HQ

We conduct experiments on CelebA-HQ and compare our method to the baselines, as listed in Table 1. In specific, our model (Avg) leverages the hierarchical structural artifacts in a combination mode as described in Eq. (4), where we set (1/2 , 1/2) as the weights for combining the final scores of the pixel and stage 5 artifacts. In Table 1, we present the results including the training and testing images are from the same generator and the images are from the different generator. For each part, we not only test on the raw images, but also test on the different challenges of spectrum regularization (+R), equalization (+E), adversarial training (+A) and a combination of perturbations (+P), as discussed before.

First, we observe our detectors achieve remarkable results in average. We emphasize: (1) Spectrum-based methods [Durall et al., 2019; Zhang et al., 2019] achieves decent performance and robustness in the cross-GAN settings. For example, the performance of [Durall et al., 2019] and [Zhang et al., 2019] deteriorates only by 10% on average in the crossGAN settings. (2) DCT-2d-based method is robust across domains and robust against processing on frequency. However, it is sensitive to the image perturbations, deteriorating the performance severely. (3) In contrast, our model with any of the features achieves better results compared to the baselines. Especially the deterioration is imperceptible when applying spectral processing or testing cross domains. (4) Introducing colorization or denoising into the re-synthesis helps learning more robust features and achieving more favorable results.

In addition to our advantageous performance, we also point out a few insightful discoveries as follows: (1) Our stage5based detector obtains the best accuracy in the cross-domain settings with image perturbations, which validates the robustness of leveraging high-level information. (2) Our average detector achieves better performance than our other two detectors in many tests, indicating the beneficial synergy between our pixel artifacts and stage 5 artifacts. We reason this as the synergy effect between pixel and stage artifacts. Therefore, we suggest considering more about high-level cues for fake detection, instead of looking at the local only.

Additionally, we also test the above detectors on recent proposed StarGAN2, where the results are visualized in Figure 6. We highlight several points: (1) Most competing detectors fail to recognize StarGAN2 generated images owing to the domain shift. Even though DCT2d detector perform well in Table 1, it hardly recognizes the fakeness from StarGAN2 images. (2) All of our detectors with different synthesis modules and visual features achieve accuracy higher than 85% and most of them are close to 100%. Besides, the improved results are achieved for ProGAN after employing colorization or denoising in the re-synthesis, which shows the effectiveness of introducing noises into the re-synthesizer.

To further demonstrate the robustness of our detectors, we visualize the class activation maps (CAMs) for our methods (SR) and previous detectors [Yu et al., 2019b] in Figure 7. From this figure, we are able to observe the CAMs of our approach are quite stable for similar input fake images, even though spectral regularization [Durall et al., 2020] is applied to reduce the spectrum distrbution between real and fake images. In contrast, the image based detector [Yu et al., 2019b] is sensitive to the changes and outputs an opposite highlighted region. Therefore, we reason performance reduction for previous approaches to their sensitivities.

4.7 Results on FFHQ

We conduct experiments for cross-domain detection with FFHQ, where the results are listed in Table 2. In the experiments, we do not train additional detectors; instead directly test the models on novel data, which are trained on CelebA-HQ. We apply the real images from FFHQ and fake images from StyleGAN and StyleGAN2 trained on FFHQ. Because this setting already challenges most detectors, we do not employ additional perturbations as CelebA-HQ. From Table 2 we observe: (1) Our model using pixel inputs achieves the best performance, and outperforms the DCT-2d based approach when StyleGAN2 is tested. (2) Our stage5-based models achieve comparable performance to DCT-2d and are better than our pixel-based when StyleGAN is tested, which further shows the complementary capability of low- and highlevel artifacts to deal with wider detection scenarios. (3) We can observe clear improvements when colorization or denoising is applied, indicating more robust features are obtained and thus achieving better results in cross-datasets detection. We owe this observation to the avoid of overfitting on the training data by the noisy inputs of our synthesizers.

4.8 Results on LSUN

We use official StyleGAN to generate images of bedroom and StyleGAN2 to generate images of church outdoor. In Table 3, we test all the models on raw images and those combined with perturbations. We observe our method reaches to competing results compared with other state of the arts. In particular, our detector using stage 5 artifacts is the most stable against perturbations, which shows the necessity of involving high-level information. For example, our SR+C and SR+D detectors based on stage 5 achieve more than 95% accuracy on church outdoor with perturbations while others are less than 70%. We conclude the proposed method is able to cope with diverse challenges using multi-level features and our re-synthesis module. It sheds the light on the new way for fake image detection, more advantageous to only analyzing the local patterns [Marra et al., 2019a; Yu et al., 2019b] or frequency distributions [Frank et al., 2020; Zhang et al., 2019]. Additionally, we show the class activation maps for LSUN in Figure 8. Unlike facial images, our CAMs are more flexible in highlighted shapes and locations for highly diverse scene images in LSUN.

5 Conclusion

Due to the unsustainable reliance of frequency based deepfake detectors, we show how these methods deteriorate when such artifacts are suppressed. Based on this insight, we present a novel feature representation for detecting Deepfakes. Instead of the limited focus on low-level local artifact characteristics, we reason that different levels of information can help detect fake images with beneficial synergy. The hierarchical artifacts from a re-synthesizer are evidenced to boost the performance, generalization, and robustness of a downstream detector. These have been validated with high-resolution Deepfakes created from state-of-the-art GANs. Further, deepfakes detection is still an open problem because many issues are unresolved. Attacks can reverse the procedure of classification to fool a detector, therefore, we believe our solution provides a promising solution that our features are from a parameterized model which can be watermarked, and thus increases the challenges of attacks on our classifier. In the end, we conclude deepfakes detection requires more than analysis in addition to low-level details, but also on higher-level visual cues which has the potential to lead to more sustainable detection schemes.

Acknowledgements

Ning Yu is partially supported by the Twitch Research Fellowship.