Dual Reweighting Domain Generalization for Face Presentation Attack Detection

1 Introduction

Face recognition techniques [Schroff et al., 2015; Deng et al., 2019] have been widely utilized in our daily lives, such as smart device authentication, access control and entrance guard systems. Although bringing convenience, it also raises public concern about the safety issue since many kinds of face presentation attacks appear, such as photos and videos, which can easily fool the face recognition systems. Therefore, many hand-crafted features based and deep learning based methods for presentation attack detection have been proposed in recent years. Hand-crafted methods utilize traditional feature descriptors, such as Local Binary Pattern

∗Equal Contribution

†Corresponding Author

(LBP) [Boulkenafet et al., 2015], Histogram of Oriented Gradients (HOG) [Komulainen et al., 2013], Scale Invariant Feature Transform (SIFT) [Patel et al., 2016], etc. Standing in the strong representation abilities, Convolutional Neural Network (CNN) tackles this issue with different supervisions [Feng et al., 2016; Zhang et al., 2021; Liu et al., 2018], such as binary cross-entropy, facial depth and remotephotoplethysmography (r-ppg) signal, etc.

Although the above state-of-the-art methods have attained remarkable performance in the intra-dataset setting where training and testing data come from the same distribution, the performance drops significantly encountering the testing data from different distributions. This is because the above methods don't take domain discrepancy into account, resulting in extracting biased features. To overcome the limitation, some researchers introduce the domain generalization (DG) methods for face presentation attack detection, which utilize several source domains to improve the generalization abilities of the model. Based on it, existing methods [Shao et al., 2019; Jia et al., 2020] attempt to align the features of multiple source data equally for a shared feature space. However, for some samples are little domain-biased and some ones are large domain-biased, training on such samples indiscriminately will corrupt the generalization abilities of the methods. Hence, how to alleviate such affection to learn a more generalizable representation remains a challenging problem.

To settle the problem, on one hand, inspired by curriculum learning, we argue that focusing on large domain-biased samples from the beginning will affect the overall optimization. However, on the other hand, solving these large domainbiased samples can further improve the generalization of the model. In order to stabilize the overall optimization direction and improve generalization, we propose a novel Dual Reweighting Domain Generalization (DRDG) framework for face anti-spoofing by iteratively reweighting the relative importance between samples, as illustrated in Figure 1. Specifically, we propose two reweighting modules, Sample Reweighting Module (SRM) and Feature Reweighting Module (FRM). Firstly, Sample Reweighting Module tries to identify samples with relatively large domain bias and reduce their impact on overall optimization. Secondly, Feature Reweighting Module (FRM) specially focuses on these large domainbiased samples and tries to extract more domain-irrelevant features via a self-distilling mechanism, which will promote generalization abilities. In general, these two modules are updated iteratively relying on the competition against the domain discriminator. And the mutual iteration of these two modules ensures the overall optimization is not excessively affected by the large domain-biased data, while at the same time, further distills extracted features, and finally achieves better generalization.

The main contributions are summarized as follows:

• We propose a novel Dual Reweighting Domain Generalization framework for face anti-spoofing, which iteratively reweights the relative importance between samples frommultiple source domains to further improve the generalization.
• We specially design Sample Reweighting Module (SRM) and Feature Reweighting Module (FRM) to ensure that the overall optimization direction is not excessively affected by the large domain-biased data, and at the same time, pay more attention to distilling the feature of these data, resulting in the more generalizable model.
• Abundant experiments and visualizations are presented, which demonstrates the effectiveness of our method against the state-of-the-art competitors.

2 RelatedWork

2.1 Face Anti-Spoofing

In recent years, researchers have made great progress in the face anti-spoofing area. The development of it is concluded into two stages. Early researchers mainly utilized handcrafted feature descriptors, such as LBP [Boulkenafet et al., 2015], HOG [Komulainen et al., 2013], SIFT [Patel et al., 2016] and trained a traditional classifier for judgment. Some methods focused on temporal features to detect movement, such as eye blinking [Pan et al., 2007], head motion and lips motion. With the rise of deep learning, [Feng et al., 2016] regarded the face anti-spoofing as a binary classification task and leveraged the CNN architecture to solve it. To further boost the performance, [Liu et al., 2018] utilized additional supervisions, such as depth map, reflect map and r-ppg signal, to boost the performance. Based on auxiliary information, [Zhang et al., 2020;

Liu et al., 2020] regularized features from the perspective of disentanglement. Since the above methods did not pay attention to the cross-dataset setting, MADDG [Shao et al., 2019] utilized multiple domain discriminators to learn a generalized feature space. While SSDG [Jia et al., 2020] only aligned the features of real samples from different datasets but not for the features of the fake ones. Moreover, [Chen et al., 2021] proposed D2AM to settle a more challenging generalizable scenario in the real world where domain labels are unknown. However, these DG-based methods all attempt to align the samples straightforwardly and uniformly to a common feature space, regardless of the complexity and bias of data distribution, which leads to the degradation of generalization.

2.2 Domain Generalization

Domain generalization aims to extract knowledge applicable to any unseen domain with multiple source domains, when target domain information is not available at training time. Several domain generalization methods have been proposed recently. [Ghifary et al., 2015] matched the feature distributions among source domains by using an auto-encoder. The work in [Li et al., 2018b] based on adversarial learning to match the multiple domains to a pre-trained distribution. MLDG [Li et al., 2018a] proposed a model-agnostic meta-learning for domain generalization. [Seo et al., 2020] utilized batch and instance normalizations to achieve the best combination for each domain. Based on the above DG methods, some face anti-spoofing methods have been proposed. However, they directly treat these samples equally, which corrupts the generalization ability. While our framework, Dual Reweighting Domain Generalization (DRDG), iteratively reweights the relative importance between samples to further improve the generalization.

3 ProposedMethod

3.1 Overview

Different from previous DG-based face anti-spoofing methods, which treat each sample from multiple domains indiscriminately during the training process, our proposed Dual Reweighting Domain Generalization framework iteratively reweights the relative importance between samples to further improve the generalization. Figure 2 illustrates the overview of our framework and the entire learning process. Specially, we design two reweighting modules, Sample Reweighting Module and Feature Reweighting Module, which are introduced in the Sec. 3.2 and Sec. 3.3. 1) Sample Reweighting Module automatically reweights the relative importance of the samples in each mini-batch via measuring the feature transferability according to their domain information, and reduces the impact of large domain-biased samples for better overall optimization. 2) Feature Reweighting Module in turn focuses on these large domain biased samples, and attempts to extract more domain-irrelevant features via a selfdistilling mechanism, which further promotes the generalization of the model. Combined with the iterative training process, these two modules facilitate the learning of generalized features for face presentation attack detection.

3.2 Sample ReweightingModule

Since some samples from multiple source domains contain large distribution discrepancies, aligning them at the early training stage inevitably introduces convergence problems and hinders the model's generalization ability. To this end, we propose to dynamically adjust weights for different samples and reduce the impact of large domain-biased samples on the overall optimization. In our assumption, the samples, which are difficult to be distinguished for the domain discriminator, are the small domain-biased ones. On the contrary, the easily distinguished samples are the large domain-biased ones. Based on the assumption, we introduce a novel module Sample Reweighting Module (SRM) to assign weights for each sample via the adversarial procedure against the domain discriminator (described in Sec. 3.4). Then, we utilize these weights to encourage the framework seeking a generalized feature space by bringing the distributions of high-weight samples much closer to each other and avoiding the influence of low-weight samples.

Particularly, SRM is responsible for assigning the weights to samples at each iteration, maximizing the loss of the domain discriminator. To keep up with the updated domain discriminator, SRM is forced to update constantly and reassign the weights across training. Considering the distribution discrepancies between the live and fake faces, we introduce two SRM modules, denoted as SRMreal and SRMfake, to reweight the samples separately for each class, which are denoted as SRM uniformly because of the same mechanism. SRM assigns weights to each sample xi, which is denoted by Wi. Given a batch of samples, x1, x2, ...xN, the number of samples in each domain is the same. SRMmaps the features of these samples to an array of scores normalized via sigmoid function as the reweighting vector. During training, the optimization objective of SRM is formulated as Equation.2, where FFRMi is the feature map obtained from FRM, D is Discriminator, N is the number of one batch samples and di is the domain label of image xi, Wi is the weight of image xi, θSRM is the parameters of SRM.

3.3 Feature ReweightingModule

To make sure that our framework tackles the large domainbiased samples to facilitate learning a more generalizable representation finally, we propose the Feature Reweighting Module (FRM) through a self-distillation method to achieve this. Specifically, this module acts as an information modulator to self-distill common information from large domain-biased samples further, automatically selecting the proper feature channels to modulate contained information. The modulated features amplify the domain-agnostic information and lessen the domain-relevant information.

Following the hypothesis in Sec. 3.2, FRM employs it to self-distill the information of each sample via selecting the channels with different weights. While interplaying with domain discriminator, FRM utilizes reverse weights 1 -Wi to aggregate the large domain-biased samples much more for gradually tackling these samples, without impacting on other modules. These two modules, FRM and SRM, facilitate each other to extract a robust representation, boosting the generalization performance. On one hand, FRM relies on the sample weights calculated from SRM. Hence FRM is updated after SRM in each iteration to fully utilize the domain-biased samples currently. On the other hand, FRM suppresses the domain-relevant information of large domain-biased samples to help SRM out of the trap, improving the relative importance of such samples.

Concretely, the inputs of FRM are the features F ∈ RN×H×W×C extracted by Feature Encoder, where N is the number of images in each mini-batch, H ×W is the spatial size of features and C is the dimension of features. To ensure that each dimension is scaled with different scales, FRM calculates the weights for them via softmax function. During training, we minimize the reweighted loss of the domain discriminator LFRM to obtain the weight Ai as follows:

where Fi is the feature map from Feature Encoder, D is the Discriminator, N is the number of samples in one mini-batch and di is the domain label of image xi, Ai is the channelwise weight of feature Fi, Wi is the weight of the sample xi, θFRM is the parameters of FRM, ⊗ denotes the element-wise multiplication.

3.4 Iterative Learning Framework

In this section, we briefly introduce other modules and the optimization strategy of our framework, as shown in Algorithm 1. Following the conventional DG-based methods for face anti-spoofing, we utilize Feature Encoder, Binary Classifier, Depth Estimator and Domain Discriminator in the framework. Concretely, Feature Encoder is proposed to extract domain-invariant features for better generalization on the unseen domains, which is optimized via LEnc. Binary Classifier aims to detect the attacks from the real faces, which is optimized via LWCls. Depth Estimator is introduced to estimate the facial depth map to guide the learning of the encoder. The pseudo-depth maps as grounding truth are produced by PRNet [Feng et al., 2018]. Depth Estimator is optimized via LWDep. Domain Discriminator is utilized to judge whether the extracted features are sufficiently domaininvariant, which is optimized via LDis. And we only utilize the output of the classifier as the final results for face presentation attack detection.

To clearly explain the iterative learning process, here we describe one complete iterative unit in detail.

Step A. We update the Discriminator (D) with the discriminator loss LDis firstly, because the two modules SRM and FRM compete against it, which requires the discriminator good enough to distinguish the domain of each sample.

Step B. Then SRM is utilized to estimate weights W of each sample for focusing on the small domain-biased ones. Combining W and the discriminator loss LDis, we update SRM to minimize the loss LSRM, which improves the estimation abilities of SRM for more accurate reweighting.

Step C. Next step, the weights W are utilized to calculate other loss functions, LWDep, LWCls and LEnc. We update Depth Estimator (Dep), Binary Classifier (BC), Feature Encoder (Enc) and (FRM) with the above loss.

Step D. Repeating the above steps K-1 times, we utilize FRM to handle the large domain-biased samples and extract more generalizable representation further via the self-distillation mechanism. Then, we update FRM only by the loss LFRM, which is calculated by the discriminator loss LDis and the reverse weights 1 -W of the corresponding iteration.

Algorithm 1: The optimization strategy of our DRDG

4 Experiments

4.1 Experimental Settings

Datasets. We strictly follow the previous works [Shao et al., 2019; Shao et al., 2020; Jia et al., 2020], utilizing four public databases, to evaluate our method: OULU-NPU (denoted as O) [Boulkenafet et al., 2017], CASIA-FASD (denoted as C) [Zhang et al., 2012], MSU-MFSD (denoted as M) [Wen et al., 2015] and Idiap Replay-Attack (denoted as I) [Chingovska et al., 2012]. Concretely, we select one dataset for testing and the remaining three for training, then four testing tasks are obtained: O&C&M to I, O&C&I to M, O&M&I to C and I&C&M to O. Since significant domain shift exists among these datasets, e.g., materials, illumination, background, resolution and etc., the tasks are challenging.

Implementation Details. Our method is implemented via PyTorch on 11G NVIDIA 2080Ti GPUs with Linux OS and trained with Adam optimizer [Kingma and Ba, 2014]. The learning rates α, β are set as 1e-3, 1e-4, respectively. We extract RGB and HSV channels of images, thus the input size is 256 × 256 × 6. In training phase, the balance coefficients λ1 and λ2 are set to 10 and 0.1 respectively. In our method, the K in Algorithm 1 is an important hyperparameter, which determines the training pace of large domain-biased samples, and we set it as 5 according to our experiments. We strictly follow the popular evaluation metrics, which contain the Half Total Error Rate (HTER) and the Area Under Curve (AUC).

4.2 Comparison Results

As shown in Table 1 and Figure 3, our method outperforms all state-of-the-art methods. Firstly, we can observe that the DGbased face anti-spoofing methods perform better than conventional methods. This proves that the distribution of target domain is different from source domain, while conventional methods only focus on the spoofing cues fitting source domain. Moreover, the proposed method outperforms other DG-based methods. Concretely, compared to MADDG [Shao et al., 2020], DRDG gains 5.26% and 5.45% improvement on the task O&C&I to M and O&M&I to C respectively with HTER. We believe the reason is that our method focuses on small domain-biased samples and self-distill the common information from large domain biased ones, the cooperation of which facilitate the generalization. As shown in Table 2, we also evaluate our method with extremely limited source domains, and it achieves the best performance, which verifies its effectiveness again. As for computational efficiency, since SRM is discarded and only FRM is retained during the inference stage, our method is comparable to others.

4.3 Ablation Study

We perform the ablation study to verify the contribution of each component on O&C&I to M task. As shown in Table 3, several observations are made as follows: 1) Compared with Baseline where both SRM and FRM are removed from DRDG, Baseline FRM gains a better performance, which demonstrates that FRM enhances the features via a selfdistillation mechanism to extract a more common feature for generalization. Moreover, Baseline FRM reverse aims to push the samples from different domains apart, which performs even worse than Baseline. 2) Baseline SRM performs better than Baseline, while Baseline SRM reverse which emphasizes large domain-biased samples first and small domainbiased ones later performs worse than it. It proves the correctness of our training strategy via starting from the small domain-biased ones than dealing with the large domainbiased ones. 3) DRDG achieves the best performance compared to the other variants, which confirms that FRM and SRM facilitate each other to extract a robust representation and boost the generalization performance.

4.4 Visualization and Analysis

Sample Reweighting Visualization. To verify the effectiveness of SRM, we randomly select a model at a certain epoch during training for visualization, which is conducted on O&C&I to M task. We first demonstrate some high-weight and low-weight samples assigned by SRM in three source domains, which are from the same mini-batch, as shown in Figure 4(a). It is obvious that the differences between lowweight samples are significantly larger than those between high-weight ones. It is attributed to their large distribution discrepancies, e.g., sample with overexposed light and one with reddish overall style (all marked with red boxes). Focusing on them may mislead the optimization of the network, hence SRM assigns the lower weights to them, alleviating their impact. As shown in Figure 5(a), We also visualize the feature distributions of these samples via t-SNE. We find that high-weight samples aggregate much more closer to each other and low-weight samples are much far away from the center like outliers. Aligning these samples equally corrupts the generalization ability of our framework. Hence, SRM indeed ensures that our framework automatically selects the most suitable samples for alignment in each mini-batch, facilitating the overall optimization for better generalization.

Feature Reweighting Visualization. Following the experiments of SRM, we utilize the same model to draw the visualization for FRM. To make an exploration about the channels with different weights, we demonstrate the feature maps of the low-weight samples selected from SRM, as illustrated in Figure 4(b). We find that the attention maps of low-weight channels are with low energy and focus on some trivial cues, such as backgrounds, clothes, etc, which are not generalizable. While attention maps of high-weight channels are always with high energy and mainly pay attention to the region of the internal face, which are more likely to be intrinsic and generalized. Hence the self-distillation mechanism of FRM indeed promotes the extraction of common features for better generalization abilities. The distribution of the features extracted by the updated FRM is also illustrated in Figure 5(b). Compared with Figure 5(a), we find that the large domainbiased samples are pulled back to the center after the update of FRM, which verifies that FRM tackles such samples to get a more compact feature space.

Feature Distribution. To study the effect of the framework containing two modules on the feature distributions, we draw the visualization of Baseline and framework DRDG, as shown in Figure 5(c),(d) separately. Since baseline tries to align all the samples regardless of their distribution discrepancies, the samples relatively scatter in the feature space, which results in the performance degradation on the target domain. While feature space constructed by framework DRDG is more compact than Baseline overall, it achieves better class boundary and generalizes well on target domain for two modules facilitating each other.

5 Conclusion

In this paper, we propose a new perspective of FAS that automatically selects the most suitable samples for domain generalization. Concretely, we present an iteratively learning framework, Dual Reweighting Domain Generalization, containing two novel modules, to boost the generalization abilities. SRM and FRM facilitate each other to establish the training strategy via reassigning the most suitable weights to samples and features respectively, and pull all the samples aggregated for better generalization results. Comprehensive experiments and visualizations are presented to demonstrate the effectiveness and interpretability of our method against the state-of-the-art competitors.