On Adversarial Robustness of Demographic Fairness in Face Attribute Recognition

1 Introduction

In recent years, deep neural networks (DNNs) have been thriving in face attribute recognition (FAR) applications (e.g., recidivism prediction and justice systems) [Ding et al., 2018; K¨arkk¨ainen and Joo, 2019; Tariq et al., 2022; Tariq et al., 2022]. Despite the impressive performance of DNNs on these applications, they usually suffer from performance bias against vulnerable demographic groups (e.g., under-represented races or gender) [Bellamy et al., 2018; Buolamwini and Gebru, 2018]. For instance, [Bellamy et al., 2018; Buolamwini and Gebru, 2018] found that AI facial services (e.g., IBMWatson Visual Recognition 1 ) often perform much better for light-skin or male images than dark-skin or female images, resulting in race/gender inequality. As such, it is vital to train models that do not discriminate for FAR services and other identity-sensitive applications.

∗Contact Author

1 https://www.ibm.com/fi-en/cloud/watson-visual-recognition

To address the bias issue, [Zhao et al., 2017; Kou et al., 2021; Zeng et al., 2023; Zeng et al., 2022] focus on creating balanced training distributions, and [Torralba and Efros, 2011; Geirhos et al., 2020; Scimeca et al., 2021] strive to suppress the shortcut learning phenomenon. However, merely pursuing a high degree of model fairness cannot automatically provide the robustness of the fairness: could the model always maintain its fairness under potential malicious fairness attacks? For instance, one can develop a fair model for the FAR-based crime prediction system. However, there might exist a malicious attacker, who can easily corrupt the system and make it biased against a specific demographic group (e.g., African-Americans). Under the fairness attack, the system is prone to produce wrong predictions for the attacked demographic group. Therefore, it is not sufficient to only develop fair models, but the robustness of the fairness should also be improved against potential adversarial threats. Compared to [Torralba and Efros, 2011; Geirhos et al., 2020; Scimeca et al., 2021; Zhao et al., 2017; Kou et al., 2021], this work studies the fairness from an adversary perspective.

With the goal of exploring robustness of fairness for FAR models, we firstly present a novel fairness attack, whose attack objective is the fairness of the model instead of its overall performance. Under our proposed attack, we show that the fairness of face attribute classifiers in different FAR applications could be easily corrupted. Then, the observed vulnerability of fairness motivates us to design a defense mechanism to enable the robustness of fairness against the potential attack. In particular, regarding our fairness attack, it is formulated as test-data-free, cross-attribute, and clean-label data poisoning at group level. Test-data-free: the test-data-free design guarantees the attacker could generate poisoned face data without accessing the test data. Compared to traditional poisoning attacks [Shafahi et al., 2018; Huang et al., 2020; Chen et al., 2017], where selected test samples must be accessible to launch attack, our test-data-free poisoning is not limited by the accessibility of test data, and thus more dangerous. Cross-attribute: the cross-attribute feature also makes the attack more insidious: the poisons are generated w.r.t. de- Demographic Attribute Data Examples Classifier before Attack Predictive Attribute

(a) Train a fair classifier.

Poisoning Examples

Classifier after Attack

Predictive Attribute

(b) Launch fairness attack.

Predictive Attribute

(c) Defense: robust-fair training.

Figure 1: Overview: (a) Assume we have a trained fair face attribute classifier. (b) Now, an attacker launches the attack against the vulnerable group (green triangles). The attacker generates cross-attribute poisons (red circles) to skew the training distribution, such that the classifier becomes biased against the attacked group. (c) To defend against the fairness attack, we propose a robust-fair training, where a re-weighted training loss is optimized to achieve robust fairness of the classifier.

mographic attributes, which are orthogonal to the predictive attribute of the FAR task. As such, the attacked demographic attribute could be one of all possible demographic attributes, making the attack highly unpredictable. Clean-label: under the clean-label constraint, the labels of all poisons remain clean as they appear to human eyes. Along with other training samples, the poisoned dataset remains visually benign but is already biased against a targeted demographic group. The clean-label design makes the attack less detectable and could easily surpass human inception [Shafahi et al., 2018]. Grouplevel: our attack focuses on an entire targeted demographic group instead of a few targeted test samples. The group-level design liberates the power of the attack, because it does not assume specific victim test samples to attack and generalizes to all test samples from the attacked demographic group by design. These four key features fundamentally distinguish our novel attack from existing poisoning attack (e.g., backdoor poisoning or test-data-required fairness poisoning). The intuition of this attack is illustrated in Figure 1. Next, motivated by the adversarial threat on model fairness, we further design an efficient defense mechanism called robust-fair training to enable the robustness of fairness against the fairness attack. Under our defense scheme, fairness adversarial examples will be generated, such that the bias within training data could be maximized. Under the biased training distribution, the face attribute classifiers are then trained to be fair using a fairness-aware loss. To this end, the face attribute classifiers learn to combat the bias introduced by the fairness adversarial examples. Eventually, the fairness of the classifier becomes robust against the fairness poisoning attack.

Consider a recidivism prediction system based on face attribute recognition. Assume the system is developed to be fair using existing methods, e.g., fair sampling. Now, a malicious fairness attacker attacks the system to be biased against a target demographic population (e.g. Asian or AfricanAmerican). Since the attacker is test-data-free, it needs not to know which exact person to attack, but the attack effect is automatically generalized to all people from the attacked demographic group by wrongly predicting them as criminals. Moreover, the clean-label attacker does not flip the groundtruth labels of the poisoned data. As such, the attack could easily surpass the data checking process, as the poisoned data is visually benign and its label is clean. In addition, the crossattribute design of the attack manipulates the demographic attribute of the poisoned data (e.g., races) instead of the predictive attribute (being criminal or not) to bias the training model. As a consequence, the attacked population would be misclassified as criminals by the biased system, leading to injustice in law enforcement.

To the best of our knowledge, our work is the first work investigating the adversarial robustness of demographic fairness in FAR applications under a novel crossattribute, test-data-free, clean-label and group-level poisoning attack. We select FAR as our study case because FAR is highly identity-sensitive and the demographic information of FAR applications is usually implicitly integrated into face data by default. As such, launching fairness attack in FAR applications becomes more challenging compared to other applications, where vectorized data format (e.g., COMPAS [Angwin et al., 2016] or Income Census [Kohavi, 1996]) is used. Under a vectorized data format, data attributes are stored in each entry of the data vector (e.g., one-hot representing the gender/race). When launching attack on vectorized data, one can easily flip the specific entry of the vector to modify the demographic information of that data point. In contrast, when the attacker tries to modify the demographic information of face images, it is non-trivial for the attacker to guarantee the efficiency of the manipulation while keeping the manipulation imperceptible.

After launching the proposed attack on different FAR tasks, we observe that the fairness property of different face attribute classifiers could be easily corrupted under our attack. Moreover, compared to existing fairness attacks (e.g., adversarial sampling, adversarial labeling, hard adversarial examples, and Fairness Poisoning (FP) [Chang et al., 2020]), our attack is more dangerous and efficient because of the novel design of our attack. Our superior attack results indicate that our method could be used as a stronger baseline for future work. Besides, we also demonstrate that our defense mechanism shows impressive performance in terms of increasing the robustness of model fairness across various FAR applications. Finally, we highlight that while our attack scheme is

Demographic Attribute designed for general multi-class classification problems, we select binary attributes in our experiments for a fair comparison against baseline attack methods as in [Chang et al., 2020]. We summarize the contributions of this work as follows:

• We introduce a new type of data poisoning attack against demographic fairness of FAR models. The key novel features (i.e., cross-attribute, test-data-free, clean-label and group-level) of our proposed fairness attack distinguish our work fundamentally from other existing poisoning attacks (e.g., backdoor attacks, single instance poisoning and test-data-required fairness poisoning).
• We present two concrete methods to launch the attack. With systematic evaluation on various FAR applications, we show that fair face attribute classifiers are severely biased by the attacker using both methods, and our attack is significantly more efficient than other baselines.
• We design an efficient defense mechanism called robustfair training to improve the robustness of the fairness against the potential fairness attack. In addition to the successful attacks, our experimental results also demonstrate the efficacy of the proposed defense method.

2 RelatedWork

Fair Machine Learning and Fairness Attacks. Efforts have been made to address the bias issue in AI models [Li et al., 2021; Wen et al., 2022]. In these studies, it is usually assumed that the data source is clean, leading to desired model fairness. However, this assumption leaves a fatal loophole: the fairness derived using existing debiasing methods could be easily broken, when the data source is hacked [Chang et al., 2020; Van et al., 2021; Mehrabi et al., 2020]. [Mehrabi et al., 2020] leverages test data to craft poisons to corrupt model fairness. Similarly, online fairness attacks are proposed in [Chang et al., 2020] and [Van et al., 2021], where poisons are continuously generated and injected into the training process. However, in this work, we consider a more practical and dangerous setting, where the attack model has no access to test data nor the labeling function of poisons. Moreover, the online fairness attacks [Chang et al., 2020; Van et al., 2021] can only 'select' training samples as poisons based on a fairness metric. Such attack is not targeted, since the attacker could not specify a demographic group to attack. Moreover, we observed that online fairness attacks on large models are extremely slow due to selection process, and does not necessarily generate a visually fair poisoned dataset, which make the attack less effective but more detectable. In contrast, our proposed cross-attribute attack directly manipulates the demographic information of poisoned data as the attacker desires, which drastically increases the targeted attack effect. We require the manipulation of poisoned data to be imperceptible, which guarantees a fair-look of the poisoned dataset and makes the attack less detectable.

Data Poisoning. To launch traditional poisoning attacks, the prerequisite of generating poisoned samples is to use the targeted test instances [Chen et al., 2017; Shafahi et al., 2018; Huang et al., 2020; Geiping et al., 2020]. That is, the attacker has to access targeted test samples at first place, then apply specific poisoning strategies, such as placing perturbed fake images geometrically [Shafahi et al., 2018; Huang et al., 2020] close to the clean test images or plugging in backdoor trigger in test samples [Chen et al., 2017]. Moreover, traditional targeted attacks focus on attacking targeted samples individually: after training the model on the poisoned data, the decision boundary near the targeted test sample is locally distorted (sample-level) to cause a wrong prediction [Chen et al., 2017; Shafahi et al., 2018; Huang et al., 2020]. Meanwhile, clean-label and test-datafree unlearning poisoning was studied in [Huang et al., 2021; Fowl et al., 2021]. In contrast, our attack is group-level and test-data-free : we propose to inject test-data-irrelevant perturbed samples to the training dataset to deform the decision boundary statistically. As such, the attacker needs no access to test data and its attack effect automatically generalizes the entire targeted demographic group rather than just a selected test sample. The test-data-free and group-level design fundamentally distinguish our attack from existing targeted poisoning attacks (including backdoor attacks).

3 Problem Statement

Notations. In an FAR application, a face dataset is denoted by DN = {(x1, y1), (x2, y2), ..., (xN, yN)} ∈ X × Y, where N is the total number of images, X refers to the input image space and Y represents the face attribute space. Technically, a dataset could be divided into a training set Xtrain and a test set Xtest. Our goal is to train a face attribute classifier fθ who predicts a face attribute y for a given face image x. However, face images are usually characterized by a set of sensitive demographic attributes such as gender or race. Without loss of generality, we consider C demographic attributes for all faces and denote them as an attribute list A = {a(1) , ..., a(C) }. Given a face image x(i) , its demographic labels are defined as A(x(i) ) = {e(i,1) , ..., e(i,C) }. For instance, we consider one specific demographic attribute for each FAR application, where we denote the demographic label as e(c) . Finally, to properly measure the bias of the models, we adopt the commonly used demographic parity ϕD and equalized odds ϕE [Hardt et al., 2016].

Threat Model. We assume the worst-case threat model, where the attacker can access all information of the attacked face attribute classifier (e.g., architecture, parameters) and its training specification (e.g., learning rate, batch size). Moreover, we aim at designing a more dangerous attacker that generates poisons even without any knowledge about the test data Xtest (no access to test data). Also, the attacker is crossattribute: perturbing demographic information w.r.t. a(c) ∈ A to corrupt face attribute predictions w.r.t. y ∈ Y. Finally, our attack is clean-label: the labels of the poisoned data remain correct and clean as it appear to human eyes.

4 Fairness Attack and Defense

4.1 Crafting Cross-attribute Poisons

How can the attacker bias the classifier in terms of a demographic attribute (e.g., gender) when the model is originally designed to predict an orthogonal face attribute (e.g., attractiveness)? Consider an attacker aiming to bias the model against smiling males, and increase gender inequality in the FAR application. In this case, we define the smiling male group as the attacked demographic group (Xatt). To bias the model, the attacker pulls the decision boundary towards the attacked demographic group by perturbing the images from the attacked demographic group away from their original data domain towards another data domain of a different demographic group (defined as the reference demographic group Xref ). To generate cross-attribute poisons, we present two different methods.

Domain-level perturbation. The first method is domainlevel: the attacker treats each demographic group as a separate data domain and will partially move the data domain of the attacked demographic group away from its original location. In other words, the attacker tries to perturb the domain of Xatt towards Xref by reducing the distance between these two domains. To measure the domain distance, we use the maximum mean discrepancy (MMD) distance. The MMD estimates the domain distance between two data domains using samples drawn from them [Gretton et al., 2012]. Given two data domains P and Q, MMD DMMD is defined as: DMMD = supk∈H (Ex∼P[k(x)] - Ex∼Q[k(x)]), where k is a function (kernel) in the reproducing kernel Hilbert space H. In our implementation, the expectation is simplified by using the latent representations of the drawn samples from two different demographic groups as in [Yue et al., 2021]. k is realized using Gaussian kernel [Yue et al., 2021], i.e., k(xi, xj) = exp(- ∥xi-xj∥2 γ ). As such, the MMD between Xatt and xref could be simplified as:

where fe is the feature extractor of the classifier fθ. xatt and xref are face images sampled from the attacked demographic group Xatt and the reference demographic group Xref . To partially perturb the data domain of Xatt towards the data domain of Xref , the attacker perturb xatt ∼ Xatt with δ by reducing the MMD distance. Since δ is independent from xref , DMMD is thus reduced to:

DMMD(δ) =

Instance-level perturbation. In addition to the domainlevel perturbations, we present another instance-level perturbation by increasing the specificity of the attacker: the attacker solves the optimal perturbation individually for each face image from the attacked demographic group. To generate instance-level perturbations, an additional neural network is trained to extract the demographic information of face images. In this way, the attacker's manipulation on demographic information could be more efficient. For a better understanding of the attack model, we propose to split a neural network into two sub-networks, a feature-extracting encoder fe (which is already used in Equation 1), and a feature classifier fy or fa (for face attribute y or any other demographic attribute a(c) ), as shown in Figure 2. To train the demographic attribute classifier fa, we initialize a classification module fa for the original face attribute classifier fθ = fy(fe). fa is the classifier w.r.t. the demographic attribute that the attacker is interested in (i.e., a(c) ). Similar to train fy, the parameter of fac is optimized using the available training dataset Xtrain:

minfa E(x,y)∼Xtrain

[l(fa(fe(x)), e(c) )],

(3)

where e(c) is the demographic label of the face images for

demographic attribute a(c) .

After deriving fa, the attacker could perturb the face images from the attacked demographic group towards another nearby demographic group. To measure the distance between individual images from different demographic groups, we use KL-divergence:

where xatt is a face image sampled from the attacked demographic group Xatt, and xref is a reference face image from reference demographic group Xref .

Crafting Cross-attribute Poisons. With both perturbation strategies introduced above, we present the objective for generating cross-attribute poisons. Note that both Equation 2 and Equation 4 only considered the perturbation along the demographic attribute dimension. However, without controlling the perturbation along the face attribute dimension, it is likely that the resulted poisons could distort the decision boundary in an undesired direction and suppress the attack efficiency. For instance, the attacker could wrongly perturb a smiling male image to be a unsmiling female without controlling the perturbation along the face attribute dimension. In this case the poison does not directly contribute to the objective of attacking smiling male group. Therefore, we propose to jointly consider the perturbation constraints from both the face attribute dimension and the demographic attribute dimension:

· Domain-level:

• Instance-level:

Note that a scaling factor α is introduced for both Equation 5 and Equation 6 to control the trade-off between the constraints from the demographic attribute and face attribute. In our experiments, we implement the iterative projected gradient descend [Madry et al., 2017] to approximate the optimal solutions to both equations. We highlight that in both poisoning strategies, the attacker does not require the access to test data to generate poisons, which differentiates our framework from previous studies on data poisoning [Chen et al., 2017; Shafahi et al., 2018; Huang et al., 2020]. Moreover, the proposed attacker is clean-label and does not manipulate the labels of the poisoned data. The poisoned data still has correct labels for both face attribute and the attacked demographic attribute as it appears to eyes. More importantly, if the original face dataset is fair, the poisoned dataset is still visually fair.

After solving Equation 5 and Equation 6 for the selected attacked demographic group Xatt and the reference demographic group Xref , the attacker obtains a poisoned dataset Z and launches the attack.

4.2 Poisoning Attack on Fairness

For face attribute classifiers in real-world FAR applications, transfer learning is a commonly used methodology to update the parameters of the ML models [Shafahi et al., 2018]. Therefore, inspired by [Shafahi et al., 2018], we use transfer learning to simulate the attack process:

where X′ = Xtrain ∪ Z is the poisoned dataset. Since the attacker is clean-label and does not change the ground truth labels of the poisoned data, the poisoned dataset X′ looks fair to human eyes if the original training set is fair. During the poisoning phase, however, X′ is severely biased w.r.t. the demographic attribute a(c) from the classifier fy's perspective.

4.3 Defending against the Fairness Attack

Given the threat above, how to defend against the fairness attack? In this section, we propose a defense named robustfair training to enable the robustness of fairness for the face attribute classifiers.

To obtain robust fairness, the robust-fair training tries to train a fair model under a worst-case adversarial distribution. The adversarial distribution is crafted to be least-fair (or most unfair) w.r.t. a fairness metric (e.g., ΦD or ΦE). We formulate this process as a minimax game between a fairness adversary and a fairness booster. The fairness adversary tries to generate fairness adversarial examples by biasing the training samples with perturbations w.r.t. a fairness metric, whereas the fairness booster tries to learn a fair model via a fairness-aware loss even if the training data is deliberately biased. Formally, the minimax game is defined as

where C denotes the number of demographic groups for the FAR application and Mj denotes the number of training samples of the j-th demographic group. η(j,i) is the bias perturbation deliberately crafted by the fairness adversary to bias the training sample x(j,i) . ˆΦ∗ is the differentiable version of any fairness notion (ΦD or ΦE) (in Section 3). Note that the 'training samples' x(j,i) in Equation 8 is sampled from the poisoned dataset X′ . As such, the defense mechanism is attack-agnostic: the robustness of fairness would still be improved no matter whether there exists poisoned data or not. Moreover, the bias perturbation η is different from the perturbation δ defined in Section 4.1 and 4.2. δ is the perturbation generated during the poisoning phase to bias the model against the attacked demographic group. In contrast, η is generated during the defense phase, so that the classifier learns to combat the bias introduced by adversarial examples.

The maximization in Equation 8 specifies the goal of the fairness adversary is to generate the fairness adversarial examples with a specific budget ϵ 2 . The budget ϵ avoids infinity solutions to the maximization problem. To obtain η(j,i) , the maximization is solved using projected gradient descent (PGD). Regarding the computation of the gradients for the perturbations, we highlight that ˆΦ∗ is the differentiable version of ΦD or ΦE. When computing ΦD or ΦE, the argmax operation will be applied to the output logits to produce the predictions, which makes ΦD or ΦE non-differentiable. Therefore, in our implementation, we compute soft scores for these metrics by plugging in the normalized output logits instead of the predictions to enable the PGD.

The minimization in Equation 8 implies that the fairness booster increase model fairness by minimizing a fairnessaware training loss. Herein, the fairness-aware training is enabled by re-weighting each training sample inversely proportional to the data frequency (IDF) within each demographic group [Han et al., 2021]. That is, the empirical risk is averaged within each demographic group, then the group-level risk is further averaged over the number of demographic groups. Moreover, note that Equation 8 is evaluated over the perturbed training samples x(j,i) + η(j,i) . Since the perturbation η(j,i) is generated in a way such that the selected fairness metric would be maximized, optimizing the fairnessaware loss over the fairness-adversarial examples is essentially performing robust-fair training. With robust-fair training, the face attribute classifier is trained to be fair under a worst-case adversarial scenario where the training data distribution is least-fair. To this end, the classifier learns to combat the bias introduced by the skewed training distribution and becomes robustly fair.

2 Note the same ϵ was used to generate poisons. As such, the power of the fairness attacker and defender is balanced for a fair comparison.

5 Experiments

5.1 Dataset and Experimental Setup

We use the large scale CelebA [Liu et al., 2015] 3 and FairFace dataset [Karkkainen and Joo, 2021] for our experiments. Due to space limit, we chose attractiveness, smiling and age as our predictive face attributes. As mentioned in [Shen et al., 2017], attractiveness prediction is highly challenging due to its subjectivity whereas smiling detection is simpler since smiling or not is easier to judge. Therefore, we pick these two FAR applications of different difficulty to understand the vulnerability of demographic fairness in FAR applications. To test the generality of our method across different datasets, we choose age detection, as only the age label is available in both Fairface and Celeba. Without loss of generality, we choose gender as the demographic attribute. To simulate the attack on fair classifiers and demonstrate the bias introduced by various fairness attacks, we sample two fair subsets to train and test the victim models. To maintain the size of the training set and guarantee the 'fair look' of the training set, when inserting the poisons into the training set, we also inject the same number of clean images from the other demographic groups. To measure model performance, we use fairness metrics ΦD and ΦE as well as the accuracy metric AB (balanced accuracy). For reproducibility, all details (e.g., experimental setup, attacker specification) and code are uploaded within the supplementary materials of this submission.

3 dataset link: https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

5.2 Baselines

Baseline Architecture. Without the loss of generality and due to the space limit, we pick LightCNN [Wu et al., 2018], ResNet [He et al., 2016] and VGGNet [Simonyan and Zisserman, 2014] as the network architectures to perform all FAR applications. Moreover, for our attack, we build the demographic attribute classification module fa (defined in Section 4) with exactly the same structure as the modified classification module for all baseline architectures. The face attribute classifiers as well as the demographic attribute classifiers for all three FAC applications are trained using our sampled fair training sets, respectively.

Baseline Poisoning Attacks. As discussed in Section 1 and Section 2, it is impractical to implement traditional data poisoning algorithms [Chen et al., 2017; Shafahi et al., 2018;

Huang et al., 2020] and the test-data-required fairness attack [Mehrabi et al., 2020] for comparison in our setting. Firstly, our attack is at group-level while the standard poisoning is at instance-level. Second, our attack does not use test data information that is used by existing fairness poisoning work [Mehrabi et al., 2020]. To build baseline comparisons, we adopt the baseline fairness poisoning attacks (Adv. sampling, Adv. labeling, Adv. Hard) and the fairness poisoning algorithm (Fairness Poisoning) in [Chang et al., 2020].

5.3 Evaluation Results

Attack Evaluation. The results on attacking fairness of face attribute classifiers are shown in Table 1. In this set of experiments, 5% of the training data is poisoned. We selected 5% because the attack efficacy of all baseline methods would be too marginal when fewer poisons are used. We observe that both our domain-level attack and instance-level attack outperform all baseline methods in terms of corrupting the fairness of the face attribute classifiers. For instance, in attractiveness prediction, the equalized odds of LightCNN-9 is raised from 0.0392 to 0.3272 (domain-level) and 0.3896 (instancelevel). Similar trends are observed for other FAR applications and datasets. In addition, we also perform a robustness study to investigate two important aspects of the proposed fairness attack, namely the number of poisons and the perturbation radius ϵ (the results of robustness study are in Appendix).

Defense Evaluation. The results on defending the proposed attack are reported in Table 2. To show the efficacy of the proposed robust-fair training method, we doubled the number of poisons to attack the fairness of the face attribute classifiers. It is observed that the defense could effectively reduce the bias introduced by the proposed fairness attack. For instance, in attractiveness prediction, the equalized odds of the LightCNN classifier was 0.5432 after the attack (instance-level). With robust-fair training (our defense), the attacked classifier becomes more fair with an equalized odds of 0.2456. The improved fairness as well as the model accuracy could also be observed for other network architectures and FAR applications. However, we acknowledge that the model's fairness after defense is still worse than the unattacked model, which indicates that future work is needed to further increase the robustness of the fairness in FAR applications.

6 Conclusion

In this work, we present a novel poisoning attack against demographic fairness of face attribute classifiers. To the best of our knowledge, our work is the first to explore test-data-free, cross-attribute and clean-label poisoning attacks against fairness of face attribute classifiers at group level. Experiments on various FAR applications show that our method could easily bias the fairly trained models. Motivated by the observed vulnerability of fairness, we further propose an efficient defense mechanism to increase the robustness of the fairness. Through this work, we stress the significance of studying robustness of fairness for AI models, as the fairness obtained with traditional fair algorithms could be vulnerable under fairness attacks. Therefore, decent efforts should be made to improve the robustness of the fairness of ML/AI models.

Acknowledgments

This research is supported in part by the National Science Foundation under Grant No. IIS-2202481, CHE-2105005, IIS-2008228, CNS-1845639, CNS-1831669. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.

Ethics Statement

Understanding the robustness of fairness is extremely important in identity-sensitive applications. In addition to fair ML and AI algorithms, we argue that it is equally important to develop robust fairness in identify-sensitive applications, as fair models could be vulnerable under the fairness attack and fair ML algorithms do not automatically enable the robustness of the fairness. Ignoring the robustness of demographic fairness in ML could harm the social equality. Therefore, this work investigates the robustness of fairness from both attacking and defense perspective, such that robust fair models could be developed for identity-sensitive applications.

Our work serves as a prototype and baseline for future studies on robust fair machine learning. In this work, we designed a prototype poisoning attack against fairness of visual models. Motivated by the vulnerability of the attack, we further present a novel defense mechanism to increase the robustness of the fairness. We stress that the core intention of this work is to inspire future work on the robustness of demographic fairness in DNNs. We believe both our attack and defense could be used as baselines for future work on robust fair ML and AI systems.