Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models

1 Introduction

Watermarking, originally designed for digital media [Petitcolas et al., 1999], has been recently applied to the protection of Deep Learning (DL) models. This technique enables the ownership verification of DL models by embedding watermarks into the Deep Neural Networks (DNNs). One promising approach is data-poisoning watermarking [Zhang et al., 2018; Adi et al., 2018; Fan et al., 2019; Zhang et al., 2020], which modifies the model to give pre-defined output on some carefully-crafted watermark samples. Then the owners with only black-box accesses to the suspicious model can extract such watermarks by querying the model.

∗Tianwei Zhang is the corresponding author.

Similar to digital watermarking, a satisfactory datapoisoning watermarking mechanism must satisfy two requirements. The first one is functionality-preserving: the embedded watermarks should not affect the performance of the target model on normal samples. The second is robustness, where the watermarks cannot be removed with common model transformation, e.g., fine-tuning, model compression, etc. Even if the adversary knows the target model is watermarked, he has no means to remove the watermarks if he does not know the details of watermark samples.

Whether existing watermarking solutions are practically robust is widely challenged. A quantity of works try to invalidate these watermarking mechanisms and can be classified into two categories. The first strategy is to detect the verification queries and manipulate the responses. Some works [Namba and Sakuma, 2019; Aiken et al., 2020] applied the strategies of backdoor attack detection to watermark detection, as some watermark designs are based on the backdoor techniques. However, these solutions fail to defeat backdoor attacks (as well as watermarks) with complex patterns [Liu et al., 2019a; Tan and Shokri, 2019]. They are not applicable to watermark schemes based on other techniques (e.g., adversarial examples, out-of-distribution samples) either.

Another strategy is to directly remove the watermarks via model transformation. Although most watermarking solutions claim to be robust against various model transformations (e.g., fine-tuning, compression), recent works [Chen et al., 2019; Shafieinejad et al., 2019; Liu et al., 2020] attempted to break such statement by introducing advanced fine-tuning methods with data augmentation, sophisticated loss function, etc. Unfortunately, these works suffer from significant limitations. For instance, [Shafieinejad et al., 2019] requires original training samples to fine-tune the watermarked model. [Chen et al., 2019] needs to know the type of the watermark to adjust the fine-tuning parameters. These assumptions make the attacks less unrealistic or practical.

It seems not promising to just use model fine-tuning for effective watermark invalidation. As a result, we propose a new watermark removal attack from a different direction: inference sample transformation. The key insight of our solution is that watermark samples are less robust than normal samples. Hence, we can design a preprocessing function to compromise the verification output while not affecting the normal output. Specifically, our novel function PST consists of a series of transformations (scaling, embedding random imperceptible patterns, spatial-level transformations). We further fine-tune the watermarked model with unlabelled out-ofdistribution data to maintain the model's performance.

We are the first to explore the possibility of inference sample transformation for watermark removal. Our solution has much fewer requirements with more reasonable assumptions than prior works: it does not need any samples from the original training set (as in [Shafieinejad et al., 2019]) or even following the original distribution (as in [Chen et al., 2019; Liu et al., 2020]). It is unified and does not need to know the adopted watermark mechanism for hyperparameter adjustment (as in [Chen et al., 2019]). Its effectiveness is also confirmed by extensive evaluations.

Given the severity and practicality of our attack, we further introduce a defense methodology to enhance the robustness of watermarks. Inspired by the data augmentation technique which can enhance the model performance and generalization, we propose to augment the watermark samples with various transformations during the embedding process. In this case, the verification results remain correct even the adversary performs different preprocessing over the inference samples. Evaluations indicate this strategy can significantly improve the robustness of existing watermarking mechanisms.

2 Background and RelatedWorks

2.1 Watermarking DNNModels

Existing watermarking schemes for DNNs can be classified into two categories. The first category is parameterembedding watermarking [Uchida et al., 2017; Rouhani et al., 2019], which embed watermarks into the parameters without decreasing the model's performance. Verification of such watermarks require white-box access to the target models, which may not be achievable in some scenarios.

We are more interested in the second category, datapoisoning watermarking [Le Merrer et al., 2019; Zhang et al., 2018; Adi et al., 2018]. These solutions take a set of carefully crafted sample-label pairs as watermarks and embed their correlation into DL models during the training process. We formally define the data-poisoning DNN watermarking scheme and illustrate the necessary properties.

Definition 1. A DNN watermarking scheme with data poisoning is a tuple of probabilistic polynomial time algorithms (WMGen, Mark, Verify), where

• WMGen generates a set of watermarks W = {(xi, yi)}ni=1 , in which xi is a secret watermark input and yi is the corresponding verification label.
• Mark embeds the watermarks into a DL model f and outputs the watermarked model ̂ f such that ̂ f(xi) = yi for ∀ (xi, yi) ∈ W.
• Verify sends {xi}ni=1 to a DL model ˜ f and obtains the predictions {y˜ f,i }ni=1 . If the probability that y˜ f,i equals yi for i ∈ [1, n] is larger than a predefined value τ , Verify outputs 1. Otherwise it outputs 0.

Let P̂ f,D be the prediction accuracy of ̂ f on the normal samples following the data distribution D; P̂ f,W be the accuracy of ̂ f on the verification watermarks W, i.e.,

A data-poisoning watermarking scheme should have the following basic properties: (1) Functionality-preserving: P ̂ f,D should be identical with Pf,D. (2) Robustness: Verify outputs 1 even if ̂ f is slightly transformed to a different model ˜ f using common techniques such as fine-tune or model compression, i.e. P˜ f,W > τ .

Various state-of-the-art data-poisoning techniques have been proposed to embed watermarks into DNN models. These methods can be classified into three categories:

• Perturbation-based : the samples are generated by slightly perturbing normal samples, where the difference between the two types of samples is bounded. They can be generated using the adversarial frontier stitching [Le Merrer et al., 2019] or GAN-based techniques [Li et al., 2019].
• Pattern-based : the samples are generated by embedding a certain pattern into normal images using backdoor techniques [Adi et al., 2018; Zhang et al., 2018].
• Out-of-distribution (OOD) : the samples are randomly selected from the internet and totally different from the normal ones [Adi et al., 2018].

2.2 AttackingWatermarking Schemes

We consider a threat model where the adversary has whitebox accesses to the target model and is able to alter the model. He aims to break the property of Equation (2) but still maintain the property of Equation (1). A good removal attack must meet the following requirements.

• Comprehensive: the attack should be able to invalidate all existing state-of-the-arts watermarking solutions, including perturbation-based, pattern-based, and OOD.
• Watermark-agnostic: the adversary has no knowledge about the employed watermarking scheme, including the technique and watermark type. He has no access to the original training samples, or the distribution.
• Efficient: the adversary should be able to remove the watermarks in a lightweight manner, i.e., with much less computation cost than training a model from scratch. Otherwise, he will lose the motivation of stealing the target model, and just train his own copy.

Existing watermark attacks fail to satisfy all these requirements. These solutions can be classified into two categories. The first approach is to distinguish the watermark samples from normal samples during the inference process, and then alter the verification results [Namba and Sakuma, 2019; Aiken et al., 2020]. However, these attacks are not watermark-agnostic, as they require the entire training dataset. They are not comprehensive as they only work for simple pattern-based watermarks.

The second is to remove watermarks from the model using fine-tuning techniques [Chen et al., 2019; Shafieinejad et al., 2019; Liu et al., 2020]. They carefully designed learning rate schedule and synthesized fine-tuning samples to make the watermarked model forget the watermark samples. Unfortunately, these approaches also suffers from several fatal limitations: they are not watermark-agnostic, as they need to access the original training data [Shafieinejad et al., 2019; Chen et al., 2019; Liu et al., 2020]. [Chen et al., 2019] also needs to know the watermark type for hyperparameter adjustment. [Liu et al., 2020] is not comprehensive to cover the OOD watermarking scheme.

3 Proposed Attack

Figure 1 illustrates the workflow of our proposed attack. The core of our solution is a powerful function, Pattern embedding and Spatial-level Transformation (PST). Given a watermarked model ̂ f, the adversary first preprocesses some OOD data samples with PST, and uses them to fine-tune ̂ f for a few epochs. Then he can serve the fine-tuned model ˜ f online. During inference, he applies PST for each sample and feeds it to ˜ f. The model output of a normal sample will remain the same while the watermark ones will be altered.

Key Insight. Our approach is based on the robustness gap between watermark and normal samples. Intuitively, a datapoisoning watermarked model is trained to memorize the relationships between {xi}ni=1 and {yi}ni=1 . During the verification, the watermarked model will predict yi given xi. However, such memory may be fragile because yi has to be different from the ground truth of xi. Based on this, PST is expected to preprocess watermark samples, and make them unrecognizable by ̂ f. To maintain the model's performance over normal samples, we leverage PST to preprocess some OOD samples and fine-tune the watermarked model.

3.1 PST

Our PST function consists of three modules: (1) scaling is to resize an input sample to a fixed size; (2) imperceptible pattern embedding introduces human-unnoticeable perturbations to reduce the impact of abnormal pixels and patterns in the watermark samples; (3) spatial-level transformations further destroy the special spatial shapes introduced by the watermarking scheme. Detailed steps are illustrated below.

Step 1: Scaling. Given a sample x of an arbitrary size, PST first scales it to a fixed size. Specifically, PST chooses the bicubic interpolation [Meijering, 2002] for scaling to reduce the correlation between watermark samples and labels. This operation Bicubic interpolates pixels by considering a small square neighborhood (4×4 in our experiments): x = Bicubic(x, β), where β is the scaling paramater. We choose this bicubic interpolation because samples with bicubic interpolation have fewer interpolation artifacts and often outperform the ones with bilinear or nearest-neighbor interpolation. Thus, this step can better preserve the performance of ̂ f on normal samples.

Step 2: Imperceptible Pattern Embedding. Besides the bicubic interpolation, PST embeds random imperceptible patterns into the scaled watermark samples to further affect the prediction of ̂ f. Due to the imperceptibility property, the embedded pattern will not impact the performance of ̂ f over normal samples. Since the adversary has no prior knowledge about the watermarking scheme and watermark samples, we design a random yet easily identifiable pattern to alter the memory of ̂ f about the watermark samples.

In particular, we adopt a random grid median filter MedianFilter to generate the desired imperceptible pattern. During the embedding, PST first selects a set of random rows and columns with the same interval size v. For each pixel xi,j in the selected rows and columns, it is replaced by the pixel with the median value in the neighborhood. The pattern size is controlled by the interval size v: a large v can lead to a small number of the selected columns and rows, and a more imperceptible pattern. This median filter can be replaced by other types of filters such as the maximum filter.

Step 3: Spatial-level Transformations. Although the above pixel-level transformations can affect the watermark samples with small perturbations, they are insufficient to remove large perturbations such as pattern-based watermarks. To this end, we propose to adopt both linear and nonlinear spatial-level transformations to further compromise the effects of watermark samples. We integrate two different transformations and use a parameter γ to uniformly control the strength of each one. Specifically, Let (i, j) be a pixel of x and (˜i,˜j) be the corresponding transformed pixel. For each operation, we require the distance between each dimension of the input and output pixels to be lower than γ, i.e.,

Our PST first adopts random affine transformations (Affine) over the pattern-embedded x. An affine transformation is linear and can be formalized as a 2×3 matrix, which preserves points, straight lines, and planes of samples. For a pixel (i, j), it calculates

For example, the matrix of the translation transformation in the horizontal direction is [1 0 φ 0 1 0]. To meet the distance constraint, we randomly choose φ such that |φ| ≤ γ for all the columns. We can combine multiple affine transformations

together such as rotation and translation.

Besides affine transformations, we also adopt an elastic transformation Elastic to further modify the samples, which is a nonlinear transformation that produces random elastic distortions. It first generates random displacement fields that are convolved with a Gaussian distribution of standard deviation σ. The displacement fields are then multiplied by a scaling factor α that controls the intensity of the deformation. We adjust the standard deviation σ and the scaling factor α to restrict the displacement to be smaller than γ.

3.2 Attack Pipeline

Algorithm 1 describes the details of the two-stage attack:

Offline fine-tuning (Lines 17-21). The adversary first adopts PST to fine-tune ̂ f to enhance the model performance. Different from prior works [Chen et al., 2019; Shafieinejad et al., 2019; Liu et al., 2020] which require the original labeled training data or distribution, our method can select an arbitrary set of unlabeled and out-of-distribution samples {x′i }. The adversary first calculates the corresponding label of each sample using ̂ f as an oracle. Then he establishes a dataset {(gpst (x′i ), ̂ f(gpst (x′i )))} to fine-tune the model ̂ f for a few epochs to reach the desired performance. This fine-tuning process takes very short time to complete, and can effectively adjust the noise and perturbations introduced by PST.

Online inference (Lines 1-7). For each sample x, the online model returns ˜ y = ˜ f(gpst (x)). When the owner executes the Verify function to query the model with a watermark sample xi, ˜ yi is different from yi with a high probability, due to the effectiveness of gpst and ˜ f. Thus, the similarity between {yi}ni=1 and {˜ yi}ni=1 would be smaller than τ , which can prevent the model owner from identifying the watermarks. In contrast, ˜ f will give the correct results for normal samples x.

4 Experiments

4.1 Experimental Setup

Datasets and DNN models. Our attack is model-agnostic and dataset-agnostic. Following existing works, we conduct experiments on the CIFAR datasets with the same number of training samples. We evaluate our attack on the ResNet model [He et al., 2016]. We adopt a batch size of 100 and the learning rate decay with an initialize value of 0.2.

Watermark samples. We evaluate our attack on four types of state-of-the-art watermark samples: 1) Perturbation: the watermark samples are generated by adding imperceptible perturbations on normal samples. They are close to the frontiers of the models [Le Merrer et al., 2019]. 2) Patch: the watermark samples are created by embedding a specific patch into the normal samples [Zhang et al., 2018]. 3) Content:

Parameters: β, v, γ, ̂ f, {x′i }, {xi, yi}ni=1 1 Function Verify({xi, yi}ni=1): 2 foreach xi ∈ {xi}ni=1 do 3 ˜ yi ← ˜ f(gpst (xi)), where ˜ f ← Fine-tune; 4 if d({yi}ni=1 , {˜ yi}ni=1 ) < τ then 5 return 1 6 else 7 return 0 8 Function gpst(x): 9 x ←Bicubic(x, β); 10 Randomly selected rows and columns with interval v; 11 foreach xi,j in the selected rows and columns do 12 xi,j ← MedianFilter(xi,j ); 13 x ←Affine(x, γ); 14 x ←Elastic(x, γ); 15 ˜ x ←Bicubic(x, 1β ); 16 return ˜ x 17 Function Fine-tune(̂ f , {x′i }): 18 foreach x′i ∈ {x′i } do 19 y′i ← ̂ f (gpst (x′i )); 20 ˜ f ← Train( ̂ f , {gpst (x′i ), y′i }); 21 return ˜ f

the watermark samples are generated by embedding the word "TEXT" into the normal samples [Zhang et al., 2018]. 4) OOD: the watermark samples are randomly selected from unrelated datasets [Adi et al., 2018].

Figure 2 (first row) illustrates an example of a normal sample and the corresponding four types of watermark samples. Following the implementation in [Adi et al., 2018; Le Merrer et al., 2019], we set the number of watermark samples as 100. We set τ as 60% for both CIFAR10 and CIFAR100 when perturbation-based watermark samples are involved. Otherwise, τ is 20% for CIFAR10 and 10% for CIFAR100. We train models from scratch to embed the watermarks. Each training process is stopped after 60 epochs if the model accuracy on watermark samples is 1. Otherwise, we continue the training till all watermarks are embedded.

Implementation Details. We uniformly set the parameters for all datasets and watermarks, which indicates our attack does not require any prior knowledge of the watermarking mechanisms. In particular, for PST, we set β as 5 and the size of the scaled images is 160×160. We set the interval size as 5 to generate the imperceptible pattern. We set the displacement bound γ in spatial-level transformations as 15. We use both in-distribution and out-of-distribution unlabeled samples in the fine-tuning phase. Specifically, the in-distribution samples are generated from the corresponding datasets using data augmentation techniques. The out-of-distribution samples of CIFAR10 (resp. CIFAR100) are randomly selected from CIFAR100 (resp. CIFAR10). We use 25000 unlabeled samples and fine-tune the watermarked models for 10 epochs.

4.2 Effectiveness of Our Attack

We consider two strategies: the first one is to just use input preprocessing to remove watermarks. The attacker can choose PST transformation, or PST without imperceptible pattern embedding (denoted as ST). For comparisons, we also implement some state-of-the-art transformations for adversarial example mitigation as baselines: BdR [Xu et al., 2017], Shield [Das et al., 2018], PD [Prakash et al., 2018], FD [Liu et al., 2019b], and GB [Hendrycks and Dietterich, 2019]. Since they are proposed to remove small perturbations in adversarial examples, we adjust the parameters of these methods to affect samples with large patterns till their accuracy is lower than 70% (resp. 50%) on CIFAR10 (resp. CIFAR100).

The second strategy is the end-to-end attack with both PST and fine-tuning. We compare it with the black-box watermark removal attack (FT) in [Shafieinejad et al., 2019] which only fine-tunes the watermarked models with unlabeled samples. FT takes a set of unlabeled samples as input and obtains their labels using the watermarked model as an oracle. Then, it fine-tunes the watermarked models for 10 epochs with the new sample-label pairs. To be comprehensive, we implement the two solutions with in-distribution data for fine-tuning as well (Proposed-In and FT-In).

Since these input transformation functions introduce high randomization, we run each experiment five times to obtain the statistical results. We calculate the average performance of the attacks on normal samples PD to reflect their usability. We select the minimum accuracy of the attacks on watermark samples PW to quantify their effectiveness, which is efficient to reflect the threat an attack can bring. The experimental results lead to the same conclusion with the average accuracy.

The experimental results are illustrated in Table 1. For the first strategy, we observe that perturbation and OOD watermarks are vulnerable to all kinds of transformations. The adversarial example transformations (except FD) and PST (also ST) can easily remove these two types of watermarks on both CIFAR10 and CIFAR100. However, adversarial example transformations are not helpful in removing pattern-based watermarks. Although GB can remove Content watermarks on CIFAR10 and CIFAR100, all adversarial example transformations can not affect Patch watermarks even PD is extremely low. In contrast, PST can significantly reduce the accuracy of the watermarked model on all types of watermarks without heavily affecting PD. ST is slightly worse than PST, but is still much better than the other transformations.

For the second strategy, we observe that although FT-In and FT-Out have almost the same performance on normal samples, they cannot remove watermarks, which was confirmed by the results in existing watermarking papers [Adi et al., 2018; Zhang et al., 2018]. Our attack (Proposed-In and Proposed-Out) can remove all watermarks and preserve acceptable PD values. Besides, compared with PST, the proposed fine-tuning process can not only improve the performance of the models on normal samples, but also further cleanse the watermarks.

4.3 Impact of Parameters

Varying the number of unlabeled samples. We evaluate the performance of our attack with different numbers of unlabeled samples on CIFAR10. Figure 3 shows the performance of our attack on Patch watermarks. We observe that there exists a trade-off: more fine-tuning samples can increase the performance, which is harder to obtain. If the adversary has enough unlabeled samples, he can obtain a high-quality model with the same performance as the watermarked one. Beside, the success rates of our attack are preserved and even slightly increases with more unlabeled samples, which indicates its severe threat to existing watermarking schemes.

Varying γ. Figure 2 visually illustrates the transformed output of normal and watermark samples with γ = 10 (second row) and γ = 20 (third row). Two observations are made from this figure. First, the distortion caused by PST increases as the displacement bound γ increases. Second, the samples are randomly preprocessed, which indicates that our attack can be adopted for various watermarking schemes with different settings. We also measure the performance of our attack with different γ values on CIFAR100. The experimental results are presented in Figure 4. We observe that due to the effect of the distortion, the model accuracy decreases (except Content) and the success rate increase on the four types of watermarks with an increased γ.

5 Watermark Augmentation for Robustness Enhancement

As discussed in previous sections, inference preprocessing with model fine-tuning can effectively remove state-of-theart watermarks from the protected model. So we aim to seek for new solutions to enhance the robustness of existing watermarking schemes. Inspired by the data augmentation techniques that can improve the generalization ability of DNN models, we propose watermark augmentation to strengthen the watermarks during the embedding process.

Methodology . A variety of transformations [Cubuk et al., 2019] have been designed for data augmentation. Given a specific watermarking scheme with a set of watermarks, we apply those transformations over each watermark sample to augment the watermark set. Then we follow the Mark procedure to embed the original and transformed samples into the model. With this watermark augmentation method, the protected model is expected to recognize the watermark samples even the model and samples are perturbed by the adversary.

Evaluations . We leverage three augmentation transformations: ShearX, Rotate, and PST. Each function has different strength values to transform images. At each iteration, we randomly sample an augmentation strength value and apply the selected transformation to the watermark samples, before embedding them to the model.

We measure the effectiveness of the watermark augmentation strategy against our PST on CIFAR10 and CIFAR100. Table 2 illustrates the experimental results of the three augmentation defenses. We observe that watermark augmentation can significantly improve the robustness of the watermarks (expect Rotate on CIFAR10) compared to the results without any defense in Table 1. Besides, watermark augmentation with PST is more effective than other transformations.

6 Conclusion and FutureWork

In this paper, we propose an efficient and practical attack to blindly remove watermarks in DL models. We introduce a novel preprocessing function PST, to transform samples for model fine-tuning and preprocess inference samples. Extensive experiments demonstrate that the proposed attack technique can remove various types of state-of-the-art watermarks without any prior knowledge about the watermarking schemes and labeled training samples. This indicates existing watermarking schemes still have high severity vulnerabilities. We also propose a new defense strategy to enhance the robustness of existing watermark mechanisms.

For the attack, we empirically and comprehensively validate our technique, following all existing relevant works. Unfortunately, we cannot formally prove the effectiveness of our solutions against all watermarking schemes. Formal and theoretical analysis about watermarks and attacks is an open and challenging problem. To our best knowledge, currently there are no works or attempts to address this problem yet. We will consider this as an important direction for future work. We will also evaluate our attack as a defense against adversarial examples and backdoors.

For the defense, evaluations show the model with augmented watermarks is more robust against simple attacks. The possibility of adaptive attacks targeting this defense is unknown and worth exploration. We hope our work can heat up the arms race between the model owner and plagiarist to inspire the designs of more advanced watermark schemes and attacks in the future.

Acknowledgments

This research was supported by the National Research Foundation, Singapore, under its National Cybersecurity R&D Programme (NCR Award Number NRF2018NCR-NCR0090001 and NRF2018NCR-NCR005-0001), Singapore Ministry of Education AcRF Tier 1 RS02/19 and 2018-T1002-069, the Singapore National Research Foundation under NCR Award Number NRF2018NCR-NSOE003-0001, NRF Investigatorship NRFI06-2020-0022, the National Natural Science Foundation of China under Grants 62072062 and U20A20176, and the Natural Science Foundation of Chongqing, China, under Grant cstc2019jcyjjqX0026.