---
title: "Enhancing the Transferability of Adversarial Examples with Random Patch"
authors:
  - "Yaoyuan Zhang"
  - "Yu-an Tan"
  - "Tian Chen"
  - "Xinrui Liu"
  - "Quanxin Zhang"
  - "Yuanzhang Li"
doi: "10.24963/ijcai.2022/233"
url: "https://ijcai.org/proceedings/2022/233"
pdf: "https://ijcai.org/proceedings/2022/0233.pdf"
venue: "Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence"
year: 2022
pages: "1672-1678"
track: "Main Track"
keywords:
  - "Computer Vision: Adversarial learning, adversarial attack and defense methods"
  - "Computer Vision: Recognition (object detection, categorization)"
---

# Enhancing the Transferability of Adversarial Examples with Random Patch

Yaoyuan Zhang, Yu-an Tan, Tian Chen, Xinrui Liu, Quanxin Zhang, Yuanzhang Li

*Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence*, pages 1672-1678. DOI: [10.24963/ijcai.2022/233](https://doi.org/10.24963/ijcai.2022/233)

## Abstract

Adversarial examples can fool deep learning models, and their transferability is critical for attacking black-box models in real-world scenarios. Existing state-of-the-art transferable adversarial attacks tend to exploit intrinsic features of objects to generate adversarial examples. This paper proposes the Random Patch Attack (RPA) to significantly improve the transferability of adversarial examples by the patch-wise random transformation that effectively highlights important intrinsic features of objects. Specifically, we introduce random patch transformations to original images to variate model-specific features. Important object-related features are preserved after aggregating the transformed images since they stay consistent in multiple transformations while model-specific elements are neutralized. The obtained essential features steer noises to perturb the object-related regions, generating the adversarial examples of superior transferability across different models. Extensive experimental results demonstrate the effectiveness of the proposed RPA. Compared to the state-of-the-art transferable attacks, our attacks improve the black-box attack success rate by 2.9\% against normally trained models, 4.7\% against defense models, and 4.6\% against vision transformers on average, reaching a maximum of 99.1\%, 93.2\%, and 87.8\%, respectively.

## Full text

## 1 Introduction

Many works [Szegedy et al., 2014; Kurakin et al., 2017a] have shown that deep neural networks (DNNs) are vulnerable to adversarial examples, i.e., input images perturbed by imperceptible noise can lead to inaccurate predictions of DNNs. The existence of adversarial examples raises concerns in security-sensitive applications, e.g., self-driving cars and face recognition. Studying adversarial attacks is helpful to evaluate and improve the robustness of different models [Goodfellow et al., 2015] to avoid future risks. Adversarial attacks are classified into white-box attacks and black-box attacks. Black-box attacks are more challenging and more practical in real-world applications. Compared to query-based black-box attacks [Ilyas et al., 2018; Shi et al., 2019], transfer-based black-box attacks [Liu et al., 2017; Zhou et al., 2018] are more realistic and flexible, which exploit the transferability of adversarial examples to execute black-box attacks.

∗Corresponding author.

Figure 1: Comparison of a previous state-of-the-art transfer-based attack (FIA) and our random patch attack (RPA). Adversarial examples are crafted on the source model (Vgg-16) to attack the target model (Resnet-50). Our RPA keeps models from capturing important features of the object and focusing on entirely irrelevant regions instead, while FIA makes models still focus on object-related areas.

![Image](https://ijcai.org/proceedings/2022/fulltext/233/image_000000_7031e1e4467535b80b08dcdf7ceac541dd3aefc0199304ff5fe2b0b67d2b01eb.png)

Many works attempt to enhance transferability by improving the optimization process [Dong et al., 2018] or by exploiting data augmentation [Xie et al., 2019] or by disrupting feature maps of intermediate layers [Huang et al., 2019]. However, these methods are easily trapped into model-specific local optimum by indiscriminately perturbing features without considering the intrinsic features of objects in images. Some works tend to disrupt inherent features of objects to improve the transferability of adversarial examples. [Wang et al., 2021b] (FIA) adopt random pixel dropping transformations and aggregate the transformed images to find these objectrelated features. Although pixel-wise transformations can neutralize some model-specific non-semantic details, these model-specific features cannot be filtered out well due to the correlation between adjacent pixels in natural images. Simultaneously, pixel dropping introduces extra pixel-wise noises that interfere with the captured object-related features, weakening the transferability of adversarial examples.

This paper proposes a Random Patch Attack (RPA), which effectively obtains inherent critical features of objects by random patch transformations, thereby significantly enhancing the transferability of adversarial examples. Specifically, we introduce random patch transformations to benign images such that the transformed images can variate the non-object noise while preserving object-related spatial structure and texture information. The regions of important object-related features are stressed by computing and aggregating the gradients of the feature maps of the transformed images at the intermediate layer. In contrast, those model-specific features are suppressed. The aggregated gradients guide adversarial perturbations distort important features, making adversarial examples more transferable. Since natural images are usually composed of smooth patches [Mahendran and Vedaldi, 2015] and model's discriminative regions are generally concentrated on several patches of them, the patch-wise transformation can better preserve object-related areas and variate patch-wise model-specific details. Different models focus on different discriminative regions during recognition [Dong et al., 2020], thus aggregating many transformed images can better neutralize model-specific information.

As shown in Figure 1, the adversarial example crafted by the proposed RPA significantly distracts models, making models focus on entirely different areas from attention on the clean image. In contrast, the models' attention on the adversarial example generated by FIA partially overlaps with that on the clean image, thus illustrating the effectiveness of patch-wise random transformations for capturing objectrelated critical features. We follow the previous works to evaluate the performance of the proposed RPA against normally trained models and defense models. The experimental results demonstrate that the proposed RPA craft adversarial examples of remarkable transferability compared to the stateof-the-art transfer-based attacks. Besides, we also evaluate the performance of the proposed RPA against vision transformers, which achieve the state-of-the-art performance on recognition tasks and are shown strong robustness against adversarial examples [Mahmood et al., 2021]. The experimental results show that our attack reaches an average black-box success rate of 62.6% against vision transformers, thus further demonstrating the superior performance of the proposed RPA. Code is available at: https://github.com/alwaysfoggy/ RPA. Our main contributions are summarized as follows:

- We propose a Random Patch Attack by patch-wise random transformations to perturbs common discriminative regions of different models, significantly improving the transferability of adversarial examples.
- The random patch transformation operation effectively highlights important object-related features, thus guiding adversarial examples towards the more transferable direction.
- Extensive experiments demonstrate that the proposed RPA generates adversarial examples of excellent trans-

ferability compared to state-of-the-art transfer-based attack methods.

## 2 RelatedWork

Following the pioneering work in [Szegedy et al., 2014], many attacks have been proposed to show the vulnerability of convolutional neural networks (CNNs) to adversarial examples. FGSM [Goodfellow et al., 2015] and BIM [Kurakin et al., 2017a], as the first generation of gradient-based attack methods, are the basis for many subsequent works, most of which focus on enhancing the transferability of adversarial examples to execute the black-box attack. [Dong et al., 2018] integrates momentum term into iterative gradient-based attacks (MIM) to boost the transferability of adversarial examples and significantly improve the attack ability of black-box attacks. [Xie et al., 2019] applies image transformation to the inputs at each iteration (DIM) to improve transferability. [Wu et al., 2021] proposes adversarial transformations to improve the robustness of synthesized adversarial examples (ATTA) to boost transferability. [Wang et al., 2021a] proposes a novel input transformation method (Admix) to enhance the transferability of adversarial examples. [Dong et al., 2020] proposes translation-invariant attack (TIM) by optimizing perturbations over an ensemble of translated images, resulting in higher transferability against defense models. [Gao et al., 2020] proposes a patch-wise iterative algorithm (PIM) to enhance the transferability of adversarial examples.

Unlike the above works perturbing the output layer, some studies focus on maximizing internal feature perturbation to improve transferability. [Ganeshan et al., 2019] disrupts features at each layer of the network to generate perturbations, resulting in highly damaged deep features to obtain higher transferability. [Huang et al., 2019] aims to fine-tune an existing adversarial example for greater black-box transferability by increasing its perturbation on a pre-specified layer of the source model. [Wang et al., 2021b] proposes a Feature Importance-aware Attack (FIA) to significantly improve the transferability of adversarial examples by disrupting the essential object-aware features. The proposed RPA belongs to the same category as FIA, which considers the importance of features rather than the indiscriminate modification of all features as in previous work.

## 3 Methodology

Given a classification model f(x) : x ∈ X → y ∈ Y that outputs a label y as the prediction for an input x, we aim to craft an adversarial example x∗ which is visually indistinguishable from x but is misclassified by the classifier, i.e., f(x∗ ) ̸= y. The generation of adversarial examples can be formulated as the following optimization problem,

<!-- formula-not-decoded -->

where the loss function J(·, ·) is often the cross-entropy loss, and the ℓp-norm is adopted to measure the distance between x and x∗ . This work uses p = ∞. However, optimizing Eq.1 needs to calculate the gradient of the loss function, which is not allowed in the black-box setting. Many methods [Dong et al., 2018; Gao et al., 2020] resort to the transferability of adversarial examples, i.e., generating adversarial examples on the source model and exploiting transferable adversarial examples attacks the target black-box model.

Figure 2: Visualization process of the patch-wise random transformation. Aggregating the gradients of masked images can emphasize object-related regions and neutralize non-object regions.

![Image](https://ijcai.org/proceedings/2022/fulltext/233/image_000001_09e7f494b63831dd762c5adccc39b919ff6d342b423bbbee3a32b4be31a007dc.png)

The key to generating adversarial examples of high transferability is to explore which features are more accessible to transfer across models. Classifiers tend to use any available signal to maximize classification accuracy [Ilyas et al., 2019]. In addition to learning intrinsic features of objects, a model also retains model-specific features. Different models may result in various model-specific features. Therefore, distorting object-related characteristics that dominate discriminative regions of different models can avoid adversarial examples falling into model-specific local optimum and enhance transferability across different models. [Wang et al., 2021b] adopt random pixel dropping to transform the input and introduce aggregate gradient to aggregate the transformed images, thereby finding and distorting object-related features to generate adversarial examples of high transferability. However, natural images are generally made of smooth patches [Mahendran and Vedaldi, 2015] and discriminative regions usually contain clustered pixels instead of scattered ones, only performing pixel-wise transformations can not filter out model-specific details well. Simultaneously, dropping pixels will introduce extra pixel-wise noises that impede obtaining important object-related features.

## 3.1 Patch-wise RandomTransformation

This paper introduces the patch-wise random transformation to effectively stress the important intrinsic features of objects and suppress model-specific elements. Specifically, we first generate a mask M of the same size as x, which is a matrix with all elements of 1. Then we divide M into regular non-overlapping patches. The patch size is n2 ; we suggest n &lt; 10, since small-size patches can eliminate the redundancy of model-specific features while trying to preserve the semantic features of an image. Then we randomly sample a subset of patches with the probability pm and modify the subset following a uniform distribution. Let f denote the source model and fk(x) represent the feature maps from the k-th layer. We calculate the gradient ∆ w.r.t. fk(x) after integrat-

## Algorithm 1 Random Patch Attack

Input: A classifier f; an original image x and ground-truth label y;

Parameter: The intermediate layer k; modify probability pm; the ensemble number N; the patch size n; the size of perturbation ϵ; the number of iteration T ; maskM.

Output: An adversarial example x∗ .

- 1: Let ∆ = 0, g0 = 0, µ = 1, α = ϵ/T;
- 2: for i = 0 to N -1 do
- 3: DivideM into non-overlapping n2 patches P ;
- 4: Random sample P with pm: Ppm = Rand(P , pm);
- 5: Modify Ppm ∼ U[0, 1);
- 6: Obtain the masked image: xM =x ⊙M;
- 7: Get the gradient of feature maps from the k-th layer:

<!-- formula-not-decoded -->

- 8: ∆ =∆+∆i;
- 9: end for
- 10: Obtain the aggregate gradient: ∆k =∆/ ∥ ∆ ∥2;
- 11: Get loss function by Equation 3;
- 12: for t = 0 to T -1 do
- 13: Update the momentumgt+1:

<!-- formula-not-decoded -->

- 14: Update x∗t+1 by applying the gradient sign:

<!-- formula-not-decoded -->

15: end for

16: return x∗T .

ing the mask into x. After N iterations, we finally adopt the aggregate gradient ∆k to aggregate the above gradients as

<!-- formula-not-decoded -->

where xM denotes the masked image, l(·, ·) denotes the logit output w.r.t. y, and C is obtained by ℓ2-norm on the corresponding summation term. Figure 2 shows the process of patch-wise random transformation, which weakens the nonobject regions that may cause local optimum and highlights the areas of object-related features that can guide the generation of adversarial examples towards the more transferable direction.

## 3.2 Attack Algorithm

This paper proposes a random patch attack (RPA) to distort object-related critical features obtained by the above patchwise random transformations, generating adversarial examples of high transferability. We utilize the obtained aggregate gradient ∆k to form the following loss function to steer the generation of x∗ .

<!-- formula-not-decoded -->

The higher intensity in ∆k represents regions of objectrelated critical features. Increasing the lower intensity and declining the higher intensity guide adversarial examples towards the more transferable direction. Therefore, the RPA method solves the following optimization problem instead of Equation 1.

<!-- formula-not-decoded -->

We adopt the momentum optimization method [Dong et al., 2018] to solve Equation 4. The details of the proposed RPA are shown in Algorithm 1.

## 4 Experiments

## 4.1 Setup

Datasets. We follow the previous works [Gao et al., 2020; Wang et al., 2021b] to conduct our experiments on the ImageNet-compatible dataset 1 , containing 1000 images used for the NIPS 2017 adversarial competition.

Models. We consider fourteen classification models as target models to evaluate the performance of different attacks, six of which are normally trained models-Vgg-16 [Simonyan and Zisserman, 2015], Resnet-v1-50 (Res-50) and Resnet-v1-152 (Res-152) [He et al., 2016], Inceptionv3 (Inc-v3) [Szegedy et al., 2016], Inception-v4 (Inc-v4) and Inception-Resnet-v2 (IncRes-v2) [Szegedy et al., 2017], four of which are defense models-Inc-v3adv and IncResv2adv [Kurakin et al., 2017b], Inc-v3ens3 and IncRes-v2ens [Tram`er et al., 2018], and four of which are state-of-theart vision transformers-PiT-S [Heo et al., 2021], CaiT-S24 (CaiT-S) [Touvron et al., 2021b], DeiT-B [Touvron et al., 2021a], Swin-B [Liu et al., 2021].

Baseline Attacks. We compare the proposed RPA to diverse state-of-the-art attack methods-MIM [Dong et al., 2018], DIM [Xie et al., 2019], TIM [Dong et al., 2020], PIM [Gao et al., 2020], Admix [Wang et al., 2021a], FIA [Wang et al., 2021b], and combinations of these methods-TIDIM [Dong et al., 2020], PIDIM [Gao et al., 2020], PIDITIM [Gao et al., 2020], PIDIM-FIA [Wang et al., 2021b], PIDITIM-FIA [Wang et al., 2021b].

Implementation Details. For the settings of parameters, we set the maximum perturbation to be ϵ = 16, the number of iteration as T = 10 and the step size as α = ϵ/T = 1.6 among all experiments. For the baseline attacks, we adopt the default settings. For MIM, the decay factor µ = 1.0. For DIM, the transform probability is 0.7. For TIM, the kernel size is 15. For PIM, settings vary with target models. When attacking normally trained models, the amplification factor β = 10, the project factor γ = 16, and project kernel size kω = 3. while its combined versions replace β = 2.5, γ = 2. When attacking defense models, PIM remove the momentum term, PIM and PIDITIM set β = 10, γ = 16, kω = 7. For Admix, the number of admixed copies m1 = 5, sampled images m2 = 3, and the strength of sampled image η = 0.2.

[1 https://github.com/cleverhans-lab/cleverhans/tree/master/ cleverhans v3.1.0/examples/nips17 adversarial competition/dataset](https://github.com/cleverhans-lab/cleverhans/tree/master/cleverhans_v3.1.0/examples/nips17_adversarial_competition/dataset)

Table 1: Success rate (%) of transferable attacks against normally trained models. The first column lists source models, and the first row shows target models. Our methods are RPA and PIDIM-RPA. ∗ indicates the white-box attacks. The best results are stressed in bold.

| Attack        | Vgg-16   |   Res-50 | Res-152   | Inc-v3   |   Inc-v4 |   IncRes-v2 |
|---------------|----------|----------|-----------|----------|----------|-------------|
| MIM           | 100.0∗   |     89.8 | 84.8      | 80.7     |     81.5 |        74.3 |
| DIM           | 99.9∗    |     91.9 | 88.5      | 86.0     |     84.9 |        80.2 |
| PIM           | 100.0∗   |     91.5 | 85.8      | 83.7     |     82.6 |        75.9 |
| Vgg-16 Admix  | 99.9∗    |     95.7 | 93.6      | 93.8     |     94.7 |        89.5 |
| FIA           | 99.9∗    |     97.0 | 95.0      | 95.0     |     95.2 |        92.1 |
| RPA           | 99.9∗    |     97.6 | 96.6      | 96.3     |     97.2 |        95.1 |
| PIDIM         | 99.9∗    |     93.9 | 92.0      | 88.3     |     89.0 |        82.7 |
| PIDIM-FIA     | 100.0∗   |     98.0 | 97.2      | 97.1     |     97.3 |        94.4 |
| PIDIM-RPA     | 100.0∗   |     99.1 | 98.0      | 98.2     |     98.2 |        96.6 |
| MIM           | 73.5     |     90.3 | 99.8∗     | 57.7     |     49.1 |        45.9 |
| DIM           | 83.0     |     93.6 | 99.6∗     | 74.1     |     65.9 |        65.4 |
| PIM           | 83.3     |     91.9 | 100.0∗    | 64.8     |     56.7 |        52.5 |
| Res-152 Admix | 89.0     |     96.3 | 99.9∗     | 84.0     |     76.5 |        75.4 |
| FIA           | 90.6     |     97.4 | 99.5∗     | 85.8     |     80.1 |        77.4 |
| RPA           | 94.3     |     97.9 | 99.6∗     | 89.6     |     85.8 |        85.7 |
| PIDIM         | 90.3     |     96.0 | 99.6∗     | 80.4     |     70.7 |        72.0 |
| PIDIM-FIA     | 95.5     |     98.0 | 99.5∗     | 91.0     |     86.1 |        86.0 |
| PIDIM-RPA     | 97.6     |     98.5 | 99.7∗     | 94.3     |     90.5 |        91.2 |
| MIM           | 38.5     |     33.6 | 28.9      | 100.0∗   |     42.1 |        39.2 |
| DIM           | 50.0     |     47.3 | 40.2      | 99.8∗    |     67.5 |        62.1 |
| PIM           | 61.5     |     51.1 | 44.7      | 100.0∗   |     53.2 |        48.9 |
| Inc-v3 Admix  | 70.6     |     71.6 | 63.0      | 100.0∗   |     78.2 |        77.1 |
| FIA           | 71.9     |     69.6 | 64.2      | 98.3∗    |     83.5 |        79.3 |
| RPA           | 75.3     |     72.7 | 68.4      | 98.6∗    |     85.7 |        84.0 |
| PIDIM         | 60.3     |     53.2 | 47.2      | 99.9∗    |     70.9 |        66.0 |
| PIDIM-FIA     | 81.4     |     78.9 | 74.9      | 98.5∗    |     87.9 |        85.0 |
| PIDIM-RPA     | 83.7     |     83.0 | 79.8      | 98.5∗    |     89.6 |        88.7 |

For FIA, the ensemble number N = 30, when attacking normally trained models, the drop probability pd = 0.3; when attacking defense models, pd = 0.1. For the proposed RPA, the patch size alternately sets n = 1, 3, 5, 7, the ensemble number N is set to 60, the modify probability pm is set to 0.3, 0.2 and 0.2 when attacking normally trained models, defense models and vision transformers, respectively. The chosen feature layer and extra parameters of combined versions follow the settings in [Wang et al., 2021b] for a fair comparison. We set random seed in our experiments to guarantee that the proposed method is reproducible, the random seed is 1234.

## 4.2 Comparison of Transferability

This section compares the transferability between the proposed RPA and the baseline attacks quantitatively. We evaluate the success rate of different attacks against normally trained models, defense models, and visual transformers, respectively.

## Attacking Normally Trained Models

Table 1 reports the black-box attack success rates of the proposed RPA and the other state-of-the-art transfer-based attacks against normally trained models. We craft adversarial examples on Vgg-16, Res-152, and Inc-v3, respectively. As shown in Table 1, our attacks outperform the other attacks, with an average improvement of 2.9% compared to the best results of other methods. When our RPA and PIDIM-RPA generate adversarial examples on Res-152 to attack IncResv2, transferability can be increased by 8.3% and 5.2%, respectively. Notably, combining our RPA with PIDIM can further improve the transferability, resulting in a 4.4% increase in transferability on average. In particular, the black-box attack success rate of PIDIM-RPA can achieve an average of about 92.5%. When the source model is Vgg-16, PIDIMRPA attacking Res-50 reaches 99.1%, close to the white-box attack success rate.

Table 2: Success rate (%) of transferable attacks against defense models (left) and vision transformers (right). The first column lists source models, and the first row shows target models. Our RPA, PIDIM-RPA and PIDITIM-RPA significantly outperform other attacks. The best results are stressed in bold.

| Attack                            | Inc-v3adv           | IncRes-v2adv        | Inc-v3ens3                    | IncRes-v2ens                            | PiT-S                                   | CaiT-S                                  | DeiT-B              | Swin-B              |
|-----------------------------------|---------------------|---------------------|-------------------------------|-----------------------------------------|-----------------------------------------|-----------------------------------------|---------------------|---------------------|
| Vgg-16 TIM PIM FIA RPA            | 53.6 58.5 87.9 90.3 | 46.0 51.4 86.7 87.9 | 55.6 55.4 85.4 87.7           | 40.7 38.2 70.9 73.3                     | 34.3 54.7 81.6 84.4                     | 27.9 48.3 78.7 83.3                     | 28.1 42.5 67.0 71.2 | 13.9 27.9 57.2 61.2 |
| PIDITIM PIDIM-FIA TIM PIM FIA RPA | 54.5 91.3 74.0 93.2 | 46.9 89.8 70.4      | 57.3 88.4 77.5 90.1 80.7 43.0 | 45.5 75.6 66.7 77.5 70.1 35.0 30.4 41.4 | 84.6 45.4 87.8 45.7 26.7 38.1 63.4 70.0 | 83.7 33.5 87.0 35.4 18.9 31.2 61.4 65.8 | 19.5 73.5           |                     |
| TIM PIM FIA                       |                     |                     |                               | 67.2 23.1 12.0 23.4                     |                                         |                                         |                     | 10.1                |
| RPA PIDITIM PIDIM-FIA             | 54.7 59.0           | 91.2                | 45.5 44.3                     | 26.3 34.6                               | 46.6                                    | 12.6                                    | 36.0                |                     |
|                                   | 46.2 65.6 64.8      | 39.2 61.9 58.6      | 49.6                          |                                         | 19.2                                    | 15.1                                    | 17.8                | 8.2                 |
| PIDIM-RPA PIDITIM-RPA             |                     |                     |                               |                                         | 25.0                                    | 15.5                                    | 32.2                | 63.7 17.8           |
| PIDITIM-FIA                       |                     |                     |                               |                                         |                                         |                                         | 34.8                |                     |
| PIDIM-RPA                         |                     |                     |                               |                                         |                                         |                                         | 76.6                | 66.1                |
| PIDITIM-RPA                       | 78.0                | 73.8                |                               |                                         |                                         |                                         | 36.5                | 17.6                |
|                                   | 41.5                | 37.6                |                               |                                         |                                         |                                         | 23.2                | 10.7                |
|                                   | 44.9                | 41.1                | 42.9                          |                                         |                                         |                                         | 32.4                | 20.2                |
|                                   | 68.6                | 66.0                | 61.9                          |                                         |                                         |                                         | 50.3                | 40.4                |
| Res-152                           | 73.7                | 71.0                | 66.9                          | 46.1                                    |                                         |                                         | 54.8                | 46.9                |
| PIDITIM                           | 53.8                | 47.7                | 60.3                          | 48.6                                    | 22.1                                    |                                         | 19.6                | 8.9 52.7            |
| PIDIM-FIA                         | 78.0                | 77.5                | 71.8                          | 50.8                                    | 75.8                                    | 73.0                                    | 64.9                |                     |
| PIDITIM-FIA                       | 67.2                | 63.5                | 69.1                          | 60.4                                    | 37.6                                    | 27.9                                    |                     | 15.0                |
| PIDIM-RPA                         | 80.8                | 80.5                | 75.8                          | 54.4                                    | 78.3                                    | 76.6                                    | 67.3                | 57.2                |
| PIDITIM-RPA                       | 71.1                | 69.5                | 74.4                          |                                         | 40.4                                    | 32.0                                    | 32.6                | 17.1                |
|                                   | 31.5                | 25.9                | 30.4                          |                                         | 17.0                                    | 14.5                                    | 15.9                | 8.7                 |
|                                   | 33.2                | 27.2                | 21.8                          |                                         | 21.5                                    | 15.1                                    | 17.4                | 12.3                |
|                                   |                     |                     |                               |                                         |                                         |                                         |                     | 28.2                |
|                                   |                     | 55.2                | 43.5                          |                                         |                                         | 37.0                                    | 32.0                |                     |
| Inc-v3                            |                     | 59.5                |                               |                                         | 50.9                                    | 41.4                                    |                     | 32.4                |
| PIDITIM-FIA                       | 60.0                |                     | 46.3 62.9                     | 26.8 52.6                               | 57.8 37.0                               | 48.5 36.0                               | 44.1 31.8           | 36.3                |
|                                   |                     |                     |                               |                                         |                                         |                                         |                     | 16.4                |
|                                   |                     | 65.6                |                               | 29.4                                    | 62.4                                    | 51.1                                    |                     | 40.2                |
|                                   |                     |                     |                               |                                         |                                         |                                         | 47.5                |                     |
|                                   | 72.1                | 64.2                | 69.6                          | 59.2                                    | 42.4                                    | 40.9                                    | 35.9                | 17.3                |
| TIM                               | 39.2                | 44.9                | 40.4                          | 37.5                                    | 20.2                                    | 19.1                                    | 17.7                | 11.1                |
| PIM                               | 38.0                | 34.3                | 25.0                          | 16.7                                    | 22.7                                    | 17.9                                    | 19.6                | 13.8                |
| FIA                               | 55.1                | 57.9                | 48.0                          | 36.7                                    | 42.8                                    | 37.5                                    | 33.1                | 31.4                |
| RPA                               | 60.5                | 64.1                | 55.2                          | 41.1                                    | 51.1                                    | 44.9                                    | 38.7                | 37.3                |
| IncRes-v2 PIDITIM                 | 57.5                | 55.1                | 57.0                          | 52.5                                    | 20.8                                    | 19.4                                    | 19.1                | 8.5                 |
| PIDIM-FIA PIDITIM-FIA             | 59.8 55.2           | 61.9 53.6           | 51.6 53.3                     | 38.4 50.3                               | 52.8 30.2                               | 48.0 27.5                               | 41.6 25.5           | 38.6 11.6           |
| PIDIM-RPA                         | 65.4                | 68.3                | 56.9                          | 42.4                                    | 59.0                                    | 54.4                                    | 49.8                | 40.8                |
| PIDITIM-RPA                       | 66.0                | 65.7                | 64.6                          | 61.4                                    | 37.6                                    | 40.9                                    | 32.2                | 16.8                |

## Attacking Defense Models

Table 2 (left) reports the black-box attack success rates of the proposed RPA and baseline attacks against defense models. We choose Vgg-16, Res-152, Inc-v3, and IncRes-v2 as the source model, respectively. Other methods with poor performance are not listed in Table 2 due to the space limit. The transferability of Table 2 (left) is consistently lower than that of Table 1 since defense models have stronger robustness to adversarial examples. However, our methods still significantly outperform the other state-of-the-art transferbased attacks, improving the black-box attack success rate by 4.7% on average. As shown in Table 2 (left), PIDIMRPA and PIDITIM-RPA further improve the transferability by 10.8% on average, the black-box attack success rate achieving 73.9% on average. In particular, when the source model is Vgg-16, the success rate of PIDIM-RPA against all defense models achieves 88.0% on average, especially the success rate against Inc-v3adv reaches 93.2%. When IncResv2 is the source model, PIDITIM-RPA against all defense models results in an average 11.4% increase in success rate compared to PIDITIM-FIA.

## Attacking Vision Transformers

Table 2 (right) reports the black-box attack success rates of different attacks against state-of-the-art vision transformers (ViTs). Previous works [Mahmood et al., 2021] demonstrate that ViTs possess better adversarial robustness when compared with CNNs, and the transferability between CNNs and ViTs is, in general, remarkably low. Therefore, attacking ViTs are more arduous by the generated adversarial examples on CNNs. From the right four columns of Table 2, we can observe that the proposed RPA significantly outperforms other methods, improving the black-box attack success rate by 4.6% on average. The combination PIDIM-RPA further enhances the transferability by 8.2% on average, achieving an average black-box attack success rate of 62.6%. When the source model is Vgg-16, our PIDIM-RPA obtains the highest attack success rate of 87.8% against PiT-S.

## 4.3 Effect of Parameters

In the proposed RPA, the modify probability pm, the ensemble number N of aggregate gradient and the patch size n2 play a critical role in affecting the black-box attack success rate. We adopt Res-152 as the source model for generating adversarial examples by the proposed RPA with different pm and N. pm ranges from 0.1 to 0.5 with a granularity of 0.1, and N changes from 10 to 70 with a step of 10. We then evaluate the transferability of the generated adversarial examples by attacking Vgg-16, Inc-v3, IncRes-v2adv and Swin-B with the results shown in Figure 3.

Figure 3: Success rate (%) of the proposed RPA with different parameters-the modify probability pm changing from 0.1 to 0.5 with a granularity of 0.1, the ensemble number N ranging from 10 to 70 with a step of 10. The source model is Res-152. The target models are two normally trained models-Vgg-16 (column-1), Inc-v3 (column-2), a defense model-IncRes-v2adv (column-3) and a vision transformerSwin-B (column-4).

![Image](https://ijcai.org/proceedings/2022/fulltext/233/image_000002_83cb12ac8d1fe12d85b268ebb56883dd814517003db0900e28b7c475fe305cdf.png)

Figure 4: Effect of patch size on black-box attack success rate against defense models and vision transformations. The source model is Inception-Resnet-v2.

![Image](https://ijcai.org/proceedings/2022/fulltext/233/image_000003_bd05910d5ebadb80d7d654ec5804be7cbef7d29c6399d1138743414687432def.png)

In general, a larger ensemble number does increase the success rate. However, when the ensemble number is large enough, the success rate goes saturate and may even decrease. We thus determine the ensemble number N = 60. When attacking normally trained models, an optimal modify probability pm is between 0.3 and 0.4. While an optimal pm is between 0.1 to 0.3 for attacking defense models and vision transformers. We thus suggest the modify probability pm = 0.3 for attacking normally trained models and pm = 0.3 for attacking defense models and vision transformers. To study the effect of n2 on success rate, we adopt IncRes-v2 as the source model to craft adversarial examples by P1, P2, P3 and P4, respectively. P1 sets n = 1, P2 alternately uses n = 1, 3, P3 alternately uses n = 1, 3, 5 and P4 alternately sets n = 1, 3, 5, 7. As shown in Figure 4, P4 leads to a higher success rate, we thus suggest the setting of P4.

## 5 Conclusion

In this paper, we propose a novel Random Patch Attack (RPA) that generates adversarial examples of superior transferability. The proposed RPA effectively captures the raw image's important object-related features and eliminates the redundancy of model-specific features by the patch-wise random transformation. Perturbing these features steers the adversarial examples towards a more transferable direction. We conduct extensive experiments that demonstrate the excellent transferability of adversarial examples generated by the proposed RPA compared to state-of-the-art transfer-based blackbox attacks. Our attack strategy can serve as a benchmark for evaluating the effectiveness of different defense methods in the future and various models' robustness.

## Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant (No. 61876019, 62072037, U1936218), Beijing Municipal Science and Technology Project Z211100004121009.

## References

1. [Dong et al., 2018] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9185-9193, 2018.
2. [Dong et al., 2020] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. In IEEE Confer- ence on Computer Vision and Pattern Recognition, 2020.
3. [Ganeshan et al., 2019] Aditya Ganeshan, Vivek B.S., and R. Venkatesh Babu. Fda: feature disruptive attack. In IEEE International Conference on Computer Vision, 2019.
4. [Gao et al., 2020] Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In European Conference on Computer Vision, pages 307-322, 2020.
5. [Goodfellow et al., 2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
6. [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pages 630-645, 2016.
7. [Heo et al., 2021] Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. In International Conference on Computer Vision, 2021.
8. [Huang et al., 2019] Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transferability with an intermediate level attack. In IEEE International Conference on Com- puter Vision, 2019.
9. [Ilyas et al., 2018] Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Confer- ence on Machine Learning, pages 2137-2146, 2018.
10. [Ilyas et al., 2019] Andrew Ilyas, Shibani Santurkar, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adver- sarial examples are not bugs, they are features. In An- nual Conference on Neural Information Processing Sys- tems, 2019.
11. [Kurakin et al., 2017a] Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Repre- sentations, 2017.
12. [Kurakin et al., 2017b] Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations, 2017.
13. [Liu et al., 2017] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. In International Conference on Learning Representations, 2017.
14. [Liu et al., 2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In International Conference on Com- puter Vision, 2021.
15. [Mahendran and Vedaldi, 2015] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representa- tions by inverting them. In IEEE Conference on Computer Vision and Pattern Recognition, pages 5188-5196, 2015.
16. [Mahmood et al., 2021] Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision trans- formers to adversarial examples. In IEEE International Conference on Computer Vision, 2021.
17. [Shi et al., 2019] Yucheng Shi, Siyu Wang, and Yahong Han. Curls & whey: Boosting black-box adversarial attacks. In IEEE Conference on Computer Vision and Pattern Recog- nition, pages 6519-6527, 2019.
18. [Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Confer- ence on Learning Representations, 2015.
19. [Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
20. [Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2818-2826, 2016.
21. [Szegedy et al., 2017] Christian Szegedy, Sergey Ioffe, Vin- cent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI conference on artificial intelligence, 2017.
22. [Touvron et al., 2021a] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablay- rolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In Interna- tional Conference on Machine Learning, 2021.
23. [Touvron et al., 2021b] Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. arXiv preprint arXiv:2103.17239, 2021.
24. [Tramèr et al., 2018] Florian Tramèr, Alexey Kurakin, Nico- las Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: attacks and de- fenses. In International Conference on Learning Repre- sentations, 2018.
25. [Wang et al., 2021a] Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In IEEE International Conference on Computer Vision, pages 16158-16167, 2021.
26. [Wang et al., 2021b] Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren. Fea- ture importance-aware transferable adversarial attacks. In IEEE International Conference on Computer Vision, pages 7639-7648, 2021.
27. [Wu et al., 2021] Weibin Wu, Yuxin Su, Michael R Lyu, and Irwin King. Improving the transferability of adversarial samples with adversarial transformations. In IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 9024-9033, 2021.
28. [Xie et al., 2019] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Im- proving transferability of adversarial examples with input diversity. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
29. [Zhou et al., 2018] Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, and Yong Yang. Transferable adversarial perturbations. In European Conference on Computer Vision, 2018.