Robustness of Autoencoders for Anomaly Detection Under Adversarial Impact

1 Introduction

Anomalies are defined in [Hawkins, 1980] as "observations which deviates so much from other observations as to arouse suspicion it was generated by a different mechanism". Anomaly detection is the task of identifying these anomalies in a set of data. It is a crucial task in many applications, from detecting fraudulent credit card usage [Bolton and Hand, 1999] to malignant tumours [Spence et al., 2001], and a plethora of approaches have been developed over time. A common challenge is the absence of labelled data, making supervised classification models ineffective. As such, many of the most successful attempts use a one-class approach; where the model is trained with only normal data and then determines whether test data belongs to the normal class or not. Recently, deep learning methods have given the best performance; particularly popular is to train an autoencoder to reconstruct data of the normal class, using the reconstruction error to determine whether a point is anomalous.

Despite their successes, the highly complex operations of deep learning models can make their outputs fragile to small perturbations in input data. This makes them vulnerable to an adversary who may exploit perturbations that are purposefully designed to greatly hinder model performance. This phenomena, known as an adversarial attack, has drawn a huge amount of attention in related literature since being discovered in [Szegedy et al., 2013] due to the potential risks to model performance and security. Unfortunately, studies have shown that attacks designed for a particular model can generalize to other models with different architectures too [Kurakin et al., 2017]. Many different attacks formulations have been studied in a variety of contexts, a key categorization being between white-box and black-box attacks: whether the adversary uses internal information and computations of the model such as loss gradients, or only inputs and outputs values, respectively. The robustness of these models, how tolerant they are to these input perturbations, is important in a variety of applications. Within the context of anomaly detection, an adversary may wish to perturb input data in such a way that as many anomalies as possible appear normal.

A number of defense mechanisms have been devised against adversarial attacks. Perhaps the most well-known, [Goodfellow et al., 2014] proposes 'adversarial training', which introduces data that has been perturbed with an adversarial attack into the training set along with their correct labels. In doing so, the model is trained to correctly classify attacked examples too. In the case of unsupervised anomaly detection, no labels are supplied to the model and anomalies are only seen during test time, meaning this defense is inapplicable. Therefore, it is necessary to develop new defense mechanisms against adversarial attacks in the unsupervised anomaly detection setting.

With this in mind, the contributions of this work are as follows:

1. We study the vulnerability of anomaly-detecting autoencoders to different types of adversarial attacks.

1. We propose the Approximate Projection Autoencoder (APAE) which includes two developments to improve both model performance and robustness under the impact of adversarial attacks.

In experiments on real data, these techniques led to a median improvement in AUC score of 9% in the presence of adversarial attacks in the range of those tested and 8% in their absence.

2 Background

2.1 Anomaly Detection with Autoencoders

Autoencoders are a very popular approach for anomaly detection. They are neural networks that are trained to reconstruct the input data, with the error between the original and the reconstruction used in the loss function for training:

where x and A(x) is the input and output of the autoencoder respectively and || · || is typically some type of norm.

There are usually fewer neurons in the hidden layers than the input (and output) layers in order to encourage the model to learn a more compressed representation of the data; a form of dimensionality reduction. As mentioned, it is common to train the autoencoder on only normal data in anomaly detection. In doing so, normal data should be reconstructed with low error during inference whilst anomalies will be reconstructed with higher error as they follow a different distribution. This reconstruction error can be used as an anomaly score; those below or above a given threshold are determined to be normal or anomalous respectively.

2.2 Adversarial Attack

As mentioned, input data can be perturbed as to hinder model performance through adversarial attacks. Formulated in [Kurakin et al., 2017], the 'fast gradient sign method' (FGSM) is particularly prominent. In this attack, inputs are moved in the direction of the gradient of the loss function with respect to these inputs which acts to increase the loss function with respect to the true class and encourage a misclassification. This is extended to the 'basic iterative method', in which this step is performed for multiple iterations as follows:

where X is the original input data, J the loss function and ytrue the true class. This is controlled by a rate α and a clipping function which keeps values in the range [-ϵ, ϵ].

3 RelatedWork

In the supervised setting, anomaly detection is a typical classification problem; the labels corresponding to whether data is normal or anomalous. Well established methods like decision trees and neural networks are applicable in this setting

[Zhang et al., 2001], however imbalanced datasets and difficulty obtaining accurate labels make this approach unpopular. As such, unsupervised versions of these types of algorithms have been developed, most notably the One-class SVM [Eskin et al., 2002]. [Goldstein and Uchida, 2016] evaluates a variety of unsupervised anomaly detection methods such as kNN, Local Outlier Factor (LOF) and One-Class SVM.

[Hawkins et al., 2002] first proposes autoencoders for anomaly detection, using a step-function as activation to separate inputs into normal and anomalous clusters. Since then, the reconstruction error as anomaly score has become the norm in more recent studies [Dau et al., 2014]. [Sakurada and Yairi, 2014] show that the non-linearity of autoencoders allows for better detection of anomalies than linear PCA, whilst being computationally cheaper than kernel PCA. Autoencoder variants are also commonly used, including recurrent [Chauhan and Vig, 2015; Malhotra et al., 2015], convolutional [Chen et al., 2018], denoising [Feng and Han, 2015] and variational models [An, 2015]. They have also been used just for feature learning, with a separate prediction network, such as a Gaussian Mixture Model [Bo et al., 2018], using the learnt features to make predictions.

Some studies improve the robustness of anomaly detection models to noisy data, such as Robust SVM [Hu et al., 2003] and Robust PCA. The approach taken in the latter; filtering noise out of input data via matrix decomposition, has also been adopted for autoencoders [Zhou and Paffenroth, 2017]. However, no similar techniques have been developed in the case of adversarial perturbations. [Kloft and Laskov, 2010] studies adversarial poisoning attacks on online centroid anomaly detectors, where the anomaly score is the distance of a point to its nearest centroid. The authors find that, in their intrusion detection scenario, the adversary needs to control 5-20% of incoming traffic in order to subvert an online centroid learner to a target position. [Paudice et al., 2018] uses anomaly detection methods to detect adversarial examples in training sets in order to improve model performance in other tasks as a preparation step, however they do not consider the effect of attacks to the anomaly detection model itself.

4 Methods

4.1 Attacks

The first task is to design adversarial attacks that hinder the performance of autoencoders for anomaly detection. In particular, we consider perturbations to anomalous points in the test set that cause the model to misclassify as many of them as possible as normal points. This is achieved by reducing reconstruction error. Two such attacks are tested: a 'randomized' attack and an 'FGSM' attack.

In the first, a randomly generated vector is added to the original point. If the reconstruction error of this perturbed point is reduced compared to the original, then the perturbation is kept, otherwise it is discarded and another random vector is tested. This is repeated iteratively, so that multiple vectors can be added and the reconstruction error reduces further over iterations.

In the second, we implement an adaptation of the basic iterative method presented in Section 2, which we refer to as

'FGSM' from here for brevity. The adaptation ensures the reconstruction error i.e. the loss function, decreases rather than increases, by moving in the opposite direction of the loss gradient as follows:

where X refers to anomalous points. The clipping function could outweigh the impact of the perturbations and is therefore removed.

4.2 Defenses

We now describe our Approximate Projection Autoencoder (APAE), which combines two defenses: Approximate Projection (AP) and Feature Weighting (FW).

Approximate Projection

Autoencoders reconstruct complex data using lower dimensional representations. Adversarial vulnerability can arise because they can learn highly unstable functions that change rapidly in response to small input perturbations.

In contrast, we consider a class of projection functions which are much more robust; moreover, their level of robustness can be characterized theoretically. Formally, let A : Rd → Rd be an autoencoder, and define the image M = {A(x) : x ∈ Rd }, i.e. the set of points that the autoencoder can map to. Define the projection f(x) as the closest point inM to x; i.e. f(x) = argminx′∈M‖x - x′ ‖, as visualized in Figure 1. Like in autoencoders, the reconstruction error Err(x) = ‖x- f(x)‖ is used as an anomaly score; where higher error values indicate more anomalous points. Then the following theorem shows that f(x) is provably robust against adversaries:

Theorem 1. If an adversary perturbs a data point from x to y such that ||x-y|| ≤ ϵ, then we have:

i.e. the adversary can change the reconstruction error of any point x by at most ϵ.

Proof. By the triangle inequality we have:

f(x) is the projection of x ontoM, therefore, it is the closest point to x onM, so:

Combining this with (4) gives:

or Err(x) ≤ Err(y)+ϵ. By symmetry, the same result holds when swapping x and y, completing the proof.

Theorem 1 shows that in the worst-case scenario, an adversary moving a point by ϵ distance can decrease the reconstruction error under f by at most ϵ. In reality, the projection f is not accessible as the image M is unknown and highly complex. Instead, after fitting an autoencoder A, we approximate a projection by performing gradient descent on the latent embedding of the data, encoded at the bottleneck layer. Upon convergence, this will lead to a reconstruction that is closer to the optimum; the projection. Gradient descent updates are made via the following formula:

where Z is the original latent embeddings of test set points under A.

Feature Weighting

In the second defense, we acknowledge that different features vary in their discriminative capabilities. In particular, regardless of whether points are normal or anomalous, different features tend to be more accurately reconstructed than others through the autoencoder. It is useful to normalize reconstruction error across the different features in order to account for these differences:

where Ji is the reconstruction error associated with feature i for a single point, ˜ Ji is the normalizing factor and ϵ is a small constant. Various formulations of ˜ Ji were considered; the median provides robustness against outliers and good empirical performance. The optimal value of ϵ varies between 10-4 and 10-6 depending on the dataset.

This is a form of normalization which prevents the reconstruction error, and therefore the anomaly score, from being dominated by those features which are reconstructed most poorly regardless of the class. Whilst improving detection performance, this step alone does not necessarily improve robustness against adversarial attacks. However, in combination with the gradient descent defense, the autoencoder would be more accurate as well as more robust in both the presence and absence of adversarial attacks.

5 Experiments

Deep autoencoders are trained using only data of the normal class, using the reconstruction error as the anomaly score. Normal data should be reconstructed with lower error than a chosen threshold, whilst those of the anomalous data should exceed it. We use the kth percentile of reconstruction errors associated with the training set to define this threshold. The value of k is varied within the range of 90 to 100 to explore the difference it makes to the performance.

5.1 Datasets

The Secure Water Treatment (SWaT) is a water treatment testbed resembling those used by Singapore's Public Utility Board. It is an example of a Cyber-Physical System (CPS), integrating digital and physical infrastructure to control and monitor system behaviour. These systems are increasingly used in important sectors, such as utilities or transportation, and are therefore potential targets to attack by malicious actors. The Water Distribution (WADI) is an extension of this setup to include a network of distribution pipelines. There are two weeks worth of data from normal operations, which are used as training data for the respective models. A number of controlled, physical attacks are implemented at different intervals in the following days, which correspond to the anomalies in the test set. More information about the systems and datasets can be found at their websites [iTrust Centre for Research in Cyber Security, 2019].

Table 1 summarises the two datasets, showing the number of features, training and test set sizes as well as proportion of anomalies in the test set. In both cases, a 10 second sliding window (with a stride of one) were concatenated into one feature vector for each data sample - the corresponding label coming from the latest timestamp within the current window. For WADI, 120 sensors are used meaning a total of 1220 features per example. For SWaT, 50 sensors instead make for 500 features.

5.2 Training

Autoencoders with three hidden layers were trained for each dataset. The bottleneck hidden layer had 100 (50) neurons for the WADI (SWaT) dataset, reflecting their difference in input dimensonality. The models are trained using the Adam optimization scheme with learning rate 1 × 10-3 and (β1, β2) = (0.5, 0.99). Early stopping was used once the loss function, Huber loss, is sufficiently small for the validation set, which was 20% of total training data. We use N = 2000 iterations and α = 0.001 for the gradient descent step (Eq. (7)).

In Table 2, we compare the performance of a standard autoencoder, as well as an autoencoder with our proposed defenses, with various other existing methods in a non-attack setting. These are the following:

APAE: The proposed Approximate Projection Autoencoder with defenses.

AE: Autoencoder as formulated above without defenses.

PCA: Principal Component Analysis finds the set of k orthogonal axes that retain the greatest variance within the data. The anomaly score is the reconstruction error after transforming and inverse-transforming data using these components.

OC-SVM: One-class support vector machine [Chen et al., 2001] adapts the SVM for anomaly detection by maximising the margin between the one-class training data and the origin.

DAGMM: Deep Autoencoding Gaussian Mixture Model trains an autoencoder and Gaussian mixture model end-toend, using energy as the anomaly score as seen in [Bo et al., 2018].

MAD-GAN: A GAN is trained on normal data and the output of the trained discriminator is used to determine anomaly score, as seen in [Liu et al., 2019].

All models were built and implemented using the PyTorch library, except for DAGMM and MAD-GAN where the publicly available codes were used. Even without optimization of hyperparameters, a basic autoencoder is the best of the existing deep methods tested. Even PCA and OC-SVM outperform the other deep methods shown, suggesting that the latter require more meticulous preparation and model calibration to achieve their state-of-the-art performance shown in the original works. We see that in a non-attack setting, the techniques used in the APAE significantly improve the performance of the autoencoder and is easily the most successful model tested. In the following sections, the performance of this model in both attack and non-attack settings is comprehensively studied.

5.3 Attacks

Table 3 show the precision, recall and F1 measures when different reconstruction error percentiles are used as the anomaly score threshold for the two datasets respectively. In the first case, the performance is recorded for the original data without adversarial attack. In this case, we see a general pattern of increasing precision and decreasing recall as the threshold increases. A higher threshold means more anomalies do not exceed it and therefore fewer are correctly detected, however fewer normal points are falsely flagged too. The optimal performance on WADI is achieved with the 99.9th percentile threshold, though it seems a much higher threshold would give better results for SWaT. To avoid this problem, the AUC score is also presented, which eliminates the need for manually-set thresholds. Using this metric, the model for SWaT achieves a score of 0.896 compared to 0.805 for WADI.

In the second case, we apply the randomized attack detailed in Section 4. We conducted 1000 iterations, and therefore 1000 random vectors were generated. These vectors vectors were sampled from a N(0, 0.01) distribution, as this showed to result in the largest perturbations. This attacks caused a slight reduction in performance, however the most severe setting of the FGSM attack tested is much more impactful, after which the AUC scores reduce by approximately

80% in both datasets. A range of attack severities are tested and discussed in Section 5.4. We see a drastic reduction in the recall, rather than the precision, as the perturbations decrease reconstruction error of anomalies and increase the false negative rate only. Interestingly, the optimal threshold also changes as a result of the FGSM attack; the 100th percentile threshold now giving the best F1 measure for WADI and the 95th for SWaT.

Figure 2 visualises the impact of the randomized attack and a less severe FGSM attack on the ROC curves for both datasets. As the randomized attack had less impact, it will be disregarded from here on and only the FGSM attack will be studied. In the following section, a wide range of implementations of the attack and their quantitative result will be presented, along with the effect of the proposed defenses.

5.4 Defense

Three different attack scenarios are considered. In the first, we consider the case where an adversary can only perturb a limited subset of features of cardinality k, analogous to a subset of sensors in the network. We test a variety of values for k, expressed as a percentage of the total number of features. The attack is iterated until convergence for the k features that had the highest reconstruction error before the first iteration. In the second case, all features can be perturbed; however, the total change, in percentage terms, is limited to a given budget ∆. This percentage is calculated as follows:

where xorig i , xadv i are a point before and after perturbations respectively and N is the total number of features. Summation over all features is taken in order to avoid division by zero. A range of perturbation budgets are tested. The final case is similar to this, however it uses the mean squared error to calculate reconstruction error instead of Huber. As this is differs from the loss function used in model training, this corresponds to a scenario where an adversary does not know how the model was trained and therefore uses the wrong error measure. The three cases are named L0, Lhuber and L2 respectively for convenience.

Table 4 shows the performance of the new method with defenses in these attack scenarios, as well as a no attack case, for the WADI dataset. The best performance is highlighted in bold and underlined and the second best is bold only. The approximate projection (AP) and feature weighting (FW) techniques are tested separately to examine their individual effects, as well as in combination (AP + FW). Furthermore, we distinguish between FWTrain and FWTest, which relate to whether the weights are calculated from the feature-wise median reconstruction errors from the training set or the test set. The latter is more applicable to scenarios where data is accessible in batches, whilst the former would be more useful where data is received point-wise, such as in streaming applications. The performance without such defenses is also shown (No Defense).

Firstly without defenses, some general patterns can be observed. As would be expected, performance worsens further with more "powerful" attacks, i.e. with larger k in the case of L0 and larger ∆ in the others. The L0 attack is noticeably less impactful than the others, despite having an unrestricted perturbation budget. Surprisingly, the AUC score actually increases from perturbing 0% (i.e. no attack) to 1% of features. For SWaT, perturbing 1% or 10% of features made negligible difference.

Lhuber attacks caused greater deterioration in performance than L2 attacks for smaller ∆ in both datasets, however the inverse is true for the highest budgets tested. It was noticed that in both datasets, some anomalous intervals had drastically larger reconstruction errors than others, suggesting a significantly larger perturbation would be required to reduce it to a normal level. The L2 loss more heavily penalises these extreme values, therefore it is more likely to reduce these extreme errors sufficiently when given a large enough budget compared with Lhuber.

Moving onto the defenses, both gradient descent and feature weighting improve the performance across the range of attacks. Considered in isolation, AP fares better than both forms of FW, and FWTest is better than FWTrain overall. The best performance is found when combining AP + FWTest.

Table 5 shows the same results for SWaT. Generally, the same pattern applies here as to WADI. However, it was noticed that much larger perturbation budgets were needed for a significant impact to performance. Furthermore, the AP defense is less successful on this dataset; performing worse than no defense in the L0 attacks. In these cases, the best performance is found using FWTest without AP.

6 Conclusion

We have examined the problem of adversarial impact to deep autoencoders for anomaly detection. Deep autoencoders are trained to learn patterns from a set of data of only the normal class. The reconstruction error associated with a point in the test set is used to determine whether it is normal or anomalous. We have considered the context in which an adversary perturbs this data in order to have as many anomalies go undetected as possible. We have shown that the model is indeed vulnerable to this kind of attack, especially to a gradient-based attack like the basic iterative method. We have also shown that through analysing the worst-case scenario of an attack, the change in reconstruction error associated with an attacked point is bounded by the size of the perturbation if we assume the autoencoder has learnt an optimal mapping, i.e. projection. We propose the Approximate Projection Autoencoder, which involves implementing gradient descent on latent embeddings as a way to optimise reconstructions. We also account for the natural variability in reconstruction errors between different features through a feature-weighting normalization step. Experiments with real data show that combining both defenses almost always improves anomaly detection performance by the largest amount, regardless of the presence or severity of an attack.

Acknowledgements

This work was supported in part by NUS ODPRT Grant R252-000-A81-133.