Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection
Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection
Chen Liu, Shibo He, Qihang Zhou, Shizhong Li, Wenchao Meng
Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Main Track. Pages 2162-2170.
https://doi.org/10.24963/ijcai.2024/239
Self-supervised methods have gained prominence in time series anomaly detection due to the scarcity of available annotations. Nevertheless, they typically demand extensive training data to acquire a generalizable representation map, which conflicts with scenarios of a few available samples, thereby limiting their performance. To overcome the limitation, we propose AnomalyLLM, a knowledge distillation-based time series anomaly detection approach where the student network is trained to mimic the features of the large language model (LLM)-based teacher network that is pretrained on large-scale datasets. During the testing phase, anomalies are detected when the discrepancy between the features of the teacher and student networks is large. To circumvent the student network from learning the teacher network’s feature of anomalous samples, we devise two key strategies. 1) Prototypical signals are incorporated into the student network to consolidate the normal feature extraction. 2) We use synthetic anomalies to enlarge the representation gap between the two networks. AnomalyLLM demonstrates state-of-the-art performance on 15 datasets, improving accuracy by at least 14.5% in the UCR dataset.
Keywords:
Data Mining: DM: Anomaly/outlier detection
Data Mining: DM: Mining spatial and/or temporal data
Machine Learning: ML: Unsupervised learning