{"type":"ScholarlyArticle","title":"Outlier Detection for Time Series with Recurrent Autoencoder Ensembles","authors":[{"name":"Tung Kieu","slug":"tung-kieu","given":"Tung","family":"Kieu","affiliation":"Department of Computer Science , Aalborg University , Denmark"},{"name":"Bin Yang","slug":"bin-yang","given":"Bin","family":"Yang","affiliation":"Department of Computer Science , Aalborg University , Denmark"},{"name":"Chenjuan Guo","slug":"chenjuan-guo","given":"Chenjuan","family":"Guo","affiliation":"Department of Computer Science , Aalborg University , Denmark"},{"name":"Christian S. Jensen","slug":"christian-s-jensen","given":"Christian S.","family":"Jensen","affiliation":"Department of Computer Science , Aalborg University , Denmark"}],"abstract":"We propose two solutions to outlier detection in time series based on recurrent autoencoder ensembles. The solutions exploit autoencoders built using sparsely-connected recurrent neural networks (S-RNNs). Such networks make it possible to generate multiple autoencoders with different neural network connection structures. The two solutions are ensemble frameworks, specifically an independent framework and a shared framework, both of which combine multiple S-RNN based autoencoders to enable outlier detection.  This ensemble-based approach aims to reduce the effects of some autoencoders being overfitted to outliers, this way improving overall detection quality. Experiments with two large real-world time series data sets, including univariate and multivariate time series, offer insight into the design properties of the proposed frameworks and demonstrate that the resulting solutions are capable of outperforming both baselines and the state-of-the-art methods.","doi":"10.24963/ijcai.2019/378","url":"https://ijcai.org/proceedings/2019/378","pdf":"https://ijcai.org/proceedings/2019/0378.pdf","markdown":"https://ijcai.org/proceedings/2019/378.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/378","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":378,"pages":{"first":2725,"last":2732},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Time-series;Data Streams","Machine Learning Applications: Applications of Unsupervised Learning","Machine Learning: Ensemble Methods","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Aggarwal and Sathe, 2017] Charu C. Aggarwal and Saket Sathe. Outlier Ensembles -An Introduction. Springer, 2017.","title":"Outlier Ensembles -An Introduction","authors":"C Charu, Saket Aggarwal, Sathe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aggarwal, 2013] Charu C. Aggarwal. Outlier Analysis. Springer, 2013.","title":"Outlier Analysis","authors":"C Charu, Aggarwal","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Akhtar et al., 2017] Md. Shad Akhtar, Abhishek Kumar, Deepanway Ghosal, Asif Ekbal, and Pushpak Bhat- tacharyya. A multilayer perceptron based ensemble tech- nique for fine-grained financial sentiment analysis. In EMNLP, pages 540-546, 2017.","title":"A multilayer perceptron based ensemble technique for fine-grained financial sentiment analysis","authors":"Shad Md, Abhishek Akhtar, Deepanway Kumar, Asif Ghosal, Pushpak Ekbal, Bhattacharyya","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Breunig et al., 2000] Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. LOF: identi- fying density-based local outliers. In SIGMOD, pages 93-104, 2000.","title":"LOF: identifying density-based local outliers","authors":"Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, Jörg Sander","year":"2000","venue":"SIGMOD","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2016] Ting Chen, Lu An Tang, Yizhou Sun, Zhengzhang Chen, and Kai Zhang. Entity embedding- based anomaly detection for heterogeneous categorical events. In IJCAI, pages 1396-1403, 2016.","title":"Entity embeddingbased anomaly detection for heterogeneous categorical events","authors":"Ting Chen, Lu, An Tang, Yizhou Sun, Zhengzhang Chen, Kai Zhang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2017] Jinghui Chen, Saket Sathe, Charu C. Ag- garwal, and Deepak S. Turaga. Outlier detection with autoencoder ensembles. In SDM, pages 90-98, 2017.","title":"Outlier detection with autoencoder ensembles","authors":"Jinghui Chen, Saket Sathe, Charu C Aggarwal, Deepak S Turaga","year":"2017","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chia and Syed, 2014] Chih-Chun Chia and Zeeshan Syed. Scalable noise mining in long-term electrocardiographic time series to predict death following heart attacks. In KDD, pages 125-134, 2014.","title":"Scalable noise mining in long-term electrocardiographic time series to predict death following heart attacks","authors":"Chia, Syed ; ] Chih-Chun Chia, Zeeshan Syed","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chung et al., 2014] Junyoung Chung, C ¸aglar Gülc ¸ehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cirstea et al., 2018] Razvan-Gabriel Cirstea, Darius-Valer Micu, Gabriel-Marcel Muresan, Chenjuan Guo, and Bin Yang. Correlated time series forecasting using multi-task deep neural networks. In CIKM, pages 1527-1530, 2018.","title":"Correlated time series forecasting using multi-task deep neural networks","authors":"Razvan-Gabriel Cirstea, Darius-Valer Micu, Gabriel-Marcel Muresan, Chenjuan Guo, Bin Yang","year":"2018","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Deng and Platt, 2014] Li Deng and John C. Platt. Ensemble deep learning for speech recognition. In INTERSPEECH, pages 1915-1919, 2014.","title":"Ensemble deep learning for speech recognition","authors":"Li Deng, John C Platt","year":"2014","venue":"INTERSPEECH","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ding et al., 2016] Zhiming Ding, Bin Yang, Yuanying Chi, and Limin Guo. Enabling smart transportation systems: A parallel spatio-temporal database approach. IEEE Trans. Computers, 65(5):1377-1391, 2016.","title":"Enabling smart transportation systems: A parallel spatio-temporal database approach","authors":"Zhiming Ding, Bin Yang, Yuanying Chi, Limin Guo","year":"2016","venue":"IEEE Trans. Computers","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gal and Ghahramani, 2016] Yarin Gal and Zoubin Ghahra- mani. A theoretically grounded application of dropout in recurrent neural networks. In NIPS, pages 1019-1027, 2016.","title":"Yarin Gal and Zoubin Ghahramani. A theoretically grounded application of dropout in recurrent neural networks","authors":"Ghahramani Gal","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo et al., 2015] Haiyun Guo, Jinqiao Wang, and Hanqing Lu. Learning deep compact descriptor with bagging auto- encoders for object retrieval. In ICIP, pages 3175-3179, 2015.","title":"Learning deep compact descriptor with bagging autoencoders for object retrieval","authors":"Haiyun Guo, Jinqiao Wang, Hanqing Lu","year":"2015","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2018] Jilin Hu, Bin Yang, Chenjuan Guo, and Christian S. Jensen. Risk-aware path selection with time- varying, uncertain travel costs: a time series approach. VLDB J., 27(2):179-200, 2018.","title":"Risk-aware path selection with timevarying, uncertain travel costs: a time series approach","authors":"Hu","year":"2018","venue":"VLDB J","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kieu et al., 2018a] Tung Kieu, Bin Yang, Chenjuan Guo, and Christian S. Jensen. Distinguishing trajectories from different drivers using incompletely labeled trajectories. In CIKM, pages 863-872, 2018.","title":"Distinguishing trajectories from different drivers using incompletely labeled trajectories","authors":"Kieu","year":"2018","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kieu et al., 2018b] Tung Kieu, Bin Yang, and Christian S. Jensen. Outlier detection for multidimensional time series using deep neural networks. In MDM, pages 125-134, 2018.","title":"Outlier detection for multidimensional time series using deep neural networks","authors":"Kieu","year":"2018","venue":"MDM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2017] Inwoong Lee, Doyoung Kim, Seoungyoon Kang, and Sanghoon Lee. Ensemble deep learning for skeleton-based action recognition using temporal sliding LSTM networks. In ICCV, pages 1012-1020, 2017.","title":"Ensemble deep learning for skeleton-based action recognition using temporal sliding LSTM networks","authors":"Lee","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liao et al., 2018] Weixian Liao, Yifan Guo, Xuhui Chen, and Pan Li. A unified unsupervised Gaussian mixture vari- ational autoencoder for high dimensional outlier detection. In BigData, pages 1208-1217, 2018.","title":"A unified unsupervised Gaussian mixture variational autoencoder for high dimensional outlier detection","authors":"Liao","year":"2018","venue":"BigData","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2008] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In ICDM, pages 413-422, 2008.","title":"Isolation forest","authors":"Tony Fei, Kai Liu, Ming Ting, Zhi-Hua Zhou","year":"2008","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Long et al., 2017] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Philip S. Yu. Learning multiple tasks with multilinear relationship networks. In NIPS, pages 1593- 1602, 2017.","title":"Learning multiple tasks with multilinear relationship networks","authors":"Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S Yu","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Luo and Nagarajan, 2018] Tie Luo and Sai G. Nagarajan. Distributed anomaly detection using autoencoder neural networks in WSN for IoT. In ICC, pages 1-6, 2018.","title":"Distributed anomaly detection using autoencoder neural networks in WSN for IoT","authors":"Nagarajan ; ] Tie Luo, Sai G Luo, Nagarajan","year":"2018","venue":"ICC","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D. Manning. Effective approaches to attention-based neural machine translation. In EMNLP, pages 1412-1421, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Luong","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Malhotra et al., 2016] Pankaj Malhotra, Anusha Ramakrish- nan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. LSTM-based encoder-decoder for multi- sensor anomaly detection. CoRR, abs/1607.00148, 2016.","title":"LSTM-based encoder-decoder for multisensor anomaly detection","authors":"Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, Gautam Shroff","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Manevitz and Yousef, 2001] Larry M. Manevitz and Malik Yousef. One-class SVMs for document classification. JMLR, 2:139-154, 2001.","title":"One-class SVMs for document classification","authors":"Larry M Manevitz, Malik Yousef","year":"2001","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Park et al., 2016] Daehyung Park, Zackory M. Erickson, Tapomayukh Bhattacharjee, and Charles C. Kemp. Multi- modal execution monitoring for anomaly detection during robot manipulation. In ICRA, pages 407-414, 2016.","title":"Multimodal execution monitoring for anomaly detection during robot manipulation","authors":"Park","year":"2016","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sammut and Webb, 2017] Claude Sammut and Geoffrey I. Webb, editors. Encyclopedia of Machine Learning and Data Mining. Springer, 2017.","title":"Encyclopedia of Machine Learning and Data Mining","authors":"Webb Sammut","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104-3112, 2014.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tagawa et al., 2014] Takaaki Tagawa, Yukihiro Tadokoro, and Takehisa Yairi. Structured denoising autoencoder for fault detection and analysis. In ACML, 2014.","title":"Structured denoising autoencoder for fault detection and analysis","authors":"Takaaki Tagawa, Yukihiro Tadokoro, Takehisa Yairi","year":"2014","venue":"ACML","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Vincent et al., 2008] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In ICML, pages 1096-1103, 2008.","title":"Extracting and composing robust features with denoising autoencoders","authors":"Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang and Tian, 2016] Yiren Wang and Fei Tian. Recurrent residual learning for sequence classification. In EMNLP, pages 938-943, 2016.","title":"Recurrent residual learning for sequence classification","authors":"Wang, Tian ; ] Yiren Wang, Fei Tian","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xia et al., 2015] Yan Xia, Xudong Cao, Fang Wen, Gang Hua, and Jian Sun. Learning discriminative reconstructions for unsupervised outlier removal. In ICCV, pages 1511- 1519, 2015.","title":"Learning discriminative reconstructions for unsupervised outlier removal","authors":"Yan Xia, Xudong Cao, Fang Wen, Gang Hua, Jian Sun","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yang et al., 2013] Bin Yang, Chenjuan Guo, and Christian S. Jensen. Travel cost inference from sparse, spatio-temporally correlated time series using markov models. PVLDB, 6(9):769-780, 2013.","title":"Travel cost inference from sparse, spatio-temporally correlated time series using markov models","authors":"Yang","year":"2013","venue":"PVLDB","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2018] Bin Yang, Jian Dai, Chenjuan Guo, Chris- tian S. Jensen, and Jilin Hu. PACE: a path-centric paradigm for stochastic path finding. VLDB J., 27(2):153-178, 2018.","title":"PACE: a path-centric paradigm for stochastic path finding","authors":"Bin Yang, Jian Dai, Chenjuan Guo, Christian S Jensen, Jilin Hu","year":"2018","venue":"VLDB J","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yeh et al., 2016] Chin-Chia Michael Yeh, Yan Zhu, Liud- mila Ulanova, Nurjahan Begum, Yifei Ding, Hoang Anh Dau, Diego Furtado Silva, Abdullah Mueen, and Eamonn J. Keogh. Matrix profile I: all pairs similarity joins for time series: A unifying view that includes motifs, discords and shapelets. In ICDM, pages 1317-1322, 2016.","title":"Matrix profile I: all pairs similarity joins for time series: A unifying view that includes motifs, discords and shapelets","authors":"Chin-Chia Michael Yeh, Yan Zhu, Liudmila Ulanova, Nurjahan Begum, Yifei Ding, Anh Hoang, Diego Furtado Dau, Abdullah Silva, Eamonn J Mueen, Keogh","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zeiler, 2012] Matthew D. Zeiler. ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701, 2012.","title":"ADADELTA: an adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhao et al., 2017] Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua. Spatio-temporal autoencoder for video anomaly detection. In ACM MM, pages 1933-1941, 2017.","title":"Spatio-temporal autoencoder for video anomaly detection","authors":"Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, Xian-Sheng Hua","year":"2017","venue":"ACM MM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}