{"type":"ScholarlyArticle","title":"Online Evasion Attacks on Recurrent Models:The Power of Hallucinating the Future","authors":[{"name":"Byunggill Joe","slug":"byunggill-joe","given":"Byunggill","family":"Joe","affiliation":"School of Computing , KAIST , Daejeon , South Korea"},{"name":"Insik Shin","slug":"insik-shin","given":"Insik","family":"Shin","affiliation":"School of Computing , KAIST , Daejeon , South Korea"},{"name":"Jihun Hamm","slug":"jihun-hamm","given":"Jihun","family":"Hamm","affiliation":"Department of Computer Science , Tulane University , Louisiana , USA"}],"abstract":"Recurrent models are frequently being used in online tasks such as autonomous driving, and a comprehensive study of their vulnerability is called for. Existing research is limited in generality only addressing application-specific vulnerability or making implausible assumptions such as the knowledge of future input. In this paper, we present a general attack framework for online tasks incorporating the unique constraints of the online setting different from offline tasks. Our framework is versatile in that it covers time-varying adversarial objectives and various optimization constraints, allowing for a comprehensive study of robustness. Using the framework, we also present a novel white-box attack called Predictive Attack that `hallucinates' the future. The attack achieves 98 percent of the performance of the ideal but infeasible clairvoyant attack on average. We validate the effectiveness of the proposed framework and attacks through various experiments.","doi":"10.24963/ijcai.2022/433","url":"https://ijcai.org/proceedings/2022/433","pdf":"https://ijcai.org/proceedings/2022/0433.pdf","markdown":"https://ijcai.org/proceedings/2022/433.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/433","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":433,"pages":{"first":3121,"last":3127},"keywords":["Machine Learning: Recurrent Networks","Machine Learning: Robustness","Computer Vision: Adversarial learning, adversarial attack and defense methods","Machine Learning: Adversarial Machine Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986483","length_sec":90,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984626","length_sec":352,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Candanedo et al., 2017] Luis Miguel Candanedo, Véronique Feldheim, and Dominique Deramaix. Data driven predic- tion models of energy use of appliances in a low-energy house. Energy and Buildings, 140:81-97, 2017.","title":"Data driven prediction models of energy use of appliances in a low-energy house","authors":"Luis Miguel Candanedo, Véronique Feldheim, Dominique Deramaix","year":"2017","venue":"Energy and Buildings","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Casale, 2014] Pierluigi Casale. User Identification From Walking Activity. UCI Machine Learning Repository, 2014.","title":"User Identification From Walking Activity","authors":"Pierluigi Casale","year":"2014","venue":"UCI Machine Learning Repository","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Croce and Hein, 2020] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adap- tive boundary attack. In International Conference on Ma- chine Learning, 2020.","title":"Minimally distorted adversarial examples with a fast adaptive boundary attack","authors":"Francesco Croce, Matthias Hein","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dang-Nhu et al., 2020] Raphaël Dang-Nhu, Gagandeep Singh, Pavol Bielik, and Martin Vechev. Adversarial at- tacks on probabilistic autoregressive forecasting models. In International Conference on Machine Learning, pages 2356-2365. PMLR, 2020.","title":"Adversarial attacks on probabilistic autoregressive forecasting models","authors":"Raphaël Dang-Nhu, Gagandeep Singh, Pavol Bielik, Martin Vechev","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fawaz et al., 2019] Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Adversarial attacks on deep neural networks for time series classification. In 2019 International Joint Con- ference on Neural Networks (IJCNN), pages 1-8. IEEE, 2019.","title":"Adversarial attacks on deep neural networks for time series classification","authors":"Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller","year":"2019","venue":"International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gondim-ribeiro et al., 2018] George Gondim-ribeiro, Pedro Tabacof, and Eduardo Valle. Adversarial Attacks on Varia- tional Autoencoders. arXiv:1806.04646, 2018.","title":"Adversarial Attacks on Variational Autoencoders","authors":"George Gondim-Ribeiro, Pedro Tabacof, Eduardo Valle","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong et al., 2019] Yuan Gong, Boyang Li, Christian Poellabauer, and Yiyu Shi. Real-time adversarial attacks. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI'19, page 4672-4680, 2019.","title":"Real-time adversarial attacks","authors":"Yuan Gong, Boyang Li, Christian Poellabauer, Yiyu Shi","year":"2019","venue":"Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI'19","doi":null,"url":null,"internal_url":"/proceedings/2019/649"},{"position":8,"raw":"[Gonzalez et al., 2017] Eric Gonzalez, MacCallister Higgins, and Oliver Cameron. Udacity self-driving car chal- lenge. https://github.com/udacity/self-driving-car, 2017. Accessed: 2022-06-05.","title":"Udacity self-driving car challenge","authors":"Eric Gonzalez, Maccallister Higgins, Oliver Cameron","year":"2022","venue":null,"doi":null,"url":"https://github.com/udacity/self-driving-car,2017","internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In International Conference on Learning Representations (ICLR), 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Harutyunyan et al., 2019] Hrayr Harutyunyan, Hrant Khachatrian, David C Kale, Greg Ver Steeg, and Aram Galstyan. Multitask learning and benchmarking with clinical time series data. Scientific data, 6(1):1-18, 2019.","title":"Multitask learning and benchmarking with clinical time series data","authors":"Hrayr Harutyunyan, Hrant Khachatrian, Greg David C Kale, Aram Ver Steeg, Galstyan","year":"2019","venue":"Scientific data","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2020] Wenhui Huang, Jason Gu, Xin Ma, and Yibin Li. End-to-end multitask siamese network with resid- ual hierarchical attention for real-time object tracking. Ap- plied Intelligence, 50(6):1908-1921, 2020.","title":"End-to-end multitask siamese network with residual hierarchical attention for real-time object tracking","authors":"Wenhui Huang, Jason Gu, Xin Ma, Yibin Li","year":"2020","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kiran et al., 2021] Bangalore Ravi Kiran, Ibrahim Sobh, Vic- tor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021.","title":"Deep reinforcement learning for autonomous driving: A survey","authors":"Kiran","year":"2021","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In International Conference on Learning Representations (ICLR), 2017.","title":"Delving into transferable adversarial examples and black-box attacks","authors":"Liu","year":"2017","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma ¸dry et al., 2018] Aleksander Ma ¸dry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations (ICLR), 2018.","title":"Towards Deep Learning Models Resistant to Adversarial Attacks","authors":"Aleksander Ma ¸dry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Oregi et al., 2018] Izaskun Oregi, Javier Del Ser, Aritz Perez, and Jose A Lozano. Adversarial sample crafting for time series classification with elastic similarity measures. In International Symposium on Intelligent and Distributed Computing, pages 26-39. Springer, 2018.","title":"Adversarial sample crafting for time series classification with elastic similarity measures","authors":"Izaskun Oregi, Javier Del Ser, Aritz Perez, Jose A Lozano","year":"2018","venue":"International Symposium on Intelligent and Distributed Computing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pinto et al., 2021] Tiago Pinto, Isabel Prac ¸a, Zita Vale, and Jose Silva. Ensemble learning for electricity consumption forecasting in office buildings. Neurocomputing, 423:747- 755, 2021.","title":"Ensemble learning for electricity consumption forecasting in office buildings","authors":"Tiago Pinto, Isabel Prac ¸a, Zita Vale, Jose Silva","year":"2021","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Suradhaniwar et al., 2021] Saurabh Suradhaniwar, Soumyashree Kar, Surya S Durbha, and Adinarayana Jagarlapudi. Time series forecasting of univariate agrome- teorological data: A comparative performance evaluation via one-step and multi-step ahead forecasting strategies. Sensors, 21(7):2430, 2021.","title":"Time series forecasting of univariate agrometeorological data: A comparative performance evaluation via one-step and multi-step ahead forecasting strategies","authors":"Saurabh Suradhaniwar, Soumyashree Kar, Adinarayana Surya S Durbha, Jagarlapudi","year":"2021","venue":"Sensors","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Szegedy et al., 2014] Christian Szegedy, Joan Bruna, Du- mitru Erhan, and Ian Goodfellow. Intriguing properties of neural networks. In In International Conference on Learning Representations (ICLR), 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Voll- graf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv, cs.LG/1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xie et al., 2020] Yi Xie, Cong Shi, Zhuohang Li, Jian Liu, Yingying Chen, and Bo Yuan. Real-time, universal, and ro- bust adversarial attacks against speaker recognition systems. In ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 1738-1742. IEEE, 2020.","title":"Real-time, universal, and robust adversarial attacks against speaker recognition systems","authors":"Yi Xie, Cong Shi, Zhuohang Li, Jian Liu, Yingying Chen, Bo Yuan","year":"2020","venue":"ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu et al., 2020] Wei Yu, Yichao Lu, Steve Easterbrook, and Sanja Fidler. Efficient and information-preserving future frame prediction and beyond. In International Conference on Learning Representations, 2020.","title":"Efficient and information-preserving future frame prediction and beyond","authors":"Wei Yu, Yichao Lu, Steve Easterbrook, Sanja Fidler","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2019] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jor- dan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.","title":"Theoretically principled trade-off between robustness and accuracy","authors":"Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, Michael I Jordan","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}