{"type":"ScholarlyArticle","title":"A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space","authors":[{"name":"Thibault Simonetto","slug":"thibault-simonetto","given":"Thibault","family":"Simonetto","affiliation":"University of Luxembourg"},{"name":"Salijona Dyrmishi","slug":"salijona-dyrmishi","given":"Salijona","family":"Dyrmishi","affiliation":"University of Luxembourg"},{"name":"Salah Ghamizi","slug":"salah-ghamizi","given":"Salah","family":"Ghamizi","affiliation":"University of Luxembourg"},{"name":"Maxime Cordy","slug":"maxime-cordy","given":"Maxime","family":"Cordy","affiliation":"University of Luxembourg"},{"name":"Yves Le Traon","slug":"yves-le-traon","given":"Yves","family":"Le Traon","affiliation":"University of Luxembourg"}],"abstract":"The generation of feasible adversarial examples is necessary for properly assessing models that work in constrained feature space. However, it remains a challenging task to enforce constraints into attacks that were designed for computer vision. We propose a unified framework to generate feasible adversarial examples that satisfy given domain constraints. Our framework can handle both linear and non-linear constraints. We instantiate our framework into two algorithms: a gradient-based attack that introduces constraints in the loss function to maximize, and a multi-objective search algorithm that aims for misclassification, perturbation minimization, and constraint satisfaction. We show that our approach is effective in four different domains, with a success rate of up to 100%, where state-of-the-art attacks fail to generate a single feasible example. In addition to adversarial retraining, we propose to introduce engineered non-convex constraints to improve model adversarial robustness. We demonstrate that this new defense is as effective as adversarial retraining. Our framework forms the starting point for research on constrained adversarial attacks and provides relevant baselines and datasets that future research can exploit.","doi":"10.24963/ijcai.2022/183","url":"https://ijcai.org/proceedings/2022/183","pdf":"https://ijcai.org/proceedings/2022/0183.pdf","markdown":"https://ijcai.org/proceedings/2022/183.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/183","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":183,"pages":{"first":1313,"last":1319},"keywords":["Constraint Satisfaction and Optimization: Constraints and Machine Learning","Search: Evolutionary Computation","Computer Vision: Adversarial learning, adversarial attack and defense methods","Constraint Satisfaction and Optimization: Constraint Satisfaction","Constraint Satisfaction and Optimization: Constraint Optimization"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986893","length_sec":94,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985036","length_sec":347,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aghakhani et al., 2020] Hojjat Aghakhani, Fabio Gritti, Francesco Mecca, Martina Lindorfer, Stefano Ortolani, Davide Balzarotti, Giovanni Vigna, and Christopher Kruegel. When malware is packin'heat; limits of machine learning classifiers based on static analysis features. In Network and Distributed Systems Security (NDSS) Symposium 2020, 2020.","title":"When malware is packin'heat; limits of machine learning classifiers based on static analysis features","authors":"Hojjat Aghakhani, Fabio Gritti, Francesco Mecca, Martina Lindorfer, Stefano Ortolani, Davide Balzarotti, Giovanni Vigna, Christopher Kruegel","year":"2020","venue":"Network and Distributed Systems Security (NDSS) Symposium 2020","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alzantot et al., 2018] Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998, 2018.","title":"Generating natural language adversarial examples","authors":"Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, Kai-Wei Chang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carlini et al., 2019] Nicholas Carlini, Anish Athalye, Nicolas Pa- pernot, Wieland Brendel, Jonas Rauber, D. Tsipras, I. Goodfel- low, A. Madry, and A. Kurakin. On evaluating adversarial ro- bustness. ArXiv, abs/1902.06705, 2019.","title":"On evaluating adversarial robustness","authors":"Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, D Tsipras, I Goodfellow, A Madry, A Kurakin","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chernikova and Oprea, 2019] Alesia Chernikova and Alina Oprea. Fence: Feasible evasion attacks on neural networks in con- strained environments. arXiv preprint arXiv:1909.10480, 2019.","title":"Fence: Feasible evasion attacks on neural networks in constrained environments","authors":"Alesia Chernikova, Alina Oprea","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dalvi et al., 2004] Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, and Deepak Verma. Adversarial classification. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 99-108, 2004.","title":"Adversarial classification","authors":"Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, Deepak Verma","year":"2004","venue":"Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Du et al., 2020] Tianyu Du, Shouling Ji, Jinfeng Li, Qinchen Gu, Ting Wang, and Raheem Beyah. Sirenattack: Generating adver- sarial audio for end-to-end acoustic systems. In Proceedings of the 15th ACM Asia Conference on Computer and Communica- tions Security, pages 357-369, 2020.","title":"Sirenattack: Generating adversarial audio for end-to-end acoustic systems","authors":"Tianyu Du, Shouling Ji, Jinfeng Li, Qinchen Gu, Ting Wang, Raheem Beyah","year":"2020","venue":"Proceedings of the 15th ACM Asia Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Erdemir et al., 2021] Ecenaz Erdemir, Jeffrey Bickford, Luca Melis, and Sergul Aydore. Adversarial robustness with non- uniform perturbations. arXiv preprint arXiv:2102.12002, 2021.","title":"Adversarial robustness with nonuniform perturbations","authors":"Erdemir","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ghamizi et al., 2020] Salah Ghamizi, Maxime Cordy, Martin Gubri, Mike Papadakis, Andrey Boystov, Yves Le Traon, and Anne Goujon. Search-based adversarial testing and improvement of constrained credit scoring systems. In Proc. of ESEC/FSE '20, pages 1089-1100, 2020.","title":"Search-based adversarial testing and improvement of constrained credit scoring systems","authors":"Salah Ghamizi, Maxime Cordy, Martin Gubri, Mike Papadakis, Andrey Boystov, Yves Le Traon, Anne Goujon","year":"2020","venue":"Proc. of ESEC/FSE '20","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ghamizi et al., 2021] Salah Ghamizi, Maxime Cordy, Mike Pa- padakis, and Yves Le Traon. Adversarial robustness in multi-task learning: Promises and illusions. arXiv preprint arXiv:2110.15053, 2021.","title":"Adversarial robustness in multi-task learning: Promises and illusions","authors":"Salah Ghamizi, Maxime Cordy, Mike Papadakis, Yves Le Traon","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2022. https: //www.gurobi.com/wp-content/plugins/hd documentations/ documentation/9.5/refman.pdf.","title":"LLC. Gurobi Optimizer Reference Manual","authors":null,"year":"2022","venue":null,"doi":null,"url":"https://www.gurobi.com/wp-content/plugins/hddocumentations/documentation/9.5/refman.pdf","internal_url":null},{"position":11,"raw":"[Hannousse and Yahiouche, 2021] Abdelhakim Hannousse and Salima Yahiouche. Towards benchmark datasets for machine learning based website phishing detection: An experimental study. Engineering Applications of Artificial Intelligence, 104:104347, 2021.","title":"Abdelhakim Hannousse and Salima Yahiouche. Towards benchmark datasets for machine learning based website phishing detection: An experimental study","authors":"Yahiouche Hannousse","year":"2021","venue":"Engineering Applications of Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kaggle, 2019] Kaggle. All Lending Club loan data, 2019.","title":"All Lending Club loan data","authors":"Kaggle","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kurakin et al., 2016] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016.","title":"Adversarial machine learning at scale","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2020] Jiangnan Li, Jin Young Lee, Yingyuan Yang, Jinyuan Stella Sun, and Kevin Tomsovic. Conaml: Constrained adversarial machine learning for cyber-physical systems. arXiv preprint arXiv:2003.05631, 2020.","title":"Conaml: Constrained adversarial machine learning for cyber-physical systems","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mode and Hoque, 2020] Gautam Raj Mode and Khaza Anuarul Hoque. Crafting adversarial examples for deep learn- ing based prognostics (extended version). arXiv preprint arXiv:2009.10149, 2020.","title":"Crafting adversarial examples for deep learning based prognostics","authors":"Raj Gautam, Khaza Anuarul Mode, Hoque","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Papernot et al., 2016] Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phe- nomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016.","title":"Transferability in machine learning: from phenomena to black-box attacks using adversarial samples","authors":"Papernot","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pierazzi et al., 2020] Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. Intriguing properties of ad- versarial ml attacks in the problem space. In 2020 IEEE Sym- posium on Security and Privacy (SP), pages 1332-1349. IEEE, 2020.","title":"Intriguing properties of adversarial ml attacks in the problem space","authors":"Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, Lorenzo Cavallaro","year":"2020","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sadeghi and Larsson, 2019] Meysam Sadeghi and Erik G Larsson. Physical adversarial attacks against end-to-end autoencoder com- munication systems. IEEE Communications Letters, 23:847- 850, 2019.","title":"Physical adversarial attacks against end-to-end autoencoder communication systems","authors":"Meysam Sadeghi, Erik G Larsson","year":"2019","venue":"IEEE Communications Letters","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sheatsley et al., 2020] Ryan Sheatsley, Nicolas Papernot, Michael Weisman, Gunjan Verma, and Patrick McDaniel. Adver- sarial examples in constrained domains. arXiv preprint arXiv:2011.01183, 2020.","title":"Adversarial examples in constrained domains","authors":"Ryan Sheatsley, Nicolas Papernot, Michael Weisman, Gunjan Verma, Patrick Mcdaniel","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shrikumar et al., 2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagat- ing activation differences. In International Conference on Ma- chine Learning. PMLR, 2017.","title":"Learning important features through propagating activation differences","authors":"Shrikumar","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Song et al., 2018] Qingquan Song, Haifeng Jin, Xiao Huang, and Xia Hu. Multi-label adversarial perturbations. In 2018 IEEE International Conference on Data Mining (ICDM), pages 1242- 1247. IEEE, 2018.","title":"Multi-label adversarial perturbations","authors":"Qingquan Song, Haifeng Jin, Xiao Huang, Xia Hu","year":"2018","venue":"2018 IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tian et al., 2020] Yunzhe Tian, Yingdi Wang, Endong Tong, Wen- jia Niu, Liang Chang, Qi Alfred Chen, Gang Li, and Jiqiang Liu. Exploring data correlation between feature pairs for generating constraint-based adversarial examples. In 2020 IEEE 26th In- ternational Conference on Parallel and Distributed Systems (IC- PADS), pages 430-437. IEEE, 2020.","title":"Exploring data correlation between feature pairs for generating constraint-based adversarial examples","authors":"Yunzhe Tian, Yingdi Wang, Endong Tong, Wenjia Niu, Liang Chang, Qi Alfred Chen, Gang Li, Jiqiang Liu","year":"2020","venue":"IEEE 26th International Conference on Parallel and Distributed Systems (IC-PADS)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vesikar et al., 2018] Yash Vesikar, Kalyanmoy Deb, and Julian Blank. Reference point based nsga-iii for preferred solutions. In 2018 IEEE symposium series on computational intelligence (SSCI), pages 1587-1594. IEEE, 2018.","title":"Reference point based nsga-iii for preferred solutions","authors":"Yash Vesikar, Kalyanmoy Deb, Julian Blank","year":"2018","venue":"IEEE symposium series on computational intelligence (SSCI)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yefet et al., 2020] Noam Yefet, Uri Alon, and Eran Yahav. Adver- sarial examples for models of code. Proceedings of the ACM on Programming Languages, 4:1-30, 2020.","title":"Adversarial examples for models of code","authors":"Yefet","year":"2020","venue":"Proceedings of the ACM on Programming Languages","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}