{"type":"ScholarlyArticle","title":"CAT: Customized Adversarial Training for Improved Robustness","authors":[{"name":"Minhao Cheng","slug":"minhao-cheng","given":"Minhao","family":"Cheng","affiliation":"Department of Computer Science and Engineering , HKUST"},{"name":"Qi Lei","slug":"qi-lei","given":"Qi","family":"Lei","affiliation":"Department of Electrical and Computer Engineering , Princeton University"},{"name":"Pin-Yu Chen","slug":"pin-yu-chen","given":"Pin-Yu","family":"Chen","affiliation":"IBM Research AI"},{"name":"Inderjit Dhillon","slug":"inderjit-dhillon","given":"Inderjit","family":"Dhillon","affiliation":"Department of Computer Science , UT Austin; Amazon"},{"name":"Cho-Jui Hsieh","slug":"cho-jui-hsieh","given":"Cho-Jui","family":"Hsieh","affiliation":"Department of Computer Science , UCLA"}],"abstract":"Adversarial training has become one of the most effective methods for improving robustness of neural networks. However, it often suffers from poor generalization on both clean and perturbed data. Current robust training method always use a uniformed perturbation strength for every samples to generate  adversarial examples during model training for improving adversarial robustness. However, we show it would lead worse training and generalizaiton error and forcing the prediction to match one-hot label.\r\nIn this paper, therefore, we propose a new algorithm, named Customized Adversarial Training (CAT), which adaptively customizes the perturbation level and the corresponding label for each training sample in adversarial training. We first show theoretically the CAT scheme improves the generalization. Also, through extensive experiments, we show that the proposed algorithm achieves better clean and robust accuracy than previous adversarial training methods. The full version of this paper is available at https://arxiv.org/abs/2002.06789.","doi":"10.24963/ijcai.2022/95","url":"https://ijcai.org/proceedings/2022/95","pdf":"https://ijcai.org/proceedings/2022/0095.pdf","markdown":"https://ijcai.org/proceedings/2022/95.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/95","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":95,"pages":{"first":673,"last":679},"keywords":["AI Ethics, Trust, Fairness: Safety & Robustness"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986584","length_sec":73,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984727","length_sec":351,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Athalye et al., 2018] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial exam- ples. International Coference on International Conference on Machine Learning, 2018.","title":"Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples","authors":"Anish Athalye, Nicholas Carlini, David Wagner","year":"2018","venue":"International Coference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Balaji et al., 2019] Yogesh Balaji, Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Im- proved accuracy tradeoffs in neural nets. arXiv preprint arXiv:1910.08051, 2019.","title":"Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets","authors":"Yogesh Balaji, Tom Goldstein, Judy Hoffman","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pages 39-57, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ding et al., 2018] Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin ad- versarial (mma) training: Direct input space margin max- imization through adversarial training. arXiv preprint arXiv:1812.02637, 2018.","title":"Max-margin adversarial (mma) training: Direct input space margin maximization through adversarial training","authors":"Gavin Weiguang, Ding, Yash Sharma, Kry Yik, Chau Lui, Ruitong Huang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Engstrom et al., 2018] Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the ro- bustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272, 2018.","title":"Evaluating and understanding the robustness of adversarial logit pairing","authors":"Engstrom","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2019] Ruiqi Gao, Tianle Cai, Haochuan Li, Cho- Jui Hsieh, Liwei Wang, and Jason D Lee. Convergence of adversarial training in overparametrized neural networks. In Advances in Neural Information Processing Systems, pages 13009-13020, 2019.","title":"Convergence of adversarial training in overparametrized neural networks","authors":"Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, Jason D Lee","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kurakin et al., 2017] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. International Conference on Learning Representations, 2017.","title":"Adversarial machine learning at scale","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Stutz et al., 2019] David Stutz, Matthias Hein, and Bernt Schiele. Confidence-calibrated adversarial training: To- wards robust models generalizing beyond the attack used during training. arXiv preprint arXiv:1910.06259, 2019.","title":"Confidence-calibrated adversarial training: Towards robust models generalizing beyond the attack used during training","authors":"David Stutz, Matthias Hein, Bernt Schiele","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Su et al., 2018] Dong Su, Huan Zhang, Hongge Chen, Jin- feng Yi, Pin-Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy?-a comprehensive study on the ro- bustness of 18 deep image classification models. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 631-648, 2018.","title":"Is robustness the cost of accuracy?-a comprehensive study on the robustness of 18 deep image classification models","authors":"Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, Yupeng Gao","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Tsipras et al., 2019] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In Inter- national Conference on Learning Representations, 2019.","title":"Robustness may be at odds with accuracy","authors":"Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, Aleksander Madry","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2019a] Yisen Wang, Xingjun Ma, James Bai- ley, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In International Conference on Machine Learning, pages 6586-6595, 2019.","title":"On the convergence and robustness of adversarial training","authors":"Wang","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2019b] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improv- ing adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Rep- resentations, 2019.","title":"Improving adversarial robustness requires revisiting misclassified examples","authors":"Wang","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang, 2019] Jianyu Wang. Bilateral adversarial training: Towards fast training of more robust models against ad- versarial attacks. International Conference on Computer Vision, 2019.","title":"Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks","authors":"Jianyu Wang","year":"2019","venue":"International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wei and Ma, 2019] Colin Wei and Tengyu Ma. Im- proved sample complexities for deep networks and ro- bust classification via an all-layer margin. arXiv preprint arXiv:1910.04284, 2019.","title":"Improved sample complexities for deep networks and robust classification via an all-layer margin","authors":"Colin Wei, Tengyu Ma","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yin et al., 2018] Dong Yin, Kannan Ramchandran, and Pe- ter Bartlett. Rademacher complexity for adversarially robust generalization. arXiv preprint arXiv:1810.11914, 2018.","title":"Rademacher complexity for adversarially robust generalization","authors":"Dong Yin, Kannan Ramchandran, Peter Bartlett","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2019] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jor- dan. Theoretically principled trade-off between robust- ness and accuracy. International Conference on Machine Learning, 2019.","title":"Theoretically principled trade-off between robustness and accuracy","authors":"Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, Michael I Jordan","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}