{"type":"ScholarlyArticle","title":"Curriculum Adversarial Training","authors":[{"name":"Qi-Zhi Cai","slug":"qi-zhi-cai","given":"Qi-Zhi","family":"Cai","affiliation":"Nanjing University"},{"name":"Chang Liu","slug":"chang-liu","given":"Chang","family":"Liu","affiliation":"UC Berkeley"},{"name":"Dawn Song","slug":"dawn-song","given":"Dawn","family":"Song","affiliation":"UC Berkeley"}],"abstract":"Recently, deep learning has been applied to many security-sensitive applications, such as facial authentication. The existence of adversarial examples hinders such applications. The state-of-the-art result on defense shows that adversarial training can be applied to train a robust model on MNIST against adversarial examples; but it fails to achieve a high empirical worst-case accuracy on a more complex task, such as CIFAR-10 and SVHN. \r\n\r\nIn our work, we propose curriculum adversarial training (CAT) to resolve this issue. The basic idea is to develop a curriculum of adversarial examples generated by attacks with a wide range of strengths. With two techniques to mitigate the catastrophic forgetting and the generalization issues, we demonstrate that CAT can improve the prior art's empirical worst-case accuracy by a large margin of 25% on CIFAR-10 and 35% on SVHN. At the same, the model's performance on non-adversarial inputs is comparable to the state-of-the-art models.","doi":"10.24963/ijcai.2018/520","url":"https://ijcai.org/proceedings/2018/520","pdf":"https://ijcai.org/proceedings/2018/0520.pdf","markdown":"https://ijcai.org/proceedings/2018/520.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/520","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":520,"pages":{"first":3740,"last":3747},"keywords":["Machine Learning: Deep Learning","Multidisciplinary Topics and Applications: Security and Privacy"],"videos":[],"references":[{"position":1,"raw":"[Aditi Raghunathan, 2018] Percy Liang Aditi Raghunathan, Jacob Steinhardt. Certified defenses against adversarial ex- amples. Proc. of ICLR, 2018.","title":"Certified defenses against adversarial examples","authors":"Percy Liang, Aditi Raghunathan, Jacob Steinhardt","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aman Sinha, 2018] John Duchi Aman Sinha, Hongseok Namkoong. Certifiable distributional ro- bustness with principled adversarial training. Proc. of ICLR, 2018.","title":"Certifiable distributional robustness with principled adversarial training","authors":"John Duchi, Aman Sinha, Hongseok Namkoong","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Anish Athalye, 2018] David Wagner Anish Athalye, Nicholas Carlini. Obfuscated gradients give a false sense of security. Proc. of ICML, 2018.","title":"Obfuscated gradients give a false sense of security","authors":"David Wagner, Anish Athalye, Nicholas Carlini","year":"2018","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Buckman et al., 2018] Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In Submissions to International Conference on Learning Representations, 2018.","title":"Thermometer encoding: One hot way to resist adversarial examples","authors":"Buckman","year":"2018","venue":"Submissions to International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carlini and Wagner, 2017a] Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: By- passing ten detection methods. In Proc. of ACM Workshop on Artificial Intelligence and Security, pages 3-14, 2017.","title":"Adversarial examples are not easily detected: Bypassing ten detection methods","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"Proc. of ACM Workshop on Artificial Intelligence and Security","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carlini and Wagner, 2017b] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural net- works. In Proc. of Euro S & P, pages 39-57, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"Proc. of Euro S & P","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2015] Chenyi Chen, Ari Seff, Alain Korn- hauser, and Jianxiong Xiao. Deepdriving: Learning af- fordance for direct perception in autonomous driving. In Proc. of ICCV, pages 2722-2730, 2015.","title":"Deepdriving: Learning affordance for direct perception in autonomous driving","authors":"Chen","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. Proc. of ICLR, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2017] Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are not strong. In Proc. of WOOT, 2017.","title":"Adversarial example defense: Ensembles of weak defenses are not strong","authors":"Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song","year":"2017","venue":"Proc. of WOOT","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hinton et al., 2012] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acous- tic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, pages 82-97, 2012.","title":"Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups","authors":"Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-Rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath","year":"2012","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely con- nected convolutional networks. In Proc. of CVPR, pages 2261-2269, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kolter and Wong, 2017] J Zico Kolter and Eric Wong. Prov- able defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint, 2017.","title":"Provable defenses against adversarial examples via the convex outer adversarial polytope","authors":"Wong ; ] J Zico Kolter, Eric Kolter, Wong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kurakin et al., 2017] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. Proc. of ICLR, 2017.","title":"Adversarial machine learning at scale","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun, 1998] Yann LeCun. The mnist database of hand- written digits. 1998.","title":"The mnist database of handwritten digits","authors":"Yann Lecun","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2016] Bo Li, Yevgeniy Vorobeychik, and Xinyun Chen. A general retraining framework for scalable adver- sarial classification. arXiv preprint, 2016.","title":"A general retraining framework for scalable adversarial classification","authors":"Bo Li, Yevgeniy Vorobeychik, Xinyun Chen","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2017] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. Proc. of ICLR, 2017.","title":"Delving into transferable adversarial examples and black-box attacks","authors":"Liu","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. Proc. of ICLR, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Moosavi-Dezfooli et al., 2017] Seyed-Mohsen Moosavi- Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proc. of CVPR, pages 86-94, 2017.","title":"Universal adversarial perturbations","authors":"Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard","year":"2017","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learn- ing. 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Papernot et al., 2016a] Nicolas Papernot, Patrick Mc- Daniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against deep learning systems using adversarial examples. arXiv preprint, 2016.","title":"Practical black-box attacks against deep learning systems using adversarial examples","authors":"Papernot","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Papernot et al., 2016b] Nicolas Papernot, Patrick Mc- Daniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In Proc. of Euro S & P, 2016 IEEE Symposium on, pages 582-597, 2016.","title":"Distillation as a defense to adversarial perturbations against deep neural networks","authors":"Papernot","year":"2016","venue":"Proc. of Euro S & P","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sharma and Chen, 2018] Yash Sharma and Pin-Yu Chen. Attacking the madry defense model with l 1-based adver- sarial examples. Proc. of AAAI, 2018.","title":"Attacking the madry defense model with l 1-based adversarial examples","authors":"Chen ; Sharma, Yash Sharma, Pin-Yu Chen","year":"2018","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J Maddi- son, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, pages 484-489, 2016.","title":"Mastering the game of go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Julian Driessche, Ioannis Schrittwieser, Veda Antonoglou, Marc Panneershelvam, Lanctot","year":"2016","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2014] Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation from predicting 10,000 classes. In Proc. of CVPR, pages 1891-1898, 2014.","title":"Deep learning face representation from predicting 10,000 classes","authors":"Yi Sun, Xiaogang Wang, Xiaoou Tang","year":"2014","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2018] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. Proc. of NDSS, 2018.","title":"Feature squeezing: Detecting adversarial examples in deep neural networks","authors":"Weilin Xu, David Evans, Yanjun Qi","year":"2018","venue":"Proc. of NDSS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}