{"type":"ScholarlyArticle","title":"Towards Adversarially Robust Deep Image Denoising","authors":[{"name":"Hanshu Yan","slug":"hanshu-yan","given":"Hanshu","family":"Yan","affiliation":""},{"name":"Jingfeng Zhang","slug":"jingfeng-zhang","given":"Jingfeng","family":"Zhang","affiliation":"RIKEN-AIP"},{"name":"Jiashi Feng","slug":"jiashi-feng","given":"Jiashi","family":"Feng","affiliation":"ByteDance Inc."},{"name":"Masashi Sugiyama","slug":"masashi-sugiyama","given":"Masashi","family":"Sugiyama","affiliation":"RIKEN-AIP; GSFS , UTokyo"},{"name":"Vincent Y. F. Tan","slug":"vincent-y-f-tan","given":"Vincent Y. F.","family":"Tan","affiliation":"Math , NUS"}],"abstract":"This work systematically investigates the adversarial robustness of deep image denoisers (DIDs), i.e, how well DIDs can recover the ground truth from noisy observations degraded by adversarial perturbations. Firstly, to evaluate DIDs’ robustness, we propose a novel adversarial attack, namely Observation-based Zero-mean Attack (OBSATK), to craft adversarial zero-mean perturbations on given noisy images. We find that existing DIDs are vulnerable to the adversarial noise generated by OBSATK. Secondly, to robustify DIDs, we pro- pose an adversarial training strategy, hybrid adversarial training (HAT), that jointly trains DIDs with adversarial and non-adversarial noisy data to ensure that the reconstruction quality is high and the denoisers around non-adversarial data are locally smooth. The resultant DIDs can effectively remove various types of synthetic and adversarial noise. We also uncover that the robustness of DIDs benefits their generalization capability on unseen real-world noise. Indeed, HAT-trained DIDs can recover high-quality clean images from real-world noise even without training on real noisy data. Extensive experiments on benchmark datasets, including Set68, PolyU, and SIDD, corroborate the effectiveness of OBSATK and HAT.","doi":"10.24963/ijcai.2022/211","url":"https://ijcai.org/proceedings/2022/211","pdf":"https://ijcai.org/proceedings/2022/0211.pdf","markdown":"https://ijcai.org/proceedings/2022/211.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/211","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":211,"pages":{"first":1516,"last":1522},"keywords":["Computer Vision: Adversarial learning, adversarial attack and defense methods","AI Ethics, Trust, Fairness: Safety & Robustness","AI Ethics, Trust, Fairness: Trustworthy AI","Computer Vision: Applications"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986585","length_sec":67,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984728","length_sec":357,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abdelhamed et al., 2018] Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. A High-Quality Denoising Dataset for Smartphone Cameras. In CVPR, 2018.","title":"A High-Quality Denoising Dataset for Smartphone Cameras","authors":"Abdelrahman Abdelhamed, Stephen Lin, Michael S Brown","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anwar and Barnes, 2019] Saeed Anwar and Nick Barnes. Real Image Denoising with Feature Attention. In ICCV, 2019.","title":"Real Image Denoising with Feature Attention","authors":"Saeed Anwar, Nick Barnes","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Batson and Royer, 2019] Joshua Batson and Loic Royer. Noise2Self: Blind Denoising by Self-Supervision. In ICML, 2019.","title":"Noise2Self: Blind Denoising by Self-Supervision","authors":"Royer Batson, Joshua Batson, Loic Royer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. arXiv, 2017.","title":"Towards Evaluating the Robustness of Neural Networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2019] Jun-Ho Choi, Huan Zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, and Jong-Seok Lee. Evaluating Ro- bustness of Deep Image Super-Resolution against Adver- sarial Attacks. In ICCV, 2019.","title":"Evaluating Robustness of Deep Image Super-Resolution against Adversarial Attacks","authors":"Choi","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2021] Jun-Ho Choi, Huan Zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, and Jong-Seok Lee. Deep Image Destruction: A Comprehensive Study on Vulnerability of Deep Image-to-Image Models against Adversarial At- tacks. arXiv, 2021.","title":"Deep Image Destruction: A Comprehensive Study on Vulnerability of Deep Image-to-Image Models against Adversarial Attacks","authors":"Choi","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dabov et al., 2007] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image De- noising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE TIP, 16, 2007.","title":"Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering","authors":"Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, Karen Egiazarian","year":"2007","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2019] Ruiqi Gao, Tianle Cai, Haochuan Li, Cho- Jui Hsieh, Liwei Wang, and Jason D. Lee. Convergence of Adversarial Training in Overparametrized Neural Net- works. In NeurIPS, 2019.","title":"Convergence of Adversarial Training in Overparametrized Neural Networks","authors":"Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, Jason D Lee","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gondara, 2016] Lovedeep Gondara. Medical image de- noising using convolutional denoising autoencoders. In ICDMW, 2016.","title":"Lovedeep Gondara. Medical image denoising using convolutional denoising autoencoders","authors":"Gondara","year":"2016","venue":"ICDMW","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harness- ing Adversarial Examples. arXiv, 2015.","title":"Explaining and Harnessing Adversarial Examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2019] Shi Guo, Zifei Yan, Kai Zhang, Wang- meng Zuo, and Lei Zhang. Toward Convolutional Blind Denoising of Real Photographs. In CVPR, 2019.","title":"Toward Convolutional Blind Denoising of Real Photographs","authors":"Zifei Shi Guo, Kai Yan, Wangmeng Zhang, Lei Zuo, Zhang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kurakin et al., 2017] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv, 2017.","title":"Adversarial examples in the physical world","authors":"Alexey Kurakin, Ian Goodfellow, Samy Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lehtinen et al., 2018] Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2Noise: Learning Image Restoration without Clean Data. In ICML, 2018.","title":"Noise2Noise: Learning Image Restoration without Clean Data","authors":"Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, Timo Aila","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. In ICLR, 2018.","title":"Towards Deep Learning Models Resistant to Adversarial Attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shafahi et al., 2019] Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Ad- versarial Training for Free! In NeurIPS, 2019.","title":"Adversarial Training for Free! In NeurIPS","authors":"Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, Tom Goldstein","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tramer et al., 2020] Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On Adaptive Attacks to Adversarial Example Defenses. arXiv, 2020.","title":"On Adaptive Attacks to Adversarial Example Defenses","authors":"Florian Tramer, Nicholas Carlini, Wieland Brendel, Aleksander Madry","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ulyanov et al., 2018] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep Image Prior. In CVPR, 2018.","title":"Deep Image Prior","authors":"Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2017] Jun Xu, Lei Zhang, David Zhang, and Xi- angchu Feng. Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising. In ICCV, 2017.","title":"Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising","authors":"Jun Xu, Lei Zhang, David Zhang, Xiangchu Feng","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2018] Jun Xu, Hui Li, Zhetong Liang, David Zhang, and Lei Zhang. Real-world Noisy Image Denois- ing: A New Benchmark. arXiv, 2018.","title":"Real-world Noisy Image Denoising: A New Benchmark","authors":"Jun Xu, Hui Li, Zhetong Liang, David Zhang, Lei Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yan et al., 2019] Hanshu Yan, Jiawei Du, Vincent Tan, and Jiashi Feng. On Robustness of Neural Ordinary Differen- tial Equations. In ICLR, 2019.","title":"On Robustness of Neural Ordinary Differential Equations","authors":"Hanshu Yan, Jiawei Du, Vincent Tan, Jiashi Feng","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yan et al., 2021] Hanshu Yan, Jingfeng Zhang, Gang Niu, Jiashi Feng, Vincent Tan, and Masashi Sugiyama. CIFS: Improving Adversarial Robustness of CNNs via Channel- wise Importance-based Feature Selection. In ICML, 2021.","title":"CIFS: Improving Adversarial Robustness of CNNs via Channelwise Importance-based Feature Selection","authors":"Hanshu Yan, Jingfeng Zhang, Gang Niu, Jiashi Feng, Vincent Tan, Masashi Sugiyama","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2022] Hanshu Yan, Jingfeng Zhang, Jiashi Feng, Masashi Sugiyama, and Vincent Y. F. Tan. Towards adver- sarially robust deep image denoising. arXiv, 2022.","title":"Towards adversarially robust deep image denoising","authors":"Hanshu Yan, Jingfeng Zhang, Jiashi Feng, Masashi Sugiyama, Y F Vincent, Tan","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2017] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image De- noising. IEEE TIP, 2017.","title":"Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising","authors":"Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, Lei Zhang","year":"2017","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2019a] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically Principled Trade-off between Robustness and Accuracy. In ICML, 2019.","title":"Theoretically Principled Trade-off between Robustness and Accuracy","authors":"Zhang","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2019b] Yide Zhang, Yinhao Zhu, Evan Nichols, Qingfei Wang, Siyuan Zhang, Cody Smith, and Scott Howard. A Poisson gaussian Denoising Dataset with Real Fluorescence Microscopy Images. In CVPR, 2019.","title":"A Poisson gaussian Denoising Dataset with Real Fluorescence Microscopy Images","authors":"Zhang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2020a] Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks Which Do Not Kill Training Make Adversarial Learning Stronger. In ICML, 2020.","title":"Attacks Which Do Not Kill Training Make Adversarial Learning Stronger","authors":"Zhang","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2020b] Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankan- halli. Geometry-aware Instance-reweighted Adversarial Training. In ICLR, 2020.","title":"Geometry-aware Instance-reweighted Adversarial Training","authors":"Zhang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}