{"type":"ScholarlyArticle","title":"Universal Adaptive Data Augmentation","authors":[{"name":"Xiaogang Xu","slug":"xiaogang-xu","given":"Xiaogang","family":"Xu","affiliation":"Zhejiang Lab"},{"name":"Hengshuang Zhao","slug":"hengshuang-zhao","given":"Hengshuang","family":"Zhao","affiliation":"The University of Hong Kong"}],"abstract":"Existing automatic data augmentation (DA) methods either ignore updating DA's parameters according to the target model's state during training or adopt update strategies that are not effective enough. In this work, we design a novel data augmentation strategy called ``Universal Adaptive Data Augmentation\" (UADA). Different from existing methods, UADA would adaptively update DA's parameters according to the target model's gradient information during training: given a pre-defined set of DA operations, we randomly decide types and magnitudes of DA operations for every data batch during training, and adaptively update DA's parameters along the gradient direction of the loss concerning DA's parameters. In this way, UADA can increase the training loss of the target networks, and the target networks would learn features from harder samples to improve the generalization. Moreover, UADA is very general and can be utilized in numerous tasks, e.g., image classification, semantic segmentation and object detection. Extensive experiments with various models are conducted on CIFAR-10, CIFAR-100, ImageNet, tiny-ImageNet, Cityscapes, and VOC07+12 to prove the significant performance improvements brought by UADA.","doi":"10.24963/ijcai.2023/177","url":"https://ijcai.org/proceedings/2023/177","pdf":"https://ijcai.org/proceedings/2023/0177.pdf","markdown":"https://ijcai.org/proceedings/2023/177.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/177","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":177,"pages":{"first":1596,"last":1603},"keywords":["Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: Scene analysis and understanding","Computer Vision: CV: Segmentation"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2017] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In ACM workshop on artificial intelligence and security, 2017.","title":"Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models","authors":"Chen","year":"2017","venue":"ACM workshop on artificial intelligence and security","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The Cityscapes dataset for semantic urban scene under- standing. In CVPR, 2016.","title":"The Cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cubuk et al., 2019] Ekin D. Cubuk, Barret Zoph, Dandelion Mane, and et. al. Autoaugment: Learning augmentation strategies from data. In CVPR, 2019.","title":"Autoaugment: Learning augmentation strategies from data","authors":"D Ekin, Barret Cubuk, Dandelion Zoph, Mane","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cubuk et al., 2020] Ekin D. Cubuk, Barret Zoph, and et. al. Randaugment: Practical automated data augmentation with a reduced search space. In CVPRW, 2020.","title":"Randaugment: Practical automated data augmentation with a reduced search space","authors":"D Ekin, Barret Cubuk, Zoph","year":"2020","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Fei-Fei Li. ImageNet: A large-scale hierarchical image database. In CVPR, 2009.","title":"ImageNet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei Li","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[DeVries and Taylor, 2017] Terrance DeVries and Gra- ham W. Taylor. Improved regularization of convolutional neural networks with cutout. arXiv:1708.04552, 2017.","title":"Improved regularization of convolutional neural networks with cutout","authors":"Terrance Devries, Graham W Taylor","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Everingham et al., 2012] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 Results, 2012.","title":"The PASCAL Visual Object Classes Challenge 2012 Results","authors":"M Everingham, L Van Gool, C K I Williams, J Winn, A Zisserman","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gastaldi, 2017] Xavier Gastaldi. Shake-shake regularization. arXiv:1705.07485, 2017.","title":"Shake-shake regularization","authors":"Xavier Gastaldi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2015] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2019] Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial attacks. In ICML, 2019.","title":"Simple black-box adversarial attacks","authors":"Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, Kilian Weinberger","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hataya et al., 2020] Ryuichiro Hataya, Jan Zdenek, and et. al. Faster autoaugment: Learning augmentation strategies using backpropagation. In ECCV, 2020.","title":"Faster autoaugment: Learning augmentation strategies using backpropagation","authors":"Ryuichiro Hataya, Jan Zdenek","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016a] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016b] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016.","title":"Identity mappings in deep residual networks","authors":"He","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ho et al., 2019] Daniel Ho, Eric Liang, Xi Chen, and et. al. Population based augmentation: Efficient learning of aug- mentation policy schedules. In ICML, 2019.","title":"Population based augmentation: Efficient learning of augmentation policy schedules","authors":"Daniel Ho, Eric Liang, Xi Chen","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. In NIPS, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Le and Yang, 2015] Ya Le and Xuan Yang. Tiny ImageNet visual recognition challenge. CS 231N, 2015.","title":"Ya Le and Xuan Yang. Tiny ImageNet visual recognition challenge","authors":"Yang Le","year":"2015","venue":"CS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lee et al., 2019] Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Rethinking data augmentation: Self-supervision and self-distillation. arXiv:1910.05872, 2019.","title":"Rethinking data augmentation: Self-supervision and self-distillation","authors":"Lee","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2020] Yonggang Li, Guosheng Hu, Yongtao Wang, Timothy Hospedales, and et. al. Differentiable automatic data augmentation. In ECCV, 2020.","title":"Differentiable automatic data augmentation","authors":"Yonggang Li, Guosheng Hu, Yongtao Wang, Timothy Hospedales","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lim et al., 2019] Sungbin Lim, Ildoo Kim, Taesup Kim, and et. al. Fast autoaugment. In NIPS, 2019.","title":"Fast autoaugment","authors":"Lim","year":"2019","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2019] Chen Lin, Minghao Guo, Chuming Li, Xin Yuan, Wei Wu, Junjie Yan, Dahua Lin, and Wanli Ouyang. Online hyper-parameter learning for auto-augmentation strategy. In ICCV, 2019.","title":"Online hyper-parameter learning for auto-augmentation strategy","authors":"Chen Lin, Minghao Guo, Chuming Li, Xin Yuan, Wei Wu, Junjie Yan, Dahua Lin, Wanli Ouyang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[LingChen et al., 2020] Tom Ching LingChen, Ava Khon- sari, Amirreza Lashkari, Mina Rafi Nazari, Jaspreet Singh Sambee, and Mario A Nascimento. Uniformaugment: A search-free probabilistic data augmentation approach. arXiv:2003.14348, 2020.","title":"Uniformaugment: A search-free probabilistic data augmentation approach","authors":"Lingchen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2016] Wei Liu, Dragomir Anguelov, Dumitru Er- han, Christian Szegedy, Scott Reed, and et. al. Ssd: Single shot multibox detector. In ECCV, 2016.","title":"Ssd: Single shot multibox detector","authors":"Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2018] Songtao Liu, Di Huang, and et. al. Recep- tive field block net for accurate and fast object detection. In ECCV, 2018.","title":"Receptive field block net for accurate and fast object detection","authors":"Songtao Liu, Di Huang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu et al., 2021a] Aoming Liu, Zehao Huang, Zhiwu Huang, and Naiyan Wang. Direct differentiable augmentation search. In ICCV, 2021.","title":"Direct differentiable augmentation search","authors":"Liu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2021b] Zirui Liu, Haifeng Jin, Ting-Hsiang Wang, and et. al. Divaug: Plug-in automated data augmentation with explicit diversity maximization. In ICCV, 2021.","title":"Divaug: Plug-in automated data augmentation with explicit diversity maximization","authors":"Liu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Madry et al., 2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Müller and Hutter, 2021] Samuel G Müller and Frank Hut- ter. Trivialaugment: Tuning-free yet state-of-the-art data augmentation. In ICCV, 2021.","title":"Trivialaugment: Tuning-free yet state-of-the-art data augmentation","authors":"Hutter ; Müller, G Samuel, Frank Müller, Hutter","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS, 2017.","title":"Automatic differentiation in PyTorch","authors":"Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Devito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Riba et al., 2020] Edgar Riba, Dmytro Mishkin, Daniel Ponsa, and et. al. Kornia: an open source differentiable computer vision library for PyTorch. In WACV, 2020.","title":"Kornia: an open source differentiable computer vision library for PyTorch","authors":"Riba","year":"2020","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tang et al., 2020] Zhiqiang Tang, Yunhe Gao, Leonid Kar- linsky, Prasanna Sattigeri, Rogerio Feris, and Dimitris Metaxas. Onlineaugment: Online data augmentation with less domain knowledge. In ECCV, 2020.","title":"Onlineaugment: Online data augmentation with less domain knowledge","authors":"Zhiqiang Tang, Yunhe Gao, Leonid Karlinsky, Prasanna Sattigeri, Rogerio Feris, Dimitris Metaxas","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2020] Sen Wu, Hongyang Zhang, Gregory Valiant, and Christopher Ré. On the generalization effects of linear transformations in data augmentation. In ICML, 2020.","title":"On the generalization effects of linear transformations in data augmentation","authors":"Sen Wu, Hongyang Zhang, Gregory Valiant, Christopher Ré","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. BMVC, 2016.","title":"Wide residual networks","authors":"Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2020] Xinyu Zhang, Qiang Wang, Jian Zhang, and Zhao Zhong. Adversarial autoaugment. In ICLR, 2020.","title":"Adversarial autoaugment","authors":"Xinyu Zhang, Qiang Wang, Jian Zhang, Zhao Zhong","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}