{"type":"ScholarlyArticle","title":"Test-time Fourier Style Calibration for Domain Generalization","authors":[{"name":"Xingchen Zhao","slug":"xingchen-zhao","given":"Xingchen","family":"Zhao","affiliation":"Northeastern University"},{"name":"Chang Liu","slug":"chang-liu","given":"Chang","family":"Liu","affiliation":"Northeastern University"},{"name":"Anthony Sicilia","slug":"anthony-sicilia","given":"Anthony","family":"Sicilia","affiliation":"University of Pittsburgh"},{"name":"Seong Jae Hwang","slug":"seong-jae-hwang","given":"Seong Jae","family":"Hwang","affiliation":"University of Pittsburgh"},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Northeastern University"}],"abstract":"The topic of generalizing machine learning models learned on a collection of source domains to unknown target domains is challenging. While many domain generalization (DG) methods have achieved promising results, they primarily rely on the source domains at train-time without manipulating the target domains at test-time. Thus, it is still possible that those methods can overfit to source domains and perform poorly on target domains. Driven by the observation that domains are strongly related to styles, we argue that reducing the gap between source and target styles can boost models’ generalizability. To solve the dilemma of having no access to the target domain during training, we introduce Test-time Fourier Style Calibration (TF-Cal) for calibrating the target domain style on the fly during testing. To access styles, we utilize Fourier transformation to decompose features into amplitude (style) features and phase (semantic) features. Furthermore, we present an effective technique to Augment Amplitude Features (AAF) to complement TF-Cal. Extensive experiments on several popular DG benchmarks and a segmentation dataset for medical images demonstrate that our method outperforms state-of-the-art methods.","doi":"10.24963/ijcai.2022/240","url":"https://ijcai.org/proceedings/2022/240","pdf":"https://ijcai.org/proceedings/2022/0240.pdf","markdown":"https://ijcai.org/proceedings/2022/240.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/240","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":240,"pages":{"first":1721,"last":1727},"keywords":["Computer Vision: Machine Learning for Vision","Computer Vision: Transfer, low-shot, semi- and un- supervised learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986856","length_sec":102,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984999","length_sec":355,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Albuquerque et al., 2019] Isabela Albuquerque, João Mon- teiro, Mohammad Javad Darvishi Bayazi, Tiago H. Falk, and Ioannis Mitliagkas. Generalizing to unseen domains via distribution matching. arXiv: Learning, 2019.","title":"Isabela Albuquerque, João Monteiro","authors":"Albuquerque","year":"2019","venue":"Generalizing to unseen domains via distribution matching","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Balaji et al., 2018] Yogesh Balaji, Swami Sankara- narayanan, and Rama Chellappa. Metareg: Towards domain generalization using meta-regularization. In NeurIPS, 2018.","title":"Metareg: Towards domain generalization using meta-regularization","authors":"Yogesh Balaji, Swami Sankaranarayanan, Rama Chellappa","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ben-David et al., 2009] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando C Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79:151-175, 2009.","title":"A theory of learning from different domains","authors":"Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando C Pereira, Jennifer Wortman Vaughan","year":"2009","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cha et al., 2021] Junbum Cha, Han-Cheol Cho, Kyungjae Lee, Seunghyun Park, Yunsung Lee, and Sungrae Park. Domain generalization needs stochastic weight averaging for robustness on domain shifts. ArXiv, abs/2102.08604, 2021.","title":"Domain generalization needs stochastic weight averaging for robustness on domain shifts","authors":"Cha","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dou et al., 2019] Qi Dou, Daniel Coelho de Castro, Kon- stantinos Kamnitsas, and Ben Glocker. Domain general- ization via model-agnostic learning of semantic features. In NeurIPS, 2019.","title":"Domain generalization via model-agnostic learning of semantic features","authors":"Dou","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ganin et al., 2016] Yaroslav Ganin, E. Ustinova, Hana Ajakan, Pascal Germain, H. Larochelle, Franc ¸ois Lavio- lette, Mario Marchand, and Victor S. Lempitsky. Domain- adversarial training of neural networks. In J. Mach. Learn. Res., 2016.","title":"Domainadversarial training of neural networks","authors":"E Yaroslav Ganin, Hana Ustinova, Pascal Ajakan, H Germain, Larochelle, Mario Franc ¸ois Laviolette, Victor S Marchand, Lempitsky","year":"2016","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Geirhos et al., 2019] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased to- wards texture; increasing shape bias improves accuracy and robustness. ArXiv, abs/1811.12231, 2019.","title":"Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness","authors":"Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix Wichmann, Wieland Brendel","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"X Kaiming He, Shaoqing Zhang, Jian Ren, Sun","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2020] Zeyi Huang, Haohan Wang, Eric P. Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In ECCV, 2020.","title":"Self-challenging improves crossdomain generalization","authors":"Zeyi Huang, Haohan Wang, Eric P Xing, Dong Huang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kuijf et al., 2019] Hugo J. Kuijf, Adrià Casamitjana, D. Louis Collins, M. Dadar, Achilleas Georgiou, and et al. Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge. IEEE Transactions on Medical Imaging, 38:2556-2568, 2019.","title":"Standardized assessment of automatic segmentation of white matter hyperintensities and results of the wmh segmentation challenge","authors":"J Hugo, Adrià Kuijf, D Louis Casamitjana, M Collins, Achilleas Dadar, Georgiou","year":"2019","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier do- main generalization. 2017 IEEE International Conference on Computer Vision (ICCV), pages 5543-5551, 2017.","title":"Deeper, broader and artier domain generalization","authors":"Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M Hospedales","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018a] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Learning to general- ize: Meta-learning for domain generalization. ArXiv, abs/1710.03463, 2018.","title":"Learning to generalize: Meta-learning for domain generalization","authors":"Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018b] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex Chichung Kot. Domain generalization with ad- versarial feature learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5400- 5409, 2018.","title":"Domain generalization with adversarial feature learning","authors":"Li","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2020] Hongwei Li, Timo Loehr, and et al. Do- main adaptive medical image segmentation via adversarial learning of disease-specific spatial patterns. arXiv: Com- puter Vision and Pattern Recognition, 2020.","title":"Domain adaptive medical image segmentation via adversarial learning of disease-specific spatial patterns","authors":"Hongwei Li, Timo Loehr","year":"2020","venue":"arXiv: Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Matsuura and Harada, 2020] Toshihiko Matsuura and Tat- suya Harada. Domain generalization using a mixture of multiple latent domains. ArXiv, abs/1911.07661, 2020.","title":"Domain generalization using a mixture of multiple latent domains","authors":"Harada Matsuura, Toshihiko Matsuura, Tatsuya Harada","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nussbaumer, 1982] Henri J. Nussbaumer. The fast fourier transform. 1982.","title":"The fast fourier transform","authors":"Henri J Nussbaumer","year":"1982","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Piotrowski and Campbell, 1982] Lisa Piotrowski and Fer- gus William Campbell. A demonstration of the visual im- portance and flexibility of spatial-frequency amplitude and phase. Perception, 11:337 -346, 1982.","title":"A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase","authors":"Campbell Piotrowski, William Campbell","year":"1982","venue":"Perception","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sicilia et al., 2021] Anthony Sicilia, Xingchen Zhao, and Seong Jae Hwang. Domain adversarial neural networks for domain generalization: When it works and how to im- prove. ArXiv, abs/2102.03924, 2021.","title":"Domain adversarial neural networks for domain generalization: When it works and how to improve","authors":"Anthony Sicilia, Xingchen Zhao, Seong Jae Hwang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Venkateswara et al., 2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Pan- chanathan. Deep hashing network for unsupervised do- main adaptation. 2017 IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 5385-5394, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2021] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based framework for domain generalization. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14378-14387, 2021.","title":"A fourier-based framework for domain generalization","authors":"Xu","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2020] Ling Zhang, Xiaosong Wang, and et al. Generalizing deep learning for medical image segmenta- tion to unseen domains via deep stacked transformation. IEEE Transactions on Medical Imaging, 39:2531-2540, 2020.","title":"Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation","authors":"Ling Zhang, Xiaosong Wang","year":"2020","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2021] Jian Zhang, Lei Qi, Yinghuan Shi, and Yang Gao. More is better: A novel multi-view framework for domain generalization. ArXiv, abs/2112.12329, 2021.","title":"More is better: A novel multi-view framework for domain generalization","authors":"Jian Zhang, Lei Qi, Yinghuan Shi, Yang Gao","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2021] Xingchen Zhao, Anthony Sicilia, and et al. Robust white matter hyperintensity segmentation on unseen domain. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 1047-1051, 2021.","title":"Robust white matter hyperintensity segmentation on unseen domain","authors":"Zhao","year":"2021","venue":"IEEE 18th International Symposium on Biomedical Imaging (ISBI)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2020] Kaiyang Zhou, Yongxin Yang, Timo- thy M. Hospedales, and Tao Xiang. Deep domain- adversarial image generation for domain generalisation. ArXiv, abs/2003.06054, 2020.","title":"Deep domainadversarial image generation for domain generalisation","authors":"Zhou","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2021] Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. ArXiv, abs/2104.02008, 2021.","title":"Domain generalization with mixstyle","authors":"Kaiyang Zhou, Yongxin Yang, Yu Qiao, Tao Xiang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}