{"type":"ScholarlyArticle","title":"Domain Generalization through the Lens of Angular Invariance","authors":[{"name":"Yujie Jin","slug":"yujie-jin","given":"Yujie","family":"Jin","affiliation":"Peking University , Beijing , China"},{"name":"Xu Chu","slug":"xu-chu","given":"Xu","family":"Chu","affiliation":"Peking University , Beijing , China; Tsinghua University , Beijing , China"},{"name":"Yasha Wang","slug":"yasha-wang","given":"Yasha","family":"Wang","affiliation":"Peking University , Beijing , China"},{"name":"Wenwu Zhu","slug":"wenwu-zhu","given":"Wenwu","family":"Zhu","affiliation":"Tsinghua University , Beijing , China"}],"abstract":"Domain generalization (DG) aims at generalizing a classifier trained on multiple source domains to an unseen target domain with domain shift. A common pervasive theme in existing DG literature is domain-invariant representation learning with various invariance assumptions. However, prior works restrict themselves to an impractical assumption for real-world challenges: If a mapping induced by a deep neural network (DNN) could align the source domains well, then such a mapping aligns a target domain as well. In this paper, we simply take DNNs as feature extractors to relax the requirement of distribution alignment. Specifically, we put forward a novel angular invariance and the accompanied norm shift assumption. Based on the proposed term of invariance, we propose a novel deep DG method dubbed Angular Invariance Domain Generalization Network (AIDGN). The optimization objective of AIDGN is developed with a von-Mises Fisher (vMF) mixture model. Extensive experiments on multiple DG benchmark datasets validate the effectiveness of the proposed AIDGN method.","doi":"10.24963/ijcai.2022/139","url":"https://ijcai.org/proceedings/2022/139","pdf":"https://ijcai.org/proceedings/2022/0139.pdf","markdown":"https://ijcai.org/proceedings/2022/139.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/139","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":139,"pages":{"first":995,"last":1001},"keywords":["Computer Vision: Transfer, low-shot, semi- and un- supervised learning","Machine Learning: Classification","Computer Vision: Machine Learning for Vision"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986871","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985014","length_sec":196,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arjovsky et al., 2020] Martin Arjovsky, Léon Bottou, Ishaan Gul- rajani, and David Lopez-Paz. Invariant risk minimization. stat, 1050:27, 2020.","title":"Invariant risk minimization","authors":"Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, David Lopez-Paz","year":"2020","venue":"stat","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beery et al., 2018] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In ECCV, pages 456-473, 2018.","title":"Recognition in terra incognita","authors":"Sara Beery, Grant Van Horn, Pietro Perona","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ben-David et al., 2007] Shai Ben-David, John Blitzer, Koby Cram- mer, Fernando Pereira, et al. Analysis of representations for domain adaptation. In Neurips, volume 19, 2007.","title":"Analysis of representations for domain adaptation","authors":"Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira","year":"2007","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blanchard et al., 2011] Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. In Neurips, volume 24, 2011.","title":"Generalizing from several related classification tasks to a new unlabeled sample","authors":"Gilles Blanchard, Gyemin Lee, Clayton Scott","year":"2011","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blanchard et al., 2021] Gilles Blanchard, Aniket Anand Desh- mukh, Ürün Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. JMLR, 22(2), 2021.","title":"Domain generalization by marginal transfer learning","authors":"Gilles Blanchard, Aniket Anand Deshmukh, Ürün Dogan, Gyemin Lee, Clayton Scott","year":"2021","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Blumenson, 1960] LE Blumenson. A derivation of n-dimensional spherical coordinates. The American Mathematical Monthly, 67(1), 1960.","title":"A derivation of n-dimensional spherical coordinates","authors":"Blumenson","year":"1960","venue":"The American Mathematical Monthly","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2013] Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In ICCV, pages 1657-1664, 2013.","title":"Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias","authors":"Chen Fang, Ye Xu, Daniel N Rockmore","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17(1), 2016.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Mario Franc ¸ois Laviolette, Victor Marchand, Lempitsky","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gopal and Yang, 2014] Siddharth Gopal and Yiming Yang. Von mises-fisher clustering models. In ICML, pages 154-162, 2014.","title":"Von mises-fisher clustering models","authors":"Siddharth Gopal, Yiming Yang","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gulrajani and Lopez-Paz, 2021] Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In ICLR, 2021.","title":"Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization","authors":"Lopez-Paz Gulrajani","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2020] Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross-domain general- ization. In ECCV, pages 124-140, 2020.","title":"Self-challenging improves cross-domain generalization","authors":"Zeyi Huang, Haohan Wang, Eric P Xing, Dong Huang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Iwasawa and Matsuo, 2021] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Neurips, volume 34, 2021.","title":"Test-time classifier adjustment module for model-agnostic domain generalization","authors":"Matsuo Iwasawa, Yusuke Iwasawa, Yutaka Matsuo","year":"2021","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jaynes, 1957] Edwin T Jaynes. Information theory and statistical mechanics. Physical Review, 106(4), 1957.","title":"Information theory and statistical mechanics","authors":"T Edwin, Jaynes","year":"1957","venue":"Physical Review","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E Hinton. Imagenet classification with deep convolutional neural networks. In Neurips, volume 25, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Neurips","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krueger et al., 2021] David Krueger, Ethan Caballero, Joern- Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution gen- eralization via risk extrapolation. In ICML, pages 5815-5826, 2021.","title":"Out-of-distribution generalization via risk extrapolation","authors":"David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, Aaron Courville","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2017] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In ICCV, pages 5542-5550, 2017.","title":"Deeper, broader and artier domain generalization","authors":"Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M Hospedales","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li et al., 2018a] Da Li, Yongxin Yang, Yi-Zhe Song, and Timo- thy M Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, pages 5542-5550, 2018.","title":"Learning to generalize: Meta-learning for domain generalization","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2018b] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learn- ing. In CVPR, pages 5400-5409, 2018.","title":"Domain generalization with adversarial feature learning","authors":"Li","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2018c] Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao. Domain generalization via conditional invariant representations. In AAAI, volume 32, 2018.","title":"Domain generalization via conditional invariant representations","authors":"Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2021] Xiaofeng Liu, Bo Hu, Linghao Jin, Xu Han, Fangxu Xing, Jinsong Ouyang, Jun Lu, Georges EL Fakhri, and Jonghye Woo. Domain generalization under conditional and label shifts via variational bayesian inference. In IJCAI, 2021.","title":"Domain generalization under conditional and label shifts via variational bayesian inference","authors":"Xiaofeng Liu, Bo Hu, Linghao Jin, Xu Han, Fangxu Xing, Jinsong Ouyang, Jun Lu, Georges El Fakhri, Jonghye Woo","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/122"},{"position":24,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Dar- rell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Meng et al., 2021] Qiang Meng, Shichao Zhao, Zhida Huang, and Feng Zhou. Magface: A universal representation for face recog- nition and quality assessment. In CVPR, pages 14225-14234, 2021.","title":"Magface: A universal representation for face recognition and quality assessment","authors":"Qiang Meng, Shichao Zhao, Zhida Huang, Feng Zhou","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Nam et al., 2021] Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap by reducing style bias. In CVPR, pages 8690-8699, 2021.","title":"Reducing domain gap by reducing style bias","authors":"Nam","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sagawa et al., 2019] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural net- works for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint:1911.08731, 2019.","title":"Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization","authors":"Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, Percy Liang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sun and Saenko, 2016] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV, pages 443-450, 2016.","title":"Deep coral: Correlation alignment for deep domain adaptation","authors":"Saenko ; Sun, Baochen Sun, Kate Saenko","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vapnik, 1999] Vladimir N Vapnik. An overview of statistical learn- ing theory. IEEE Transactions on Neural Networks, 10(5), 1999.","title":"An overview of statistical learning theory","authors":"N Vladimir, Vapnik","year":"1999","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Venkateswara et al., 2017] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hash- ing network for unsupervised domain adaptation. In CVPR, pages 5018-5027, 2017.","title":"Deep hashing network for unsupervised domain adaptation","authors":"Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2020a] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adap- tation by entropy minimization. In ICLR, 2020.","title":"Tent: Fully test-time adaptation by entropy minimization","authors":"Wang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2020b] Yufei Wang, Haoliang Li, and Alex C Kot. Het- erogeneous domain generalization via domain mixup. In ICASSP, pages 3622-3626, 2020.","title":"Heterogeneous domain generalization via domain mixup","authors":"Wang","year":"2020","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zeiler and Fergus, 2014] Matthew D Zeiler and Rob Fergus. Vi- sualizing and understanding convolutional networks. In ECCV, pages 818-833, 2014.","title":"Visualizing and understanding convolutional networks","authors":"D Matthew, Rob Zeiler, Fergus","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2018] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.","title":"mixup: Beyond empirical risk minimization","authors":"Hongyi Zhang, Moustapha Cisse, David Yann N Dauphin, Lopez-Paz","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2020] Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive risk minimization: A meta-learning approach for tack- ling group distribution shift. arXiv preprint:2007.02931, 2020.","title":"Adaptive risk minimization: A meta-learning approach for tackling group distribution shift","authors":"Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, Chelsea Finn","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhao et al., 2019] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant represen- tations for domain adaptation. In ICML, pages 7523-7532, 2019.","title":"On learning invariant representations for domain adaptation","authors":"Han Zhao, Remi Tachet, Des Combes, Kun Zhang, Geoffrey Gordon","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}