{"type":"ScholarlyArticle","title":"Bidirectional Adversarial Training for Semi-Supervised Domain Adaptation","authors":[{"name":"Pin Jiang","slug":"pin-jiang","given":"Pin","family":"Jiang","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China; Tianjin Key Lab of Machine Learning , Tianjin University , Tianjin , China"},{"name":"Aming Wu","slug":"aming-wu","given":"Aming","family":"Wu","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China; Tianjin Key Lab of Machine Learning , Tianjin University , Tianjin , China"},{"name":"Yahong Han","slug":"yahong-han","given":"Yahong","family":"Han","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China; Tianjin Key Lab of Machine Learning , Tianjin University , Tianjin , China"},{"name":"Yunfeng Shao","slug":"yunfeng-shao","given":"Yunfeng","family":"Shao","affiliation":"Huawei Noah's Ark Lab {jpin ,"},{"name":"Meiyu Qi","slug":"meiyu-qi","given":"Meiyu","family":"Qi","affiliation":"Huawei Noah's Ark Lab {jpin ,"},{"name":"Bingshuai Li","slug":"bingshuai-li","given":"Bingshuai","family":"Li","affiliation":"Huawei Noah's Ark Lab {jpin ,"}],"abstract":"Semi-supervised domain adaptation (SSDA) is a novel branch of machine learning that scarce labeled target examples are available, compared with unsupervised domain adaptation. To make effective use of these additional data so as to bridge the domain gap, one possible way is to generate adversarial examples, which are images with additional perturbations, between the two domains and fill the domain gap. Adversarial training has been proven to be a powerful method for this purpose. However, the traditional adversarial training adds noises in arbitrary directions, which is inefficient to migrate between domains, or generate directional noises from the source to target domain and reverse. In this work, we devise a general bidirectional adversarial training method and employ gradient to guide adversarial examples across the domain gap, i.e., the Adaptive Adversarial Training (AAT) for source to target domain and Entropy-penalized Virtual Adversarial Training (E-VAT) for target to source domain. Particularly, we devise a Bidirectional Adversarial Training (BiAT) network to perform diverse adversarial trainings jointly. We evaluate the effectiveness of BiAT on three benchmark datasets and experimental results demonstrate the proposed method achieves the state-of-the-art.","doi":"10.24963/ijcai.2020/130","url":"https://ijcai.org/proceedings/2020/130","pdf":"https://ijcai.org/proceedings/2020/0130.pdf","markdown":"https://ijcai.org/proceedings/2020/130.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/130","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":130,"pages":{"first":934,"last":940},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24843/file_24843.m3u8","length_sec":274,"provider":"hls"}],"references":[{"position":1,"raw":"[Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi- supervised learning. In Advances in Neural Information Processing Systems, pages 5050-5060, 2019.","title":"Mixmatch: A holistic approach to semisupervised learning","authors":"David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin A Raffel","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, pages 39-57. IEEE, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"IEEE Symposium on Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ganin and Lempitsky, 2014] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backprop- agation. arXiv preprint arXiv:1409.7495, 2014.","title":"Unsupervised domain adaptation by backpropagation","authors":"Yaroslav Ganin, Victor Lempitsky","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grandvalet and Bengio, 2005] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. In Advances in Neural Information Processing Sys- tems, pages 529-536, 2005.","title":"Semi-supervised learning by entropy minimization","authors":"Bengio Grandvalet, Yves Grandvalet, Yoshua Bengio","year":"2005","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural In- formation Processing Systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Laine and Aila, 2016] Samuli Laine and Timo Aila. Tem- poral ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ;, Samuli Laine, Timo Aila","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2019] Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable adversarial train- ing: A general approach to adapting deep classifiers. In International Conference on Machine Learning, pages 4013-4022, 2019.","title":"Transferable adversarial training: A general approach to adapting deep classifiers","authors":"Liu","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable fea- tures with deep adaptation networks. arXiv preprint arXiv:1502.02791, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael I Jordan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Long et al., 2017] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In International Conference on Machine Learning, pages 2208-2217, 2017.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2018] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial do- main adaptation. In Advances in Neural Information Pro- cessing Systems, pages 1640-1650, 2018.","title":"Conditional adversarial domain adaptation","authors":"Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I Jordan","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miyato et al., 2016] Takeru Miyato, Andrew M Dai, and Ian Goodfellow. Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725, 2016.","title":"Adversarial training methods for semi-supervised text classification","authors":"Miyato","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Miyato et al., 2018] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):1979-1993, 2018.","title":"Virtual adversarial training: a regularization method for supervised and semi-supervised learning","authors":"Miyato","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Park et al., 2018] Sungrae Park, JunKeon Park, Su-Jin Shin, and Il-Chul Moon. Adversarial dropout for supervised and semi-supervised learning. In Thirty-Second AAAI Confer- ence on Artificial Intelligence, 2018.","title":"Il-Chul Moon. Adversarial dropout for supervised and semi-supervised learning","authors":"Park","year":"2018","venue":"Thirty-Second AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2019] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment match- ing for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1406-1415, 2019.","title":"Moment matching for multi-source domain adaptation","authors":"Peng","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rasmus et al., 2015] Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi- supervised learning with ladder networks. In Advances in Neural Information Processing Systems, pages 3546- 3554, 2015.","title":"Semisupervised learning with ladder networks","authors":"Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, Tapani Raiko","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Saito et al., 2017] Kuniaki Saito, Yoshitaka Ushiku, Tat- suya Harada, and Kate Saenko. Adversarial dropout regu- larization. arXiv preprint arXiv:1711.01575, 2017.","title":"Adversarial dropout regularization","authors":"Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, Kate Saenko","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Saito et al., 2018] Kuniaki Saito, Kohei Watanabe, Yoshi- taka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3723-3732, 2018.","title":"Maximum classifier discrepancy for unsupervised domain adaptation","authors":"Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, Tatsuya Harada","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Saito et al., 2019] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi- supervised domain adaptation via minimax entropy. In Proceedings of the IEEE International Conference on Computer Vision, pages 8050-8058, 2019.","title":"Semisupervised domain adaptation via minimax entropy","authors":"Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, Kate Saenko","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems, pages 1195-1204, 2017.","title":"Mean teachers are better role models: Weightaveraged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen, Antti Tarvainen, Harri Valpola","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}