{"type":"ScholarlyArticle","title":"Learning Disentangled Semantic Representation for Domain Adaptation","authors":[{"name":"Ruichu Cai","slug":"ruichu-cai","given":"Ruichu","family":"Cai","affiliation":"School of Computers , Guangdong University of Technology , China"},{"name":"Zijian Li","slug":"zijian-li","given":"Zijian","family":"Li","affiliation":"School of Computers , Guangdong University of Technology , China"},{"name":"Pengfei Wei","slug":"pengfei-wei","given":"Pengfei","family":"Wei","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Jie Qiao","slug":"jie-qiao","given":"Jie","family":"Qiao","affiliation":"School of Computers , Guangdong University of Technology , China"},{"name":"Kun Zhang","slug":"kun-zhang","given":"Kun","family":"Zhang","affiliation":"Department of Philosophy , Carnegie Mellon University , USA"},{"name":"Zhifeng Hao","slug":"zhifeng-hao","given":"Zhifeng","family":"Hao","affiliation":"School of Mathematics and Big Data , Foshan University , China"}],"abstract":"Domain adaptation is an important but challenging task. Most of the existing domain adaptation methods struggle to extract the domain-invariant representation on the feature space with entangling domain information and semantic information. Different from previous efforts on the entangled feature space, we aim to extract the domain invariant semantic information in the latent disentangled semantic representation (DSR) of the data. In DSR, we assume the data generation process is controlled by two independent sets of variables, i.e., the semantic latent variables and the domain latent variables. Under the above assumption, we employ a variational auto-encoder to reconstruct the semantic latent variables and domain latent variables behind the data. We further devise a dual adversarial network to disentangle these two sets of reconstructed latent variables. The disentangled semantic latent variables are finally adapted across the domains. Experimental studies testify that our model yields state-of-the-art performance on several domain adaptation benchmark datasets.","doi":"10.24963/ijcai.2019/285","url":"https://ijcai.org/proceedings/2019/285","pdf":"https://ijcai.org/proceedings/2019/0285.pdf","markdown":"https://ijcai.org/proceedings/2019/285.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/285","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":285,"pages":{"first":2060,"last":2066},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"ResNet-50 [He et al., 2016] 34.9 50.0 58.0 37.4 41.9 46.2 38.5 31.2 60.4 53.9 41.2 59.9 46.1 DAN [Long et al., 2015] 43.6 57.0 67.9 45.8 56.5 60.4 44.0 43.6 67.7 63.1 51.5 74.3 56.3 DANN [Ganin et al., 2016] 45.6 59.3 70.1 47.0 58.5 60.9 46.1 43.7 68.5 63.2 51.8 76.8 57.6 JAN [Long et al., 2017] 45.9 61.2 68.9 50.4 59.7 61.0 45.8 43.4 70.3 63.9 52.4 76.8 58.3 MSTN [Xie et al., 2018] 49.3 67.6 74.7 49.6 63.7 64.5 50.7 43.7 73.0 62.7 52.2 77.9 60.9 CDAN-M [Long et al., 2018] 50.6 65.9 73.4 55.7 62.7 64.2 51.8 49.1 74.5 68.2 56.9 80.7 62.8 DSR DM(δ = 1) 50.5 69.2 75.5 52.9 64.0 65.6 53.0 45.0 73.5 63.2 50.8 78.3 61.8 DSR DM(δ = 2) 52.3 70.9 76.5 54.0 65.2 67.0 53.5 45.0 73.7 62.7 50.7 78.7 62.5 DSR 53.4 71.6 77.4 57.1 66.8 69.3 56.7 49.2 75.7 68.0 54.0 79.5 64","title":"ResNet","authors":"He","year":"2015","venue":"ResNet","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"9 Table 2: Accuracy (%) on Office-home for unsupervised domain adaptation (ResNet) (a)Adapted (b)MSTN Adapted (c)DSR Adapted References","title":"9 Table 2: Accuracy (%) on Office-home for unsupervised domain adaptation (ResNet) (a)Adapted (b)MSTN Adapted (c)DSR Adapted References","authors":null,"year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ben-David et al., 2007] Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in neural information processing systems, pages 137-144, 2007.","title":"Analysis of representations for domain adaptation","authors":"Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira","year":"2007","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2018] Chaoqi Chen, Weiping Xie, Tingyang Xu, Wenbing Huang, Yu Rong, Xinghao Ding, Yue Huang, and Junzhou Huang. Progressive feature align- ment for unsupervised domain adaptation. arXiv preprint arXiv:1811.08585, 2018.","title":"Progressive feature alignment for unsupervised domain adaptation","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2018] Weijian Deng, Liang Zheng, and Jian- bin Jiao. Domain alignment with triplets. arXiv preprint arXiv:1812.00893, 2018.","title":"Domain alignment with triplets","authors":"Weijian Deng, Liang Zheng, Jianbin Jiao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dinh et al., 2014] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent compo- nents estimation. arXiv preprint arXiv:1410.8516, 2014.","title":"Nice: Non-linear independent components estimation","authors":"Laurent Dinh, David Krueger, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ganin and Lempitsky, 2015] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backprop- agation. In Proceedings of the 32nd International Confer- ence on International Conference on Machine Learning- Volume 37, pages 1180-1189. JMLR. org, 2015.","title":"Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation","authors":"Lempitsky Ganin","year":"2015","venue":"Proceedings of the 32nd International Conference on International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1):2096-2030, 2016.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Mario Franc ¸ois Laviolette, Victor Marchand, Lempitsky","year":"2016","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsuper- vised domain adaptation. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2066-2073. IEEE, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Long et al., 2015] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable fea- tures with deep adaptation networks. arXiv preprint arXiv:1502.02791, 2015.","title":"Learning transferable features with deep adaptation networks","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Michael I Jordan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2016] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Unsupervised domain adap- tation with residual transfer networks. In Advances in Neu- ral Information Processing Systems, pages 136-144, 2016.","title":"Unsupervised domain adaptation with residual transfer networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Long et al., 2017] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th In- ternational Conference on Machine Learning-Volume 70, pages 2208-2217. JMLR. org, 2017.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Long et al., 2018] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial do- main adaptation. In Advances in Neural Information Pro- cessing Systems, pages 1647-1657, 2018.","title":"Conditional adversarial domain adaptation","authors":"Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I Jordan","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lowry, 2014] Richard Lowry. Concepts and applications of inferential statistics. 2014.","title":"Concepts and applications of inferential statistics","authors":"Richard Lowry","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan et al., 2011] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Net- works, 22(2):199-210, 2011.","title":"Domain adaptation via transfer component analysis","authors":"Ivor W Sinno Jialin Pan, James T Tsang, Qiang Kwok, Yang","year":"2011","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pei et al., 2018] Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Multi-adversarial domain adap- tation. In AAAI Conference on Artificial Intelligence, 2018.","title":"Multi-adversarial domain adaptation","authors":"Zhongyi Pei, Zhangjie Cao, Mingsheng Long, Jianmin Wang","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tzeng et al., 2014] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confu- sion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014.","title":"Deep domain confusion: Maximizing for domain invariance","authors":"Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tzeng et al., 2015] Eric Tzeng, Judy Hoffman, Trevor Dar- rell, and Kate Saenko. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 4068-4076, 2015.","title":"Simultaneous deep transfer across domains and tasks","authors":"Eric Tzeng, Judy Hoffman, Trevor Darrell, Kate Saenko","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tzeng et al., 2017] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition (CVPR), volume 1, page 4, 2017.","title":"Adversarial discriminative domain adaptation","authors":"Eric Tzeng, Judy Hoffman, Kate Saenko, Trevor Darrell","year":"2017","venue":"Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xie et al., 2018] Shaoan Xie, Zibin Zheng, Liang Chen, and Chuan Chen. Learning semantic representations for unsu- pervised domain adaptation. In International Conference on Machine Learning, pages 5419-5428, 2018.","title":"Learning semantic representations for unsupervised domain adaptation","authors":"Xie","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2013] Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adapta- tion under target and conditional shift. In International Conference on Machine Learning, pages 819-827, 2013.","title":"Domain adaptation under target and conditional shift","authors":"Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, Zhikun Wang","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2018] Yexun Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. Domain-invariant adversarial learn- ing for unsupervised domain adaption. arXiv preprint arXiv:1811.12751, 2018.","title":"Domain-invariant adversarial learning for unsupervised domain adaption","authors":"Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}