{"type":"ScholarlyArticle","title":"Dual-View Variational Autoencoders for Semi-Supervised Text Matching","authors":[{"name":"Zhongbin Xie","slug":"zhongbin-xie","given":"Zhongbin","family":"Xie","affiliation":"SKLSDE Lab , Beihang University , Beijing , China Beijing Advanced Innovation Center for Big Data and Brain Computing , Beijing , China"},{"name":"Shuai Ma","slug":"shuai-ma","given":"Shuai","family":"Ma","affiliation":"SKLSDE Lab , Beihang University , Beijing , China Beijing Advanced Innovation Center for Big Data and Brain Computing , Beijing , China"}],"abstract":"Semantically matching two text sequences (usually two sentences) is a fundamental problem in NLP. Most previous methods either encode each of the two sentences into a vector representation (sentence-level embedding) or leverage word-level interaction features between the two sentences. In this study, we propose to take the sentence-level embedding features and the word-level interaction features as two distinct views of a sentence pair, and unify them with a framework of Variational Autoencoders such that the sentence pair is matched in a semi-supervised manner. The proposed model is referred to as Dual-View Variational AutoEncoder (DV-VAE), where the optimization of the variational lower bound can be interpreted as an implicit Co-Training mechanism for two matching models over distinct views. Experiments on SNLI, Quora and a Community Question Answering dataset demonstrate the superiority of our DV-VAE over several strong semi-supervised and supervised text matching models.","doi":"10.24963/ijcai.2019/737","url":"https://ijcai.org/proceedings/2019/737","pdf":"https://ijcai.org/proceedings/2019/0737.pdf","markdown":"https://ijcai.org/proceedings/2019/737.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/737","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":737,"pages":{"first":5306,"last":5312},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Natural Language Processing: Natural Language Semantics","Machine Learning: Learning Generative Models","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Blum and Mitchell, 1998] Avrim Blum and Tom M. Mitchell. Combining labeled and unlabeled data with co-training. In COLT, pages 92-100, 1998.","title":"Combining labeled and unlabeled data with co-training","authors":"Avrim Blum, Tom M Mitchell","year":"1998","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2015] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In EMNLP, pages 632-642, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"R Samuel, Gabor Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowman et al., 2016a] Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. A fast unified model for parsing and sentence understanding. In ACL, pages 1466-1477, 2016.","title":"A fast unified model for parsing and sentence understanding","authors":"Bowman","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bowman et al., 2016b] Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentences from a continuous space. In CoNLL, pages 10-21, 2016.","title":"Generating sentences from a continuous space","authors":"Bowman","year":"2016","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dolan and Brockett, 2005] Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential para- phrases. In IWP, 2005.","title":"Automatically constructing a corpus of sentential paraphrases","authors":"Brockett Dolan, Bill Dolan, Chris Brockett","year":"2005","venue":"IWP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gong et al., 2018] Yichen Gong, Heng Luo, and Jian Zhang. Neural language inference over interaction space. In ICLR, 2018.","title":"Neural language inference over interaction space","authors":"Yichen Gong, Heng Luo, Jian Zhang","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma et al., 2014] Diederik P. Kingma, Shakir Mo- hamed, Danilo Jimenez Rezende, and Max Welling. Semi- supervised learning with deep generative models. In NIPS, pages 3581-3589, 2014.","title":"Semisupervised learning with deep generative models","authors":"P Diederik, Shakir Kingma, Danilo Mohamed, Max Jimenez Rezende, Welling","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lan and Xu, 2018] Wuwei Lan and Wei Xu. Neural net- work models for paraphrase identification, semantic textu- al similarity, natural language inference, and question an- swering. In COLING, pages 3890-3902, 2018.","title":"Neural network models for paraphrase identification, semantic textual similarity, natural language inference, and question answering","authors":"Lan, Xu ; ] Wuwei Lan, Wei Xu","year":"2018","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Miao and Blunsom, 2016] Yishu Miao and Phil Blunsom. Language as a latent variable: Discrete generative models for sentence compression. In EMNLP, 2016.","title":"Language as a latent variable: Discrete generative models for sentence compression","authors":"Blunsom Miao, Yishu Miao, Phil Blunsom","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Miao et al., 2016] Yishu Miao, Lei Yu, and Phil Blunsom. Neural variational inference for text processing. In ICML, pages 1727-1736, 2016.","title":"Neural variational inference for text processing","authors":"Yishu Miao, Lei Yu, Phil Blunsom","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mnih and Gregor, 2014] Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief net- works. In ICML, pages 1791-1799, 2014.","title":"Neural variational inference and learning in belief networks","authors":"Gregor ; Mnih, Andriy Mnih, Karol Gregor","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mueller and Thyagarajan, 2016] Jonas Mueller and Aditya Thyagarajan. Siamese recurrent architectures for learning sentence similarity. In AAAI, pages 2786-2792, 2016.","title":"Siamese recurrent architectures for learning sentence similarity","authors":"Thyagarajan Mueller, Jonas Mueller, Aditya Thyagarajan","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nakov et al., 2015] Preslav Nakov, Lluís Màrquez, Walid Magdy, Alessandro Moschitti, Jim Glass, and Bilal Ran- deree. Semeval-2015 task 3: Answer selection in commu- nity question answering. In SemEval, 2015.","title":"Semeval-2015 task 3: Answer selection in community question answering","authors":"Nakov","year":"2015","venue":"SemEval","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rocktäschel et al., 2016] Tim Rocktäschel, Edward Grefen- stette, Karl Moritz Hermann, Tomáš Kočiský, and Phil Blunsom. Reasoning about entailment with neural atten- tion. In ICLR, 2016.","title":"Reasoning about entailment with neural attention","authors":"Rocktäschel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Serban et al., 2017] Iulian Vlad Serban, Alessandro Sor- doni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. A hierarchical latent variable encoder-decoder model for generating dia- logues. In AAAI, pages 3295-3301, 2017.","title":"A hierarchical latent variable encoder-decoder model for generating dialogues","authors":"Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, Yoshua Bengio","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shen et al., 2018] Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, and Lawrence Carin. Deconvolution- al latent-variable model for text sequence matching. In AAAI, pages 5438-5445, 2018.","title":"Deconvolutional latent-variable model for text sequence matching","authors":"Shen","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sindhwani et al., 2005] Vikas Sindhwani, Partha Niyogi, and Mikhail Belkin. A co-regularization approach to semi- supervised learning with multiple views. In ICML Work- shop, 2005.","title":"A co-regularization approach to semisupervised learning with multiple views","authors":"Vikas Sindhwani, Partha Niyogi, Mikhail Belkin","year":"2005","venue":"ICML Workshop","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tan et al., 2018] Chuanqi Tan, Furu Wei, Wenhui Wang, Weifeng Lv, and Ming Zhou. Multiway attention networks for modeling sentence pairs. In IJCAI, 2018.","title":"Multiway attention networks for modeling sentence pairs","authors":"Tan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/613"},{"position":22,"raw":"[Tay et al., 2018] Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. Co-stack residual affinity networks with multi-level attention refinement for matching text sequences. In EMNLP, pages 4492-4502, 2018.","title":"Co-stack residual affinity networks with multi-level attention refinement for matching text sequences","authors":"Yi Tay, Anh Tuan Luu, Siu Cheung, Hui","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang and Jiang, 2017] Shuohang Wang and Jing Jiang. A compare-aggregate model for matching text sequences. In ICLR, 2017.","title":"A compare-aggregate model for matching text sequences","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Zhiguo Wang, Wael Hamza, and Radu Florian. Bilateral multi-perspective matching for natural language sentences. In IJCAI, pages 4144-4150, 2017.","title":"Bilateral multi-perspective matching for natural language sentences","authors":"Wang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/579"},{"position":25,"raw":"[Williams, 1992] Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine Learning, 8:229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Ronald J Williams","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2018] Yu Wu, Wei Wu, Can Xu, and Zhoujun Li. Knowledge enhanced hybrid neural network for text matching. In AAAI, pages 5586-5593, 2018.","title":"Knowledge enhanced hybrid neural network for text matching","authors":"Yu Wu, Wei Wu, Can Xu, Zhoujun Li","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2013] Chang Xu, Dacheng Tao, and Chao Xu. A survey on multi-view learning. arXiv:1304.5634, 2013.","title":"A survey on multi-view learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xu et al., 2017] Weidi Xu, Haoze Sun, Chao Deng, and Y- ing Tan. Variational autoencoder for semi-supervised text classification. In AAAI, pages 3358-3364, 2017.","title":"Variational autoencoder for semi-supervised text classification","authors":"Xu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2015] Yi Yang, Wen-tau Yih, and Christopher Meek. Wikiqa: A challenge dataset for open-domain ques- tion answering. In EMNLP, pages 2013-2018, 2015.","title":"Wikiqa: A challenge dataset for open-domain question answering","authors":"Yi Yang, Wen-Tau Yih, Christopher Meek","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2017] Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick. Improved variational autoencoders for text modeling using dilated convolutions. In ICML, pages 3881-3890, 2017.","title":"Improved variational autoencoders for text modeling using dilated convolutions","authors":"Yang","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2016] Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, and Min Zhang. Variational neural machine translation. In EMNLP, pages 521-530, 2016.","title":"Variational neural machine translation","authors":"Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, Min Zhang","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhao et al., 2018] Junbo Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, and Yann LeCun. Adversari- ally regularized autoencoders. In ICML, pages 5897-5906, 2018.","title":"Adversarially regularized autoencoders","authors":"Jake Junbo, Yoon Zhao, Kelly Kim, Alexander M Zhang, Yann Rush, Lecun","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}