{"type":"ScholarlyArticle","title":"Bilateral Multi-Perspective Matching for Natural Language Sentences","authors":[{"name":"Zhiguo Wang","slug":"zhiguo-wang","given":"Zhiguo","family":"Wang","affiliation":"Research Center"},{"name":"Wael Hamza","slug":"wael-hamza","given":"Wael","family":"Hamza","affiliation":"Research Center"},{"name":"Radu Florian","slug":"radu-florian","given":"Radu","family":"Florian","affiliation":"Research Center"}],"abstract":"Natural language sentence matching is a fundamental technology for a variety of tasks. Previous approaches either match sentences from a single direction or only apply single granular (word-by-word or sentence-by-sentence) matching. In this work, we propose a bilateral multi-perspective matching (BiMPM) model. Given two sentences P and Q, our model first encodes them with a BiLSTM encoder. Next, we match the two encoded sentences in two directions P against Q and P against Q. In each matching direction, each time step of one sentence is matched against all time-steps of the other sentence from multiple perspectives. Then, another BiLSTM layer is utilized to aggregate the matching results into a fix-length matching vector. Finally, based on the matching vector, a decision is made through a fully connected layer. We evaluate our model on three tasks: paraphrase identification, natural language inference and answer sentence selection. Experimental results on standard benchmark datasets show that our model achieves the state-of-the-art performance on all tasks.","doi":"10.24963/ijcai.2017/579","url":"https://ijcai.org/proceedings/2017/579","pdf":"https://ijcai.org/proceedings/2017/0579.pdf","markdown":"https://ijcai.org/proceedings/2017/579.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/579","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":579,"pages":{"first":4144,"last":4150},"keywords":["Natural Language Processing: Information Retrieval","Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Semantics","Natural Language Processing: Question Answering"],"videos":[],"references":[{"position":1,"raw":"[Bowman et al., 2015] Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"Gabor Samuel R Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bromley et al., 1993] Jane Bromley, James W. Bentz, Léon Bottou, Isabelle Guyon, Yann LeCun, Cliff Moore, Eduard Säckinger, and Roopak Shah. Signature verification using a \"siamese\" time delay neural network. IJPRAI, 7(4):669- 688, 1993.","title":"Signature verification using a \"siamese\" time delay neural network","authors":"Jane Bromley, James W Bentz, Léon Bottou, Isabelle Guyon, Yann Lecun, Cliff Moore, Eduard Säckinger, Roopak Shah","year":"1993","venue":"IJPRAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang. Enhancing and combining sequen- tial and tree lstm for natural language inference. arXiv preprint arXiv:1609.06038, 2016.","title":"Enhancing and combining sequential and tree lstm for natural language inference","authors":"Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2016] Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733, 2016.","title":"Long short-term memory-networks for machine reading","authors":"Cheng","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He and Lin, 2016] Hua He and Jimmy Lin. Pairwise word interaction modeling with deep neural networks for se- mantic similarity measurement. In NAACL, 2016.","title":"Pairwise word interaction modeling with deep neural networks for semantic similarity measurement","authors":"Lin ; ] Hua He, Jimmy He, Lin","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Heilman and Smith, 2010] Michael Heilman and Noah A Smith. Tree edit models for recognizing textual entail- ments, paraphrases, and answers to questions. In NAACL, 2010.","title":"Tree edit models for recognizing textual entailments, paraphrases, and answers to questions","authors":"Smith ; Heilman, Michael Heilman, Noah A Smith","year":"2010","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. Nature, 521(7553):436- 444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2016a] Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. Modelling interaction of sentence pair with coupled-lstms. arXiv preprint arXiv:1605.05573, 2016.","title":"Modelling interaction of sentence pair with coupled-lstms","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2016b] Yang Liu, Chengjie Sun, Lei Lin, and Xi- aolong Wang. Learning natural language inference us- ing bidirectional lstm model and inner-attention. arXiv preprint arXiv:1605.09090, 2016.","title":"Learning natural language inference using bidirectional lstm model and inner-attention","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Miao et al., 2016] Yishu Miao, Lei Yu, and Phil Blunsom. Neural variational inference for text processing. In ICML, 2016.","title":"Neural variational inference for text processing","authors":"Yishu Miao, Lei Yu, Phil Blunsom","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in neural information processing systems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mou et al., 2015] Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. Natural language in- ference by tree-based convolution and heuristic matching. arXiv preprint arXiv:1512.08422, 2015.","title":"Natural language inference by tree-based convolution and heuristic matching","authors":"Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, Zhi Jin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Munkhdalai and Yu, 2016] Tsendsuren Munkhdalai and Hong Yu. Neural tree indexers for text understanding. arXiv preprint arXiv:1607.04492, 2016.","title":"Tsendsuren Munkhdalai and Hong Yu","authors":"Yu Munkhdalai","year":"2016","venue":"Neural tree indexers for text understanding","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Parikh et al., 2016] Ankur P Parikh, Oscar Täckström, Di- panjan Das, and Jakob Uszkoreit. A decomposable atten- tion model for natural language inference. arXiv preprint arXiv:1606.01933, 2016.","title":"A decomposable attention model for natural language inference","authors":"P Ankur, Oscar Parikh, Dipanjan Täckström, Jakob Das, Uszkoreit","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peng et al., 2015a] Xi Peng, Junzhou Huang, Qiong Hu, Shaoting Zhang, Ahmed Elgammal, and Dimitris Metaxas. From circle to 3-sphere: Head pose estimation by instance parameterization. Computer Vision and Image Understanding, 136:92-102, 2015.","title":"From circle to 3-sphere: Head pose estimation by instance parameterization","authors":"Peng","year":"2015","venue":"Computer Vision and Image Understanding","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Peng et al., 2015b] Xi Peng, Shaoting Zhang, Yu Yang, and Dimitris N Metaxas. Piefa: Personalized incremental and ensemble face alignment. In Proceedings of the IEEE In- ternational Conference on Computer Vision, pages 3880- 3888, 2015.","title":"Piefa: Personalized incremental and ensemble face alignment","authors":"Peng","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peng et al., 2016] Xi Peng, Rogerio S Feris, Xiaoyu Wang, and Dimitris N Metaxas. A recurrent encoder-decoder net- work for sequential face alignment. In European Confer- ence on Computer Vision, pages 38-56. Springer Interna- tional Publishing, 2016.","title":"A recurrent encoder-decoder network for sequential face alignment","authors":"Xi Peng, S Rogerio, Xiaoyu Feris, Dimitris N Wang, Metaxas","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Peng et al., 2017] Xi Peng, Shaoting Zhang, Yang Yu, and Dimitris N Metaxas. Toward personalized modeling: In- cremental and ensemble alignment for sequential faces in the wild. International Journal of Computer Vision, pages 1-14, 2017.","title":"Toward personalized modeling: Incremental and ensemble alignment for sequential faces in the wild","authors":"Xi Peng, Shaoting Zhang, Yang Yu, Dimitris N Metaxas","year":"2017","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rao et al., 2016] Jinfeng Rao, Hua He, and Jimmy Lin. Noise-contrastive estimation for answer selection with deep neural networks. In CIKM, 2016.","title":"Noise-contrastive estimation for answer selection with deep neural networks","authors":"Rao","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rocktäschel et al., 2015] Tim Rocktäschel, Edward Grefen- stette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. Reasoning about entailment with neural atten- tion. arXiv preprint arXiv:1509.06664, 2015.","title":"Tim Rocktäschel, Edward Grefenstette","authors":"Rocktäschel","year":"2015","venue":"Reasoning about entailment with neural attention","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Santos et al., 2016] Cicero dos Santos, Ming Tan, Bing Xi- ang, and Bowen Zhou. Attentive pooling networks. arXiv preprint arXiv:1602.03609, 2016.","title":"Cicero dos Santos, Ming Tan, Bing Xi","authors":"Santos","year":"2016","venue":"Attentive pooling networks","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sha et al., 2016] Lei Sha, Baobao Chang, Zhifang Sui, and Sujian Li. Reading and thinking: Re-read lstm unit for textual entailment recognition. In COLING, 2016.","title":"Reading and thinking: Re-read lstm unit for textual entailment recognition","authors":"Lei Sha, Baobao Chang, Zhifang Sui, Sujian Li","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tan et al., 2015] Ming Tan, Cicero dos Santos, Bing Xi- ang, and Bowen Zhou. Lstm-based deep learning mod- els for non-factoid answer selection. arXiv preprint arXiv:1511.04108, 2015.","title":"Lstm-based deep learning models for non-factoid answer selection","authors":"Ming Tan, Bing Cicero Dos Santos, Bowen Xiang, Zhou","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vendrov et al., 2015] Ivan Vendrov, Ryan Kiros, Sanja Fi- dler, and Raquel Urtasun. Order-embeddings of images and language. arXiv preprint arXiv:1511.06361, 2015.","title":"Order-embeddings of images and language","authors":"Ivan Vendrov, Ryan Kiros, Sanja Fidler, Raquel Urtasun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang and Ittycheriah, 2015] Zhiguo Wang and Abraham It- tycheriah. Faq-based question answering via word align- ment. arXiv preprint arXiv:1507.02628, 2015.","title":"Faq-based question answering via word alignment","authors":"Ittycheriah Wang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang and Jiang, 2015] Shuohang Wang and Jing Jiang. Learning natural language inference with lstm. arXiv preprint arXiv:1512.08849, 2015.","title":"Learning natural language inference with lstm","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and Jiang, 2016] Shuohang Wang and Jing Jiang. A compare-aggregate model for matching text sequences. arXiv preprint arXiv:1611.01747, 2016.","title":"A compare-aggregate model for matching text sequences","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2007] Mengqiu Wang, Noah A Smith, and Teruko Mitamura. What is the jeopardy model? a quasi- synchronous grammar for qa. In EMNLP, 2007.","title":"What is the jeopardy model? a quasisynchronous grammar for qa","authors":"Wang","year":"2007","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2016a] Bingning Wang, Kang Liu, and Jun Zhao. Inner attention based recurrent neural networks for answer selection. In ACL, 2016.","title":"Inner attention based recurrent neural networks for answer selection","authors":"Wang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2016b] Zhiguo Wang, Haitao Mi, Wael Hamza, and Radu Florian. Multi-perspective context match- ing for machine comprehension. arXiv preprint arXiv:1612.04211, 2016.","title":"Multi-perspective context matching for machine comprehension","authors":"Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2016c] Zhiguo Wang, Haitao Mi, and Abraham Ittycheriah. Semi-supervised clustering for short text via deep representation learning. In CoNLL, 2016.","title":"Semi-supervised clustering for short text via deep representation learning","authors":"Wang","year":"2016","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2016d] Zhiguo Wang, Haitao Mi, and Abra- ham Ittycheriah. Sentence similarity learning by lexical decomposition and composition. In COLING, 2016.","title":"Sentence similarity learning by lexical decomposition and composition","authors":"Wang","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2015] Yi Yang, Wen-tau Yih, and Christopher Meek. Wikiqa: A challenge dataset for open-domain ques- tion answering. In EMNLP, 2015.","title":"Wikiqa: A challenge dataset for open-domain question answering","authors":"Yi Yang, Wen-Tau Yih, Christopher Meek","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yin et al., 2015] Wenpeng Yin, Hinrich Schütze, Bing Xi- ang, and Bowen Zhou. Abcnn: Attention-based convolu- tional neural network for modeling sentence pairs. arXiv preprint arXiv:1512.05193, 2015.","title":"Abcnn: Attention-based convolutional neural network for modeling sentence pairs","authors":"Yin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}