{"type":"ScholarlyArticle","title":"Attention-Fused Deep Matching Network for Natural Language Inference","authors":[{"name":"Chaoqun Duan","slug":"chaoqun-duan","given":"Chaoqun","family":"Duan","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Lei Cui","slug":"lei-cui","given":"Lei","family":"Cui","affiliation":"Microsoft Research Asia , Beijing , China"},{"name":"Xinchi Chen","slug":"xinchi-chen","given":"Xinchi","family":"Chen","affiliation":"School of Computer Science , Fudan University , Shanghai , China"},{"name":"Furu Wei","slug":"furu-wei","given":"Furu","family":"Wei","affiliation":""},{"name":"Conghui Zhu","slug":"conghui-zhu","given":"Conghui","family":"Zhu","affiliation":"Harbin Institute of Technology , Harbin , China"},{"name":"Tiejun Zhao","slug":"tiejun-zhao","given":"Tiejun","family":"Zhao","affiliation":"Harbin Institute of Technology , Harbin , China"}],"abstract":"Natural language inference aims to predict whether a premise sentence can infer another hypothesis sentence. Recent progress on this task only relies on a shallow interaction between sentence pairs, which is insufficient for modeling complex relations. In this paper, we present an attention-fused deep matching network (AF-DMN) for natural language inference. Unlike existing models, AF-DMN takes two sentences as input and iteratively learns the attention-aware representations for each side by multi-level interactions. Moreover, we add a self-attention mechanism to fully exploit local context information within each sentence. Experiment results show that AF-DMN achieves state-of-the-art performance and outperforms strong baselines on Stanford natural language inference (SNLI), multi-genre natural language inference (MultiNLI), and Quora duplicate questions datasets.","doi":"10.24963/ijcai.2018/561","url":"https://ijcai.org/proceedings/2018/561","pdf":"https://ijcai.org/proceedings/2018/0561.pdf","markdown":"https://ijcai.org/proceedings/2018/561.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/561","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":561,"pages":{"first":4033,"last":4040},"keywords":["Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Bowman et al., 2015] Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326, 2015.","title":"A large annotated corpus for learning natural language inference","authors":"Gabor Samuel R Bowman, Christopher Angeli, Christopher D Potts, Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman et al., 2016] Samuel R Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D Manning, and Christopher Potts. A fast unified model for parsing and sentence understanding. arXiv preprint arXiv:1603.06021, 2016.","title":"A fast unified model for parsing and sentence understanding","authors":"Jon Samuel R Bowman, Abhinav Gauthier, Raghav Rastogi, Christopher D Gupta, Christopher Manning, Potts","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017a] Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Enhanced lstm for natural language inference. In Proc. ACL, 2017.","title":"Enhanced lstm for natural language inference","authors":"Chen","year":"2017","venue":"Proc. ACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017b] Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Recur- rent neural network-based sentence encoder with gated attention for natural language inference. arXiv preprint arXiv:1708.01353, 2017.","title":"Recurrent neural network-based sentence encoder with gated attention for natural language inference","authors":"Chen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. arXiv preprint arXiv:1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cui et al., 2016] Yiming Cui, Zhipeng Chen, Si Wei, Shi- jin Wang, Ting Liu, and Guoping Hu. Attention-over- attention neural networks for reading comprehension. arXiv preprint arXiv:1607.04423, 2016.","title":"Attention-overattention neural networks for reading comprehension","authors":"Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, Guoping Hu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Heilman and Smith, 2010] Michael Heilman and Noah A Smith. Tree edit models for recognizing textual entail- ments, paraphrases, and answers to questions. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Com- putational Linguistics, pages 1011-1019. Association for Computational Linguistics, 2010.","title":"Tree edit models for recognizing textual entailments, paraphrases, and answers to questions","authors":"Smith ; Heilman, Michael Heilman, Noah A Smith","year":"2010","venue":"Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2017] Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130, 2017.","title":"A structured self-attentive sentence embedding","authors":"Lin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016a] Pengfei Liu, Xipeng Qiu, Jifan Chen, and Xuanjing Huang. Deep fusion lstms for text semantic matching. In ACL (1), 2016.","title":"Deep fusion lstms for text semantic matching","authors":"Liu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2016b] Yang Liu, Chengjie Sun, Lei Lin, and Xi- aolong Wang. Learning natural language inference us- ing bidirectional lstm model and inner-attention. arXiv preprint arXiv:1605.09090, 2016.","title":"Learning natural language inference using bidirectional lstm model and inner-attention","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[MacCartney, 2009] Bill MacCartney. Natural language in- ference. Stanford University, 2009.","title":"Natural language inference","authors":"Maccartney","year":"2009","venue":"Bill MacCartney","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mou et al., 2015] Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. Natural language in- ference by tree-based convolution and heuristic matching. arXiv preprint arXiv:1512.08422, 2015.","title":"Natural language inference by tree-based convolution and heuristic matching","authors":"Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, Zhi Jin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Munkhdalai and Yu, 2016] Tsendsuren Munkhdalai and Hong Yu. Neural tree indexers for text understanding. arXiv preprint arXiv:1607.04492, 2016.","title":"Tsendsuren Munkhdalai and Hong Yu","authors":"Yu Munkhdalai","year":"2016","venue":"Neural tree indexers for text understanding","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Parikh et al., 2016] Ankur P Parikh, Oscar Täckström, Di- panjan Das, and Jakob Uszkoreit. A decomposable atten- tion model for natural language inference. arXiv preprint arXiv:1606.01933, 2016.","title":"A decomposable attention model for natural language inference","authors":"P Ankur, Oscar Parikh, Dipanjan Täckström, Jakob Das, Uszkoreit","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rocktäschel et al., 2015] Tim Rocktäschel, Edward Grefen- stette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. Reasoning about entailment with neural atten- tion. arXiv preprint arXiv:1509.06664, 2015.","title":"Tim Rocktäschel, Edward Grefenstette","authors":"Rocktäschel","year":"2015","venue":"Reasoning about entailment with neural attention","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sha et al., 2016] Lei Sha, Baobao Chang, Zhifang Sui, and Sujian Li. Reading and thinking: Re-read lstm unit for textual entailment recognition. In COLING, pages 2870- 2879, 2016.","title":"Reading and thinking: Re-read lstm unit for textual entailment recognition","authors":"Lei Sha, Baobao Chang, Zhifang Sui, Sujian Li","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 1-9, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tan et al., 2015] Ming Tan, Cicero dos Santos, Bing Xi- ang, and Bowen Zhou. Lstm-based deep learning mod- els for non-factoid answer selection. arXiv preprint arXiv:1511.04108, 2015.","title":"Lstm-based deep learning models for non-factoid answer selection","authors":"Ming Tan, Bing Cicero Dos Santos, Bowen Xiang, Zhou","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang and Jiang, 2015] Shuohang Wang and Jing Jiang. Learning natural language inference with lstm. arXiv preprint arXiv:1512.08849, 2015.","title":"Learning natural language inference with lstm","authors":"Wang, Jiang, Shuohang Wang, Jing Jiang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017a] Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching net- works for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Pa- pers), volume 1, pages 189-198, 2017.","title":"Gated self-matching networks for reading comprehension and question answering","authors":"Wang","year":"2017","venue":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2017b] Zhiguo Wang, Wael Hamza, and Radu Florian. Bilateral multi-perspective matching for natu- ral language sentences. arXiv preprint arXiv:1702.03814, 2017.","title":"Bilateral multi-perspective matching for natural language sentences","authors":"Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/579"},{"position":27,"raw":"[Williams et al., 2017] Adina Williams, Nikita Nangia, and Samuel R Bowman. A broad-coverage challenge cor- pus for sentence understanding through inference. arXiv preprint arXiv:1704.05426, 2017.","title":"A broad-coverage challenge corpus for sentence understanding through inference","authors":"Adina Williams, Nikita Nangia, Samuel R Bowman","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2016] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation sys- tem: Bridging the gap between human and machine trans- lation. arXiv preprint arXiv:1609.08144, 2016.","title":"Google's neural machine translation system: Bridging the gap between human and machine translation","authors":"Yonghui Wu, Mike Schuster, Zhifeng Chen, V Quoc, Mohammad Le, Wolfgang Norouzi, Maxim Macherey, Yuan Krikun, Qin Cao, Klaus Gao, Macherey","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}