{"type":"ScholarlyArticle","title":"ACV-tree: A New Method for Sentence Similarity Modeling","authors":[{"name":"Yuquan Le","slug":"yuquan-le","given":"Yuquan","family":"Le","affiliation":"† College of Computer Science and Electronic Engineering , Hunan University , Changsha , China"},{"name":"Zhi-Jie Wang","slug":"zhi-jie-wang","given":"Zhi-Jie","family":"Wang","affiliation":"‡ Guangdong Key Lab. of Big Data Anal. and Proc. , Sun Yat-Sen University , Guangzhou , China"},{"name":"Zhe Quan","slug":"zhe-quan","given":"Zhe","family":"Quan","affiliation":"† College of Computer Science and Electronic Engineering , Hunan University , Changsha , China"},{"name":"Jiawei He","slug":"jiawei-he","given":"Jiawei","family":"He","affiliation":"† College of Computer Science and Electronic Engineering , Hunan University , Changsha , China"},{"name":"Bin Yao","slug":"bin-yao","given":"Bin","family":"Yao","affiliation":"# Department of Computer Science and Engineering , Shanghai Jiao Tong University , Shanghai , China"}],"abstract":"Sentence similarity modeling lies at the core of many natural language processing applications, and thus has received much attention. Owing to the success of word embeddings, recently, popular neural network methods have achieved sentence embedding, obtaining attractive performance. Nevertheless, most of them focused on learning semantic information and modeling it as a continuous vector, while the syntactic information of sentences has not been fully exploited. On the other hand, prior works have shown the benefits of structured trees that include syntactic information, while few methods in this branch utilized the advantages of word embeddings and another powerful technique ? attention weight mechanism. This paper makes the first attempt to absorb their advantages by merging these techniques in a unified structure, dubbed as ACV-tree. Meanwhile, this paper develops a new tree kernel, known as ACVT kernel, that is tailored for sentence similarity measure based on the proposed structure. The experimental results, based on 19 widely-used datasets, demonstrate that our model is effective and competitive, compared against state-of-the-art models.","doi":"10.24963/ijcai.2018/575","url":"https://ijcai.org/proceedings/2018/575","pdf":"https://ijcai.org/proceedings/2018/0575.pdf","markdown":"https://ijcai.org/proceedings/2018/575.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/575","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":575,"pages":{"first":4137,"last":4143},"keywords":["Natural Language Processing: Embeddings","Natural Language Processing: Natural Language Processing","Natural Language Processing: Natural Language Semantics"],"videos":[],"references":[{"position":1,"raw":"[Arora et al., 2017] Sanjeev Arora, Yingyu Liang, and Tengyu Ma. A simple but tough-to-beat baseline for sen- tence embeddings. In ICLR, 2017.","title":"A simple but tough-to-beat baseline for sentence embeddings","authors":"Sanjeev Arora, Yingyu Liang, Tengyu Ma","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barrett et al., 2016] Maria Barrett, Joachim Bingel, Frank Keller, and Anders Søgaard. Weakly supervised part-of- speech tagging using eye-tracking data. In ACL, pages 579-584, 2016.","title":"Weakly supervised part-ofspeech tagging using eye-tracking data","authors":"Maria Barrett, Joachim Bingel, Frank Keller, Anders Søgaard","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2003] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilis- tic language model. JMLR, 3:1137-1155, 2003.","title":"A neural probabilistic language model","authors":"Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carnie, 2012] Andrew Carnie. Syntax: A Generative Intro- duction, 3rd Edition. Wiley-Blackwell, 2012.","title":"Syntax: A Generative Introduction","authors":"Andrew Carnie","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Collins and Duffy, 2001] Michael Collins and Nigel Duffy. Convolution kernels for natural language. In NIPS, pages 625-632, 2001.","title":"Convolution kernels for natural language","authors":"Michael Collins, Nigel Duffy","year":"2001","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Conneau et al., 2017] Alexis Conneau, Douwe Kiela, Hol- ger Schwenk, Loic Barrault, and Antoine Bordes. Super- vised learning of universal sentence representations from natural language inference data. pages 670-680, 2017.","title":"Supervised learning of universal sentence representations from natural language inference data","authors":"Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, Antoine Bordes","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Croce et al., 2011] Danilo Croce, Alessandro Moschitti, and Roberto Basili. Structured lexical similarity via con- volution kernels on dependency trees. In EMNLP, pages 1034-1046, 2011.","title":"Structured lexical similarity via convolution kernels on dependency trees","authors":"Danilo Croce, Alessandro Moschitti, Roberto Basili","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fellbaum, 1998] Christiane Fellbaum. WordNet: An Elec- tronic Lexical Database. The MIT Press, Cambridge, 1998.","title":"WordNet: An Electronic Lexical Database","authors":null,"year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gabrilovich and Markovitch, 2007] Evgeniy Gabrilovich and Shaul Markovitch. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In IJCAI, pages 1606-1611, 2007.","title":"Computing semantic relatedness using wikipedia-based explicit semantic analysis","authors":"Evgeniy Gabrilovich, Shaul Markovitch","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/259"},{"position":10,"raw":"[Gan et al., 2017] Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, and Lawrence Carin. Learn- ing generic sentence representations using convolutional neural networks. In EMNLP, pages 2390-2400, 2017.","title":"Learning generic sentence representations using convolutional neural networks","authors":"Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gers and Schraudolph, 2002] Felix A. Gers and Nicol N. Schraudolph. Learning precise timing with lstm recurrent networks. JMLR, 3:115-143, 2002.","title":"Learning precise timing with lstm recurrent networks","authors":"Felix A Gers, Nicol N Schraudolph","year":"2002","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo and Diab, 2012] Weiwei Guo and Mona T. Diab. Mod- eling sentences in the latent space. In ACL, pages 864-872, 2012.","title":"Modeling sentences in the latent space","authors":"Diab Guo, Weiwei Guo, Mona T Diab","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Iyyer et al., 2015] Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé Iii. Deep unordered com- position rivals syntactic methods for text classification. In ACL, pages 1681-1691, 2015.","title":"Deep unordered composition rivals syntactic methods for text classification","authors":"Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, Hal Daumé Iii","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jiang and Conrath, 1997] Jay J Jiang and David W Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In ROCLING, page 19-33, 1997.","title":"Semantic similarity based on corpus statistics and lexical taxonomy","authors":"J Jay, David W Jiang, Conrath","year":"1997","venue":"ROCLING","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jiang and Wu, 2017] Wenjun Jiang and Jie Wu. Active opinion-formation in online social networks. In INFO- COM, pages 1-9, 2017.","title":"Active opinion-formation in online social networks","authors":"Wenjun Jiang, Jie Wu","year":"2017","venue":"INFO-COM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2015] Wenjun Jiang, Jie Wu, and Guojun Wang. On selecting recommenders for trust evaluation in online social networks. ACM Trans. Internet Techn., 15(4):1-21, 2015.","title":"On selecting recommenders for trust evaluation in online social networks","authors":"Wenjun Jiang, Jie Wu, Guojun Wang","year":"2015","venue":"ACM Trans. Internet Techn","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kalchbrenner et al., 2014] Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. In ACL, pages 655-665, 2014.","title":"A convolutional neural network for modelling sentences","authors":"Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kenter et al., 2016] Tom Kenter, Alexey Borisov, and Maarten De Rijke. Siamese cbow: Optimizing word em- beddings for sentence representations. In ACL, pages 941- 951, 2016.","title":"Siamese cbow: Optimizing word embeddings for sentence representations","authors":"Kenter","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kiros et al., 2015] Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. Skip-thought vectors. In NIPS, pages 3294-3302, 2015.","title":"Skip-thought vectors","authors":"Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Le and Mikolov, 2014] Quoc V. Le and Tomas Mikolov. Distributed representations of sentences and documents. In ICML, pages 1188-1196, 2014.","title":"Distributed representations of sentences and documents","authors":"V Quoc, Tomas Le, Mikolov","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Leacock and Chodorow, 1998] Claudia Leacock and Martin Chodorow. Combining local context and WordNet similar- ity for word sense identification. The MIT Press, Cam- bridge, 1998.","title":"Combining local context and WordNet similarity for word sense identification","authors":"Chodorow Leacock, Claudia Leacock, Martin Chodorow","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lin and others, 1998] Dekang Lin et al. An information- theoretic definition of similarity. In ICML, pages 296-304, 1998.","title":"An informationtheoretic definition of similarity","authors":"Dekang Lin","year":"1998","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Manning et al., 2017] Chris Manning, Dan Jurafsky, and Percy Liang. Stanford parsing tool. https://nlp.stanford. edu/software/, 2017.","title":"Stanford parsing tool","authors":"Chris Manning, Dan Jurafsky, Percy Liang","year":null,"venue":null,"doi":null,"url":"https://nlp.stanford.edu/software/,2017","internal_url":null},{"position":24,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Moschitti, 2006] Alessandro Moschitti. Efficient convolu- tion kernels for dependency and constituent syntactic trees. In ECML, pages 318-329, 2006.","title":"Efficient convolution kernels for dependency and constituent syntactic trees","authors":"Alessandro Moschitti","year":"2006","venue":"ECML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pilehvar and Navigli, 2015] Mohammad Taher Pilehvar and Roberto Navigli. From senses to texts: An all-in-one graph-based approach for measuring semantic similarity. Artif. Intell., 228:95-128, 2015.","title":"From senses to texts: An all-in-one graph-based approach for measuring semantic similarity","authors":"Mohammad Taher, Pilehvar, Roberto Navigli","year":"2015","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Resnik, 1995] Philip Resnik. Using information content to evaluate semantic similarity in a taxonomy. In IJCAI, pages 448-453, 1995.","title":"Using information content to evaluate semantic similarity in a taxonomy","authors":"Philip Resnik, Resnik","year":"1995","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/1995/59"},{"position":29,"raw":"[Severyn et al., 2013] Aliaksei Severyn, Massimo Nicosia, and Alessandro Moschitti. Building structures from clas- sifiers for passage reranking. In CIKM, pages 969-978, 2013.","title":"Building structures from classifiers for passage reranking","authors":"Aliaksei Severyn, Massimo Nicosia, Alessandro Moschitti","year":"2013","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christopher D. Manning. Improved semantic represen- tations from tree-structured long short-term memory net- works. In ACL, pages 1556-1566, 2015.","title":"Improved semantic representations from tree-structured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2016] Yashen Wang, Heyan Huang, Chong Feng, Qiang Zhou, Jiahui Gu, and Xiong Gao. Cse: Con- ceptual sentence embeddings based on attention model. In ACL, pages 505-515, 2016.","title":"Cse: Conceptual sentence embeddings based on attention model","authors":"Wang","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2017] Shaonan Wang, Jiajun Zhang, and Chengqing Zong. Learning sentence representation with guidance of human attention. In IJCAI, pages 4137-4143, 2017.","title":"Learning sentence representation with guidance of human attention","authors":"Wang","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/578"},{"position":33,"raw":"[Wieting et al., 2015] John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu, and Dan Roth. From paraphrase database to compositional paraphrase model and back. TCAL, 3:98-104, 2015.","title":"From paraphrase database to compositional paraphrase model and back","authors":"John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu, Dan Roth","year":"2015","venue":"TCAL","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wieting et al., 2016] John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. Towards universal paraphras- tic sentence embeddings. In ICLR, 2016.","title":"Towards universal paraphrastic sentence embeddings","authors":"John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wu and Lowery, 2006] Andi Wu and Kirk Lowery. From prosodic trees to syntactic trees. In ACL, pages 898-904, 2006.","title":"From prosodic trees to syntactic trees","authors":"Andi Wu, Kirk Lowery","year":"2006","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wu and Palmer, 1994] Zhibiao Wu and Martha Palmer. Verbs semantics and lexical selection. In ACL, pages 133- 138, 1994.","title":"Zhibiao Wu and Martha Palmer. Verbs semantics and lexical selection","authors":"Palmer Wu","year":"1994","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2014] Wei Xu, Alan Ritter, Chris Callison-Burch, William B. Dolan, and Yangfeng Ji. Extracting lexically divergent paraphrases from twitter. TCAL, 2:435-448, 2014.","title":"Extracting lexically divergent paraphrases from twitter","authors":"Wei Xu, Alan Ritter, Chris Callison-Burch, William B Dolan, Yangfeng Ji","year":"2014","venue":"TCAL","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yu and Dredze, 2015] Mo Yu and Mark Dredze. Learning composition models for phrase embeddings. TACL, 3:227- 242, 2015.","title":"Learning composition models for phrase embeddings","authors":"Mo Yu, Mark Dredze","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}