{"type":"ScholarlyArticle","title":"A Latent Variable Model for Learning Distributional Relation Vectors","authors":[{"name":"Jose Camacho-Collados","slug":"jose-camacho-collados","given":"Jose","family":"Camacho-Collados","affiliation":"School of Computer Science and Informatics , Cardiff University , United Kingdom"},{"name":"Luis Espinosa-Anke","slug":"luis-espinosa-anke","given":"Luis","family":"Espinosa-Anke","affiliation":"School of Computer Science and Informatics , Cardiff University , United Kingdom"},{"name":"Shoaib Jameel","slug":"shoaib-jameel","given":"Shoaib","family":"Jameel","affiliation":"School of Computing , University of Kent , United Kingdom"},{"name":"Steven Schockaert","slug":"steven-schockaert","given":"Steven","family":"Schockaert","affiliation":"School of Computer Science and Informatics , Cardiff University , United Kingdom"}],"abstract":"Recently a number of unsupervised approaches have been proposed for learning vectors that capture the relationship between two words. Inspired by word embedding models, these approaches rely on co-occurrence statistics that are obtained from sentences in which the two target words appear. However, the number of such sentences is often quite small, and most of the words that occur in them are not relevant for characterizing the considered relationship. As a result, standard co-occurrence statistics typically lead to noisy relation vectors. To address this issue, we propose a latent variable model that aims to explicitly determine what words from the given sentences best characterize the relationship between the two target words. Relation vectors then correspond to the parameters of a simple unigram language model which is estimated from these words.","doi":"10.24963/ijcai.2019/682","url":"https://ijcai.org/proceedings/2019/682","pdf":"https://ijcai.org/proceedings/2019/0682.pdf","markdown":"https://ijcai.org/proceedings/2019/682.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/682","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":682,"pages":{"first":4911,"last":4917},"keywords":["Natural Language Processing: Embeddings","Natural Language Processing: Natural Language Semantics","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Agichtein and Gravano, 2000] Eugene Agichtein and Luis Gravano. Snowball: Extracting relations from large plain- text collections. In Proc. of Digital libraries, pages 85-94, 2000.","title":"Eugene Agichtein and Luis Gravano. Snowball: Extracting relations from large plaintext collections","authors":"Gravano Agichtein","year":"2000","venue":"Proc. of Digital libraries","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arora et al., 2016] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A latent variable model approach to pmi-based word embeddings. TACL, 4:385-399, 2016.","title":"A latent variable model approach to pmi-based word embeddings","authors":"Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, Andrej Risteski","year":"2016","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arora et al., 2017] Sanjeev Arora, Yingyu Liang, and Tengyu Ma. A simple but tough-to-beat baseline for sen- tence embeddings. In Proc. of ICLR, 2017.","title":"A simple but tough-to-beat baseline for sentence embeddings","authors":"Sanjeev Arora, Yingyu Liang, Tengyu Ma","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Banko et al., 2007] Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Et- zioni. Open information extraction from the web. In Proc. of IJCAI, pages 2670-2676, 2007.","title":"Open information extraction from the web","authors":"Michele Banko, J Michael, Stephen Cafarella, Matthew Soderland, Oren Broadhead, Etzioni","year":"2007","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baroni and Lenci, 2011] Marco Baroni and Alessandro Lenci. How we blessed distributional semantic evaluation. In Proc. GEMS Workshop, pages 1-10, 2011.","title":"How we blessed distributional semantic evaluation","authors":"Marco Baroni, Alessandro Lenci","year":"2011","venue":"Proc. GEMS Workshop","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bojanowski et al., 2017] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vec- tors with subword information. TACL, 5:135-146, 2017.","title":"Enriching word vectors with subword information","authors":"Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov","year":"2017","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camacho-Collados and Pilehvar, 2018] Jose Camacho- Collados and Mohammad Taher Pilehvar. On the role of text preprocessing in neural network architectures: An evaluation study on text categorization and sentiment analysis. In Proc. of Blackbox EMNLP Workshop, 2018.","title":"On the role of text preprocessing in neural network architectures: An evaluation study on text categorization and sentiment analysis","authors":"Jose Camacho, -Collados, Mohammad Taher, Pilehvar","year":"2018","venue":"Proc. of Blackbox EMNLP Workshop","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Carlson et al., 2010] Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. Toward an architecture for never-ending language learning. In Proc. of AAAI, pages 1306-1313, 2010.","title":"Toward an architecture for never-ending language learning","authors":"Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, Tom M Mitchell","year":"2010","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[dos Santos et al., 2015] Cícero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. Classifying relations by rank- ing with convolutional neural networks. In Proc. of ACL, pages 626-634, 2015.","title":"Classifying relations by ranking with convolutional neural networks","authors":"Cícero Nogueira Dos Santos, Bing Xiang, Bowen Zhou","year":"2015","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Espinosa-Anke and Schockaert, 2018] Luis Espinosa-Anke and Steven Schockaert. SeVeN: Augmenting word em- beddings with unsupervised relation vectors. In Proc. of COLING, pages 2653-2665, 2018.","title":"SeVeN: Augmenting word embeddings with unsupervised relation vectors","authors":"Luis Espinosa-Anke, Steven Schockaert","year":"2018","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Greene and Cunningham, 2006] Derek Greene and Pádraig Cunningham. Practical solutions to the problem of diago- nal dominance in kernel document clustering. In Proc. of ICML, pages 377-384, 2006.","title":"Practical solutions to the problem of diagonal dominance in kernel document clustering","authors":"Cunningham Greene, Derek Greene, Pádraig Cunningham","year":"2006","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hashimoto et al., 2015] Kazuma Hashimoto, Pontus Stene- torp, Makoto Miwa, and Yoshimasa Tsuruoka. Task- oriented learning of word embeddings for semantic rela- tion classification. In Proc. of CoNLL, 2015.","title":"Taskoriented learning of word embeddings for semantic relation classification","authors":"Kazuma Hashimoto, Pontus Stenetorp, Makoto Miwa, Yoshimasa Tsuruoka","year":"2015","venue":"Proc. of CoNLL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Marti A. Hearst. Automatic acquisition of hy- ponyms from large text corpora. In Proc. of COLING, pages 539-545, 1992.","title":"Automatic acquisition of hyponyms from large text corpora","authors":"Marti A Hearst","year":"1992","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jameel et al., 2018] Shoaib Jameel, Zied Bouraoui, and Steven Schockaert. Unsupervised learning of distribu- tional relation vectors. In Proc. of ACL, 2018.","title":"Unsupervised learning of distributional relation vectors","authors":"Shoaib Jameel, Zied Bouraoui, Steven Schockaert","year":"2018","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Joshi et al., 2019] Mandar Joshi, Eunsol Choi, Omer Levy, Daniel S. Weld, and Luke Zettlemoyer. pair2vec: Com- positional word-pair embeddings for cross-sentence infer- ence. In Proc. of NAACL, pages 3597-3608, 2019.","title":"pair2vec: Compositional word-pair embeddings for cross-sentence inference","authors":"Mandar Joshi, Eunsol Choi, Omer Levy, Daniel S Weld, Luke Zettlemoyer","year":"2019","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jurgens et al., 2012] David Jurgens, Saif Mohammad, Pe- ter D. Turney, and Keith J. Holyoak. Semeval-2012 task 2: Measuring degrees of relational similarity. In Proc. of SemEval, pages 356-364, 2012.","title":"Semeval-2012 task 2: Measuring degrees of relational similarity","authors":"David Jurgens, Saif Mohammad, Peter D Turney, Keith J Holyoak","year":"2012","venue":"Proc. of SemEval","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lang, 1995] Ken Lang. Newsweeder: Learning to filter net- news. In Machine Learning Proceedings, pages 331-339, 1995.","title":"Newsweeder: Learning to filter netnews","authors":"Lang ;, Ken Lang","year":"1995","venue":"Machine Learning Proceedings","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Levy et al., 2015a] Omer Levy, Yoav Goldberg, and Ido Dagan. Improving distributional similarity with lessons learned from word embeddings. TACL, 3:211-225, 2015.","title":"Improving distributional similarity with lessons learned from word embeddings","authors":"Levy","year":"2015","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Levy et al., 2015b] Omer Levy, Steffen Remus, Chris Bie- mann, Ido Dagan, and Israel Ramat-Gan. Do supervised distributional methods really learn lexical inference rela- tions? In Proc. of NAACL, pages 970-976, 2015.","title":"Do supervised distributional methods really learn lexical inference relations?","authors":"Levy","year":"2015","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lewis et al., 2004] David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. Rcv1: A new benchmark collection for text categorization research. JMLR, 5:361-397, 2004.","title":"Rcv1: A new benchmark collection for text categorization research","authors":"Yiming David D Lewis, Tony G Yang, Fan Rose, Li","year":"2004","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their composi- tionality. In Proc. of NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Riedel et al., 2013] Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. Relation extraction with matrix factorization and universal schemas. In Proc. of HLT-NAACL, pages 74-84, 2013.","title":"Relation extraction with matrix factorization and universal schemas","authors":"Sebastian Riedel, Limin Yao, Andrew Mccallum, Benjamin M Marlin","year":"2013","venue":"Proc. of HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shinyama and Sekine, 2006] Yusuke Shinyama and Satoshi Sekine. Preemptive information extraction using unre- stricted relation discovery. In Proc. of NAACL-HLT, pages 304-311, 2006.","title":"Preemptive information extraction using unrestricted relation discovery","authors":"Sekine Shinyama, Yusuke Shinyama, Satoshi Sekine","year":"2006","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Turney, 2005] Peter D. Turney. Measuring semantic similar- ity by latent relational analysis. In Proc. of IJCAI, pages 1136-1141, 2005.","title":"Measuring semantic similarity by latent relational analysis","authors":"D Peter, Turney","year":"2005","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vu and Shwartz, 2018] Tu Vu and Vered Shwartz. Integrat- ing multiplicative features into supervised distributional methods for lexical entailment. In Proc. of *SEM, 2018.","title":"Integrating multiplicative features into supervised distributional methods for lexical entailment","authors":"Tu Vu, Vered Shwartz","year":"2018","venue":"Proc. of *SEM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vulić et al., 2017] Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. Hyperlex: A large-scale evaluation of graded lexical entailment. Computational Linguistics, 43(4):781-835, 2017.","title":"Hyperlex: A large-scale evaluation of graded lexical entailment","authors":"Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen","year":"2017","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vylomova et al., 2016] Ekaterina Vylomova, Laura Rimell, Trevor Cohn, and Timothy Baldwin. Take and took, gaggle and goose, book and read: Evaluating the utility of vector differences for lexical relation learning. In Proc. of ACL, pages 1671-1682, 2016.","title":"Take and took, gaggle and goose, book and read: Evaluating the utility of vector differences for lexical relation learning","authors":"Ekaterina Vylomova, Laura Rimell, Trevor Cohn, Timothy Baldwin","year":"2016","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Washio and Kato, 2018a] Koki Washio and Tsuneaki Kato. Filling missing paths: Modeling co-occurrences of word pairs and dependency paths for recognizing lexical seman- tic relations. In Proc. of NAACL, pages 1123-1133, 2018.","title":"Filling missing paths: Modeling co-occurrences of word pairs and dependency paths for recognizing lexical semantic relations","authors":"Kato Washio, Koki Washio, Tsuneaki Kato","year":"2018","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Washio and Kato, 2018b] Koki Washio and Tsuneaki Kato. Neural latent relational analysis to capture lexical semantic relations in a vector space. In Proc. of EMNLP, 2018.","title":"Neural latent relational analysis to capture lexical semantic relations in a vector space","authors":"Washio, Koki Kato ; B], Tsuneaki Washio, Kato","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Weeds et al., 2014] Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. Learning to distin- guish hypernyms and co-hyponyms. In Proc. of COLING, pages 2249-2259, 2014.","title":"Learning to distinguish hypernyms and co-hyponyms","authors":"Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, Bill Keller","year":"2014","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2015] Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. Classifying relations via long short term memory networks along shortest dependency paths. In Proc. of EMNLP, pages 1785-1794, 2015.","title":"Classifying relations via long short term memory networks along shortest dependency paths","authors":"Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, Zhi Jin","year":"2015","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}