{"type":"ScholarlyArticle","title":"A Survey on Word Meta-Embedding Learning","authors":[{"name":"Danushka Bollegala","slug":"danushka-bollegala","given":"Danushka","family":"Bollegala","affiliation":"University of Liverpool; Amazon"},{"name":"James O' Neill","slug":"james-o-neill","given":"James","family":"O' Neill","affiliation":"University of Liverpool"}],"abstract":"Meta-embedding (ME) learning is an emerging approach that attempts to learn more accurate word embeddings given existing (source) word embeddings as the sole input.\r\n  Due to their ability to incorporate semantics from multiple source embeddings in a compact manner with superior performance, ME learning has gained popularity among practitioners in NLP.\r\n  To the best of our knowledge, there exist no prior systematic survey on ME learning and this paper\r\n attempts to fill this need.\r\n  We classify ME learning methods according to multiple factors such as whether they \r\n  (a) operate on static or contextualised embeddings, (b) trained in an unsupervised manner or (c) fine-tuned for a particular task/domain.\r\n  Moreover, we discuss the limitations of existing ME learning methods and highlight potential future research directions.","doi":"10.24963/ijcai.2022/758","url":"https://ijcai.org/proceedings/2022/758","pdf":"https://ijcai.org/proceedings/2022/0758.pdf","markdown":"https://ijcai.org/proceedings/2022/758.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/758","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Survey Track","track":"Survey Track","number":758,"pages":{"first":5402,"last":5409},"keywords":["Survey Track: Natural Language Processing","Survey Track: Machine Learning","Survey Track: -"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987301","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985444","length_sec":735,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Alsuhaibani et al., 2019] Mohammed Alsuhaibani, Takanori Maehara, and Danushka Bollegala. Joint learning of hierar- chical word embeddings from a corpus and a taxonomy. In Proc. of AKBC, 2019.","title":"Joint learning of hierarchical word embeddings from a corpus and a taxonomy","authors":"Mohammed Alsuhaibani, Takanori Maehara, Danushka Bollegala","year":"2019","venue":"Proc. of AKBC","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arora et al., 2016] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A latent variable model approach to pmi-based word embeddings. TACL, 4:385-399, 2016.","title":"A latent variable model approach to pmi-based word embeddings","authors":"Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, Andrej Risteski","year":"2016","venue":"TACL","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Artetxe et al., 2016] Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In Proc. of EMNLP, pages 2289-2294, 2016.","title":"Learning principled bilingual mappings of word embeddings while preserving monolingual invariance","authors":"Mikel Artetxe, Gorka Labaka, Eneko Agirre","year":"2016","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bao and Bollegala, 2018] Cong Bao and Danushka Bolle- gala. Learning word meta-embeddings by autoencoding. In Proc. of COLING, pages 1650-1661, 2018.","title":"Learning word meta-embeddings by autoencoding","authors":"Cong Bao, Danushka Bollegala","year":"2018","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bollegala et al., 2016] Danushka Bollegala, Alsuhaibani Mohammed, Takanori Maehara, and Ken-ichi Kawarabayashi. Joint word representation learning using a corpus and a semantic lexicon. In Proc. of AAAI, pages 2690-2696, 2016.","title":"Joint word representation learning using a corpus and a semantic lexicon","authors":"Danushka Bollegala, Alsuhaibani Mohammed, Takanori Maehara, Ken-Ichi Kawarabayashi","year":"2016","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bollegala et al., 2017] Danushka Bollegala, Kohei Hayashi, and Ken-ichi Kawarabayashi. Learning linear transforma- tions between counting-based and prediction-based word embeddings. PLoS ONE, 12(9):1-21, September 2017.","title":"Learning linear transformations between counting-based and prediction-based word embeddings","authors":"Danushka Bollegala, Kohei Hayashi, Ken-Ichi Kawarabayashi","year":"2017","venue":"PLoS ONE","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bollegala et al., 2018a] Danushka Bollegala, Kohei Hayashi, and Ken-ichi Kawarabayashi. Think globally, embed lo- cally -locally linear meta-embedding of words. In Proc. of IJCAI-EACI, pages 3970-3976, 2018.","title":"Think globally, embed locally -locally linear meta-embedding of words","authors":"Bollegala","year":"2018","venue":"Proc. of IJCAI-EACI","doi":null,"url":null,"internal_url":"/proceedings/2018/552"},{"position":8,"raw":"[Bollegala et al., 2018b] Danushka Bollegala, Yuichi Yoshida, and Ken-ichi Kawarabayashi. Using k-way Co-occurrences for Learning Word Embeddings. In Proc. of AAAI, pages 5037-5044, 2018.","title":"Using k-way Co-occurrences for Learning Word Embeddings","authors":"Bollegala","year":"2018","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bollegala, 2022] Danushka Bollegala. Learning meta word embeddings by unsupervised weighted concatenation of source embeddings. In Proc. of IJCAI, 2022.","title":"Learning meta word embeddings by unsupervised weighted concatenation of source embeddings","authors":"Danushka Bollegala","year":"2022","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/563"},{"position":10,"raw":"[Bolukbasi et al., 2016] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. In Proc. of NeurIPS, pages 4349-4357, 2016.","title":"Man is to computer programmer as woman is to homemaker? debiasing word embeddings","authors":"Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, Adam T Kalai","year":"2016","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Coates and Bollegala, 2018] Joshua Coates and Danushka Bollegala. Frustratingly easy meta-embedding -computing meta-embeddings by averaging source word embeddings. In Proc. of NAACL-HLT, pages 194-198, 2018.","title":"Frustratingly easy meta-embedding -computing meta-embeddings by averaging source word embeddings","authors":"Bollegala Coates, Joshua Coates, Danushka Bollegala","year":"2018","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuska. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493 -2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuska","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Conneau et al., 2017] Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. arXiv preprint arXiv:1710.04087, 2017.","title":"Word translation without parallel data","authors":"Alexis Conneau, Guillaume Lample, Marc'aurelio Ranzato, Ludovic Denoyer, Hervé Jégou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL-HLT, pages 4171-4186, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Dhillon et al., 2015] Paramveer S. Dhillon, Dean P. Foster, and Lyle H. Ungar. Eigenwords: Spectral word embeddings. Journal of Machine Learning Research, 16:3035-3078, 2015.","title":"Eigenwords: Spectral word embeddings","authors":"S Paramveer, Dean P Dhillon, Lyle H Foster, Ungar","year":"2015","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Dietterich, 2002] Thomas Dietterich. The Handbook of Brain Theory and Neural Networks. MIT press Cambridge, 2002.","title":"The Handbook of Brain Theory and Neural Networks","authors":"Thomas Dietterich","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Doval et al., 2018] Yerai Doval, Jose Camacho-Collados, Luis Espinosa-Anke, and Steven Schockaert. Improving cross-lingual word embeddings by meeting in the middle. arXiv preprint arXiv:1808.08780, 2018.","title":"Improving cross-lingual word embeddings by meeting in the middle","authors":"Doval","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-Adversarial Training of Neural Networks. JMLR, 17:1-35, 2016.","title":"Domain-Adversarial Training of Neural Networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Mario Franc ¸ois Laviolette, Victor Marchand, Lempitsky","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[García et al., 2020] Iker García, Rodrigo Agerri, and Ger- man Rigau. A common semantic space for monolin- gual and cross-lingual meta-embeddings. arXiv preprint arXiv:2001.06381, 2020.","title":"A common semantic space for monolingual and cross-lingual meta-embeddings","authors":"Iker García, Rodrigo Agerri, German Rigau","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[García-Ferrero et al., 2021] Iker García-Ferrero, Rodrigo Agerri, and German Rigau. Benchmarking meta- embeddings: What works and what does not. In Find- ings of EMNLP, pages 3957-3972, Punta Cana, Dominican Republic, 2021.","title":"Benchmarking metaembeddings: What works and what does not","authors":"Iker García-Ferrero, Rodrigo Agerri, German Rigau","year":"2021","venue":"Findings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Goikoetxea et al., 2016] Josu Goikoetxea, Eneko Agirre, and Aitor Soroa. Single or multiple? combining word representations independently learned from text and word- net. In Proc. of AAAI, pages 2608-2614, 2016.","title":"Single or multiple? combining word representations independently learned from text and wordnet","authors":"Josu Goikoetxea, Eneko Agirre, Aitor Soroa","year":"2016","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[He et al., 2020] Jingyi He, KC Tsiolis, Kian Kenyon-Dean, and Jackie Chi Kit Cheung. Learning Efficient Task- Specific Meta-Embeddings with Word Prisms. In Proc. of COLING, 2020.","title":"Learning Efficient Task-Specific Meta-Embeddings with Word Prisms","authors":"Jingyi He, Kian Tsiolis, Jackie Kenyon-Dean, Kit Chi, Cheung","year":"2020","venue":"Proc. of COLING","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Hinton et al., 2015] Geoffery E. Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv, 10.48550/ARXIV.1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"E Geoffery, Oriol Hinton, Jeff Vinyals, Dean","year":"2015","venue":null,"doi":"arxiv,10.48550/arxiv.1503.02531","url":null,"internal_url":null},{"position":24,"raw":"[Huang et al., 2012] Eric H. Huang, Richard Socher, Christo- pher D. Manning, and Andrew Y. Ng. Improving word representations via global context and multiple word proto- types. In Proc. of ACL, pages 873-882, 2012.","title":"Improving word representations via global context and multiple word prototypes","authors":"Eric H Huang, Richard Socher, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Jawanpuria et al., 2020] Pratik Jawanpuria, Satya Dev N T V, Anoop Kunchukuttan, and Bamdev Mishra. Learn- ing geometric word meta-embeddings. In Proc. of RepL4NLP, pages 39-44, 2020.","title":"Learning geometric word meta-embeddings","authors":"Pratik Jawanpuria, Satya Dev, N T V, Anoop Kunchukuttan, Bamdev Mishra","year":"2020","venue":"Proc. of RepL4NLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Joulin et al., 2017] A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov. Bag of Tricks for Efficient Text Classification. In Proc. of EACL, pages 427-431, July 2017.","title":"Bag of Tricks for Efficient Text Classification","authors":"A Joulin, E Grave, P Bojanowski, T Mikolov","year":"2017","venue":"Proc. of EACL","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Kaneko and Bollegala, 2019] Masahiro Kaneko and Danushka Bollegala. Gender-preserving debiasing for pre-trained word embeddings. In Proc. of ACL, pages 1641-1650, 2019.","title":"Gender-preserving debiasing for pre-trained word embeddings","authors":"Masahiro Kaneko, Danushka Bollegala","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Kiela et al., 2018] Douwe Kiela, Changhan Wang, and Kyunghyun Cho. Dynamic meta-embeddings for improved sentence representations. In Proc. of EMNLP, pages 1466- 1477, 2018.","title":"Dynamic meta-embeddings for improved sentence representations","authors":"Kiela","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lan et al., 2020] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Sori- cut. Albert: A lite bert for self-supervised learning of language representations. In Proc. of ICLR, 2020.","title":"Albert: A lite bert for self-supervised learning of language representations","authors":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Lange et al., 2021] Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. FAME: Feature-based ad- versarial meta-embeddings for robust input representations. In Proc. of EMNLP, pages 8382-8395, 2021.","title":"FAME: Feature-based ad-versarial meta-embeddings for robust input representations","authors":"Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow","year":"2021","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Levy et al., 2015] Omer Levy, Yoav Goldberg, and Ido Da- gan. Improving distributional similarity with lessons learned from word embeddings. Transactions of Associa- tion for Computational Linguistics, 3:211-225, 2015.","title":"Improving distributional similarity with lessons learned from word embeddings","authors":"Omer Levy, Yoav Goldberg, Ido Dagan","year":"2015","venue":"Transactions of Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Ap- proach. 10.48550/ARXIV.1907.11692, 2019.","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","authors":"Liu","year":"2019","venue":null,"doi":"10.48550/arxiv.1907.11692","url":null,"internal_url":null},{"position":33,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, and Jeffrey Dean. Efficient estimation of word representation in vector space. In Proc. of ICLR, 2013.","title":"Efficient estimation of word representation in vector space","authors":"Mikolov","year":"2013","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Miller and Charles, 1998] G. Miller and W. Charles. Con- textual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1-28, 1998.","title":"Contextual correlates of semantic similarity","authors":"G Miller, W Charles","year":"1998","venue":"Language and Cognitive Processes","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Mnih and Hinton, 2009] Andriy Mnih and Geoffrey E. Hin- ton. A scalable hierarchical distributed language model. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Proc. of NIPS, pages 1081-1088, 2009.","title":"A scalable hierarchical distributed language model","authors":"Andriy Mnih, Geoffrey E Hinton","year":"2009","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Mu et al., 2018] J. Mu, S. Bhat, and P. Viswanath. All-but- the-Top: Simple and Effective Postprocessing for Word Representations. In Proc. of ICLR, 2018.","title":"All-butthe-Top: Simple and Effective Postprocessing for Word Representations","authors":"J Mu, S Bhat, P Viswanath","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Muromägi et al., 2017] Avo Muromägi, Kairit Sirts, and Sven Laur. Linear ensembles of word embedding mod- els. In Proc. of the Nordic Conference on Computational Linguistics, pages 96-104, 2017.","title":"Avo Muromägi, Kairit Sirts, and Sven Laur. Linear ensembles of word embedding models","authors":"Muromägi","year":"2017","venue":"Proc. of the Nordic Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Neelakantan et al., 2014] Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. Effi- cient non-parametric estimation of multiple embeddings per word in vector space. In Proc. of EMNLP, pages 1059-1069, October 2014.","title":"Efficient non-parametric estimation of multiple embeddings per word in vector space","authors":"Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, Andrew Mccallum","year":"2014","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[O'Neill and Bollegala, 2018] James O'Neill and Danushka Bollegala. Meta-embedding as auxiliary task regularization. In Proc. of EACI, 2018.","title":"Meta-embedding as auxiliary task regularization","authors":"O' Neill, Bollegala ; James O' Neill, Danushka Bollegala","year":"2018","venue":"Proc. of EACI","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowl- edge and Data Engineering, 22(10):1345 -1359, October 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Passos et al., 2018] Alexandre Tachard Passos, Gabriel Pereyra, Geoffrey Hinton, George Dahl, Robert Ormandi, and Rohan Anil. Large scale distributed neural network training through online distillation. In Proc. of ICLR, 2018.","title":"Large scale distributed neural network training through online distillation","authors":"Alexandre Tachard Passos, Gabriel Pereyra, Geoffrey Hinton, George Dahl, Robert Ormandi, Rohan Anil","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Pennington et al., 2014] Jeffery Pennington, Richard Socher, and Christopher D. Manning. Glove: global vectors for word representation. In Proc. of EMNLP, pages 1532-1543, 2014.","title":"Glove: global vectors for word representation","authors":"Jeffery Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Peters et al., 2018] Matthew E. Peters, Mark Neumann, Mo- hit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word represen- tations. In Proc. of NAACL-HLT, 2018.","title":"Deep contextualized word representations","authors":"Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer","year":"2018","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Poerner et al., 2020] Nina Poerner, Ulli Waltinger, and Hin- rich Schütze. Sentence meta-embeddings for unsupervised semantic textual similarity. In Proc. of ACL, pages 7027- 7034, 2020.","title":"Sentence meta-embeddings for unsupervised semantic textual similarity","authors":"Nina Poerner, Ulli Waltinger, Hinrich Schütze","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Polikar, 2012] Robi Polikar. Ensemble learning. In Ensem- ble Machine Learning, pages 1-34. Springer US, 2012.","title":"Robi Polikar. Ensemble learning","authors":"Polikar","year":"2012","venue":"Ensemble Machine Learning","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Reisinger and Mooney, 2010] Joseph Reisinger and Ray- mond J. Mooney. Multi-prototype vector-space models of word meaning. In Proc. of HLT-NAACL, pages 109-117, 2010.","title":"Multi-prototype vector-space models of word meaning","authors":"Joseph Reisinger, Raymond J Mooney","year":"2010","venue":"Proc. of HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Takahashi and Bollegala, 2022] Keigo Takahashi and Danushka Bollegala. Unsupervised attention-based sentence-level meta-embeddings from contextualised language models. In Proc. of LREC, 2022.","title":"Unsupervised attention-based sentence-level meta-embeddings from contextualised language models","authors":"Keigo Takahashi, Danushka Bollegala","year":"2022","venue":"Proc. of LREC","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Tissier et al., 2017] Julien Tissier, Christopher Gravier, and Amaury Habrard. Dict2vec : Learning word embeddings using lexical dictionaries. In Proc. of EMNLP, pages 254- 263, 2017.","title":"Dict2vec : Learning word embeddings using lexical dictionaries","authors":"Julien Tissier, Christopher Gravier, Amaury Habrard","year":"2017","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proc of NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Proc of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Wang et al., 2018] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proc. of BlackboxNLP, pages 353-355, 2018.","title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","authors":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel Bowman","year":"2018","venue":"Proc. of BlackboxNLP","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Winata et al., 2019a] Genta Indra Winata, Zhaojiang Lin, and Pascale Fung. Learning multilingual meta-embeddings for code-switching named entity recognition. In Proc. of RepL4NLP, pages 181-186, 2019.","title":"Learning multilingual meta-embeddings for code-switching named entity recognition","authors":"Winata","year":"2019","venue":"Proc. of RepL4NLP","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Winata et al., 2019b] Genta Indra Winata, Zhaojiang Lin, Jamin Shin, Zihan Liu, and Pascale Fung. Hierarchical meta-embeddings for code-switching named entity recog- nition. arXiv preprint arXiv:1909.08504, 2019.","title":"Hierarchical meta-embeddings for code-switching named entity recognition","authors":"Winata","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Wu et al., 2020] Xin Wu, Yi Cai, Yang Kai, Tao Wang, and Qing Li. Task-oriented domain-specific meta-embedding for text classification. In Proc. of EMNLP, pages 3508- 3513, 2020.","title":"Task-oriented domain-specific meta-embedding for text classification","authors":"Xin Wu, Yi Cai, Yang Kai, Tao Wang, Qing Li","year":"2020","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Xie et al., 2019] Yuqiang Xie, Yue Hu, Luxi Xing, and Xi- angpeng Wei. Dynamic task-specific factors for meta- embedding. In International Conference on Knowledge Science, Engineering and Management, pages 63-74. Springer, 2019.","title":"Dynamic task-specific factors for metaembedding","authors":"Yuqiang Xie, Yue Hu, Luxi Xing, Xiangpeng Wei","year":"2019","venue":"International Conference on Knowledge Science, Engineering and Management","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Yang et al., 2019] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. In Proc. of NeurIPS, 2019.","title":"Xlnet: Generalized autoregressive pretraining for language understanding","authors":"Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, Quoc V Le","year":"2019","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Yin and Schütze, 2016] Wenpeng Yin and Hinrich Schütze. Learning word meta-embeddings. In Proc. of ACL, pages 1351-1360, 2016.","title":"Learning word meta-embeddings","authors":"Schütze Yin, Wenpeng Yin, Hinrich Schütze","year":"2016","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Yin and Shen, 2018] Zi Yin and Yuanyuan Shen. On the di- mensionality of word embedding. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett, editors, Proc. of NeurIPS, pages 887-898.","title":"On the dimensionality of word embedding","authors":"Zi Yin, Yuanyuan Shen","year":null,"venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}