{"type":"ScholarlyArticle","title":"Enhancing Entity Representations with Prompt Learning for Biomedical Entity Linking","authors":[{"name":"Tiantian Zhu","slug":"tiantian-zhu","given":"Tiantian","family":"Zhu","affiliation":"Harbin Institute of Technology (Shenzhen) , Shenzhen , China; Peng Cheng Laboratory , Shenzhen , China"},{"name":"Yang Qin","slug":"yang-qin","given":"Yang","family":"Qin","affiliation":"Harbin Institute of Technology (Shenzhen) , Shenzhen , China"},{"name":"Qingcai Chen","slug":"qingcai-chen","given":"Qingcai","family":"Chen","affiliation":"Harbin Institute of Technology (Shenzhen) , Shenzhen , China; Peng Cheng Laboratory , Shenzhen , China"},{"name":"Baotian Hu","slug":"baotian-hu","given":"Baotian","family":"Hu","affiliation":"Harbin Institute of Technology (Shenzhen) , Shenzhen , China"},{"name":"Yang Xiang","slug":"yang-xiang","given":"Yang","family":"Xiang","affiliation":"Peng Cheng Laboratory , Shenzhen , China"}],"abstract":"Biomedical entity linking aims to map mentions in biomedical text to standardized concepts or entities in a curated knowledge base (KB) such as Unified Medical Language System (UMLS). The latest research tends to solve this problem in a unified framework solely based on surface form matching between mentions and entities. Specifically, these methods focus on addressing the variety challenge of the heterogeneous naming of biomedical concepts. Yet, the ambiguity challenge that the same word under different contexts may refer to distinct entities is usually ignored. To address this challenge, we propose a two-stage linking algorithm to enhance the entity representations based on prompt learning. The first stage includes a coarser-grained retrieval from a representation space defined by a bi-encoder that independently embeds the mention and entity’s surface forms. Unlike previous one-model-fits-all systems, each candidate is then re-ranked with a finer-grained encoder based on prompt-tuning that utilizes the contextual information. Extensive experiments show that our model achieves promising performance improvements compared with several state-of-the-art techniques on the largest biomedical public dataset MedMentions and the NCBI disease corpus. We also observe by cases that the proposed prompt-tuning strategy is effective in solving both the variety and ambiguity challenges in the linking task.","doi":"10.24963/ijcai.2022/560","url":"https://ijcai.org/proceedings/2022/560","pdf":"https://ijcai.org/proceedings/2022/0560.pdf","markdown":"https://ijcai.org/proceedings/2022/560.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/560","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":560,"pages":{"first":4036,"last":4042},"keywords":["Multidisciplinary Topics and Applications: Bioinformatics","Natural Language Processing: Applications","Multidisciplinary Topics and Applications: Health and Medicine"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986691","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984834","length_sec":345,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Brown et al., 2020] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Win- ter, Christopher Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few- shot learners. In Proc. of NeurIPS, pages 1877-1901, 2020.","title":"Language models are fewshot learners","authors":"B Tom, Benjamin Brown, Nick Mann, Melanie Ryder, Jared Subbiah, Prafulla Kaplan, Arvind Dhariwal, Pranav Neelakantan, Girish Shyam, Amanda Sastry, Sandhini Askell, Ariel Agarwal, Gretchen Herbert-Voss, Tom Krueger, Rewon Henighan, Aditya Child, Daniel M Ramesh, Jeffrey Ziegler, Clemens Wu, Christopher Winter, Mark Hesse, Eric Chen, Mateusz Sigler, Scott Litwin, Benjamin Gray, Jack Chess, Christopher Clark, Sam Berner, Alec Mccandlish, Ilya Radford, Dario Sutskever, Amodei","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Davis et al., 2012] Allan Peter Davis, Thomas C. Wiegers, Michael C. Rosenstein, and Carolyn J. Mattingly. MEDIC: a practical disease vocabulary used at the comparative tox- icogenomics database. Database J. Biol. Databases Cura- tion, 2012, 2012.","title":"MEDIC: a practical disease vocabulary used at the comparative toxicogenomics database","authors":"Allan Peter Davis, Thomas C Wiegers, Michael C Rosenstein, Carolyn J Mattingly","year":"2012","venue":"Database J. Biol. Databases Curation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Davison et al., 2019] Joe Davison, Joshua Feldman, and Alexander M. Rush. Commonsense knowledge mining from pretrained models. In Proc. of EMNLP-IJCNLP, pages 1173-1178, 2019.","title":"Commonsense knowledge mining from pretrained models","authors":"Joe Davison, Joshua Feldman, Alexander M Rush","year":"2019","venue":"Proc. of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In Proc. of NAACL-HLT, pages 4171-4186, 2019.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dogan et al., 2014] Rezarta Islamaj Dogan, Robert Lea- man, and Zhiyong Lu. NCBI disease corpus: A resource for disease name recognition and concept normalization. J. Biomed. Informatics, 47:1-10, 2014.","title":"Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. NCBI disease corpus: A resource for disease name recognition and concept normalization","authors":"Dogan","year":"2014","venue":"J. Biomed. Informatics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[D'Souza and Ng, 2015] Jennifer D'Souza and Vincent Ng. Sieve-based entity linking for the biomedical domain. In Proc. of ACL, pages 297-302, 2015.","title":"Sieve-based entity linking for the biomedical domain","authors":"D'souza And Ng ; Jennifer, D' Souza, Vincent Ng","year":"2015","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2021] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few- shot learners. In Proc. of ACL/IJCNLP, pages 3816-3830, 2021.","title":"Making pre-trained language models better fewshot learners","authors":"Tianyu Gao, Adam Fisch, Danqi Chen","year":"2021","venue":"Proc. of ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ji et al., 2020] Zongcheng Ji, Qiang Wei, and Hua Xu. Bert- based ranking for biomedical entity normalization. AMIA Jt Summits Transl Sci Proc., 2020:269-277, 2020.","title":"Bertbased ranking for biomedical entity normalization","authors":"Ji","year":"2020","venue":"AMIA Jt Summits Transl Sci Proc","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2020] Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How can we know what language models know. Trans. Assoc. Comput. Linguistics, 8:423- 438, 2020.","title":"How can we know what language models know","authors":"Zhengbao Jiang, Frank F Xu, Jun Araki, Graham Neubig","year":"2020","venue":"Trans. Assoc. Comput. Linguistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Leaman and Lu, 2016] Robert Leaman and Zhiyong Lu. Taggerone: joint named entity recognition and normaliza- tion with semi-markov models. Bioinform., 32(18):2839- 2846, 2016.","title":"Taggerone: joint named entity recognition and normalization with semi-markov models","authors":"Lu ; Leaman, Robert Leaman, Zhiyong Lu","year":"2016","venue":"Bioinform","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2020] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioin- form., 36(4):1234-1240, 2020.","title":"Biobert: a pre-trained biomedical language representation model for biomedical text mining","authors":"Lee","year":"2020","venue":"Bioinform","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation. In Proc. of ACL/IJCNLP, pages 4582-4597, 2021.","title":"Prefixtuning: Optimizing continuous prompts for generation","authors":"Lisa Xiang, Percy Li, Liang","year":"2021","venue":"Proc. of ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2021] Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. Self- alignment pretraining for biomedical entity representa- tions. In Proc. of NAACL, pages 4228-4238, 2021.","title":"Selfalignment pretraining for biomedical entity representations","authors":"Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier","year":"2021","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luca and Goharian, 2016] Soldaini Luca and Nazli Gohar- ian. Quickumls: a fast, unsupervised approach for medical concept extraction. In MedIR workshop, sigir, pages 1-4, 2016.","title":"Soldaini Luca and Nazli Goharian. Quickumls: a fast, unsupervised approach for medical concept extraction","authors":"Goharian Luca","year":"2016","venue":"MedIR workshop, sigir","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mohan and Li, 2019] Sunil Mohan and Donghui Li. Med- mentions: A large biomedical corpus annotated with UMLS concepts. In Proc. of AKBC, 2019.","title":"Medmentions: A large biomedical corpus annotated with UMLS concepts","authors":"Sunil Mohan, Donghui Li","year":"2019","venue":"Proc. of AKBC","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Niu et al., 2019] Jinghao Niu, Yehui Yang, Siheng Zhang, Zhengya Sun, and Wensheng Zhang. Multi-task character- level attentional networks for medical concept normaliza- tion. Neural Process. Lett., 49(3):1239-1256, 2019.","title":"Multi-task characterlevel attentional networks for medical concept normalization","authors":"Niu","year":"2019","venue":"Neural Process. Lett","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Phan et al., 2019] Minh C. Phan, Aixin Sun, and Yi Tay. Robust representation learning of biomedical names. In Proc. of ACL, pages 3275-3285, 2019.","title":"Robust representation learning of biomedical names","authors":"Minh C Phan, Aixin Sun, Yi Tay","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schick and Schütze, 2021] Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proc. of EACL, pages 255-269, 2021.","title":"Exploiting cloze-questions for few-shot text classification and natural language inference","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"Proc. of EACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sung et al., 2020] Mujeen Sung, Hwisang Jeon, Jinhyuk Lee, and Jaewoo Kang. Biomedical entity representations with synonym marginalization. In Proc. of ACL, pages 3641-3650, 2020.","title":"Biomedical entity representations with synonym marginalization","authors":"Sung","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wright et al., 2019] Dustin Wright, Yannis Katsis, Raghav Mehta, and Chun-Nan Hsu. Normco: Deep disease nor- malization for biomedical knowledge base construction. In Proc. of AKBC, 2019.","title":"Normco: Deep disease normalization for biomedical knowledge base construction","authors":"Dustin Wright, Yannis Katsis, Raghav Mehta, Chun-Nan Hsu","year":"2019","venue":"Proc. of AKBC","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2020] Ledell Wu, Fabio Petroni, Martin Josi- foski, Sebastian Riedel, and Luke Zettlemoyer. Scalable zero-shot entity linking with dense entity retrieval. In Proc. of EMNLP, pages 6397-6407, 2020.","title":"Scalable zero-shot entity linking with dense entity retrieval","authors":"Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, Luke Zettlemoyer","year":"2020","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhu et al., 2020] Ming Zhu, Busra Celikkaya, Parminder Bhatia, and Chandan K. Reddy. LATTE: latent type mod- eling for biomedical entity linking. In Proc. of AAAI, pages 9757-9764, 2020.","title":"LATTE: latent type modeling for biomedical entity linking","authors":"Ming Zhu, Busra Celikkaya, Parminder Bhatia, Chandan K Reddy","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}