{"type":"ScholarlyArticle","title":"TaxoPrompt: A Prompt-based Generation Method with Taxonomic Context for Self-Supervised Taxonomy Expansion","authors":[{"name":"Hongyuan Xu","slug":"hongyuan-xu","given":"Hongyuan","family":"Xu","affiliation":"College of Computer Science , Nankai University; Tianjin Media Computing Center , Nankai University"},{"name":"Yunong Chen","slug":"yunong-chen","given":"Yunong","family":"Chen","affiliation":"College of Computer Science , Nankai University; Tianjin Media Computing Center , Nankai University"},{"name":"Zichen Liu","slug":"zichen-liu","given":"Zichen","family":"Liu","affiliation":"College of Computer Science , Nankai University; Tianjin Media Computing Center , Nankai University"},{"name":"Yanlong Wen","slug":"yanlong-wen","given":"Yanlong","family":"Wen","affiliation":"College of Computer Science , Nankai University; Tianjin Media Computing Center , Nankai University"},{"name":"Xiaojie Yuan","slug":"xiaojie-yuan","given":"Xiaojie","family":"Yuan","affiliation":"College of Computer Science , Nankai University; Tianjin Media Computing Center , Nankai University"}],"abstract":"Taxonomies are hierarchical classifications widely exploited to facilitate downstream natural language processing tasks. The taxonomy expansion task aims to incorporate emergent concepts into the existing taxonomies. Prior works focus on modeling the local substructure of taxonomies but neglect the global structure. In this paper, we propose TaxoPrompt, a framework that learns the global structure by prompt tuning with taxonomic context. Prompt tuning leverages a template to formulate downstream tasks into masked language model form for better distributed semantic knowledge use. To further infuse global structure knowledge into language models, we enhance the prompt template by exploiting the taxonomic context constructed by a variant of the random walk algorithm. Experiments on seven public benchmarks show that our proposed TaxoPrompt is effective and efficient in automatically expanding taxonomies and achieves state-of-the-art performance.","doi":"10.24963/ijcai.2022/615","url":"https://ijcai.org/proceedings/2022/615","pdf":"https://ijcai.org/proceedings/2022/0615.pdf","markdown":"https://ijcai.org/proceedings/2022/615.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/615","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":615,"pages":{"first":4432,"last":4438},"keywords":["Natural Language Processing: Natural Language Semantics","Natural Language Processing: Knowledge Extraction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986744","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984887","length_sec":349,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bordea et al., 2016] Georgeta Bordea, Els Lefever, and Paul Buite- laar. Semeval-2016 task 13: Taxonomy extraction evaluation (texeval-2). In SemEval-2016, pages 1081-1091, 2016.","title":"Semeval-2016 task 13: Taxonomy extraction evaluation (texeval-2)","authors":"Georgeta Bordea, Els Lefever, Paul Buitelaar","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Fenxiao Chen, Yun-Cheng Wang, Bin Wang, and C-C Jay Kuo. Graph representation learning: a survey. AP- SIPA, 9(1):1-21, 2020.","title":"Graph representation learning: a survey","authors":"Fenxiao Chen, Yun-Cheng Wang, Bin Wang, C-C Jay Kuo","year":"2020","venue":"AP-SIPA","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Clark et al., 2020] Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: pre-training text en- coders as discriminators rather than generators. In ICLR, pages 1-18, 2020.","title":"ELECTRA: pre-training text encoders as discriminators rather than generators","authors":"Kevin Clark, Minh-Thang Luong, Quoc V Le, Christopher D Manning","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In NAACL, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2020] Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, and James Caverlee. Infusing disease knowledge into BERT for health question answering, medical inference and disease name recognition. In EMNLP, pages 4604-4614, 2020.","title":"Infusing disease knowledge into BERT for health question answering, medical inference and disease name recognition","authors":"Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, James Caverlee","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2019] Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In WSDM, pages 573-581, 2019.","title":"Taxonomy-aware multi-hop reasoning networks for sequential recommendation","authors":"Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, Daxiang Dong","year":"2019","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang et al., 2020] Zhengbao Jiang, Antonios Anastasopoulos, Jun Araki, Haibo Ding, and Graham Neubig. X-FACTR: multilingual factual knowledge retrieval from pretrained language models. In EMNLP, pages 5943-5959, 2020.","title":"X-FACTR: multilingual factual knowledge retrieval from pretrained language models","authors":"Jiang","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jurgens and Pilehvar, 2016] David Jurgens and Mohammad Taher Pilehvar. Semeval-2016 task 14: Semantic taxonomy enrichment. In SemEval-2016, pages 1092-1102, 2016.","title":"Semeval-2016 task 14: Semantic taxonomy enrichment","authors":"David Jurgens, Mohammad Taher, Pilehvar","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karamanolakis et al., 2020] Giannis Karamanolakis, Jun Ma, and Xin Luna Dong. Txtract: Taxonomy-aware knowledge extraction for thousands of product categories. In ACL, pages 8489-8502, 2020.","title":"Txtract: Taxonomy-aware knowledge extraction for thousands of product categories","authors":"Giannis Karamanolakis, Jun Ma, Xin Luna, Dong","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A ro- bustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2021a] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586, 2021.","title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","authors":"Liu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2021b] Zichen Liu, Hongyuan Xu, Yanlong Wen, Ning Jiang, Haiying Wu, and Xiaojie Yuan. TEMP: taxonomy ex- pansion with dynamic margin loss through taxonomy-paths. In EMNLP, pages 3854-3863, 2021.","title":"TEMP: taxonomy expansion with dynamic margin loss through taxonomy-paths","authors":"Liu","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, pages 1-18, 2019.","title":"Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization","authors":"Hutter Loshchilov","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2021] Mingyu Derek Ma, Muhao Chen, Te-Lin Wu, and Nanyun Peng. Hyperexpan: Taxonomy expansion with hyper- bolic representation learning. In EMNLP, pages 4182-4194, 2021.","title":"Hyperexpan: Taxonomy expansion with hyperbolic representation learning","authors":"Derek Mingyu, Muhao Ma, Te-Lin Chen, Nanyun Wu, Peng","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Manzoor et al., 2020] Emaad Manzoor, Rui Li, Dhananjay Shrouty, and Jure Leskovec. Expanding taxonomies with implicit edge semantics. In WWW, pages 2044-2054, 2020.","title":"Expanding taxonomies with implicit edge semantics","authors":"Manzoor","year":"2020","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nickel and Kiela, 2017] Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In NeurIPS, pages 6338-6347, 2017.","title":"Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sanh et al., 2019] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.","title":"Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter","authors":"Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schick and Schütze, 2021] Timo Schick and Hinrich Schütze. Ex- ploiting cloze-questions for few-shot text classification and natu- ral language inference. In EACL, pages 255-269, 2021.","title":"Exploiting cloze-questions for few-shot text classification and natural language inference","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schuster and Nakajima, 2012] Mike Schuster and Kaisuke Naka- jima. Japanese and korean voice search. In ICASSP, pages 5149- 5152, 2012.","title":"Japanese and korean voice search","authors":"Mike Schuster, Kaisuke Nakajima","year":"2012","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen et al., 2018] Jiaming Shen, Zeqiu Wu, Dongming Lei, Chao Zhang, Xiang Ren, Michelle T Vanni, Brian M Sadler, and Jiawei Han. Hiexpan: Task-guided taxonomy construction by hierarchi- cal tree expansion. In SIGKDD, pages 2180-2189, 2018.","title":"Hiexpan: Task-guided taxonomy construction by hierarchical tree expansion","authors":"Jiaming Shen, Zeqiu Wu, Dongming Lei, Chao Zhang, Xiang Ren, Michelle T Vanni, Brian M Sadler, Jiawei Han","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2020] Jiaming Shen, Zhihong Shen, Chenyan Xiong, Chi Wang, Kuansan Wang, and Jiawei Han. Taxoexpan: Self- supervised taxonomy expansion with position-enhanced graph neural network. In WWW, pages 486-497, 2020.","title":"Taxoexpan: Selfsupervised taxonomy expansion with position-enhanced graph neural network","authors":"Jiaming Shen, Zhihong Shen, Chenyan Xiong, Chi Wang, Kuansan Wang, Jiawei Han","year":"2020","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sinha et al., 2015] Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June Hsu, and Kuansan Wang. An overview of microsoft academic service (MAS) and applications. In WWW, pages 243-246, 2015.","title":"An overview of microsoft academic service (MAS) and applications","authors":"Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June Hsu, Kuansan Wang","year":"2015","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2021] Xiangchen Song, Jiaming Shen, Jieyu Zhang, and Jiawei Han. Who should go first? a self-supervised concept sorting model for improving taxonomy expansion. arXiv preprint arXiv:2104.03682, 2021.","title":"Who should go first? a self-supervised concept sorting model for improving taxonomy expansion","authors":"Song","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Takeoka et al., 2021] Kunihiro Takeoka, Kosuke Akimoto, and Masafumi Oyamada. Low-resource taxonomy enrichment with pretrained language models. In EMNLP, pages 2747-2758, 2021.","title":"Low-resource taxonomy enrichment with pretrained language models","authors":"Kunihiro Takeoka, Kosuke Akimoto, Masafumi Oyamada","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2021] Suyuchen Wang, Ruihui Zhao, Xi Chen, Yefeng Zheng, and Bang Liu. Enquire one's parent and child before decision: Fully exploit hierarchical structure for self- supervised taxonomy expansion. In WWW, pages 3291-3304, 2021.","title":"Enquire one's parent and child before decision: Fully exploit hierarchical structure for selfsupervised taxonomy expansion","authors":"Wang","year":"2021","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2017] Shuo Yang, Lei Zou, Zhongyuan Wang, Jun Yan, and Ji-Rong Wen. Efficiently answering technical ques- tions-a knowledge graph approach. In AAAI, pages 3111-3118, 2017.","title":"Efficiently answering technical questions-a knowledge graph approach","authors":"Shuo Yang, Lei Zou, Zhongyuan Wang, Jun Yan, Ji-Rong Wen","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2020] Yue Yu, Yinghao Li, Jiaming Shen, Hao Feng, Ji- meng Sun, and Chao Zhang. STEAM: self-supervised taxonomy expansion with mini-paths. In KDD, pages 1026-1035, 2020.","title":"STEAM: self-supervised taxonomy expansion with mini-paths","authors":"Yue Yu, Yinghao Li, Jiaming Shen, Hao Feng, Jimeng Sun, Chao Zhang","year":"2020","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zeng et al., 2021] Qingkai Zeng, Jinfeng Lin, Wenhao Yu, Jane Cleland-Huang, and Meng Jiang. Enhancing taxonomy comple- tion with concept generation via fusing relational representations. In KDD, pages 2104-2113, 2021.","title":"Enhancing taxonomy completion with concept generation via fusing relational representations","authors":"Qingkai Zeng, Jinfeng Lin, Wenhao Yu, Jane Cleland-Huang, Meng Jiang","year":"2021","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2021] Jieyu Zhang, Xiangchen Song, Ying Zeng, Ji- aze Chen, Jiaming Shen, Yuning Mao, and Lei Li. Taxonomy completion via triplet matching network. In AAAI, pages 4662- 4670, 2021.","title":"Taxonomy completion via triplet matching network","authors":"Zhang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}