{"type":"ScholarlyArticle","title":"Low-Resource NER by Data Augmentation With Prompting","authors":[{"name":"Jian Liu","slug":"jian-liu","given":"Jian","family":"Liu","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University , School of Computer and Information Technology , China"},{"name":"Yufeng Chen","slug":"yufeng-chen","given":"Yufeng","family":"Chen","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University , School of Computer and Information Technology , China"},{"name":"Jinan Xu","slug":"jinan-xu","given":"Jinan","family":"Xu","affiliation":"Beijing Key Lab of Traffic Data Analysis and Mining Beijing Jiaotong University , School of Computer and Information Technology , China"}],"abstract":"Named entity recognition (NER) is a fundamental information extraction task that seeks to identify entity mentions of certain types in text.\r\nDespite numerous advances, the existing NER methods rely on extensive supervision for model training, which struggle in a low-resource scenario with limited training data.\r\nIn this paper, we propose a new data augmentation method for low-resource NER, by eliciting knowledge from BERT with prompting strategies.\r\nParticularly, we devise a label-conditioned word replacement strategy that can produce more label-consistent examples by capturing the underlying word-label dependencies, and a prompting with question answering method to generate new training data from unlabeled texts. \r\nThe experimental results have widely confirmed the effectiveness of our approach.\r\nParticularly, in a low-resource scenario with only 150 training sentences, our approach outperforms previous methods without data augmentation by over 40% in F1 and prior best data augmentation methods by over 2.0% in F1. Furthermore, our approach also fits with a zero-shot scenario, yielding promising results without using any human-labeled data for the task.","doi":"10.24963/ijcai.2022/590","url":"https://ijcai.org/proceedings/2022/590","pdf":"https://ijcai.org/proceedings/2022/0590.pdf","markdown":"https://ijcai.org/proceedings/2022/590.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/590","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":590,"pages":{"first":4252,"last":4258},"keywords":["Natural Language Processing: Information Extraction","Natural Language Processing: Knowledge Extraction"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987004","length_sec":55,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985147","length_sec":453,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Chen et al., 2020] Hannah Chen, Yangfeng Ji, and David Evans. Finding Friends and flipping frenemies: Automatic paraphrase dataset augmentation using graph theory. In Findings of EMNLP, pages 4741-4751, 2020.","title":"Finding Friends and flipping frenemies: Automatic paraphrase dataset augmentation using graph theory","authors":"Hannah Chen, Yangfeng Ji, David Evans","year":"2020","venue":"Findings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dai and Adel, 2020] Xiang Dai and Heike Adel. An analy- sis of simple data augmentation for named entity recogni- tion. In COLING, pages 3861-3867, 2020.","title":"An analysis of simple data augmentation for named entity recognition","authors":"Xiang Dai, Heike Adel","year":"2020","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL, pages 4171-4186, 2019.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ding et al., 2020] Bosheng Ding, Linlin Liu, Lidong Bing, Canasai Kruengkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si, and Chunyan Miao. DAGA: Data augmentation with a generation approach for low-resource tagging tasks. In EMNLP, pages 6045-6057, 2020.","title":"DAGA: Data augmentation with a generation approach for low-resource tagging tasks","authors":"Ding","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gal and Ghahramani, 2016] Yarin Gal and Zoubin Ghahra- mani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. PMLR, pages 1050- 1059, 2016.","title":"Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning","authors":"Ghahramani Gal","year":"2016","venue":"PMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Grishman and Sundheim, 1996] Ralph Grishman and Beth Sundheim. Message Understanding Conference-6: A brief history. In COLING, pages 1-20, 1996.","title":"Message Understanding Conference-6: A brief history","authors":"Sundheim Grishman, Ralph Grishman, Beth Sundheim","year":"1996","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., page 1735-1780, 1997.","title":"Long short-term memory","authors":"Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Comput","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hou et al., 2020] Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, and Ting Liu. Few-shot slot tagging with collapsed dependency transfer and label- enhanced task-adaptive projection network. In ACL, pages 1381-1393, 2020.","title":"Few-shot slot tagging with collapsed dependency transfer and labelenhanced task-adaptive projection network","authors":"Yutai Hou, Wanxiang Che, Yongkui Lai, Zhihan Zhou, Yijia Liu, Han Liu, Ting Liu","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hovy et al., 2006] Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. OntoNotes: The 90% solution. In NAACL, pages 57-60, 2006.","title":"OntoNotes: The 90% solution","authors":"Hovy","year":"2006","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2020] Zhengbao Jiang, Antonios Anastasopou- los, Jun Araki, Haibo Ding, and Graham Neubig. X- FACTR: Multilingual factual knowledge retrieval from pretrained language models. In EMNLP, pages 5943- 5959, 2020.","title":"X-FACTR: Multilingual factual knowledge retrieval from pretrained language models","authors":"Jiang","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kumar et al., 2020] Varun Kumar, Ashutosh Choudhary, Eunahothers Cho, et al. Data augmentation using pre- trained transformer models. In Life-long Learning for Spo- ken Language Systems, pages 18-26, 2020.","title":"Data augmentation using pretrained transformer models","authors":"Varun Kumar, Ashutosh Choudhary, Eunahothers Cho","year":"2020","venue":"Life-long Learning for Spoken Language Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lafferty et al., 2001] John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, page 282-289, 2001.","title":"Conditional random fields: Probabilistic models for segmenting and labeling sequence data","authors":"John D Lafferty, Andrew Mccallum, Fernando C N Pereira","year":"2001","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lample et al., 2016] Guillaume Lample, Miguel Balles- teros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recog- nition. In NAACL, pages 260-270, 2016.","title":"Neural architectures for named entity recognition","authors":"Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer","year":"2016","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2021] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. In CoRR, volume abs/2107.13586, 2021.","title":"Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing","authors":"Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig","year":"2021","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mayhew et al., 2017] Stephen Mayhew, Chen-Tse Tsai, and Dan Roth. Cheap translation for cross-lingual named en- tity recognition. In EMNLP, pages 2536-2545, 2017.","title":"Cheap translation for cross-lingual named entity recognition","authors":"Stephen Mayhew, Chen-Tse Tsai, Dan Roth","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mysore et al., 2019] Sheshera Mysore, Zachary Jensen, Ed- ward Kim, Kevin Huang, Haw-Shiuan Chang, Emma Strubell, Jeffrey Flanigan, Andrew McCallum, and Elsa Olivetti. The materials science procedural text corpus: An- notating materials synthesis procedures with shallow se- mantic structures. In LA, pages 56-64, 2019.","title":"The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures","authors":"Mysore","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"GloVe: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Plank and Agić, 2018] Barbara Plank and Željko Agić. Dis- tant supervision from disparate sources for low-resource part-of-speech tagging. In EMNLP, pages 614-620, 2018.","title":"Distant supervision from disparate sources for low-resource part-of-speech tagging","authors":"Agić Plank, Barbara Plank, Željko Agić","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rajpurkar et al., 2018] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for SQuAD. In ACL, pages 784-789, 2018.","title":"Know what you don't know: Unanswerable questions for SQuAD","authors":"Pranav Rajpurkar, Robin Jia, Percy Liang","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schick and Schütze, 2021] Timo Schick and Hinrich Schütze. Exploiting cloze-questions for few-shot text classification and natural language inference. In EACL, pages 255-269, 2021.","title":"Exploiting cloze-questions for few-shot text classification and natural language inference","authors":"Schütze Schick, Timo Schick, Hinrich Schütze","year":"2021","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Scudder, 1965] H. Scudder. Probability of error of some adaptive pattern-recognition machines. IEEE Transactions on Information Theory, pages 363-371, 1965.","title":"Probability of error of some adaptive pattern-recognition machines","authors":"H Scudder","year":"1965","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tjong Kim Sang and De Meulder, 2003] Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In NAACL, pages 142-147, 2003.","title":"Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition","authors":"Tjong Kim, Sang, De Meulder, ; ] Erik, F Tjong, Kim Sang, Fien De, Meulder","year":"2003","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wei and Zou, 2019] Jason Wei and Kai Zou. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In EMNLP, pages 6382-6388, 2019.","title":"EDA: Easy data augmentation techniques for boosting performance on text classification tasks","authors":"Zou ; ] Wei, Jason Wei, Kai Zou","year":"2019","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2018] Jiateng Xie, Zhilin Yang, Graham Neu- big, Noah A. Smith, and Jaime Carbonell. Neural cross- lingual named entity recognition with minimal resources. In EMNLP, pages 369-379, 2018.","title":"Neural crosslingual named entity recognition with minimal resources","authors":"Xie","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2019] Qizhe Xie, Eduard Hovy, Minh-Thang Lu- ong, and Quoc V. Le. Self-training with noisy student im- proves imagenet classification. In CVPR, 2019.","title":"Self-training with noisy student improves imagenet classification","authors":"Xie","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2019] Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, and Ming Zhou. BERT-based lexical substi- tution. In ACL, pages 3368-3373, 2019.","title":"BERT-based lexical substitution","authors":"Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2021] Jing Zhou, Yanan Zheng, Jie Tang, Jian Li, and Zhilin Yang. Flipda: Effective and robust data aug- mentation for few-shot learning. abs/2108.06332, 2021.","title":"Flipda: Effective and robust data augmentation for few-shot learning","authors":"Jing Zhou, Yanan Zheng, Jie Tang, Jian Li, Zhilin Yang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}