{"type":"ScholarlyArticle","title":"Function-words Adaptively Enhanced Attention Networks for Few-Shot Inverse Relation Classification","authors":[{"name":"Chunliu Dou","slug":"chunliu-dou","given":"Chunliu","family":"Dou","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China, 300350"},{"name":"Shaojuan Wu","slug":"shaojuan-wu","given":"Shaojuan","family":"Wu","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China, 300350"},{"name":"Xiaowang Zhang","slug":"xiaowang-zhang","given":"Xiaowang","family":"Zhang","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China, 300350; Tianjin University-Aishu Data Intelligence Joint Laboratory , Tianjin , China"},{"name":"Zhiyong Feng","slug":"zhiyong-feng","given":"Zhiyong","family":"Feng","affiliation":"College of Intelligence and Computing , Tianjin University , Tianjin , China, 300350"},{"name":"Kewen Wang","slug":"kewen-wang","given":"Kewen","family":"Wang","affiliation":"School of Information and Communication Technology , Griffith University , Brisbane , Australia"}],"abstract":"The relation classification is to identify semantic relations between two entities in a given text. While existing models perform well for classifying inverse relations with large datasets, their performance is significantly reduced for few-shot learning. In this paper, we propose a function words adaptively enhanced attention framework (FAEA) for few-shot inverse relation classification, in which a hybrid attention model is designed to attend class-related function words based on meta-learning. As the involvement of function words brings in significant intra-class redundancy, an adaptive message passing mechanism is introduced to capture and transfer inter-class differences.We mathematically analyze the negative impact of function words from dot-product measurement, which explains why the message passing mechanism effectively reduces the impact. Our experimental results show that FAEA outperforms strong baselines, especially the inverse relation accuracy is improved by 14.33% under 1-shot setting in FewRel1.0.","doi":"10.24963/ijcai.2022/407","url":"https://ijcai.org/proceedings/2022/407","pdf":"https://ijcai.org/proceedings/2022/0407.pdf","markdown":"https://ijcai.org/proceedings/2022/407.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/407","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":407,"pages":{"first":2937,"last":2943},"keywords":["Natural Language Processing: Text Classification","Machine Learning: Few-shot learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986901","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985044","length_sec":354,"provider":"slideslive"}],"references":[{"position":1,"raw":"Encoder Model 5-way-1-shot 5-way-5-shot 10-way-1-shot 10-way-5-shot CNN Proto-CNN [Snell et al., 2017] ▷ 72.65 / 74.52 86.15 / 88.40 60.13 / 62.38 76.20 / 80.45","title":"Encoder Model 5-way-1-shot 5-way-5-shot 10-way-1-shot 10-way-5-shot CNN Proto-CNN","authors":"Snell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Proto-HATT [Gao et al., 2019a] 75.01 / -- 87.09 / 90.12 62.48 / -- 77.50 / 83.05 MLMAN [Ye and Ling, 2019] ⋄ 78.85 / 82.98 88.32 / 92.66 67.54 / 73.59 79.44 / 87.29","title":null,"authors":"Proto-Hatt [ Gao","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Bert Proto-Bert [Snell et al., 2017] • 82.92 / 80.68 91.32 / 89.60 73.24 / 71.48 83.68 / 82.89 MAML [Finn et al., 2017] • 82.93 / 89.70 86.21 / 93.55 73.20 / 83.17 76.06 / 88.51 GNN [Satorras and Estrach, 2018] 74.21 / 75.66 86.16 / 89.06 67.98 / 70.08 73.65 / 76.93","title":"Satorras and Estrach","authors":"Bert Proto-Bert, [ Snell","year":"2017","venue":"Satorras and Estrach","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"BERT-PAIR [Gao et al., 2019b] ▷ 85.66 / 88.32 89.48 / 93.22 76.84 / 80.63 81.76 / 87.02 REGRAB [Qu et al., 2020] • 87.93 / 90.30 92.58 / 94.25 80.52 / 84.09 87.02 / 89.93","title":null,"authors":"Bert-Pair [ Gao","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"TD-Proto [Sun et al., 2019] 83.43 / 84.53 90.26 / 92.38 72.45 / 74.32 82.10 / 85.19 ConceptFERE [Yang et al., 2021] 87.21 / 89.21 90.53 / 93.98","title":null,"authors":"Td-Proto [sun","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Abiola and Andreas, 2019] Obamuyide Abiola and Vlachos Andreas. Model-agnostic meta-learning for relation clas- sification with limited supervision. In Proc. of ACL, pages 5873-5879, 2019.","title":"Obamuyide Abiola and Vlachos Andreas. Model-agnostic meta-learning for relation classification with limited supervision","authors":"Andreas Abiola","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bao et al., 2020] Yujia Bao, Menghua Wu, Shiyu Chang, and Regina Barzilay. Few-shot text classification with dis- tributional signatures. In Proc. of ICLR, 2020.","title":"Few-shot text classification with distributional signatures","authors":"Yujia Bao, Menghua Wu, Shiyu Chang, Regina Barzilay","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bo et al., 2021] Deyu Bo, Xiao Wang, Chuan Shi, and Huawei Shen. Beyond low-frequency information in graph convolutional networks. In Proc. of AAAI, pages 3950- 3957, 2021.","title":"Beyond low-frequency information in graph convolutional networks","authors":"Deyu Bo, Xiao Wang, Chuan Shi, Huawei Shen","year":"2021","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Devlin et al., 2019] Jacob Devlin, Mingwei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Mingwei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proc. of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dong et al., 2020] Hang Dong, Wei Wang, Frans Coenen, and Kaizhu Huang. Knowledge base enrichment by relation learning from social tagging data. Inf. Sci., 526(4):203-220, 2020.","title":"Knowledge base enrichment by relation learning from social tagging data","authors":"Dong","year":"2020","venue":"Inf. Sci","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. of ICML, pages 1126-1135, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2019a] Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. Hybrid attention-based prototypical net- works for noisy few-shot relation classification. In Proc. of AAAI, pages 6407-6414, 2019.","title":"Hybrid attention-based prototypical networks for noisy few-shot relation classification","authors":"Gao","year":"2019","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gao et al., 2019b] Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Fewrel 2.0: Towards more challenging few-shot relation classification. In Proc. of EMNLP-IJCNLP, pages 6249-6254, 2019.","title":"Fewrel 2.0: Towards more challenging few-shot relation classification","authors":"Gao","year":"2019","venue":"Proc. of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Han et al., 2018] Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proc. of EMNLP, pages 4803-4809, 2018.","title":"Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation","authors":"Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, Maosong Sun","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee, 2021] Yohan Lee. Improving end-to-end task-oriented dialog system with a simple auxiliary task. In Proc. of EMNLP, pages 1296-1303, 2021.","title":"Improving end-to-end task-oriented dialog system with a simple auxiliary task","authors":"Yohan Lee","year":"2021","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2020] Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Learning from context or names? An empirical study on neural relation extraction. In Proc. of EMNLP, pages 3661-3672, 2020.","title":"Learning from context or names? An empirical study on neural relation extraction","authors":"Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou","year":"2020","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qu et al., 2020] Meng Qu, Tianyu Gao, Louis Pascal A. C. Xhonneux, and Jian Tang. Few-shot relation extraction via bayesian meta-learning on relation graphs. In Proc. of ICML, pages 7867-7876, 2020.","title":"Few-shot relation extraction via bayesian meta-learning on relation graphs","authors":"Meng Qu, Tianyu Gao, Louis Pascal, A C Xhonneux, Jian Tang","year":"2020","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Satorras and Estrach, 2018] Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In Proc. of ICLR, 2018.","title":"Few-shot learning with graph neural networks","authors":"Estrach ; Satorras, Garcia Victor, Joan Satorras, Estrach Bruna","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few- shot learning. In Proc. of NeurIPS, pages 4077-4087, 2017.","title":"Prototypical networks for fewshot learning","authors":"Jake Snell, Kevin Swersky, Richard S Zemel","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Soares et al., 2019] Livio Baldini Soares, Nicholas FitzGer- ald, Jeffrey Ling, and Tom Kwiatkowski. Matching the blanks: Distributional similarity for relation learning. In Proc. of ACL, pages 2895-2905, 2019.","title":"Matching the blanks: Distributional similarity for relation learning","authors":"Baldini Livio, Nicholas Soares, Jeffrey Fitzgerald, Tom Ling, Kwiatkowski","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2019] Shengli Sun, Qingfeng Sun, Kevin Zhou, and Tengchao Lv. Hierarchical attention prototypical net- works for few-shot text classification. In Proc. of EMNLP- IJCNLP, pages 476-485, 2019.","title":"Hierarchical attention prototypical networks for few-shot text classification","authors":"Shengli Sun, Qingfeng Sun, Kevin Zhou, Tengchao Lv","year":"2019","venue":"Proc. of EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2020] Yuxia Wang, Karin Verspoor, and Tim- othy Baldwin. Learning from unlabelled data for clinical semantic textual similarity. In Proc. of ClinicalNLP, pages 227-233, 2020.","title":"Learning from unlabelled data for clinical semantic textual similarity","authors":"Yuxia Wang, Karin Verspoor, Timothy Baldwin","year":"2020","venue":"Proc. of ClinicalNLP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wen et al., 2021] Wen Wen, Yongbin Liu, Chunping Ouyang, Qiang Lin, and Tonglee Chung. Enhanced proto- typical network for few-shot relation extraction. Inf. Pro- cess. Manag., 58(4):102596, 2021.","title":"Enhanced prototypical network for few-shot relation extraction","authors":"Wen Wen, Yongbin Liu, Chunping Ouyang, Qiang Lin, Tonglee Chung","year":"2021","venue":"Inf. Process. Manag","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2021] Aming Wu, Yahong Han, Linchao Zhu, and Yi Yang. Universal-prototype enhancing for few-shot object detection. In Proc. of ICCV, pages 9567-9576, 2021.","title":"Universal-prototype enhancing for few-shot object detection","authors":"Aming Wu, Yahong Han, Linchao Zhu, Yi Yang","year":"2021","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2021] Shan Yang, Yongfei Zhang, Guanglin Niu, Qinghua Zhao, and Shiliang Pu. Entity concept- enhanced few-shot relation extraction. In Proc. of ACL, pages 987-991, 2021.","title":"Entity conceptenhanced few-shot relation extraction","authors":"Shan Yang, Yongfei Zhang, Guanglin Niu, Qinghua Zhao, Shiliang Pu","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2022] Fengyuan Yang, Ruiping Wang, and Xilin Chen. Sega: Semantic guided attention on visual prototype for few-shot learning. In Proc. of WACV, pages 1586-1596, 2022.","title":"Sega: Semantic guided attention on visual prototype for few-shot learning","authors":"Yang","year":"2022","venue":"Proc. of WACV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ye and Ling, 2019] Zhixiu Ye and Zhenhua Ling. Multi- level matching and aggregation network for few-shot re- lation classification. In Proc. of ACL, pages 2872-2881, 2019.","title":"Zhixiu Ye and Zhenhua Ling. Multilevel matching and aggregation network for few-shot relation classification","authors":"Ling Ye","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2020] Ji Zhang, Chengyao Chen, Pengfei Liu, Chao He, and Cane Wing-Ki Leung. Target-guided struc- tured attention network for target-dependent sentiment analysis. Trans. Assoc. Comput. Linguistics, 8(3):172- 182, 2020.","title":"Target-guided structured attention network for target-dependent sentiment analysis","authors":"Ji Zhang, Chengyao Chen, Pengfei Liu, Chao He, Cane Wing, -Ki Leung","year":"2020","venue":"Trans. Assoc. Comput. Linguistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}