{"type":"ScholarlyArticle","title":"Diversity Features Enhanced Prototypical Network for Few-shot Intent Detection","authors":[{"name":"Fengyi Yang","slug":"fengyi-yang","given":"Fengyi","family":"Yang","affiliation":"Xinjiang Technical Institute of Physics & Chemistry , Chinese Academy of Sciences , Urumqi 830011 , China; University of Chinese Academy of Sciences , Beijing 100049 , China; Xinjiang Laboratory of Minority Speech and Language Information Processing , Urumqi 830011 , China"},{"name":"Xi Zhou","slug":"xi-zhou","given":"Xi","family":"Zhou","affiliation":"Xinjiang Technical Institute of Physics & Chemistry , Chinese Academy of Sciences , Urumqi 830011 , China; University of Chinese Academy of Sciences , Beijing 100049 , China; Xinjiang Laboratory of Minority Speech and Language Information Processing , Urumqi 830011 , China"},{"name":"Yi Wang","slug":"yi-wang","given":"Yi","family":"Wang","affiliation":"Xinjiang Technical Institute of Physics & Chemistry , Chinese Academy of Sciences , Urumqi 830011 , China; University of Chinese Academy of Sciences , Beijing 100049 , China; Xinjiang Laboratory of Minority Speech and Language Information Processing , Urumqi 830011 , China"},{"name":"Abibulla Atawulla","slug":"abibulla-atawulla","given":"Abibulla","family":"Atawulla","affiliation":"Xinjiang Technical Institute of Physics & Chemistry , Chinese Academy of Sciences , Urumqi 830011 , China; University of Chinese Academy of Sciences , Beijing 100049 , China; Xinjiang Laboratory of Minority Speech and Language Information Processing , Urumqi 830011 , China"},{"name":"Ran Bi","slug":"ran-bi","given":"Ran","family":"Bi","affiliation":"Xinjiang Technical Institute of Physics & Chemistry , Chinese Academy of Sciences , Urumqi 830011 , China; University of Chinese Academy of Sciences , Beijing 100049 , China; Xinjiang Laboratory of Minority Speech and Language Information Processing , Urumqi 830011 , China"}],"abstract":"Few-shot Intent Detection (FSID) is a challenging task in dialogue systems due to the scarcity of available annotated utterances. Although existing few-shot learning approaches have made remarkable progress, they fall short in adapting to the Generalized Few-shot Intent Detection (GFSID) task where both seen and unseen classes are present. A core problem of the simultaneous existence of these two tasks is that limited training samples fail to cover the diversity of user expressions. In this paper, we propose an effective Diversity Features Enhanced Prototypical Network (DFEPN) to enhance diversity features for novel intents by fully exploiting the diversity of known intent samples. Specially, DFEPN generates diversity features of samples in the hidden space via a diversity feature generator module and then fuses these features with original support vectors to get a more suitable prototype vector of each class. To evaluate the effectiveness of our model on both FSID and GFSID tasks, we carry out sufficient experiments on two benchmark intent detection datasets. Results demonstrate that our proposed model outperforms existing state-of-the-art methods and keeps stable performance on both two tasks.","doi":"10.24963/ijcai.2022/617","url":"https://ijcai.org/proceedings/2022/617","pdf":"https://ijcai.org/proceedings/2022/0617.pdf","markdown":"https://ijcai.org/proceedings/2022/617.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/617","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Natural Language Processing","track":"Main Track","number":617,"pages":{"first":4447,"last":4453},"keywords":["Natural Language Processing: Applications","Machine Learning: Meta-Learning","Machine Learning: Few-shot learning","Natural Language Processing: Dialogue and Interactive Systems","Natural Language Processing: Text Classification"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986635","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984778","length_sec":315,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ba et al., 2016] Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. CoRR, abs/1607.06450, 2016.","title":"Layer normalization","authors":"Jimmy Lei, Jamie Ba, Geoffrey E Ryan Kiros, Hinton","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Mengting Chen, Yuxin Fang, Xinggang Wang, Heng Luo, Yifeng Geng, Xinyu Zhang, Chang Huang, Wenyu Liu, and Bo Wang. Diversity transfer network for few-shot learning. In Proc. of AAAI, pages 10559-10566, 2020.","title":"Diversity transfer network for few-shot learning","authors":"Chen","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Coucke et al., 2018] Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Maël Primet, and Joseph Dureau. Snips voice platform: an embedded spoken lan- guage understanding system for private-by-design voice interfaces. CoRR, abs/1805.10190, 2018.","title":"Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces","authors":"Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Maël Primet, Joseph Dureau","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2020] Shumin Deng, Ningyu Zhang, Zhanlin Sun, Jiaoyan Chen, and Huajun Chen. When low re- source NLP meets unsupervised language model: Meta- pretraining then meta-learning for few-shot text classifica- tion (student abstract). In Proc. of AAAI, pages 13773- 13774, 2020.","title":"When low resource NLP meets unsupervised language model: Metapretraining then meta-learning for few-shot text classification","authors":"Shumin Deng, Ningyu Zhang, Zhanlin Sun, Jiaoyan Chen, Huajun Chen","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In Proc. of NAACL, pages 4171-4186, 2019.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proc. of NAACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dopierre et al., 2021] Thomas Dopierre, Christophe Gravier, and Wilfried Logerais. A neural few-shot text classification reality check. In Proc. of EACL, pages 935-943, 2021.","title":"A neural few-shot text classification reality check","authors":"Thomas Dopierre, Christophe Gravier, Wilfried Logerais","year":"2021","venue":"Proc. of EACL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. of ICML, pages 1126-1135, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2019] Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. Hybrid attention-based prototypical net- works for noisy few-shot relation classification. In Proc. of AAAI, pages 6407-6414, 2019.","title":"Hybrid attention-based prototypical networks for noisy few-shot relation classification","authors":"Tianyu Gao, Xu Han, Zhiyuan Liu, Maosong Sun","year":"2019","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2021] Chengcheng Han, Zeqiu Fan, Dongxiang Zhang, Minghui Qiu, Ming Gao, and Aoying Zhou. Meta- learning adversarial domain adaptation network for few- shot text classification. In Proc. of ACL Findings, pages 1664-1673, 2021.","title":"Metalearning adversarial domain adaptation network for fewshot text classification","authors":"Han","year":"2021","venue":"Proc. of ACL Findings","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proc. of CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2019] Aoxue Li, Tiange Luo, Tao Xiang, Weiran Huang, and Liwei Wang. Few-shot learning with global class representations. In Proc. of ICCV, pages 9714-9723, 2019.","title":"Few-shot learning with global class representations","authors":"Li","year":"2019","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Xingkun Liu, Arash Eshghi, Pawel Swieto- janski, and Verena Rieser. Benchmarking natural language understanding services for building conversational agents. In International Workshop on Spoken Dialogue Systems, pages 165-183, 2019.","title":"Benchmarking natural language understanding services for building conversational agents","authors":"Liu","year":"2019","venue":"International Workshop on Spoken Dialogue Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nguyen et al., 2020] Hoang Nguyen, Chenwei Zhang, Con- gying Xia, and Philip S. Yu. Semantic matching and ag- gregation network for few-shot intent detection. In Proc. of EMNLP Findings, pages 1209-1218, 2020.","title":"Semantic matching and aggregation network for few-shot intent detection","authors":"Hoang Nguyen, Chenwei Zhang, Congying Xia, Philip S Yu","year":"2020","venue":"Proc. of EMNLP Findings","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rusu et al., 2018] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osin- dero, and Raia Hadsell. Meta-learning with latent embed- ding optimization. CoRR, abs/1807.05960, 2018.","title":"Meta-learning with latent embedding optimization","authors":"Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, Raia Hadsell","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shi et al., 2019] Xiahan Shi, Leonard Salewski, Martin Schiegg, Zeynep Akata, and Max Welling. Relational gen- eralized few-shot learning. CoRR, abs/1907.09557, 2019.","title":"Relational generalized few-shot learning","authors":"Shi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few- shot learning. In Proc. of NeurIPS, pages 4077-4087, 2017.","title":"Prototypical networks for fewshot learning","authors":"Jake Snell, Kevin Swersky, Richard S Zemel","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sun et al., 2019] Shengli Sun, Qingfeng Sun, Kevin Zhou, and Tengchao Lv. Hierarchical attention prototypical net- works for few-shot text classification. In Proc. of EMNLP, pages 476-485, 2019.","title":"Hierarchical attention prototypical networks for few-shot text classification","authors":"Shengli Sun, Qingfeng Sun, Kevin Zhou, Tengchao Lv","year":"2019","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2021] Pengfei Sun, Yawen Ouyang, Wenming Zhang, and Xinyu Dai. MEDA: meta-learning with data augmentation for few-shot text classification. In Proc. of IJCAI, pages 3929-3935, 2021.","title":"MEDA: meta-learning with data augmentation for few-shot text classification","authors":"Pengfei Sun, Yawen Ouyang, Wenming Zhang, Xinyu Dai","year":"2021","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/541"},{"position":20,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learn- ing. In Proc. of CVPR, pages 1199-1208, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, pages 2579-2605, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning. In Proc. of NeurIPS, pages 3630-3638, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, Daan Wierstra","year":"2016","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wolf et al., 2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Fun- towicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of- the-art natural language processing. In Proc. of EMNLP, pages 38-45, 2020.","title":"Transformers: State-ofthe-art natural language processing","authors":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Clara Patrick Von Platen, Yacine Ma, Julien Jernite, Canwen Plu, Teven Xu, Sylvain Le Scao, Mariama Gugger, Quentin Drame, Alexander M Lhoest, Rush","year":"2020","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xia et al., 2018] Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and Philip S. Yu. Zero-shot user intent de- tection via capsule neural networks. In Proc. of EMNLP, pages 3090-3099, 2018.","title":"Zero-shot user intent detection via capsule neural networks","authors":"Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, Philip S Yu","year":"2018","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xia et al., 2020] Congying Xia, Chenwei Zhang, Hoang Nguyen, Jiawei Zhang, and Philip S. Yu. CG-BERT: con- ditional text generation with BERT for generalized few- shot intent detection. CoRR, abs/2004.01881, 2020.","title":"CG-BERT: conditional text generation with BERT for generalized fewshot intent detection","authors":"Congying Xia, Chenwei Zhang, Hoang Nguyen, Jiawei Zhang, Philip S Yu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ye and Ling, 2019] Zhi-Xiu Ye and Zhen-Hua Ling. Multi- level matching and aggregation network for few-shot re- lation classification. In Proc. of ACL, pages 2872-2881, 2019.","title":"Multilevel matching and aggregation network for few-shot relation classification","authors":"Ling ; Zhi-Xiu Ye, Zhen-Hua Ye, Ling","year":"2019","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}