{"type":"ScholarlyArticle","title":"Masked Feature Generation Network for Few-Shot Learning","authors":[{"name":"Yunlong Yu","slug":"yunlong-yu","given":"Yunlong","family":"Yu","affiliation":"College of Information Science and Electronic Engineering , Zhejiang University"},{"name":"Dingyi Zhang","slug":"dingyi-zhang","given":"Dingyi","family":"Zhang","affiliation":"College of Information Science and Electronic Engineering , Zhejiang University"},{"name":"Zhong Ji","slug":"zhong-ji","given":"Zhong","family":"Ji","affiliation":"School of Electrical and Information Engineering , Tianjin University"}],"abstract":"In this paper, we present a feature-augmentation approach called Masked Feature Generation Network (MFGN) for Few-Shot Learning (FSL), a challenging task that attempts to recognize the novel classes with a few visual instances for each class. Most of the feature-augmentation approaches tackle FSL tasks via modeling the intra-class distributions. We extend this idea further to explicitly capture the intra-class variations in a one-to-many manner. Specifically, MFGN consists of an encoder-decoder architecture, with an encoder that performs as a feature extractor and extracts the feature embeddings of the available visual instances (the unavailable instances are seen to be masked), along with a decoder that performs as a feature generator and reconstructs the feature embeddings of the unavailable visual instances from both the available feature embeddings and the masked tokens. Equipped with this generative architecture, MFGN produces nontrivial visual features for the novel classes with limited visual instances. In extensive experiments on four FSL benchmarks, MFGN performs competitively and outperforms the state-of-the-art competitors on most of the few-shot classification tasks.","doi":"10.24963/ijcai.2022/513","url":"https://ijcai.org/proceedings/2022/513","pdf":"https://ijcai.org/proceedings/2022/0513.pdf","markdown":"https://ijcai.org/proceedings/2022/513.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/513","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":513,"pages":{"first":3695,"last":3701},"keywords":["Machine Learning: Few-shot learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986672","length_sec":79,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984815","length_sec":352,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bertinetto et al., 2018] Luca Bertinetto, Joao F Henriques, Philip Torr, and Andrea Vedaldi. Meta-learning with dif- ferentiable closed-form solvers. In ICLR, 2018.","title":"Meta-learning with differentiable closed-form solvers","authors":"Luca Bertinetto, Joao F Henriques, Philip Torr, Andrea Vedaldi","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2018] Qi Cai, Yingwei Pan, Ting Yao, Cheng- gang Yan, and Tao Mei. Memory matching networks for one-shot image recognition. In CVPR, pages 4080-4088, 2018.","title":"Memory matching networks for one-shot image recognition","authors":"Qi Cai, Yingwei Pan, Ting Yao, Chenggang Yan, Tao Mei","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR, 2018.","title":"A closer look at few-shot classification","authors":"Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank, Wang, Jia-Bin Huang","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dong et al., 2021] Chuanqi Dong, Wenbin Li, Jing Huo, Zheng Gu, and Yang Gao. Learning task-aware local rep- resentations for few-shot learning. In IJCAL, pages 716- 722, 2021.","title":"Learning task-aware local representations for few-shot learning","authors":"Dong","year":"2021","venue":"IJCAL","doi":null,"url":null,"internal_url":"/proceedings/2020/100"},{"position":5,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, pages 1126-1135, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2021] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. arXiv:2111.06377, 2021.","title":"Masked autoencoders are scalable vision learners","authors":"Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hou et al., 2019] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In NeurIPS, pages 4003-4014, 2019.","title":"Cross attention network for few-shot classification","authors":"Hou","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2020] Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, and Qiang Wu. Low-rank pairwise alignment bilinear network for few-shot fine-grained im- age classification. IEEE TMM, 2020.","title":"Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification","authors":"Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, Qiang Wu","year":"2020","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevap- rakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In CVPR Workshop, 2011.","title":"Novel dataset for fine-grained image categorization: Stanford dogs","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Fei-Fei Li","year":"2011","venue":"CVPR Workshop","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2019] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with dif- ferentiable convex optimization. In CVPR, pages 10657- 10665, 2019.","title":"Meta-learning with differentiable convex optimization","authors":"Lee","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019a] Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descrip- tor based image-to-class measure for few-shot learning. In CVPR, pages 7260-7268, 2019.","title":"Revisiting local descriptor based image-to-class measure for few-shot learning","authors":"Li","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019b] Wenbin Li, Jinglin Xu, Jing Huo, Lei Wang, Yang Gao, and Jiebo Luo. Distribution consistency based covariance metric networks for few-shot learning. In AAAI, pages 8642-8649, 2019.","title":"Distribution consistency based covariance metric networks for few-shot learning","authors":"Li","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2020] Kai Li, Yulun Zhang, Kunpeng Li, and Yun Fu. Adversarial feature hallucination networks for few- shot learning. In CVPR, pages 13470-13479, 2020.","title":"Adversarial feature hallucination networks for fewshot learning","authors":"Kai Li, Yulun Zhang, Kunpeng Li, Yun Fu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Munkhdalai et al., 2018] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adap- tation with conditionally shifted neurons. In ICML, pages 3664-3673, 2018.","title":"Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons","authors":"Munkhdalai","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ravi and Larochelle, 2016] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In ICLR, 2016.","title":"Optimization as a model for few-shot learning","authors":"Larochelle ; Ravi, Sachin Ravi, Hugo Larochelle","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ravichandran et al., 2019] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-shot learning with em- bedded class models and shot-free meta training. In ICCV, pages 331-339, 2019.","title":"Few-shot learning with embedded class models and shot-free meta training","authors":"Avinash Ravichandran, Rahul Bhotika, Stefano Soatto","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schwartz et al., 2018] Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, and Alex M Bronstein. δ-encoder: an effective sample synthesis method for few- shot object recognition. In NeurIPS, pages 2850-2860, 2018.","title":"δ-encoder: an effective sample synthesis method for fewshot object recognition","authors":"Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, Alex M Bronstein","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS, pages 4080-4090, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sun et al., 2019] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In CVPR, pages 403-412, 2019.","title":"Meta-transfer learning for few-shot learning","authors":"Sun","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learn- ing. In CVPR, pages 1199-1208, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tian et al., 2020] Yonglong Tian, Yue Wang, Dilip Krish- nan, Joshua B Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: a good embedding is all you need? In ECCV, pages 266-282, 2020.","title":"Rethinking few-shot image classification: a good embedding is all you need?","authors":"Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, Phillip Isola","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Timo- thy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS, pages 3630-3638, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. http://www.vision.caltech. edu/datasets/cub 200 2011, 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":"http://www.vision.caltech.edu/datasets/cub2002011","internal_url":null},{"position":25,"raw":"[Xing et al., 2019] Chen Xing, Negar Rostamzadeh, Boris Oreshkin, and Pedro O Pinheiro. Adaptive cross-modal few-shot learning. NeurIPS, 32:4847-4857, 2019.","title":"Adaptive cross-modal few-shot learning","authors":"Chen Xing, Negar Rostamzadeh, Boris Oreshkin, Pedro O Pinheiro","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xu et al., 2021] Chengming Xu, Yanwei Fu, Chen Liu, Chengjie Wang, Jilin Li, Feiyue Huang, Li Zhang, and Xi- angyang Xue. Learning dynamic alignment via meta-filter for few-shot learning. In CVPR, pages 5182-5191, 2021.","title":"Learning dynamic alignment via meta-filter for few-shot learning","authors":"Chengming Xu, Yanwei Fu, Chen Liu, Chengjie Wang, Jilin Li, Feiyue Huang, Li Zhang, Xiangyang Xue","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2020] Shuo Yang, Lu Liu, and Min Xu. Free lunch for few-shot learning: Distribution calibration. In ICLR, 2020.","title":"Free lunch for few-shot learning: Distribution calibration","authors":"Shuo Yang, Lu Liu, Min Xu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2022] Tianyuan Yu, Sen He, Yi-Zhe Song, and Tao Xiang. Hybrid graph neural networks for few-shot learning. In AAAI, 2022.","title":"Hybrid graph neural networks for few-shot learning","authors":"Tianyuan Yu, Sen He, Yi-Zhe Song, Tao Xiang","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu et al., 2020] Yaohui Zhu, Chenlong Liu, and Shuqiang Jiang. Multi-attention meta learning for few-shot fine- grained image recognition. In IJCAI, pages 1090-1096, 2020.","title":"Multi-attention meta learning for few-shot finegrained image recognition","authors":"Zhu","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}