{"type":"ScholarlyArticle","title":"Learning Task-aware Local Representations for Few-shot Learning","authors":[{"name":"Chuanqi Dong","slug":"chuanqi-dong","given":"Chuanqi","family":"Dong","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Wenbin Li","slug":"wenbin-li","given":"Wenbin","family":"Li","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Jing Huo","slug":"jing-huo","given":"Jing","family":"Huo","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Zheng Gu","slug":"zheng-gu","given":"Zheng","family":"Gu","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"},{"name":"Yang Gao","slug":"yang-gao","given":"Yang","family":"Gao","affiliation":"State Key Laboratory for Novel Software Technology , Nanjing University , China"}],"abstract":"Few-shot learning for visual recognition aims to adapt to novel unseen classes with only a few images. Recent work, especially the work based on low-level information, has achieved great progress. In these work, local representations (LRs) are typically employed, because LRs are more consistent among the seen and unseen classes. However, most of them are limited to an individual image-to-image or image-to-class measure manner, which cannot fully exploit the capabilities of LRs, especially in the context of a certain task. This paper proposes an Adaptive Task-aware Local Representations Network (ATL-Net) to address this limitation by introducing episodic attention, which can adaptively select the important local patches among the entire task, as the process of human recognition. We achieve much superior results on multiple benchmarks. On the miniImagenet, ATL-Net gains 0.93% and 0.88% improvements over the compared methods under the 5-way 1-shot and 5-shot settings. Moreover, ATL-Net can naturally tackle the problem that how to adaptively identify and weight the importance of different key local parts, which is the major concern of fine-grained recognition. Specifically, on the fine-grained dataset Stanford Dogs, ATL-Net outperforms the second best method with 5.39% and 9.69% gains under the 5-way 1-shot and 5-shot settings.","doi":"10.24963/ijcai.2020/100","url":"https://ijcai.org/proceedings/2020/100","pdf":"https://ijcai.org/proceedings/2020/0100.pdf","markdown":"https://ijcai.org/proceedings/2020/100.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/100","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":100,"pages":{"first":716,"last":722},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24625/file_24625.m3u8","length_sec":280,"provider":"hls"}],"references":[{"position":1,"raw":"[Cai et al., 2018] Qi Cai, Yingwei Pan, Ting Yao, Cheng- gang Yan, and Tao Mei. Memory matching networks for one-shot image recognition. In CVPR, pages 4080-4088, 2018.","title":"Memory matching networks for one-shot image recognition","authors":"Cai","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chu et al., 2019] Wen-Hsuan Chu, Yu-Jhe Li, Jing-Cheng Chang, and Yu-Chiang Frank Wang. Spot and learn: A maximum-entropy patch sampler for few-shot image clas- sification. In CVPR, pages 6251-6260, 2019.","title":"Spot and learn: A maximum-entropy patch sampler for few-shot image classification","authors":"Wen-Hsuan Chu, Yu-Jhe Li, Jing-Cheng Chang, Yu-Chiang Frank, Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":null,"venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, pages 1126-1135. JMLR. org, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2019] Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, and Qiang Wu. Low-rank pairwise alignment bilinear network for few-shot fine-grained im- age classification. arXiv preprint arXiv:1908.01313, 2019.","title":"Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification","authors":"Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, Qiang Wu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevap- rakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR Workshop on Fine-Grained Visual Categorization (FGVC), 2011.","title":"Novel dataset for fine-grained image categorization: Stanford dogs","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Fei-Fei Li","year":"2011","venue":"Proc. CVPR Workshop on Fine-Grained Visual Categorization (FGVC)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Koch et al., 2015] Gregory Koch, Richard Zemel, and Rus- lan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, vol- ume 2, 2015.","title":"Siamese neural networks for one-shot image recognition","authors":"Gregory Koch, Richard Zemel, Ruslan Salakhutdinov","year":"2015","venue":"ICML deep learning workshop","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In ICCV Workshops, pages 554- 561, 2013.","title":"3d object representations for finegrained categorization","authors":"Jonathan Krause, Michael Stark, Jia Deng, Li Fei-Fei","year":"2013","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, pages 1097- 1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835, 2017.","title":"Meta-sgd: Learning to learn quickly for few-shot learning","authors":"Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019a] Aoxue Li, Tiange Luo, Tao Xiang, Weiran Huang, and Liwei Wang. Few-shot learning with global class representations. In ICCV, pages 9715-9724, 2019.","title":"Few-shot learning with global class representations","authors":"Li","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019b] Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descrip- tor based image-to-class measure for few-shot learning. In CVPR, pages 7260-7268, 2019.","title":"Revisiting local descriptor based image-to-class measure for few-shot learning","authors":"Li","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2019c] Wenbin Li, Jinglin Xu, Jing Huo, Lei Wang, Yang Gao, and Jiebo Luo. Distribution consistency based covariance metric networks for few-shot learning. In AAAI, volume 33, pages 8642-8649, 2019.","title":"Distribution consistency based covariance metric networks for few-shot learning","authors":"Li","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mishra et al., 2018] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner. In ICLR, 2018.","title":"A simple neural attentive meta-learner","authors":"Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In NeurIPS, pages 8024-8035, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Santoro et al., 2016] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lill- icrap. Meta-learning with memory-augmented neural networks. In ICML, pages 1842-1850, 2016.","title":"Meta-learning with memory-augmented neural networks","authors":"Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Satorras and Estrach, 2018] Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In ICLR, 2018.","title":"Few-shot learning with graph neural networks","authors":"Estrach ; Satorras, Garcia Victor, Joan Satorras, Estrach Bruna","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS, pages 4077-4087, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2018] Ming Sun, Yuchen Yuan, Feng Zhou, and Errui Ding. Multi-attention multi-class constraint for fine- grained image recognition. In ECCV, pages 805-821, 2018.","title":"Multi-attention multi-class constraint for finegrained image recognition","authors":"Ming Sun, Yuchen Yuan, Feng Zhou, Errui Ding","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learn- ing. In CVPR, pages 1199-1208, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Timo- thy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS, pages 3630-3638, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2018] Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. In CVPR, pages 7278-7286, 2018.","title":"Low-shot learning from imaginary data","authors":"Yu-Xiong Wang, Ross Girshick, Martial Hebert, Bharath Hariharan","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Welinder et al., 2010] P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech- UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology, 2010.","title":null,"authors":"P Welinder, S Branson, T Mita, C Wah, F Schroff, S Belongie, P Perona","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2018] Ruixiang Zhang, Tong Che, Zoubin Ghahramani, Yoshua Bengio, and Yangqiu Song. Meta- gan: An adversarial approach to few-shot learning. In NeurIPS, pages 2365-2374, 2018.","title":"Metagan: An adversarial approach to few-shot learning","authors":"Zhang","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}