{"type":"ScholarlyArticle","title":"Multi-attention Meta Learning for Few-shot Fine-grained Image Recognition","authors":[{"name":"Yaohui Zhu","slug":"yaohui-zhu","given":"Yaohui","family":"Zhu","affiliation":"Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing , 100190 , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Chenlong Liu","slug":"chenlong-liu","given":"Chenlong","family":"Liu","affiliation":"Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing , 100190 , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Shuqiang Jiang","slug":"shuqiang-jiang","given":"Shuqiang","family":"Jiang","affiliation":"Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing , 100190 , China; University of Chinese Academy of Sciences , Beijing , China"}],"abstract":"The goal of few-shot image recognition is to distinguish different categories with only one or a few training samples. Previous works of few-shot learning mainly work on general object images. And current solutions usually learn a global image representation from training tasks to adapt novel tasks. However, fine-gained categories are distinguished by subtle and local parts, which could not be captured by global representations effectively. This may hinder existing few-shot learning approaches from dealing with fine-gained categories well. In this work, we propose a multi-attention meta-learning (MattML) method for few-shot fine-grained image recognition (FSFGIR). Instead of using only base learner for general feature learning, the proposed meta-learning method uses attention mechanisms of the base learner and task learner to capture discriminative parts of images. The base learner is equipped with two convolutional block attention modules (CBAM) and a classifier. The two CBAM can learn diverse and informative parts. And the initial weights of classifier are attended by the task learner, which gives the classifier a task-related sensitive initialization. For adaptation, the gradient-based meta-learning approach is employed by updating the parameters of two CBAM and the attended classifier, which facilitates the updated base learner to adaptively focus on discriminative parts. We experimentally analyze the different components of our method, and experimental results on four benchmark datasets demonstrate the effectiveness and superiority of our method.","doi":"10.24963/ijcai.2020/152","url":"https://ijcai.org/proceedings/2020/152","pdf":"https://ijcai.org/proceedings/2020/0152.pdf","markdown":"https://ijcai.org/proceedings/2020/152.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/152","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":152,"pages":{"first":1090,"last":1096},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24937/file_24937.m3u8","length_sec":280,"provider":"hls"}],"references":[{"position":1,"raw":"[Chai et al., 2013] Yuning Chai, Victor Lempitsky, and Andrew Zisserman. Symbiotic segmentation and part localization for fine- grained categorization. In ICCV, pages 321-328, 2013. 2.","title":"Symbiotic segmentation and part localization for finegrained categorization","authors":"Chai","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Conneau et al., 2017] Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. Supervised learn- ing of universal sentence representations from natural language inference data. In EMNLP, pages 670-680, 2017. 4.","title":"Supervised learning of universal sentence representations from natural language inference data","authors":"Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, Antoine Bordes","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fei-Fei et al., 2006] Li Fei-Fei, Rob Fergus, and Pietro Perona. One-shot learning of object categories. IEEE TPAMI, 28(4):594- 611, 2006. 2.","title":"One-shot learning of object categories","authors":"Li Fei-Fei, Rob Fergus, Pietro Perona","year":"2006","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep net- works. In ICML, 2017. 1, 2.2, 3.3, 5.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fu et al., 2017] Jianlong Fu, Heliang Zheng, and Tao Mei. Look closer to see better: Recurrent attention convolutional neural net- work for fine-grained image recognition. In CVPR, pages 4438- 4446, 2017. 1, 1, 2.","title":"Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition","authors":"Jianlong Fu, Heliang Zheng, Tao Mei","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2019] Huaxi Huang, Junjie Zhang, Jian Zhang, Jing- song Xu, and Qiang Wu. Low-rank pairwise alignment bilin- ear network for few-shot fine-grained image classification. arXiv preprint arXiv:1908.01313, 2019. 2.2, 5.","title":"Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification","authors":"Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, Qiang Wu","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In CVPR Workshops, 2011. 1, 5.","title":"Novel dataset for fine-grained image categorization","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Li Fei-Fei","year":"2011","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 5.3","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained cate- gorization. In ICCV Workshops, pages 554-561, 2013. 5.","title":"3d object representations for fine-grained categorization","authors":"Jonathan Krause, Michael Stark, Jia Deng, Li Fei-Fei","year":"2013","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lake et al., 2015] B. M. Lake, R Salakhutdinov, and J. B. Tenen- baum. Human-level concept learning through probabilistic pro- gram induction. Science, 350(6266):1332-1338, 2015. 2.","title":"Human-level concept learning through probabilistic program induction","authors":"B M Lake, J B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee and Choi, 2018] Yoonho Lee and Seungjin Choi. Gradient- based meta-learning with learned layerwise metric and subspace. In ICML, pages 2927-2936, 2018. 2.","title":"Yoonho Lee and Seungjin Choi. Gradientbased meta-learning with learned layerwise metric and subspace","authors":"Choi Lee","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019a] Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descriptor based image-to- class measure for few-shot learning. In CVPR, 2019. 5.2, 3","title":"Revisiting local descriptor based image-toclass measure for few-shot learning","authors":"Li","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019b] Wenbin Li, Jinglin Xu, Jing Huo, Lei Wang, Yang Gao, and Jiebo Luo. Distribution consistency based covariance metric networks for few-shot learning. In AAAI, 2019. 5.1, 5.2, 3","title":"Distribution consistency based covariance metric networks for few-shot learning","authors":"Li","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luo et al., 2019] Wei Luo, Xitong Yang, Xianjie Mo, Yuheng Lu, Larry S Davis, Jun Li, Jian Yang, and Ser-Nam Lim. Cross-x learning for fine-grained visual categorization. In ICCV, pages 8242-8251, 2019. 1, 2.","title":"Cross-x learning for fine-grained visual categorization","authors":"Wei Luo, Xitong Yang, Xianjie Mo, Yuheng Lu, Larry S Davis, Jun Li, Jian Yang, Ser-Nam Lim","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maji et al., 2013] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013. 5.1","title":"Fine-grained visual classification of aircraft","authors":"Maji","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Munkhdalai and Yu, 2017] Tsendsuren Munkhdalai and Hong Yu. Meta networks. In ICML, 2017. 2.2","title":"Tsendsuren Munkhdalai and Hong Yu. Meta networks","authors":"Yu Munkhdalai","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Munkhdalai et al., 2018] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with con- ditionally shifted neurons. In ICML, 2018. 1, 2.2, 5.","title":"Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons","authors":"Munkhdalai","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, pages 618-626, 2017. 5.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Selvaraju","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Smith and Slone, 2017] Linda B Smith and Lauren K Slone. A developmental approach to machine learning? Frontiers in psy- chology, 8:2124, 2017.","title":"A developmental approach to machine learning","authors":"Slone ; ] Smith, B Linda, Lauren K Smith, Slone","year":"2017","venue":"Frontiers in psychology","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NIPS, pages 4080-4090, 2017. 1, 2.2, 5.2, 5.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xi- ang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In CVPR, 2018. 1, 2.2, 5.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim Lilli- crap, Daan Wierstra, et al. Matching networks for one shot learn- ing. In NIPS, pages 3630-3638, 2016. 1, 2.2, 5.2, 5.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200- 2011 dataset. 2011. 1, 5.","title":"The caltech-ucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2015] Dequan Wang, Zhiqiang Shen, Jie Shao, Wei Zhang, Xiangyang Xue, and Zheng Zhang. Multiple granularity descriptors for fine-grained categorization. In ICCV, pages 2399- 2406, 2015. 2.","title":"Multiple granularity descriptors for fine-grained categorization","authors":"Dequan Wang, Zhiqiang Shen, Jie Shao, Wei Zhang, Xiangyang Xue, Zheng Zhang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wei et al., 2019] Xiu-Shen Wei, Peng Wang, Lingqiao Liu, Chun- hua Shen, and Jianxin Wu. Piecewise classifier mappings: Learn- ing fine-grained learners for novel categories with few examples. IEEE TIP, 28(12):6116-6125, 2019. 2.","title":"Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples","authors":"Xiu-Shen Wei, Peng Wang, Lingqiao Liu, Chunhua Shen, Jianxin Wu","year":"2019","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Woo et al., 2018] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In ECCV, pages 3-19, 2018. 1, 4.1, 4.","title":"Cbam: Convolutional block attention module","authors":"Sanghyun Woo, Jongchan Park, Joon-Young Lee, In So Kweon","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xiao et al., 2015] Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiax- ing Zhang, Yuxin Peng, and Zheng Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In CVPR, pages 842-850, 2015. 1, 1, 2.","title":"The application of two-level attention models in deep convolutional neural network for fine-grained image classification","authors":"Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, Zheng Zhang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xie et al., 2013] Lingxi Xie, Qi Tian, Richang Hong, Shuicheng Yan, and Bo Zhang. Hierarchical part matching for fine-grained visual categorization. In ICCV, pages 1641-1648, 2013. 2.","title":"Hierarchical part matching for fine-grained visual categorization","authors":"Xie","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2015] Saining Xie, Tianbao Yang, Xiaoyu Wang, and Yuanqing Lin. Hyper-class augmented and regularized deep learning for fine-grained image classification. In CVPR, pages 2645-2654, 2015. 2.","title":"Hyper-class augmented and regularized deep learning for fine-grained image classification","authors":"Saining Xie, Tianbao Yang, Xiaoyu Wang, Yuanqing Lin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yoon et al., 2018] Jaesik Yoon, Taesup Kim, Ousmane Dia, Sung- woong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model- agnostic meta-learning. In NIPS, pages 7342-7352, 2018. 2.","title":"Bayesian modelagnostic meta-learning","authors":"Yoon","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zeiler and Fergus, 2014] Matthew D Zeiler and Rob Fergus. Vi- sualizing and understanding convolutional networks. In ECCV, pages 818-833. Springer, 2014. 4.","title":"Visualizing and understanding convolutional networks","authors":"D Matthew, Rob Zeiler, Fergus","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2014] Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Darrell. Part-based r-cnns for fine-grained category detec- tion. In ECCV, pages 834-849. Springer, 2014. 1, 1, 2.","title":"Part-based r-cnns for fine-grained category detection","authors":"Ning Zhang, Jeff Donahue, Ross Girshick, Trevor Darrell","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2016] Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. Picking deep filter responses for fine-grained image recognition. In CVPR, pages 1134-1142, 2016. 2.","title":"Picking deep filter responses for fine-grained image recognition","authors":"Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, Qi Tian","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zheng et al., 2017] Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. Learning multi-attention convolutional neural net- work for fine-grained image recognition. In ICCV, pages 5209- 5217, 2017. 1, 2.","title":"Learning multi-attention convolutional neural network for fine-grained image recognition","authors":"Zheng","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhou and Lin, 2016] Feng Zhou and Yuanqing Lin. Fine-grained image classification by exploring bipartite-graph labels. In CVPR, pages 1124-1133, 2016. 2.1","title":"Fine-grained image classification by exploring bipartite-graph labels","authors":"Lin ; ] Feng Zhou, Yuanqing Zhou, Lin","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}