{"type":"ScholarlyArticle","title":"StackDRL: Stacked Deep Reinforcement Learning for Fine-grained Visual Categorization","authors":[{"name":"Xiangteng He","slug":"xiangteng-he","given":"Xiangteng","family":"He","affiliation":"Institute of Computer Science and Technology , Peking University Beijing 100871 , China"},{"name":"Yuxin Peng","slug":"yuxin-peng","given":"Yuxin","family":"Peng","affiliation":"Institute of Computer Science and Technology , Peking University Beijing 100871 , China"},{"name":"Junjie Zhao","slug":"junjie-zhao","given":"Junjie","family":"Zhao","affiliation":"Institute of Computer Science and Technology , Peking University Beijing 100871 , China"}],"abstract":"Fine-grained visual categorization (FGVC) is the discrimination of similar subcategories, whose main challenge is to localize the quite subtle visual distinctions between similar subcategories. There are two pivotal problems: discovering which region is discriminative and representative, and determining how many discriminative regions are necessary to achieve the best performance. Existing methods generally solve these two problems relying on the prior knowledge or experimental validation, which extremely restricts the usability and scalability of FGVC. To address the \"which\" and \"how many\" problems adaptively and intelligently, this paper proposes a stacked deep reinforcement learning approach (StackDRL). It adopts a two-stage learning architecture, which is driven by the semantic reward function. Two-stage learning localizes the object and its parts in sequence (\"which\"), and determines the number of discriminative regions adaptively (\"how many\"), which is quite appealing in FGVC. Semantic reward function drives StackDRL to fully learn the discriminative and conceptual visual information, via jointly combining the attention-based reward and category-based reward. Furthermore, unsupervised discriminative localization avoids the heavy labor consumption of labeling, and extremely strengthens the usability and scalability of our StackDRL approach. Comparing with ten state-of-the-art methods on CUB-200-2011 dataset, our StackDRL approach achieves the best categorization accuracy.","doi":"10.24963/ijcai.2018/103","url":"https://ijcai.org/proceedings/2018/103","pdf":"https://ijcai.org/proceedings/2018/0103.pdf","markdown":"https://ijcai.org/proceedings/2018/103.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/103","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":103,"pages":{"first":741,"last":747},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Machine Learning Applications: Applications of Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2017] Sijia Cai, Wangmeng Zuo, and Lei Zhang. Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization. In CVPR, pages 511-520, 2017.","title":"Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization","authors":"Sijia Cai, Wangmeng Zuo, Lei Zhang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caicedo and Lazebnik, 2015] Juan C Caicedo and Svetlana Lazeb- nik. Active object localization with deep reinforcement learning. In ICCV, pages 2488-2496. IEEE, 2015.","title":"Juan C Caicedo and Svetlana Lazebnik. Active object localization with deep reinforcement learning","authors":"Lazebnik Caicedo","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. pages 248-255, 2009.","title":"ImageNet: A Large-Scale Hierarchical Image Database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fu et al., 2017] Jianlong Fu, Heliang Zheng, and Tao Mei. Look closer to see better: Recurrent attention convolutional neural net- work for fine-grained image recognition. In The CVPR, July 2017.","title":"Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition","authors":"Jianlong Fu, Heliang Zheng, Tao Mei","year":"2017","venue":"The CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In ICCV, December 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He and Peng, 2017a] Xiangteng He and Yuxin Peng. Fine-grained image classification via combining vision and language. In The CVPR, July 2017.","title":"Xiangteng He and Yuxin Peng. Fine-grained image classification via combining vision and language","authors":"Peng He","year":"2017","venue":"The CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He and Peng, 2017b] Xiangteng He and Yuxin Peng. Weakly su- pervised learning of part selection model with spatial constraints for fine-grained image classification. In AAAI, pages 4075-4081, 2017.","title":"Xiangteng He and Yuxin Peng. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification","authors":"Peng He","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2016] Shaoli Huang, Zhe Xu, Dacheng Tao, and Ya Zhang. Part-stacked cnn for fine-grained visual categoriza- tion. In CVPR, pages 1173-1182, 2016.","title":"Part-stacked cnn for fine-grained visual categorization","authors":"Huang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. arxiv:1502.03167, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jaderberg et al., 2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In NIPS, pages 2017-2025, 2015.","title":"Spatial transformer networks","authors":"Jaderberg","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jie et al., 2016] Zequn Jie, Xiaodan Liang, Jiashi Feng, Xiaojie Jin, Wen Lu, and Shuicheng Yan. Tree-structured reinforcement learning for sequential object localization. In NIPS, pages 127- 135, 2016.","title":"Tree-structured reinforcement learning for sequential object localization","authors":"Jie","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Khosla et al., 2011] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In CVPR, volume 2, 2011.","title":"Novel dataset for fine-grained image categorization: Stanford dogs","authors":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, Fei-Fei Li","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kong and Fowlkes, 2017] Shu Kong and Charless Fowlkes. Low- rank bilinear pooling for fine-grained classification. In CVPR, pages 7025-7034. IEEE, 2017.","title":"Lowrank bilinear pooling for fine-grained classification","authors":"Shu Kong, Charless Fowlkes","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krause et al., 2015] Jonathan Krause, Hailin Jin, Jianchao Yang, and Li Fei-Fei. Fine-grained recognition without part annota- tions. In CVPR, pages 5546-5555, 2015.","title":"Fine-grained recognition without part annotations","authors":"Jonathan Krause, Hailin Jin, Jianchao Yang, Li Fei-Fei","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Neider and Zelinsky, 2006] Mark B Neider and Gregory J Zelin- sky. Searching for camouflaged targets: Effects of target- background similarity on visual search. Vision research, 46(14):2217-2235, 2006.","title":"Searching for camouflaged targets: Effects of targetbackground similarity on visual search","authors":"B Mark, Gregory J Neider, Zelinsky","year":"2006","venue":"Vision research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Otsu, 1979] Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics, 9(1):62-66, 1979.","title":"A threshold selection method from gray-level histograms","authors":"Nobuyuki Otsu","year":"1979","venue":"IEEE transactions on systems, man, and cybernetics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simon and Rodner, 2015] Marcel Simon and Erik Rodner. Neu- ral activation constellations: Unsupervised part model discovery with convolutional networks. In ICCV, pages 1143-1151, 2015.","title":"Neural activation constellations: Unsupervised part model discovery with convolutional networks","authors":"Rodner Simon, Marcel Simon, Erik Rodner","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. arxiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton and Barto, 1998] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200- 2011 dataset. 2011.","title":"The caltech-ucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiao et al., 2015] Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiax- ing Zhang, Yuxin Peng, and Zheng Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In CVPR, pages 842-850, 2015.","title":"The application of two-level attention models in deep convolutional neural network for fine-grained image classification","authors":"Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, Zheng Zhang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2016] Hantao Yao, Shiliang Zhang, Yongdong Zhang, Jintao Li, and Qi Tian. Coarse-to-fine description for fine-grained visual categorization. IEEE TIP, 25(10):4858-4872, 2016.","title":"Coarse-to-fine description for fine-grained visual categorization","authors":"Hantao Yao, Shiliang Zhang, Yongdong Zhang, Jintao Li, Qi Tian","year":"2016","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2014] Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Darrell. Part-based r-cnns for fine-grained category detec- tion. In ICML, pages 834-849, 2014.","title":"Part-based r-cnns for fine-grained category detection","authors":"Ning Zhang, Jeff Donahue, Ross Girshick, Trevor Darrell","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016] Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. Picking deep filter responses for fine-grained image recognition. In CVPR, pages 1134-1142, 2016.","title":"Picking deep filter responses for fine-grained image recognition","authors":"Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, Qi Tian","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2017] Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. Picking neural activations for fine-grained recognition. IEEE TMM, 19(12):2736-2750, 2017.","title":"Picking neural activations for fine-grained recognition","authors":"Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, Qi Tian","year":"2017","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}