{"type":"ScholarlyArticle","title":"Centralized Ranking Loss with Weakly Supervised Localization for Fine-Grained Object Retrieval","authors":[{"name":"Xiawu Zheng","slug":"xiawu-zheng","given":"Xiawu","family":"Zheng","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , Xiamen University; School of Information Science and Engineering , Xiamen University"},{"name":"Rongrong Ji","slug":"rongrong-ji","given":"Rongrong","family":"Ji","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , Xiamen University; School of Information Science and Engineering , Xiamen University"},{"name":"Xiaoshuai Sun","slug":"xiaoshuai-sun","given":"Xiaoshuai","family":"Sun","affiliation":"Harbin Institute of Technology"},{"name":"Yongjian Wu","slug":"yongjian-wu","given":"Yongjian","family":"Wu","affiliation":"Tencent Technology (Shanghai) Co. , Ltd"},{"name":"Feiyue Huang","slug":"feiyue-huang","given":"Feiyue","family":"Huang","affiliation":"Tencent Technology (Shanghai) Co. , Ltd"},{"name":"Yanhua Yang","slug":"yanhua-yang","given":"Yanhua","family":"Yang","affiliation":"Xidian University"}],"abstract":"Fine-grained object retrieval has attracted extensive research focus recently. Its state-of-the-art schemesare typically based upon convolutional neural network (CNN) features. Despite the extensive progress, two issues remain open. On one hand, the deep features are coarsely extracted at image level rather than precisely at object level, which are interrupted by background clutters. On the other hand, training CNN features with a standard triplet loss is time consuming and incapable to learn discriminative features. In this paper, we present a novel fine-grained object retrieval scheme that conquers these issues in a unified framework. Firstly, we introduce a novel centralized ranking loss (CRL), which achieves a very efficient (1,000times training speedup comparing to the triplet loss) and discriminative feature learning by a ?centralized? global pooling. Secondly, a weakly supervised attractive feature extraction is proposed, which segments object contours with top-down saliency. Consequently, the contours are integrated into the CNN response map to precisely extract features ?within? the target object. Interestingly, we have discovered that the combination of CRL and weakly supervised learning can reinforce each other. We evaluate the performance ofthe proposed scheme on widely-used benchmarks including CUB200-2011 and CARS196. We havereported significant gains over the state-of-the-art schemes, e.g., 5.4% over SCDA [Wei et al., 2017]on CARS196, and 3.7% on CUB200-2011.","doi":"10.24963/ijcai.2018/171","url":"https://ijcai.org/proceedings/2018/171","pdf":"https://ijcai.org/proceedings/2018/0171.pdf","markdown":"https://ijcai.org/proceedings/2018/171.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/171","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":171,"pages":{"first":1226,"last":1233},"keywords":["Computer Vision: Computer Vision","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[],"references":[{"position":1,"raw":"[Bell and Bala, 2015] Sean Bell and Kavita Bala. Learning visual similarity for product design with convolutional neural networks. ACM Transactions on Graphics (TOG), 34(4):98, 2015.","title":"Learning visual similarity for product design with convolutional neural networks","authors":"Bala ; Bell, Sean Bell, Kavita Bala","year":"2015","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Carsten et al., 2004] Rother Carsten, Kolmogorov Vladimir, and Blake Andrew. Grabcut: Interactive foreground extraction using iterated graph cuts. In ACM transactions on graphics (TOG), volume 23, pages 309-314. ACM, 2004.","title":"Grabcut: Interactive foreground extraction using iterated graph cuts","authors":"Rother Carsten, Kolmogorov Vladimir, Blake Andrew","year":"2004","venue":"ACM transactions on graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248-255. IEEE, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[He and Peng, 2017] Xiangteng He and Yuxin Peng. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification. In Proceedings of the Eleventh National Conference on Artificial Intelligence, pages 4075-4081, 2017.","title":"Xiangteng He and Yuxin Peng. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification","authors":"Peng He","year":"2017","venue":"Proceedings of the Eleventh National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Huang et al., 2016] Chen Huang, Chen Change Loy, and Xiaoou Tang. Local similarity-aware deep feature embedding. In Advances in Neural Information Processing Systems, pages 1262-1270, 2016.","title":"Local similarity-aware deep feature embedding","authors":"Chen Huang, Chen Change Loy, Xiaoou Tang","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hyun et al., 2016] Oh Song Hyun, Xiang Yu, Jegelka Stefanie, and Savarese Silvio. Deep metric learning via lifted structured feature embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4004-4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jonathan et al., 2013] Krause Jonathan, Stark Michael, Deng Jia, and Fei-Fei Li. 3d object representations for fine-grained categorization. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 554-561, 2013.","title":"3d object representations for fine-grained categorization","authors":"Jonathan Krause, Stark Michael, Deng Jia, Fei-Fei Li","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision Workshops","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jonathan et al., 2015] Krause Jonathan, Hailin Jin, Jianchao Yang, and Fei-Fei Li. Fine-grained recognition without part annota- tions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5546-5555, 2015.","title":"Fine-grained recognition without part annotations","authors":"Jonathan Krause, Hailin Jin, Jianchao Yang, Fei-Fei Li","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Karen and Andrew, 2014] Simonyan Karen and Zisserman An- drew. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Simonyan Karen, Zisserman Andrew","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Karen et al., 2013] Simonyan Karen, Vedaldi Andrea, and Zisser- man Andrew. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.","title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","authors":"Simonyan Karen, Andrea Vedaldi, Zisserman Andrew","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. In Advances in neural information processing systems, pages 109-117, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3431-3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ming et al., 2017] Zuheng Ming, Joseph Chazalon, Muham- mad Muzzamil Luqman, Muriel Visani, and Jean-Christophe Burie. Simple triplet loss based on intra/inter-class metric learning for face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1656-1664, 2017.","title":"Simple triplet loss based on intra/inter-class metric learning for face verification","authors":"Ming","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Oh Song et al., 2017] Hyun Oh Song, Stefanie Jegelka, Vivek Rathod, and Kevin Murphy. Deep metric learning via facility location. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.","title":"Deep metric learning via facility location","authors":"Hyun Oh Song, Stefanie Jegelka, Vivek Rathod, Kevin Murphy","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rodner et al., 2015] Erik Rodner, Marcel Simon, Gunnar Brehm, Stephanie Pietsch, J Wolfgang Wägele, and Joachim Denzler. Fine-grained recognition datasets for biodiversity analysis. arXiv preprint arXiv:1507.00913, 2015.","title":"Fine-grained recognition datasets for biodiversity analysis","authors":"Rodner","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Simon and Rodner, 2015] Marcel Simon and Erik Rodner. Neural activation constellations: Unsupervised part model discovery with convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 1143-1151, 2015.","title":"Neural activation constellations: Unsupervised part model discovery with convolutional networks","authors":"Rodner Simon, Marcel Simon, Erik Rodner","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sohn, 2016] Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Advances in Neural Information Processing Systems, pages 1857-1865, 2016.","title":"Improved deep metric learning with multi-class n-pair loss objective","authors":"Kihyuk Sohn, Sohn","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ustinova and Lempitsky, 2016] Evgeniya Ustinova and Victor Lempitsky. Learning deep embeddings with histogram loss. In Advances in Neural Information Processing Systems, pages 4170-4178, 2016.","title":"Learning deep embeddings with histogram loss","authors":"Evgeniya Ustinova, Victor Lempitsky","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200- 2011 dataset. 2011.","title":"The caltech-ucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2014] Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. Learning fine-grained image similarity with deep ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1386-1393, 2014.","title":"Learning fine-grained image similarity with deep ranking","authors":"Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, Ying Wu","year":"2014","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wei et al., 2017] Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, and Zhi-Hua Zhou. Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE Transactions on Image Processing, 26(6):2868-2881, 2017.","title":"Selective convolutional descriptor aggregation for fine-grained image retrieval","authors":"Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, Zhi-Hua Zhou","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wen et al., 2016] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision, pages 499-515. Springer, 2016.","title":"A discriminative feature learning approach for deep face recognition","authors":"Yandong Wen, Kaipeng Zhang, Zhifeng Li, Yu Qiao","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xiao et al., 2015] Tianjun Xiao, Yichong Xu, Kuiyuan Yang, Jiaxing Zhang, Yuxin Peng, and Zheng Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 842-850, 2015.","title":"The application of two-level attention models in deep convolutional neural network for fine-grained image classification","authors":"Xiao","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2013] Lingxi Xie, Qi Tian, Richang Hong, Shuicheng Yan, and Bo Zhang. Hierarchical part matching for fine-grained visual categorization. In Proceedings of the IEEE International Conference on Computer Vision, pages 1641-1648, 2013.","title":"Hierarchical part matching for fine-grained visual categorization","authors":"Xie","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2015] Lingxi Xie, Jingdong Wang, Bo Zhang, and Qi Tian. Fine-grained image search. IEEE Transactions on Multimedia, 17(5):636-647, 2015.","title":"Fine-grained image search","authors":"Xie","year":"2015","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2016a] Xiaofan Zhang, Feng Zhou, Yuanqing Lin, and Shaoting Zhang. Embedding label structures for fine-grained feature representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1114-1123, 2016.","title":"Embedding label structures for fine-grained feature representation","authors":"Zhang","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2016b] Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. Picking deep filter responses for fine-grained image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1134-1142, 2016.","title":"Picking deep filter responses for fine-grained image recognition","authors":"Zhang","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2921-2929, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}