{"type":"ScholarlyArticle","title":"Generalized Zero-Shot Vehicle Detection in Remote Sensing Imagery via Coarse-to-Fine Framework","authors":[{"name":"Hong Chen","slug":"hong-chen","given":"Hong","family":"Chen","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China"},{"name":"Yongtan Luo","slug":"yongtan-luo","given":"Yongtan","family":"Luo","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China"},{"name":"Liujuan Cao","slug":"liujuan-cao","given":"Liujuan","family":"Cao","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China"},{"name":"Baochang Zhang","slug":"baochang-zhang","given":"Baochang","family":"Zhang","affiliation":"School of Automation Science and Electrical Engineering , Beihang University , China"},{"name":"Guodong Guo","slug":"guodong-guo","given":"Guodong","family":"Guo","affiliation":"Institute of Deep Learning , Baidu Research; National Engineering Laboratory for Deep Learning Technology and Application"},{"name":"Cheng Wang","slug":"cheng-wang","given":"Cheng","family":"Wang","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China"},{"name":"Jonathan Li","slug":"jonathan-li","given":"Jonathan","family":"Li","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China"},{"name":"Rongrong Ji","slug":"rongrong-ji","given":"Rongrong","family":"Ji","affiliation":"Fujian Key Laboratory of Sensing and Computing for Smart City , School of Information Science and Engineering , Xiamen University , China; Peng Cheng Laboratory , China"}],"abstract":"Vehicle detection and recognition in remote sensing images are challenging, especially when only limited training data are available to accommodate various target categories. In this paper, we introduce a novel coarse-to-fine framework, which decomposes vehicle detection into segmentation-based vehicle localization and generalized zero-shot vehicle classification. Particularly, the proposed framework can well handle the problem of generalized zero-shot vehicle detection, which is challenging due to the requirement of recognizing vehicles that are even unseen during training. Specifically, a hierarchical DeepLab v3 model is proposed in the framework, which fully exploits fine-grained features to locate the target on a pixel-wise level, then recognizes vehicles in a coarse-grained manner. Additionally, the hierarchical DeepLab v3 model is beneficially compatible to combine the generalized zero-shot recognition. To the best of our knowledge, there is no publically available dataset to test comparative methods, we therefore construct a new dataset to fill this gap of evaluation. The experimental results show that the proposed framework yields promising results on the imperative yet difficult task of zero-shot vehicle detection and recognition.","doi":"10.24963/ijcai.2019/97","url":"https://ijcai.org/proceedings/2019/97","pdf":"https://ijcai.org/proceedings/2019/0097.pdf","markdown":"https://ijcai.org/proceedings/2019/97.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/97","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":97,"pages":{"first":687,"last":693},"keywords":["Computer Vision: Computer Vision","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[],"references":[{"position":1,"raw":"[Ankan Bansal, 2018] Gaurav Sharma Rama Chellappa Ajay Divakaran Ankan Bansal, Karan Sikka. Zero-Shot Object Detection. arXiv preprint arXiv:1804.04340, 2018.","title":"Zero-Shot Object Detection","authors":"Gaurav Sharma, Rama Chellappa, Ajay Divakaran, Ankan Bansal, Karan Sikka","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2016] Liujuan Cao, Feng Luo, Li Chen, Yihan Sheng, Haibin Wang, Cheng Wang, and Rongrong Ji. Weakly supervised vehicle detection in satellite images via multi-instance discriminative learning. Pattern Recogni- tion, 2016.","title":"Weakly supervised vehicle detection in satellite images via multi-instance discriminative learning","authors":"Liujuan Cao, Feng Luo, Li Chen, Yihan Sheng, Haibin Wang, Cheng Wang, Rongrong Ji","year":"2016","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Liang Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. 2017.","title":"Rethinking atrous convolution for semantic image segmentation","authors":"George Liang Chieh Chen, Florian Papandreou, Hartwig Schroff, Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018a] L. C. Chen, G Papandreou, I Kokkinos, K Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous con- volution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834- 848, 2018.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Chen","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2018b] Liang Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. 2018.","title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chollet, 2017] Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251-1258, 2017.","title":"Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions","authors":"Chollet","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Demirel et al., 2018] Berkan Demirel, Ramazan Gokberk Cinbis, and Nazli Ikizler-Cinbis. Zero-shot object de- tection by hybrid region embedding. arXiv preprint arXiv:1805.06157, 2018.","title":"Zero-shot object detection by hybrid region embedding","authors":"Berkan Demirel, Ramazan Gokberk Cinbis, Nazli Ikizler-Cinbis","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Girshick et al., 2016] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Region-based convolutional networks for accurate object detection and segmentation. 2016.","title":"Region-based convolutional networks for accurate object detection and segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. arXiv preprint arXiv:1512.03385, 2015.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2017] Huajie Jiang, Ruiping Wang, Shiguang Shan, Yi Yang, and Xilin Chen. Learning discriminative latent attributes for zero-shot classification. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 4223-4232, 2017.","title":"Learning discriminative latent attributes for zero-shot classification","authors":"Huajie Jiang, Ruiping Wang, Shiguang Shan, Yi Yang, Xilin Chen","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kemker et al., 2018] Ronald Kemker, Carl Salvaggio, and Christopher Kanan. Algorithms for semantic segmenta- tion of multispectral remote sensing imagery using deep learning. Isprs Journal of Photogrammetry and Remote Sensing, page S0924271618301229, 2018.","title":"Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning","authors":"Ronald Kemker, Carl Salvaggio, Christopher Kanan","year":"2018","venue":"Isprs Journal of Photogrammetry and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lampert et al., 2014] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 36(3):453-465, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2017] Aoxue Li, Zhiwu Lu, Liwei Wang, Tao Xiang, and Ji Rong Wen. Zero-shot scene classifica- tion for high spatial resolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 55(7):4157-4167, 2017.","title":"Zero-shot scene classification for high spatial resolution remote sensing images","authors":"Aoxue Li, Zhiwu Lu, Liwei Wang, Tao Xiang, Ji Rong, Wen","year":"2017","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Yansong Liu, Sankaranarayanan Pira- manayagam, Sildomar T. Monteiro, and Eli Saber. Dense semantic labeling of very-high-resolution aerial imagery and lidar with fully-convolutional neural networks and higher-order crfs. In IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops, 2017.","title":"Dense semantic labeling of very-high-resolution aerial imagery and lidar with fully-convolutional neural networks and higher-order crfs","authors":"Yansong Liu, Sankaranarayanan Piramanayagam, T Sildomar, Eli Monteiro, Saber","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2014] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4):640-651, 2014.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Redmon and Farhadi, 2017] Joseph Redmon and Ali Farhadi. Yolo9000: Better, faster, stronger. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.","title":"Yolo9000: Better, faster, stronger","authors":"Joseph Redmon, Ali Farhadi","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Romera-Paredes and Torr, 2015] Bernardino Romera- Paredes and Philip Torr. An embarrassingly simple approach to zero-shot learning. In International Confer- ence on Machine Learning, pages 2152-2161, 2015.","title":"Bernardino Romera-Paredes and Philip Torr. An embarrassingly simple approach to zero-shot learning","authors":"Romera-Paredes, Torr","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Soleimani et al., 2018] Amir Soleimani, Nasser M Nasrabadi, Elias Griffith, Jason Ralph, and Simon Maskell. Convolutional neural networks for aerial vehicle detection and recognition. 2018.","title":"Convolutional neural networks for aerial vehicle detection and recognition","authors":"Amir Soleimani, M Nasser, Elias Nasrabadi, Jason Griffith, Simon Ralph, Maskell","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sumbul et al., 2017] Gencer Sumbul, Ramazan Gokberk Cinbis, and Selim Aksoy. Fine-grained object recogni- tion and zero-shot learning in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, PP(99):1-10, 2017.","title":"Fine-grained object recognition and zero-shot learning in remote sensing imagery","authors":"Gencer Sumbul, Ramazan Gokberk Cinbis, Selim Aksoy","year":"2017","venue":"IEEE Transactions on Geoscience and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xian et al., 2016] Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh Nguyen, Matthias Hein, and Bernt Schiele. Latent embeddings for zero-shot classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 69-77, 2016.","title":"Latent embeddings for zero-shot classification","authors":"Xian","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xian et al., 2017] Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-shot learning-the good, the bad and the ugly. arXiv preprint arXiv:1703.04394, 2017.","title":"Zero-shot learning-the good, the bad and the ugly","authors":"Xian","year":"2017","venue":"Yongqin Xian","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2018] Li Yan, Junge Zhang, Jianguo Zhang, and Kaiqi Huang. Discriminative learning of latent features for zero-shot recognition. 2018.","title":"Discriminative learning of latent features for zero-shot recognition","authors":"Li Yan, Junge Zhang, Jianguo Zhang, Kaiqi Huang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2017] Li Zhang, Tao Xiang, and Shaogang Gong. Learning a deep embedding model for zero-shot learning. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, 2017.","title":"Learning a deep embedding model for zero-shot learning","authors":"Li Zhang, Tao Xiang, Shaogang Gong","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2016] Hengshuang Zhao, Jianping Shi, Xiao- juan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. 2016.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2018] Pengkai Zhu, Hanxiao Wang, Tolga Bolukbasi, and Venkatesh Saligrama. Zero-shot detection. 2018.","title":"Zero-shot detection","authors":"Zhu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}