{"type":"ScholarlyArticle","title":"Deep Descriptor Transforming for Image Co-Localization","authors":[{"name":"Xiu-Shen Wei","slug":"xiu-shen-wei","given":"Xiu-Shen","family":"Wei","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Chen-Lin Zhang","slug":"chen-lin-zhang","given":"Chen-Lin","family":"Zhang","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Yao Li","slug":"yao-li","given":"Yao","family":"Li","affiliation":"The University of Adelaide , Adelaide , Australia"},{"name":"Chen-Wei Xie","slug":"chen-wei-xie","given":"Chen-Wei","family":"Xie","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Jianxin Wu","slug":"jianxin-wu","given":"Jianxin","family":"Wu","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"},{"name":"Chunhua Shen","slug":"chunhua-shen","given":"Chunhua","family":"Shen","affiliation":"The University of Adelaide , Adelaide , Australia"},{"name":"Zhi-Hua Zhou","slug":"zhi-hua-zhou","given":"Zhi-Hua","family":"Zhou","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing , China"}],"abstract":"Reusable model design becomes desirable with the rapid expansion of machine learning applications. In this paper, we focus on the reusability of pre-trained deep convolutional models. Specifically, different from treating pre-trained models as feature extractors, we reveal more treasures beneath convolutional layers, i.e., the convolutional activations could act as a detector for the common object in the image co-localization problem. We propose a simple but effective method, named Deep Descriptor Transforming (DDT), for evaluating the correlations of descriptors and then obtaining the category-consistent regions, which can accurately locate the common object in a set of images. Empirical studies validate the effectiveness of the proposed DDT method. On benchmark image co-localization datasets, DDT consistently outperforms existing state-of-the-art methods by a large margin. Moreover, DDT also demonstrates good generalization ability for unseen categories and robustness for dealing with noisy data.","doi":"10.24963/ijcai.2017/425","url":"https://ijcai.org/proceedings/2017/425","pdf":"https://ijcai.org/proceedings/2017/0425.pdf","markdown":"https://ijcai.org/proceedings/2017/425.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/425","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":425,"pages":{"first":3048,"last":3054},"keywords":["Machine Learning: Deep Learning","Robotics and Vision: Vision and Perception","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Bilen et al., 2015] H. Bilen, M. Pedersoli, and T. Tuytelaars. Weakly supervised object detection with convex clustering. In CVPR, pages 1081-1089, 2015.","title":"Weakly supervised object detection with convex clustering","authors":"H Bilen, M Pedersoli, T Tuytelaars","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] M. Chen, W. Li, W. Zhang, and X.-G. Wang. Dimensionality reduction with generalized linear models. In IJCAI, pages 1267-1272, 2013.","title":"Dimensionality reduction with generalized linear models","authors":"M Chen, W Li, W Zhang, X.-G Wang","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/191"},{"position":3,"raw":"[Cho et al., 2015] M. Cho, S. Kwak, C. Schmid, and J. Ponce. Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals. In CVPR, pages 1201-1210, 2015.","title":"Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals","authors":"M Cho, S Kwak, C Schmid, J Ponce","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cinbis et al., 2015] R. G. Cinbis, J. J. Verbeek, and C. Schmid. Multi-fold MIL training for weakly supervised object localization. In CVPR, pages 2409-2416, 2015.","title":"Multi-fold MIL training for weakly supervised object localization","authors":"R G Cinbis, J J Verbeek, C Schmid","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Davidson, 2009] I. Davidson. Knowledge driven dimension reduction for clustering. In IJCAI, pages 1034-1039, 2009.","title":"Knowledge driven dimension reduction for clustering","authors":"; I Davidson, Davidson","year":"2009","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2009/175"},{"position":6,"raw":"[Everingham et al., 2015] M. Everingham, S. M. Ali Eslami, L. Van Gool, C. K. L. Williams, J. Winn, and A. Zisser- man. The PASCAL visual object classes challenge: A retrospective. IJCV, 111(1):98-136, 2015.","title":"The PASCAL visual object classes challenge: A retrospective","authors":"M Everingham, S M Ali Eslami, L Van Gool, C K L Williams, J Winn, A Zisserman","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ghodrati et al., 2015] A. Ghodrati, A. Diba, M. Pedersoli, T. Tuytelaars, and L. Van Gool. Deepproposal: Hunting objects by cascading deep convolutional layers. In ICCV, pages 2578-2586, 2015.","title":"Deepproposal: Hunting objects by cascading deep convolutional layers","authors":"A Ghodrati, A Diba, M Pedersoli, T Tuytelaars, L Van Gool","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gu et al., 2011] Q.-Q. Gu, Z.-H. Li, and J.-W. Han. Joint feature selection and subspace learning. In IJCAI, pages 1294-1299, 2011.","title":"Joint feature selection and subspace learning","authors":"Q.-Q Gu, Z.-H Li, J.-W Han","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/219"},{"position":9,"raw":"[Joulin et al., 2010] A. Joulin, F. Bach, and J. Ponce. Dis- criminative clustering for image co-segmentation. In ICCV, pages 2984-2991, 2010.","title":"Discriminative clustering for image co-segmentation","authors":"A Joulin, F Bach, J Ponce","year":"2010","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Joulin et al., 2012] A. Joulin, F. Bach, and J. Ponce. Multi- class co-segmentation. In ICCV, pages 139-150, 2012.","title":"Multiclass co-segmentation","authors":"A Joulin, F Bach, J Ponce","year":"2012","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joulin et al., 2014] A. Joulin, K. Tang, and L. Fei-Fei. Ef- ficient image and video co-localization with Frank-Wolfe algorithm. In ECCV, pages 253-268, 2014.","title":"Efficient image and video co-localization with Frank-Wolfe algorithm","authors":"A Joulin, K Tang, L Fei-Fei","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kim et al., 2011] G. Kim, E. P. Xing, L. Fei-Fei, and T. Kanade. Distributed co-segmentation via submodular optimization on anisotropic diffusion. In ICCV, pages 169- 176, 2011.","title":"Distributed co-segmentation via submodular optimization on anisotropic diffusion","authors":"G Kim, E P Xing, L Fei-Fei, T Kanade","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2016] Y. Li, L. Liu, C. Shen, and A. V. D. Hengel. Image co-localization by mimicking a good detector's con- fidence score distribution. In ECCV, pages 19-34, 2016.","title":"Image co-localization by mimicking a good detector's confidence score distribution","authors":"Y Li, L Liu, C Shen, A V D Hengel","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nie et al., 2011] F.-P. Nie, H. Huang, C. Ding, D.-J. Luo, and H. Wang. Robust principal component analysis with non-greedy ℓ 1 -norm maximization. In IJCAI, pages 1433- 1438, 2011.","title":"Robust principal component analysis with non-greedy ℓ 1 -norm maximization","authors":"F.-P Nie, H Huang, C Ding, D.-J Luo, H Wang","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"K. Pearson. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2(11):559-572, 1901.","title":"On lines and planes of closest fit to systems of points in space","authors":"K Pearson","year":"1901","venue":"Philosophical Magazine","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ren et al., 2016] W. Ren, K. Huang, D. Tao, and T. Tan. Weakly supervised large scale object localization with mul- tiple instance learning and bag splitting. IEEE TPAMI, 38(2):405-416, 2016.","title":"Weakly supervised large scale object localization with multiple instance learning and bag splitting","authors":"W Ren, K Huang, D Tao, T Tan","year":"2016","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rubinstein et al., 2013] M. Rubinstein, A. Joulin, J. Kopf, and C. Liu. Unsupervised joint object discovery and seg- mentation in internet images. In CVPR, pages 1939-1946, 2013.","title":"Unsupervised joint object discovery and segmentation in internet images","authors":"M Rubinstein, A Joulin, J Kopf, C Liu","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Russakovsky et al., 2015] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Im- ageNet large scale visual recognition challenge. IJCV, 115(3):211-252, 2015.","title":"Im-ageNet large scale visual recognition challenge","authors":"O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, Z Huang, A Karpathy, A Khosla, M Bernstein, A C Berg, L Fei-Fei","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2013] Z. Shi, T. M. Hospedales, and T. Xiang. Bayesian joint topic modelling for weakly supervised object localisation. In ICCV, pages 2984-2991, 2013.","title":"Bayesian joint topic modelling for weakly supervised object localisation","authors":"Z Shi, T M Hospedales, T Xiang","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Simonyan and Zisserman, 2015] K. Simonyan and A. Zis- serman. Very deep convolutional networks for large-scale image recognition. In ICLR, pages 1-14, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Siva and Xiang, 2011] P. Siva and T. Xiang. Weakly super- vised object detector learning with model drift detection. In CVPR, pages 343-350, 2011.","title":"Weakly supervised object detector learning with model drift detection","authors":"P Siva, T Xiang","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang et al., 2014] K. Tang, A. Joulin, L. Li, and L. Fei-Fei. Co-localization in real-world images. In CVPR, pages 1464-1471, 2014.","title":"Co-localization in real-world images","authors":"K Tang, A Joulin, L Li, L Fei-Fei","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vedaldi and Lenc, 2015] A. Vedaldi and K. Lenc. MatCon- vNet -convolutional neural networks for MATLAB. In ACM MM, pages 689-692, 2015.","title":"MatCon-vNet -convolutional neural networks for MATLAB","authors":"A Vedaldi, K Lenc","year":"2015","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2014] C. Wang, W. Ren, K. Huang, and T. Tan. Weakly supervised object localization with latent category learning. In ECCV, pages 431-445, 2014.","title":"Weakly supervised object localization with latent category learning","authors":"C Wang, W Ren, K Huang, T Tan","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2015] X. Wang, Z. Zhu, C. Yao, and X. Bai. Relaxed multiple-instance SVM with application to object discovery. In ICCV, pages 1224-1232, 2015.","title":"Relaxed multiple-instance SVM with application to object discovery","authors":"X Wang, Z Zhu, C Yao, X Bai","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wei et al., 2017] X.-S. Wei, J.-H. Luo, J. Wu, and Z.-H. Zhou. Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE TIP, 26(6):2868-2881, 2017.","title":"Selective convolutional descriptor aggregation for fine-grained image retrieval","authors":"Wei","year":"2017","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2017] Y. Yang, D.-C. Zhan, Y. Fan, Y. Jiang, and Z.-H. Zhou. Deep learning for fixed model reuse. In AAAI, pages 2831-2837, 2017.","title":"Deep learning for fixed model reuse","authors":"Y Yang, D.-C Zhan, Y Fan, Y Jiang, Z.-H Zhou","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2009] T. Zhang, D. Tao, X. Li, and J. Yang. Patch alignment for dimensionality reduction. IEEE TKDE, 21(9):1299-1313, 2009.","title":"Patch alignment for dimensionality reduction","authors":"T Zhang, D Tao, X Li, J Yang","year":"2009","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2013] L. Zhang, L. Zhang, D. Tao, and X. Huang. Tensor discriminative locality alignment for hy- perspectral image spectral-spatial feature extraction. IEEE TGRS, 51(1):242-256, 2013.","title":"Tensor discriminative locality alignment for hyperspectral image spectral-spatial feature extraction","authors":"L Zhang, L Zhang, D Tao, X Huang","year":"2013","venue":"IEEE TGRS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2016] D. Zhang, D. Meng, L. Zhao, and J. Han. Bridging saliency detection to weakly supervised object detection based on self-paced curriculum. In IJCAI, pages 3538-3544, 2016.","title":"Bridging saliency detection to weakly supervised object detection based on self-paced curriculum","authors":"D Zhang, D Meng, L Zhao, J Han","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhao and Fu, 2015] H. Zhao and Y. Fu. Semantic single video segmentation with robust graph representation. In IJCAI, pages 2219-2225, 2015.","title":"Semantic single video segmentation with robust graph representation","authors":"H Zhao, Y Fu","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/314"},{"position":32,"raw":"[Zhou, 2016] Z.-H. Zhou. Learnware: On the future of ma- chine learning. Frontiers of Computer Science, 10(4):589- 590, 2016.","title":"Learnware: On the future of machine learning","authors":"Z.-H Zhou","year":"2016","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}