{"type":"ScholarlyArticle","title":"Collaborative Learning for Weakly Supervised Object Detection","authors":[{"name":"Jiajie Wang","slug":"jiajie-wang","given":"Jiajie","family":"Wang","affiliation":"Cooperative Medianet Innovation Center , Shanghai Jiao Tong University , China"},{"name":"Jiangchao Yao","slug":"jiangchao-yao","given":"Jiangchao","family":"Yao","affiliation":"Cooperative Medianet Innovation Center , Shanghai Jiao Tong University , China"},{"name":"Ya Zhang","slug":"ya-zhang","given":"Ya","family":"Zhang","affiliation":"Cooperative Medianet Innovation Center , Shanghai Jiao Tong University , China"},{"name":"Rui Zhang","slug":"rui-zhang","given":"Rui","family":"Zhang","affiliation":"Cooperative Medianet Innovation Center , Shanghai Jiao Tong University , China"}],"abstract":"Weakly supervised object detection has recently received much attention, since it only requires image-level labels instead of the bounding-box labels consumed in strongly supervised learning. Nevertheless, the save in labeling expense is usually at the cost of model accuracy.In this paper, we propose a simple but effective weakly supervised collaborative learning framework to resolve this problem, which trains a weakly supervised learner and a strongly supervised learner jointly by enforcing partial feature sharing and prediction consistency. For object detection, taking WSDDN-like architecture as weakly supervised detector sub-network and Faster-RCNN-like architecture as strongly supervised detector sub-network, we propose an end-to-end Weakly Supervised Collaborative Detection Network. As there is no strong supervision available to train the Faster-RCNN-like sub-network, a new prediction consistency loss is defined to enforce consistency of predictions between the two sub-networks as well as within the Faster-RCNN-like sub-networks. At the same time, the two detectors are designed to partially share features to further guarantee the model consistency at perceptual level. Extensive experiments on PASCAL VOC 2007 and 2012 data sets have demonstrated the effectiveness of the proposed framework.","doi":"10.24963/ijcai.2018/135","url":"https://ijcai.org/proceedings/2018/135","pdf":"https://ijcai.org/proceedings/2018/0135.pdf","markdown":"https://ijcai.org/proceedings/2018/135.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/135","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":135,"pages":{"first":971,"last":977},"keywords":["Machine Learning: Deep Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Bilen and Vedaldi, 2016] Hakan Bilen and Andrea Vedaldi. Weakly supervised deep detection networks. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 2846-2854, 2016.","title":"Weakly supervised deep detection networks","authors":"Hakan Bilen, Andrea Vedaldi","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bilen et al., 2015] Hakan Bilen, Marco Pedersoli, and Tinne Tuytelaars. Weakly supervised object detection with convex clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1081- 1089, 2015.","title":"Weakly supervised object detection with convex clustering","authors":"Hakan Bilen, Marco Pedersoli, Tinne Tuytelaars","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blum and Mitchell, 1998] Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Compu- tational learning theory, pages 92-100. ACM, 1998.","title":"Combining labeled and unlabeled data with co-training","authors":"Avrim Blum, Tom Mitchell","year":"1998","venue":"Proceedings of the eleventh annual conference on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cinbis et al., 2017] Ramazan Gokberk Cinbis, Jakob Ver- beek, and Cordelia Schmid. Weakly supervised object lo- calization with multi-fold multiple instance learning. IEEE transactions on pattern analysis and machine intelligence, 39(1):189-203, 2017.","title":"Ramazan Gokberk Cinbis, Jakob Verbeek, and Cordelia Schmid. Weakly supervised object localization with multi-fold multiple instance learning","authors":"Cinbis","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deselaers et al., 2012] Thomas Deselaers, Bogdan Alexe, and Vittorio Ferrari. Weakly supervised localization and learning with generic knowledge. International journal of computer vision, 100(3):275-293, 2012.","title":"Weakly supervised localization and learning with generic knowledge","authors":"Thomas Deselaers, Bogdan Alexe, Vittorio Ferrari","year":"2012","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dong et al., 2017] Xuanyi Dong, Deyu Meng, Fan Ma, and Yi Yang. A dual-network progressive approach to weakly supervised object detection. pages 279-287, 2017.","title":"A dual-network progressive approach to weakly supervised object detection","authors":"Dong","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440-1448, 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2014] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep con- volutional networks for visual recognition. In European Conference on Computer Vision, pages 346-361. Springer, 2014.","title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jie et al., 2017] Zequn Jie, Yunchao Wei, Xiaojie Jin, Ji- ashi Feng, and Wei Liu. Deep self-taught learning for weakly supervised object localization. arXiv preprint arXiv:1704.05188, 2017.","title":"Deep self-taught learning for weakly supervised object localization","authors":"Jie","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kantorov et al., 2016] Vadim Kantorov, Maxime Oquab, Minsu Cho, and Ivan Laptev. Contextlocnet: Context- aware deep network models for weakly supervised local- ization. In European Conference on Computer Vision, pages 350-365. Springer, 2016.","title":"Contextlocnet: Contextaware deep network models for weakly supervised localization","authors":"Vadim Kantorov, Maxime Oquab, Minsu Cho, Ivan Laptev","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2016] Dong Li, Jia-Bin Huang, Yali Li, Shengjin Wang, and Ming-Hsuan Yang. Weakly supervised object localization with progressive domain adaptation. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3512-3520, 2016.","title":"Weakly supervised object localization with progressive domain adaptation","authors":"Dong Li, Jia-Bin Huang, Yali Li, Shengjin Wang, Ming-Hsuan Yang","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Redmon and Farhadi, 2016] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. arXiv preprint arXiv:1612.08242, 2016.","title":"Yolo9000: better, faster, stronger","authors":"Farhadi Redmon, Joseph Redmon, Ali Farhadi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. In Advances in neural information processing systems, pages 91-99, 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recogni- tion challenge. International Journal of Computer Vision, 115(3):211-252, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tang et al., 2017] Peng Tang, Xinggang Wang, Xiang Bai, and Wenyu Liu. Multiple instance detection network with online instance classifier refinement. arXiv preprint arXiv:1704.00138, 2017.","title":"Multiple instance detection network with online instance classifier refinement","authors":"Peng Tang, Xinggang Wang, Xiang Bai, Wenyu Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Thomee et al., 2015] Bart Thomee, David A Shamma, Ger- ald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li Jia Li. The new data and new challenges in multimedia research. Communications of the Acm, 59(2):64-73, 2015.","title":"The new data and new challenges in multimedia research","authors":"Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, Li Jia, Li","year":"2015","venue":"Communications of the Acm","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Uijlings et al., 2013] Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders. Se- lective search for object recognition. International journal of computer vision, 104(2):154-171, 2013.","title":"Selective search for object recognition","authors":"Uijlings","year":"2013","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2014] Chong Wang, Weiqiang Ren, Kaiqi Huang, and Tieniu Tan. Weakly supervised object local- ization with latent category learning. In European Confer- ence on Computer Vision, pages 431-445. Springer, 2014.","title":"Weakly supervised object localization with latent category learning","authors":"Chong Wang, Weiqiang Ren, Kaiqi Huang, Tieniu Tan","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wu et al., 2015] Jiajun Wu, Yinan Yu, Chang Huang, and Kai Yu. Deep multiple instance learning for image classi- fication and auto-annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3460-3469, 2015.","title":"Deep multiple instance learning for image classification and auto-annotation","authors":"Jiajun Wu, Yinan Yu, Chang Huang, Kai Yu","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yan et al., 2017] Ziang Yan, Jian Liang, Weishen Pan, Jin Li, and Changshui Zhang. Weakly-and semi-supervised object detection with expectation-maximization algorithm. arXiv preprint arXiv:1702.08740, 2017.","title":"Weakly-and semi-supervised object detection with expectation-maximization algorithm","authors":"Ziang Yan, Jian Liang, Weishen Pan, Jin Li, Changshui Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2006] Cha Zhang, John C Platt, and Paul A Vi- ola. Multiple instance boosting for object detection. In Advances in neural information processing systems, pages 1417-1424, 2006.","title":"Multiple instance boosting for object detection","authors":"Cha Zhang, John C Platt, Paul A Viola","year":"2006","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}