{"type":"ScholarlyArticle","title":"Domain-Adaptive Self-Supervised Face & Body Detection in Drawings","authors":[{"name":"Barış Batuhan Topal","slug":"bar-s-batuhan-topal","given":"Barış Batuhan","family":"Topal","affiliation":"Department of Computer Engineering , KUIS AI Center , Koc ¸University"},{"name":"Deniz Yuret","slug":"deniz-yuret","given":"Deniz","family":"Yuret","affiliation":"Department of Computer Engineering , KUIS AI Center , Koc ¸University"},{"name":"Tevfik Metin Sezgin","slug":"tevfik-metin-sezgin","given":"Tevfik Metin","family":"Sezgin","affiliation":"Department of Computer Engineering , KUIS AI Center , Koc ¸University"}],"abstract":"Drawings are powerful means of pictorial abstraction and communication. Understanding diverse forms of drawings, including digital arts, cartoons, and comics, has been a major problem of interest for the computer vision and computer graphics communities. Although there are large amounts of digitized drawings from comic books and cartoons, they contain vast stylistic variations, which necessitate expensive manual labeling for training domain-specific recognizers. In this work, we show how self-supervised learning, based on a teacher-student network with a modified student network update design, can be used to build face and body detectors. Our setup allows exploiting large amounts of unlabeled data from the target domain when labels are provided for only a small subset of it. We further demonstrate that style transfer can be incorporated into our learning pipeline to bootstrap detectors using a vast amount of out-of-domain labeled images from natural images (i.e., images from the real world). Our combined architecture yields detectors with state-of-the-art (SOTA) and near-SOTA performance using minimal annotation effort. Our code can be accessed from https://github.com/barisbatuhan/DASS_Detector.","doi":"10.24963/ijcai.2023/159","url":"https://ijcai.org/proceedings/2023/159","pdf":"https://ijcai.org/proceedings/2023/0159.pdf","markdown":"https://ijcai.org/proceedings/2023/159.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/159","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":159,"pages":{"first":1432,"last":1439},"keywords":["Machine Learning: ML: Self-supervised Learning","Computer Vision: CV: Recognition (object detection, categorization)","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Bochkovskiy et al., 2020] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. ArXiv, abs/2004.10934, 2020.","title":"Yolov4: Optimal speed and accuracy of object detection","authors":"Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2019] Qi Cai, Yingwei Pan, Chong-Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao. Exploring ob- ject relation in mean teacher for cross-domain detection. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11449-11458, 2019.","title":"Exploring object relation in mean teacher for cross-domain detection","authors":"Qi Cai, Yingwei Pan, Chong-Wah Ngo, Xinmei Tian, Lingyu Duan, Ting Yao","year":"2019","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Yang Chen, Yu-Kun Lai, and Yong-Jin Liu. Cartoongan: Generative adversarial networks for photo cartoonization. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition, pages 9465-9474, 2018.","title":"Cartoongan: Generative adversarial networks for photo cartoonization","authors":"Yang Chen, Yu-Kun Lai, Yong-Jin Liu","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2021] Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain object detection. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4089-4099, 2021.","title":"Unbiased mean teacher for cross-domain object detection","authors":"Jinhong Deng, Wen Li, Yuhua Chen, Lixin Duan","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ge et al., 2021] Zheng Ge, Songtao Liu, Feng Wang, Zem- ing Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430, 2021.","title":"Yolox: Exceeding yolo series in 2021","authors":"Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hicsonmez et al., 2020] Samet Hicsonmez, Nermin Samet, Emre Akbas, and Pinar Duygulu. GANILLA: generative adversarial networks for image to illustration translation. CoRR, abs/2002.05638, 2020.","title":"GANILLA: generative adversarial networks for image to illustration translation","authors":"Samet Hicsonmez, Nermin Samet, Emre Akbas, Pinar Duygulu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Inoue et al., 2018] Naoto Inoue, Ryosuke Furuta, T. Ya- masaki, and Kiyoharu Aizawa. Cross-domain weakly- supervised object detection through progressive domain adaptation. 2018 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5001-5009, 2018.","title":"Cross-domain weaklysupervised object detection through progressive domain adaptation","authors":"Naoto Inoue, Ryosuke Furuta, T Yamasaki, Kiyoharu Aizawa","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Iyyer et al., 2016] Mohit Iyyer, Varun Manjunatha, Anu- pam Guha, Yogarshi Vyas, Jordan L. Boyd-Graber, Hal Daumé, and Larry S. Davis. The amazing mysteries of the gutter: Drawing inferences between panels in comic book narratives. 2017 IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 6478-6487, 2016.","title":"The amazing mysteries of the gutter: Drawing inferences between panels in comic book narratives","authors":"Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan L Boyd-Graber, Hal Daumé, Larry S Davis","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jiang et al., 2022] Junguang Jiang, Baixu Chen, Jianmin Wang, and Mingsheng Long. Decoupled adaptation for cross-domain object detection. In International Confer- ence on Learning Representations, 2022.","title":"Decoupled adaptation for cross-domain object detection","authors":"Jiang","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2021] Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi- supervised object detection. In Proceedings of the Inter- national Conference on Learning Representations (ICLR), 2021.","title":"Unbiased teacher for semisupervised object detection","authors":"Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, Peter Vajda","year":"2021","venue":"Proceedings of the International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Matsui et al., 2017] Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, 76(20):21811-21838, 2017.","title":"Sketch-based manga retrieval using manga109 dataset","authors":"Matsui","year":"2017","venue":"Multimedia Tools and Applications","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nguyen et al., 2018] Nhu-Van Nguyen, Christophe Rigaud, and Jean-Christophe Burie. Digital comics image indexing based on deep learning. Journal of Imaging, 4(7), 2018.","title":"Digital comics image indexing based on deep learning","authors":"Christophe Nhu-Van Nguyen, Jean-Christophe Rigaud, Burie","year":"2018","venue":"Journal of Imaging","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ogawa et al., 2018] Toru Ogawa, Atsushi Otsubo, Rei Narita, Yusuke Matsui, T. Yamasaki, and Kiyoharu Aizawa. Object detection for comics using manga109 an- notations. ArXiv, abs/1803.08670, 2018.","title":"Object detection for comics using manga109 annotations","authors":"Toru Ogawa, Atsushi Otsubo, Rei Narita, Yusuke Matsui, T Yamasaki, Kiyoharu Aizawa","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shrivastava et al., 2016] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 761-769, 2016.","title":"Training region-based object detectors with online hard example mining","authors":"Abhinav Shrivastava, Abhinav Gupta, Ross Girshick","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang and Yu, 2020] Xinrui Wang and Jinze Yu. Learning to cartoonize using white-box cartoon representations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.","title":"Learning to cartoonize using white-box cartoon representations","authors":"Wang, Yu ; ] Xinrui Wang, Jinze Yu","year":"2020","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Xu et al., 2021] Mengde Xu, Zheng Zhang, Han Hu, Jian- feng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2021.","title":"End-to-end semi-supervised object detection with soft teacher","authors":"Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, Zicheng Liu","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Xu et al., 2022] Yunqiu Xu, Yifan Sun, Zongxin Yang, Ji- axu Miao, and Yi Yang. H 2 FA R-CNN: Holistic and hi- erarchical feature alignment for cross-domain weakly su- pervised object detection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14329-14339, 2022.","title":"H 2 FA R-CNN: Holistic and hierarchical feature alignment for cross-domain weakly supervised object detection","authors":"Yunqiu Xu, Yifan Sun, Zongxin Yang, Jiaxu Miao, Yi Yang","year":"2022","venue":"Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yang et al., 2016] Shuo Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Wider face: A face detection bench- mark. In IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), 2016.","title":"Wider face: A face detection benchmark","authors":"Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2016","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2020a] Bin Zhang, Jian Li, Yabiao Wang, Zhipeng Cui, Yili Xia, Chengjie Wang, Jilin Li, and Feiyue Huang. Acfd: Asymmetric cartoon face detector. ArXiv, abs/2007.00899, 2020.","title":"Acfd: Asymmetric cartoon face detector","authors":"Zhang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2020b] Hongkai Zhang, Hong Chang, Bing- peng Ma, Naiyan Wang, and Xilin Chen. Dynamic R- CNN: Towards high quality object detection via dynamic training. In ECCV, 2020.","title":"Dynamic R-CNN: Towards high quality object detection via dynamic training","authors":"Zhang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zheng et al., 2020] Yi Zheng, Yifan Zhao, Mengyuan Ren, He Yan, Xiangju Lu, Junhui Liu, and Jia Li. Cartoon face recognition: A benchmark dataset. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2264-2272, 2020.","title":"Cartoon face recognition: A benchmark dataset","authors":"Yi Zheng, Yifan Zhao, Mengyuan Ren, He Yan, Xiangju Lu, Junhui Liu, Jia Li","year":"2020","venue":"Proceedings of the 28th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Computer Vision (ICCV), 2017 IEEE International Conference on, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A Efros","year":"2017","venue":"Computer Vision (ICCV), 2017 IEEE International Conference on","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}