{"type":"ScholarlyArticle","title":"CoSegNet: Image Co-segmentation using a Conditional Siamese Convolutional Network","authors":[{"name":"Sayan Banerjee","slug":"sayan-banerjee","given":"Sayan","family":"Banerjee","affiliation":"Indian Institute of Technology Bombay , India"},{"name":"Avik Hati","slug":"avik-hati","given":"Avik","family":"Hati","affiliation":"Istituto Italiano di Tecnologia , Genova , Italy"},{"name":"Subhasis Chaudhuri","slug":"subhasis-chaudhuri","given":"Subhasis","family":"Chaudhuri","affiliation":"Indian Institute of Technology Bombay , India"},{"name":"Rajbabu Velmurugan","slug":"rajbabu-velmurugan","given":"Rajbabu","family":"Velmurugan","affiliation":"Indian Institute of Technology Bombay , India"}],"abstract":"The objective in image co-segmentation is to jointly\r\nsegment unknown common objects from a given\r\nset of images. In this paper, we propose a novel\r\ndeep convolution neural network based end-to-end\r\nco-segmentation model. It is composed of a metric\r\nlearning and decision network leading to a novel\r\nconditional siamese encoder-decoder network for\r\nestimating a co-segmentation mask. The role of the\r\nmetric learning network is to find an optimum latent\r\nfeature space where objects of the same class\r\nare closer and that of different classes are separated\r\nby a certain margin. Depending on the\r\nextracted features, the decision network decides\r\nwhether input images have common objects or not\r\nand the encoder-decoder network produces a cosegmentation\r\nmask accordingly. Key aspects of the\r\narchitecture are as follows. First, it is completely\r\nclass agnostic and does not require any semantic\r\ninformation. Second, in addition to producing\r\nmasks, the decoder network also learns similarity\r\nacross image pairs that improves co-segmentation\r\nsignificantly. Experimental results reflect an excellent\r\nperformance of our method compared to state of-the-art\r\nmethods on challenging co-segmentation\r\ndatasets.","doi":"10.24963/ijcai.2019/95","url":"https://ijcai.org/proceedings/2019/95","pdf":"https://ijcai.org/proceedings/2019/0095.pdf","markdown":"https://ijcai.org/proceedings/2019/95.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/95","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":95,"pages":{"first":673,"last":679},"keywords":["Computer Vision: Computer Vision","Machine Learning Applications: Applications of Supervised Learning","Computer Vision: 2D and 3D Computer Vision","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Batra et al., 2010] Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen. iCoseg: Interac- tive co-segmentation with intelligent scribble guidance. In IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 3169-3176. IEEE, 2010.","title":"iCoseg: Interactive co-segmentation with intelligent scribble guidance","authors":"Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, Tsuhan Chen","year":"2010","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Batra et al., 2011] Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen. Interactively co-segmentating topically related images with intelligent scribble guidance. International Journal of Computer Vi- sion, 93(3):273-292, 2011.","title":"Interactively co-segmentating topically related images with intelligent scribble guidance","authors":"Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, Tsuhan Chen","year":"2011","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2014] Xinlei Chen, Abhinav Shrivastava, and Abhinav Gupta. Enriching visual knowledge bases via ob- ject discovery and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2035-2042. IEEE, 2014.","title":"Enriching visual knowledge bases via object discovery and segmentation","authors":"Chen","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018] Hong Chen, Yifei Huang, and Hideki Nakayama. Semantic aware attention based deep object co-segmentation. arXiv preprint arXiv:1810.06859, 2018.","title":"Semantic aware attention based deep object co-segmentation","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong et al., 2015] Xingping Dong, Jianbing Shen, Ling Shao, and Ming-Hsuan Yang. Interactive cosegmentation using global and local energy optimization. IEEE Trans- actions on Image Processing, 24(11):3966-3977, 2015.","title":"Interactive cosegmentation using global and local energy optimization","authors":"Xingping Dong, Jianbing Shen, Ling Shao, Ming-Hsuan Yang","year":"2015","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisser- man. The pascal visual object classes (voc) challenge. In- ternational Journal of Computer Vision, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Faktor and Irani, 2013] Alon Faktor and Michal Irani. Co- segmentation by composition. In IEEE International Con- ference on Computer Vision (ICCV), pages 1297-1304. IEEE, 2013.","title":"Cosegmentation by composition","authors":"Alon Faktor, Michal Irani","year":"2013","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hati et al., 2018] Avik Hati, Subhasis Chaudhuri, and Ra- jbabu Velmurugan. Co-segmentation of non-homogeneous image sets. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 266-270, Oct 2018.","title":"Co-segmentation of non-homogeneous image sets","authors":"Avik Hati, Subhasis Chaudhuri, Rajbabu Velmurugan","year":"2018","venue":"25th IEEE International Conference on Image Processing (ICIP)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hsu et al., 2018] Kuang-Jui Hsu, Yen-Yu Lin, and Yung-Yu Chuang. Co-attention CNNs for unsupervised object co- segmentation. In Proceedings of the International Joint Conference on Artificial Intelligence IJCAI, pages 748- 756, 7 2018.","title":"Co-attention CNNs for unsupervised object cosegmentation","authors":"Kuang-Jui Hsu, Yen-Yu Lin, Yung-Yu Chuang","year":"2018","venue":"Proceedings of the International Joint Conference on Artificial Intelligence IJCAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Joulin et al., 2010] Armand Joulin, Francis Bach, and Jean Ponce. Discriminative clustering for image co- segmentation. In IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 1943-1950. IEEE, 2010.","title":"Discriminative clustering for image cosegmentation","authors":"Armand Joulin, Francis Bach, Jean Ponce","year":"2010","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joulin et al., 2012] Armand Joulin, Francis Bach, and Jean Ponce. Multi-class cosegmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 542-549. IEEE, 2012.","title":"Multi-class cosegmentation","authors":"Armand Joulin, Francis Bach, Jean Ponce","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018] Weihao Li, Omid Hosseini Jafari, and Carsten Rother. Deep object co-segmentation. arXiv preprint arXiv:1804.06423, 2018.","title":"Deep object co-segmentation","authors":"Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Quan et al., 2016] Rong Quan, Junwei Han, Dingwen Zhang, and Feiping Nie. Object co-segmentation via graph optimized-flexible manifold ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 687-695, 2016.","title":"Object co-segmentation via graph optimized-flexible manifold ranking","authors":"Rong Quan, Junwei Han, Dingwen Zhang, Feiping Nie","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rother et al., 2006] Carsten Rother, Tom Minka, Andrew Blake, and Vladimir Kolmogorov. Cosegmentation of im- age pairs by histogram matching-incorporating a global constraint into MRFs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pages 993-1000. IEEE, 2006.","title":"Cosegmentation of image pairs by histogram matching-incorporating a global constraint into MRFs","authors":"Carsten Rother, Tom Minka, Andrew Blake, Vladimir Kolmogorov","year":"2006","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rubinstein et al., 2013] Michael Rubinstein, Armand Joulin, Johannes Kopf, and Ce Liu. Unsupervised joint object discovery and segmentation in internet images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1939-1946. IEEE, 2013.","title":"Unsupervised joint object discovery and segmentation in internet images","authors":"Michael Rubinstein, Armand Joulin, Johannes Kopf, Ce Liu","year":"2013","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rubio et al., 2012] Jose C Rubio, Joan Serrat, Anto- nio López, and Nikos Paragios. Unsupervised co- segmentation through region matching. In IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 749-756. IEEE, 2012.","title":"Unsupervised cosegmentation through region matching","authors":"Joan Jose C Rubio, Antonio Serrat, Nikos López, Paragios","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vicente et al., 2010] Sara Vicente, Vladimir Kolmogorov, and Carsten Rother. Cosegmentation revisited: Models and optimization. In European Conference on Computer Vision, pages 465-479. Springer, 2010.","title":"Cosegmentation revisited: Models and optimization","authors":"Sara Vicente, Vladimir Kolmogorov, Carsten Rother","year":"2010","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang and Shen, 2016] Wenguan Wang and Jianbing Shen. Higher-order image co-segmentation. IEEE Transactions on Multimedia, 18(6):1011-1021, 2016.","title":"Higher-order image co-segmentation","authors":"Wang, Shen, Wenguan Wang, Jianbing Shen","year":"2016","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2013] Fan Wang, Qixing Huang, and Leonidas J Guibas. Image co-segmentation via con- sistent functional maps. In Proceedings of the IEEE International Conference on Computer Vision, pages 849-856, 2013.","title":"Image co-segmentation via consistent functional maps","authors":"Fan Wang, Qixing Huang, Leonidas J Guibas","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Chuan Wang, Hua Zhang, Liang Yang, Xiaochun Cao, and Hongkai Xiong. Multiple seman- tic matching on augmented n-partite graph for object co- segmentation. IEEE Transactions on Image Processing, 26(12):5825-5839, 2017.","title":"Multiple semantic matching on augmented n-partite graph for object cosegmentation","authors":"Chuan Wang, Hua Zhang, Liang Yang, Xiaochun Cao, Hongkai Xiong","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yuan et al., 2017] Zehuan Yuan, Tong Lu, and Yirui Wu. Deep-dense conditional random fields for object co- segmentation. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 3371- 3377. AAAI Press, 2017.","title":"Deep-dense conditional random fields for object cosegmentation","authors":"Yuan","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}