{"type":"ScholarlyArticle","title":"Annotation-Free and One-Shot Learning for Instance Segmentation of Homogeneous Object Clusters","authors":[{"name":"Zheng Wu","slug":"zheng-wu","given":"Zheng","family":"Wu","affiliation":"Shanghai Jiao Tong University"},{"name":"Ruiheng Chang","slug":"ruiheng-chang","given":"Ruiheng","family":"Chang","affiliation":"Shanghai Jiao Tong University"},{"name":"Jiaxu Ma","slug":"jiaxu-ma","given":"Jiaxu","family":"Ma","affiliation":"Shanghai Jiao Tong University"},{"name":"Cewu Lu","slug":"cewu-lu","given":"Cewu","family":"Lu","affiliation":"Shanghai Jiao Tong University"},{"name":"Chi Keung Tang","slug":"chi-keung-tang","given":"Chi Keung","family":"Tang","affiliation":"HKUST"}],"abstract":"We propose a novel approach for instance segmentation given an image of homogeneous object cluster (HOC). Our learning approach is one-shot because a single video of an object instance is captured and it requires no human annotation. Our intuition is that images of homogeneous objects can be effectively synthesized based on structure and illumination priors derived from real images. A novel solver is proposed that iteratively maximizes our structured likelihood to generate realistic images of HOC. Illumination transformation scheme is applied to make the real and synthetic images share the same illumination condition. Extensive experiments and comparisons are performed to verify our method. We build a dataset consisting of pixel-level annotated images of HOC. The dataset and code will be released.","doi":"10.24963/ijcai.2018/144","url":"https://ijcai.org/proceedings/2018/144","pdf":"https://ijcai.org/proceedings/2018/0144.pdf","markdown":"https://ijcai.org/proceedings/2018/144.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/144","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":144,"pages":{"first":1036,"last":1042},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: 2D and 3D Computer Vision","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Caelles et al., 2017] Sergi Caelles, Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Laura Leal-Taixé, Daniel Cremers, and Luc Van Gool. One-shot video object segmentation. In CVPR 2017. IEEE, 2017.","title":"One-shot video object segmentation","authors":"Kevis-Kokitsi Sergi Caelles, Jordi Maninis, Laura Pont-Tuset, Daniel Leal-Taixé, Luc Cremers, Van Gool","year":"2017","venue":"CVPR 2017","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] Qifeng Chen, Dingzeyu Li, and Chi- Keung Tang. Knn matting. Pattern Analysis and Ma- chine Intelligence, IEEE Transactions on, 35(9):2175- 2188, Sept 2013.","title":"Knn matting. Pattern Analysis and Machine Intelligence","authors":"Qifeng Chen, Dingzeyu Li, Chi-Keung Tang","year":"2013","venue":"IEEE Transactions on","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cordts et al., 2016] Marius Cordts, Mohamed Omran, Se- bastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene under- standing. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.","title":"The cityscapes dataset for semantic urban scene understanding","authors":"Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele","year":"2016","venue":"Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fei-Fei et al., 2006] Li Fei-Fei, Rob Fergus, and Pietro Per- ona. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell., 28:594-611, 2006.","title":"One-shot learning of object categories","authors":"Li Fei-Fei, Rob Fergus, Pietro Perona","year":"2006","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hariharan et al., 2014] Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. Simultaneous detection and segmentation. In European Conference on Computer Vision, pages 297-312, 2014.","title":"Simultaneous detection and segmentation","authors":"Bharath Hariharan, Pablo Arbeláez, Ross Girshick, Jitendra Malik","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2980-2988. IEEE, 2017.","title":"Kaiming He, Georgia Gkioxari","authors":"He","year":"2017","venue":"Computer Vision (ICCV), 2017 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Isola et al., 2016] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004, 2016.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Ad- vances in Neural Information Processing Systems 25, pages 1097-1105.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Leong et al., 2003] FJ WM Leong, M Brady, and JO McGee. Correction of uneven illumination (vi- gnetting) in digital microscopy images. Journal of clinical pathology, 56(8):619-621, 2003.","title":"Correction of uneven illumination (vignetting) in digital microscopy images","authors":"M Fj Wm Leong, J O Brady, Mcgee","year":"2003","venue":"Journal of clinical pathology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. Fully convolutional instance-aware se- mantic segmentation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2359-2367, 2017.","title":"Fully convolutional instance-aware semantic segmentation","authors":"Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, Yichen Wei","year":"2017","venue":"IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In European conference on com- puter vision, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.","title":"Conditional generative adversarial nets","authors":"Osindero Mirza","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Močkus, 1975] J Močkus. On bayesian methods for seeking the extremum. In Optimization Techniques IFIP Technical Conference, pages 400-404. Springer, 1975.","title":"On bayesian methods for seeking the extremum","authors":"Močkus","year":"1975","venue":"Optimization Techniques IFIP Technical Conference","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Papon and Schoeler, 2015] Jeremie Papon and Markus Schoeler. Semantic pose using deep networks trained on synthetic rgb-d. In Proceedings of the IEEE International Conference on Computer Vision, pages 774-782, 2015.","title":"Semantic pose using deep networks trained on synthetic rgb-d","authors":"Schoeler Papon, Jeremie Papon, Markus Schoeler","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Peng et al., 2015] Xingchao Peng, Baochen Sun, Karim Ali, and Kate Saenko. Learning deep object detectors from 3d models. In Proceedings of the IEEE International Con- ference on Computer Vision, pages 1278-1286, 2015.","title":"Learning deep object detectors from 3d models","authors":"Peng","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rong and Yang, 2016] Tao Rong and Ruoyu Yang. One- shot-learning gesture segmentation and recognition using frame-based pdv features. In Pacific Rim Conference on Multimedia, pages 355-365. Springer, 2016.","title":"Tao Rong and Ruoyu Yang. Oneshot-learning gesture segmentation and recognition using frame-based pdv features","authors":"Yang Rong","year":"2016","venue":"Pacific Rim Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sixt et al., 2016] Leon Sixt, Benjamin Wild, and Tim Land- graf. Rendergan: Generating realistic labeled data. arXiv preprint arXiv:1611.01331, 2016.","title":"Rendergan: Generating realistic labeled data","authors":"Leon Sixt, Benjamin Wild, Tim Landgraf","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Su et al., 2012] Hao Su, Jia Deng, and Li Fei-Fei. Crowd- sourcing annotations for visual object detection. In Work- shops at the Twenty-Sixth AAAI Conference on Artificial Intelligence, volume 1, 2012.","title":"Crowdsourcing annotations for visual object detection","authors":"Hao Su, Jia Deng, Li Fei-Fei","year":"2012","venue":"Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Su et al., 2015] Hao Su, Charles R Qi, Yangyan Li, and Leonidas J Guibas. Render for cnn: Viewpoint estima- tion in images using cnns trained with rendered 3d model views. In Proceedings of the IEEE International Confer- ence on Computer Vision, pages 2686-2694, 2015.","title":"Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views","authors":"Hao Su, Yangyan Charles R Qi, Leonidas J Li, Guibas","year":"2015","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zitnick and Dollár, 2014] C. Lawrence Zitnick and Piotr Dollár. Edge boxes: Locating object proposals from edges. In ECCV, 2014.","title":"Edge boxes: Locating object proposals from edges","authors":"C, Lawrence Zitnick, Piotr Dollár","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}