{"type":"ScholarlyArticle","title":"One-Shot Texture Retrieval with Global Context Metric","authors":[{"name":"Kai Zhu","slug":"kai-zhu","given":"Kai","family":"Zhu","affiliation":"University of Science and Technology of China"},{"name":"Wei Zhai","slug":"wei-zhai","given":"Wei","family":"Zhai","affiliation":"University of Science and Technology of China"},{"name":"Zheng-Jun Zha","slug":"zheng-jun-zha","given":"Zheng-Jun","family":"Zha","affiliation":"University of Science and Technology of China"},{"name":"Yang Cao","slug":"yang-cao","given":"Yang","family":"Cao","affiliation":"University of Science and Technology of China"}],"abstract":"In this paper, we tackle one-shot texture retrieval: given an example of a new reference texture, detect and segment all the pixels of the same texture category within an arbitrary image. To address this problem, we present an OS-TR network to encoding both reference patch and query image, leading to achieve texture segmentation towards the reference category. Unlike the existing texture encoding methods that integrate CNN with orderless pooling, we propose a directionality-aware network to capture the texture variations at each direction, resulting in spatially invariant representation. To segment new categories given only few examples, we incorporate a self-gating mechanism into relation network to exploit global context information for adjusting per-channel modulation weights of local relation features. Extensive experiments on benchmark texture datasets and real scenarios demonstrate the above-par segmentation performance and robust generalization across domains of our proposed method.","doi":"10.24963/ijcai.2019/620","url":"https://ijcai.org/proceedings/2019/620","pdf":"https://ijcai.org/proceedings/2019/0620.pdf","markdown":"https://ijcai.org/proceedings/2019/620.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/620","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":620,"pages":{"first":4461,"last":4467},"keywords":["Machine Learning: Deep Learning","Computer Vision: Perception","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Bell et al., 2013] Sean Bell, Paul Upchurch, Noah Snavely, and Kavita Bala. Opensurfaces: A richly annotated catalog of surface appearance. ACM Transactions on Graphics (TOG), 32(4):111, 2013.","title":"Opensurfaces: A richly annotated catalog of surface appearance","authors":"Sean Bell, Paul Upchurch, Noah Snavely, Kavita Bala","year":"2013","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caelles et al., 2018] Sergi Caelles, Alberto Montes, Kevis- Kokitsi Maninis, Yuhua Chen, Luc Van Gool, Federico Perazzi, and Jordi Pont-Tuset. The 2018 davis chal- lenge on video object segmentation. arXiv preprint arXiv:1803.00557, 1(2), 2018.","title":"The 2018 davis challenge on video object segmentation","authors":"Alberto Sergi Caelles, Kevis-Kokitsi Montes, Yuhua Maninis, Luc Chen, Federico Van Gool, Jordi Perazzi, Pont-Tuset","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cimpoi et al., 2014] Mircea Cimpoi, Subhransu Maji, Ia- sonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In IEEE Conference on Computer Vision & Pattern Recognition, 2014.","title":"Describing textures in the wild","authors":"Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, Andrea Vedaldi","year":"2014","venue":"IEEE Conference on Computer Vision & Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cimpoi et al., 2015] Mircea Cimpoi, Subhransu Maji, and Andrea Vedaldi. Deep filter banks for texture recognition and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3828- 3836, 2015.","title":"Deep filter banks for texture recognition and segmentation","authors":"Mircea Cimpoi, Subhransu Maji, Andrea Vedaldi","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dong and Xing, 2018] Nanqing Dong and Eric P Xing. Few-shot semantic segmentation with prototype learning. In BMVC, volume 1, page 6, 2018.","title":"Nanqing Dong and Eric P Xing. Few-shot semantic segmentation with prototype learning","authors":"Xing Dong","year":"2018","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1126- 1135. JMLR. org, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2018] Jie Hu, Li Shen, and Gang Sun. Squeeze- and-excitation networks. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 7132-7141, 2018.","title":"Squeezeand-excitation networks","authors":"Jie Hu, Li Shen, Gang Sun","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Koch et al., 2015] Gregory Koch, Richard Zemel, and Rus- lan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2, 2015.","title":"Siamese neural networks for one-shot image recognition","authors":"Gregory Koch, Richard Zemel, Ruslan Salakhutdinov","year":"2015","venue":"ICML Deep Learning Workshop","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krishna et al., 2017] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. Inter- national Journal of Computer Vision, 123(1):32-73, 2017.","title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","authors":"Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma","year":"2017","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar et al., 2011] Neeraj Kumar, Alexander Berg, Pe- ter N Belhumeur, and Shree Nayar. Describable visual attributes for face verification and image search. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 33(10):1962-1977, 2011.","title":"Describable visual attributes for face verification and image search","authors":"Neeraj Kumar, Alexander Berg, Peter N Belhumeur, Shree Nayar","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lake et al., 2015] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learn- ing through probabilistic program induction. Science, 350(6266):1332-1338, 2015.","title":"Human-level concept learning through probabilistic program induction","authors":"Ruslan Brenden M Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qiao et al., 2019] Tingting Qiao, Jing Zhang, Duanqing Xu, and Dacheng Tao. Mirrorgan: Learning text- to-image generation by redescription. arXiv preprint arXiv:1903.05854, 2019.","title":"Mirrorgan: Learning textto-image generation by redescription","authors":"Tingting Qiao, Jing Zhang, Duanqing Xu, Dacheng Tao","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ren et al., 2018] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676, 2018.","title":"Meta-learning for semi-supervised few-shot classification","authors":"Ren","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234-241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"International Conference on Medical image computing and computerassisted intervention","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shaban et al., 2017] Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, and Byron Boots. One-shot learning for semantic segmentation. arXiv preprint arXiv:1709.03410, 2017.","title":"One-shot learning for semantic segmentation","authors":"Shaban","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Ad- vances in Neural Information Processing Systems, pages 4077-4087, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learn- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1199-1208, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Triantafillou et al., 2017] Eleni Triantafillou, Richard Zemel, and Raquel Urtasun. Few-shot learning through an information retrieval lens. In Advances in Neural Information Processing Systems, pages 2255-2265, 2017.","title":"Few-shot learning through an information retrieval lens","authors":"Eleni Triantafillou, Richard Zemel, Raquel Urtasun","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ustyuzhaninov et al., 2018] Ivan Ustyuzhaninov, Claudio Michaelis, Wieland Brendel, and Matthias Bethge. One-shot texture segmentation. arXiv preprint arXiv:1807.02654, 2018.","title":"One-shot texture segmentation","authors":"Ivan Ustyuzhaninov, Claudio Michaelis, Wieland Brendel, Matthias Bethge","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- othy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630-3638, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2018] Zheng Wu, Ruiheng Chang, Jiaxu Ma, Cewu Lu, and Chi-Keung Tang. Annotation-free and one- shot learning for instance segmentation of homogeneous object clusters. arXiv preprint arXiv:1802.00383, 2018.","title":"Annotation-free and oneshot learning for instance segmentation of homogeneous object clusters","authors":"Zheng Wu, Ruiheng Chang, Jiaxu Ma, Cewu Lu, Chi-Keung Tang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xue et al., 2018] Jia Xue, Hang Zhang, and Kristin Dana. Deep texture manifold for ground terrain recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 558-567, 2018.","title":"Deep texture manifold for ground terrain recognition","authors":"Jia Xue, Hang Zhang, Kristin Dana","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2017] Hang Zhang, Jia Xue, and Kristin Dana. Deep ten: Texture encoding network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 708-717, 2017.","title":"Deep ten: Texture encoding network","authors":"Hang Zhang, Jia Xue, Kristin Dana","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}