{"type":"ScholarlyArticle","title":"Semantic Locality-Aware Deformable Network for Clothing Segmentation","authors":[{"name":"Wei Ji","slug":"wei-ji","given":"Wei","family":"Ji","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Xi Li","slug":"xi-li","given":"Xi","family":"Li","affiliation":"Zhejiang University , Hangzhou , China; Alibaba-Zhejiang University Joint Institute of Frontier Technologies , Hangzhou , China"},{"name":"Yueting Zhuang","slug":"yueting-zhuang","given":"Yueting","family":"Zhuang","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Omar El Farouk Bourahla","slug":"omar-el-farouk-bourahla","given":"Omar El Farouk","family":"Bourahla","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Yixin Ji","slug":"yixin-ji","given":"Yixin","family":"Ji","affiliation":"Zhejiang University , Hangzhou , China"},{"name":"Shihao Li","slug":"shihao-li","given":"Shihao","family":"Li","affiliation":"Zhejiang University , Hangzhou , China; Alibaba-Zhejiang University Joint Institute of Frontier Technologies , Hangzhou , China"},{"name":"Jiabao Cui","slug":"jiabao-cui","given":"Jiabao","family":"Cui","affiliation":"Zhejiang University , Hangzhou , China"}],"abstract":"Clothing segmentation is a challenging vision problem typically implemented within a fine-grained semantic segmentation framework. Different from conventional segmentation, clothing segmentation has some domain-specific properties such as texture richness, diverse appearance variations, non-rigid geometry deformations, and small sample learning. To deal with these points, we propose a semantic locality-aware segmentation model, which adaptively attaches an original clothing image with a semantically similar (e.g., appearance or pose) auxiliary exemplar by search. Through considering the interactions of the clothing image and its exemplar, more intrinsic knowledge about the locality manifold structures of clothing images is discovered to make the learning process of small sample problem more stable and tractable. Furthermore, we present a CNN model based on the deformable convolutions to extract the non-rigid geometry-aware features for clothing images. Experimental results demonstrate the effectiveness of the proposed model against the state-of-the-art approaches.","doi":"10.24963/ijcai.2018/106","url":"https://ijcai.org/proceedings/2018/106","pdf":"https://ijcai.org/proceedings/2018/0106.pdf","markdown":"https://ijcai.org/proceedings/2018/106.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/106","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":106,"pages":{"first":764,"last":770},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Badrinarayanan et al., 2017] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutio- nal encoder-decoder architecture for image segmentation. IEEE TPAMI, 39(12):2481-2495, 2017.","title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","authors":"Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2017] Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, volume 1, page 7, 2017.","title":"Realtime multi-person 2d pose estimation using part affinity fields","authors":"Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Dee- plab: Semantic image segmentation with deep convolu- tional nets, atrous convolution, and fully connected crfs. IEEE TPAMI, 40(4):834-848, 2018.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dai et al., 2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. De- formable convolutional networks. CoRR, abs/1703.06211, 1(2):3, 2017.","title":"Deformable convolutional networks","authors":"Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei","year":"2017","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gong et al., 2017] Ke Gong, Xiaodan Liang, Dongyu Zhang, Xiaohui Shen, and Liang Lin. Look into per- son: Self-supervised structure-sensitive learning and a new benchmark for human parsing. In CVPR, pages 932-940, 2017.","title":"Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing","authors":"Ke Gong, Xiaodan Liang, Dongyu Zhang, Xiaohui Shen, Liang Lin","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hadi Kiapour et al., 2015] M Hadi Kiapour, Xufeng Han, Svetlana Lazebnik, Alexander C Berg, and Tamara L Berg. Where to buy it: Matching street clothing photos in online shops. In ICCV, pages 3343-3351, 2015.","title":"Where to buy it: Matching street clothing photos in online shops","authors":"Xufeng Hadi Kiapour, Svetlana Han, Alexander C Lazebnik, Tamara L Berg, Berg","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Dona- hue, Sergey Karayev, Jonathan Long, Ross Girshick, Ser- gio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM MM, pa- ges 675-678. ACM, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Jia","year":"2014","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. In NIPS, pages 109-117, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liang et al., 2016a] Xiaodan Liang, Liang Lin, Wei Yang, Ping Luo, Junshi Huang, and Shuicheng Yan. Clothes co-parsing via joint image segmentation and labeling with application to clothing retrieval. IEEE TMM, 18(6):1175- 1186, 2016.","title":"Clothes co-parsing via joint image segmentation and labeling with application to clothing retrieval","authors":"Liang","year":"2016","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liang et al., 2016b] Xiaodan Liang, Xiaohui Shen, Jiashi Feng, Liang Lin, and Shuicheng Yan. Semantic object par- sing with graph lstm. In ECCV, pages 125-143. Springer, 2016.","title":"Semantic object parsing with graph lstm","authors":"Liang","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liang et al., 2017] Xiaodan Liang, Liang Lin, Xiaohui Shen, Jiashi Feng, Shuicheng Yan, and Eric P Xing. Inter- pretable structure-evolving lstm. In CVPR, pages 2175- 2184, 2017.","title":"Interpretable structure-evolving lstm","authors":"Xiaodan Liang, Liang Lin, Xiaohui Shen, Jiashi Feng, Shuicheng Yan, Eric P Xing","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liew et al., 2017] JunHao Liew, Yunchao Wei, Wei Xiong, Sim-Heng Ong, and Jiashi Feng. Regional interactive image segmentation networks. In ICCV, pages 2746- 2754. IEEE, 2017.","title":"Regional interactive image segmentation networks","authors":"Liew","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2014] Si Liu, Jiashi Feng, Csaba Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, and Shuicheng Yan. Fashion parsing with weak color-category labels. IEEE TMM, 16(1):253-265, 2014.","title":"Fashion parsing with weak color-category labels","authors":"Si Liu, Jiashi Feng, Csaba Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, Shuicheng Yan","year":"2014","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2016] Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In CVPR, pages 1096-1104, 2016.","title":"Deepfashion: Powering robust clothes recognition and retrieval with rich annotations","authors":"Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, Xiaoou Tang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for seman- tic segmentation. In CVPR, pages 3431-3440, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[McAuley et al., 2015] Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based re- commendations on styles and substitutes. In SIGIR, pages 43-52. ACM, 2015.","title":"Image-based recommendations on styles and substitutes","authors":"Julian Mcauley, Christopher Targett, Qinfeng Shi, Anton Van Den, Hengel","year":"2015","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simo-Serra et al., 2014] Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. A high performance crf model for clothes parsing. In ACCV, pa- ges 64-81. Springer, 2014.","title":"A high performance crf model for clothes parsing","authors":"Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, Raquel Urtasun","year":"2014","venue":"ACCV, pages","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tangseng et al., 2017] Pongsate Tangseng, Zhipeng Wu, and Kota Yamaguchi. Looking at outfit to parse clothing. CoRR, abs/1703.01386, 2017.","title":"Looking at outfit to parse clothing","authors":"Tangseng","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Veit et al., 2015] Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. Lear- ning visual clothing style with heterogeneous dyadic co- occurrences. In ICCV, pages 4642-4650, 2015.","title":"Learning visual clothing style with heterogeneous dyadic cooccurrences","authors":"Andreas Veit, Balazs Kovacs, Sean Bell, Julian Mcauley, Kavita Bala, Serge Belongie","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vittayakorn et al., 2015] Sirion Vittayakorn, Kota Yamagu- chi, Alexander C Berg, and Tamara L Berg. Runway to realway: Visual analysis of fashion. In WACV, pages 951- 958. IEEE, 2015.","title":"Runway to realway: Visual analysis of fashion","authors":"Vittayakorn","year":"2015","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yamaguchi et al., 2012] Kota Yamaguchi, M Hadi Kiapour, Luis E Ortiz, and Tamara L Berg. Parsing clothing in fashion photographs. In CVPR, pages 3570-3577. IEEE, 2012.","title":"Parsing clothing in fashion photographs","authors":"Yamaguchi","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yamaguchi et al., 2015] Kota Yamaguchi, M Hadi Kiapour, Luis E Ortiz, and Tamara L Berg. Retrieving similar styles to parse clothing. IEEE TPAMI, 37(5):1028-1040, 2015.","title":"Retrieving similar styles to parse clothing","authors":"Yamaguchi","year":"2015","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2017] Rui Yang, Bingbing Ni, Chao Ma, Yi Xu, and Xiaokang Yang. Video segmentation via multiple gra- nularity analysis. In CVPR, pages 3010-3019, 2017.","title":"Video segmentation via multiple granularity analysis","authors":"Rui Yang, Bingbing Ni, Chao Ma, Yi Xu, Xiaokang Yang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2017] Hengshuang Zhao, Jianping Shi, Xiao- juan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, pages 2881-2890, 2017.","title":"Pyramid scene parsing network","authors":"Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2017] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017.","title":"Scene parsing through ade20k dataset","authors":"Zhou","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}