{"type":"ScholarlyArticle","title":"Auxiliary Template-Enhanced Generative Compatibility Modeling","authors":[{"name":"Jinhuan Liu","slug":"jinhuan-liu","given":"Jinhuan","family":"Liu","affiliation":"Shandong University , Qingdao , China"},{"name":"Xuemeng Song","slug":"xuemeng-song","given":"Xuemeng","family":"Song","affiliation":"Shandong University , Qingdao , China"},{"name":"Zhaochun Ren","slug":"zhaochun-ren","given":"Zhaochun","family":"Ren","affiliation":"Shandong University , Qingdao , China"},{"name":"Liqiang Nie","slug":"liqiang-nie","given":"Liqiang","family":"Nie","affiliation":"Shandong University , Qingdao , China"},{"name":"Zhaopeng Tu","slug":"zhaopeng-tu","given":"Zhaopeng","family":"Tu","affiliation":"Tencent AI Lab , Shenzhen , China"},{"name":"Jun Ma","slug":"jun-ma","given":"Jun","family":"Ma","affiliation":"Shandong University , Qingdao , China"}],"abstract":"In recent years, there has been a growing interest in the fashion analysis (e.g., clothing matching) due to the huge economic value of the fashion industry. The essential problem is to model the compatibility between the complementary fashion items, such as the top and bottom in clothing matching. The majority of existing work on fashion analysis has focused on measuring the item-item compatibility in a latent space with deep learning methods.\r\nIn this work, we aim to improve the compatibility modeling by sketching a compatible template for a given item as an auxiliary link between fashion items. Specifically, we propose an end-to-end Auxiliary Template-enhanced Generative Compatibility Modeling (AT-GCM) scheme, which introduces an auxiliary complementary template generation network equipped with the pixel-wise consistency and compatible template regularization. Extensive experiments on two real-world datasets demonstrate the superiority of the proposed approach.","doi":"10.24963/ijcai.2020/485","url":"https://ijcai.org/proceedings/2020/485","pdf":"https://ijcai.org/proceedings/2020/0485.pdf","markdown":"https://ijcai.org/proceedings/2020/485.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/485","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":485,"pages":{"first":3508,"last":3514},"keywords":["Multidisciplinary Topics and Applications: Recommender Systems","Multidisciplinary Topics and Applications: Information Retrieval"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24480/file_24480.m3u8","length_sec":290,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen and He, 2018] Long Chen and Yuhang He. Dress fashion- ably: Learn fashion collocation with deep mixed-category metric learning. In AAAI, 2018.","title":"Dress fashionably: Learn fashion collocation with deep mixed-category metric learning","authors":"Chen, He, Long Chen, Yuhang He","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cucurull et al., 2019] Guillem Cucurull, Perouz Taslakian, and David Vazquez. Context-aware visual compatibility prediction. arXiv preprint arXiv:1902.03646, 2019.","title":"Context-aware visual compatibility prediction","authors":"Guillem Cucurull, Perouz Taslakian, David Vazquez","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Han et al., 2017] Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S Davis. Learning fashion compatibility with bidirectional lstms. In MM, pages 1078-1086, 2017.","title":"Learning fashion compatibility with bidirectional lstms","authors":"Han","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He and McAuley, 2016] Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback. In AAAI, pages 144-150, 2016.","title":"Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback","authors":"Mcauley He","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, pages 630-645, 2016.","title":"Identity mappings in deep residual networks","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2019] Tianyu He, Yingce Xia, Jianxin Lin, Xu Tan, Di He, Tao Qin, and Zhibo Chen. Deliberation learning for image-to-image translation. IJCAI, 2019.","title":"Deliberation learning for image-to-image translation","authors":"Tianyu He, Yingce Xia, Jianxin Lin, Xu Tan, Di He, Tao Qin, Zhibo Chen","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/345"},{"position":8,"raw":"[Huang et al., 2019] Zhiwu Huang, Jiqing Wu, and Luc Van Gool. Manifold-valued image generation with wasserstein generative adversarial nets. In AAAI, 2019.","title":"Manifold-valued image generation with wasserstein generative adversarial nets","authors":"Zhiwu Huang, Jiqing Wu, Luc Van Gool","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In CVPR, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jin et al., 2019] Di Jin, Bingyi Li, Pengfei Jiao, Dongxiao He, and Weixiong Zhang. Network-specific variational auto-encoder for embedding in attribute networks. In IJCAI, pages 2663-2669, 2019.","title":"Network-specific variational auto-encoder for embedding in attribute networks","authors":"Di Jin, Bingyi Li, Pengfei Jiao, Dongxiao He, Weixiong Zhang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/370"},{"position":11,"raw":"[Kim, 2014] Yoon Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Yoon Kim","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2017] Chunyuan Li, Hao Liu, Changyou Chen, Yuchen Pu, Liqun Chen, Ricardo Henao, and Lawrence Carin. Alice: Towards understanding adversarial learning for joint distribution matching. In NIPS, 2017.","title":"Alice: Towards understanding adversarial learning for joint distribution matching","authors":"Chunyuan Li, Hao Liu, Changyou Chen, Yuchen Pu, Liqun Chen, Ricardo Henao, Lawrence Carin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2019] Qintong Li, Hongshen Chen, Zhaochun Ren, Zhumin Chen, Zhaopeng Tu, and Jun Ma. Empgan: Multi- resolution interactive empathetic dialogue generation. arXiv preprint arXiv:1911.08698, 2019.","title":"Empgan: Multiresolution interactive empathetic dialogue generation","authors":"Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2013] Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013.","title":"Network in network","authors":"Lin","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2019] Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten de Rijke. Improving outfit recommendation with co-supervision of fashion generation. In WWW, pages 1095-1105, 2019.","title":"Improving outfit recommendation with co-supervision of fashion generation","authors":"Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, Maarten De Rijke","year":"2019","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017] Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NIPS, pages 700-708, 2017.","title":"Unsupervised image-to-image translation networks","authors":"Ming-Yu Liu, Thomas Breuel, Jan Kautz","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ma et al., 2017] Yihui Ma, Jia Jia, Suping Zhou, Jingtian Fu, Yejun Liu, and Zijian Tong. Towards better understanding the clothing fashion styles: A multimodal deep learning approach. In AAAI, pages 38-44, 2017.","title":"Towards better understanding the clothing fashion styles: A multimodal deep learning approach","authors":"Yihui Ma, Jia Jia, Suping Zhou, Jingtian Fu, Yejun Liu, Zijian Tong","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mao et al., 2017] Xudong Mao, Qing Li, Haoran Xie, Ray- mond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In ICCV, pages 2794- 2802, 2017.","title":"Least squares generative adversarial networks","authors":"Xudong Mao, Qing Li, Haoran Xie, Raymond Yk Lau, Zhen Wang, Stephen Paul Smolley","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[McAuley et al., 2015] Julian McAuley, Christopher Targett, Qin- feng Shi, and Anton Van Den Hengel. Image-based recommen- dations on styles and substitutes. In SIGIR, pages 43-52, 2015.","title":"Image-based recommendations on styles and substitutes","authors":"Julian Mcauley, Christopher Targett, Qinfeng Shi, Anton Van Den, Hengel","year":"2015","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pu et al., 2016] Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. Variational autoencoder for deep learning of images, labels and captions. In NIPS, pages 2352-2360, 2016.","title":"Variational autoencoder for deep learning of images, labels and captions","authors":"Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, Lawrence Carin","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.","title":"U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"MICCAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2017] Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. Neurostylist: Neural compatibility modeling for clothing matching. In MM, pages 753-761, 2017.","title":"Neurostylist: Neural compatibility modeling for clothing matching","authors":"Song","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2018] Jingkuan Song, Tao He, Lianli Gao, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Binary generative adversarial networks for image retrieval. In AAAI, 2018.","title":"Binary generative adversarial networks for image retrieval","authors":"Song","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Vasileva et al., 2018] Mariya I Vasileva, Bryan A Plummer, Kr- ishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. Learning type-aware embeddings for fashion compatibility. In ECCV, pages 390-405, 2018.","title":"Learning type-aware embeddings for fashion compatibility","authors":"Mariya I Vasileva, Bryan A Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David Forsyth","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] Zihan Wang, Ziheng Jiang, Zhaochun Ren, Jiliang Tang, and Dawei Yin. A path-constrained framework for discriminating substitutable and complementary products in e- commerce. In WSDM, pages 619-627, 2018.","title":"A path-constrained framework for discriminating substitutable and complementary products in ecommerce","authors":"Zihan Wang, Ziheng Jiang, Zhaochun Ren, Jiliang Tang, Dawei Yin","year":"2018","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[White, 2016] Tom White. Sampling generative networks. arXiv preprint arXiv:1609.04468, 2016.","title":"Sampling generative networks","authors":"Tom White","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xian et al., 2018] Wenqi Xian, Patsorn Sangkloy, Varun Agrawal, Amit Raj, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. Texturegan: Controlling deep image synthesis with texture patches. In CVPR, 2018.","title":"Texturegan: Controlling deep image synthesis with texture patches","authors":"Patsorn Wenqi Xian, Varun Sangkloy, Amit Agrawal, Jingwan Raj, Chen Lu, Fisher Fang, James Yu, Hays","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2019] Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. Transnfcm: translation-based neural fashion compatibility modeling. In AAAI, 2019.","title":"Transnfcm: translation-based neural fashion compatibility modeling","authors":"Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, Tat-Seng Chua","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yi et al., 2017] Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. Dualgan: Unsupervised dual learning for image-to-image translation. In ICCV, pages 2849-2857, 2017.","title":"Dualgan: Unsupervised dual learning for image-to-image translation","authors":"Zili Yi, Hao Zhang, Ping Tan, Minglun Gong","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, pages 2223- 2232, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}