{"type":"ScholarlyArticle","title":"Dress like an Internet Celebrity: Fashion Retrieval in Videos","authors":[{"name":"Hongrui Zhao","slug":"hongrui-zhao","given":"Hongrui","family":"Zhao","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Jin Yu","slug":"jin-yu","given":"Jin","family":"Yu","affiliation":"Alibaba Group"},{"name":"Yanan Li","slug":"yanan-li","given":"Yanan","family":"Li","affiliation":"Institute of Artificial Intelligence , Zhejiang Lab {hrzhao ,"},{"name":"Donghui Wang","slug":"donghui-wang","given":"Donghui","family":"Wang","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Jie Liu","slug":"jie-liu","given":"Jie","family":"Liu","affiliation":"Alibaba Group"},{"name":"Hongxia Yang","slug":"hongxia-yang","given":"Hongxia","family":"Yang","affiliation":"Alibaba Group"},{"name":"Fei Wu","slug":"fei-wu","given":"Fei","family":"Wu","affiliation":"College of Computer Science and Technology , Zhejiang University"}],"abstract":"Nowadays, both online shopping and video sharing have grown exponentially. Although internet celebrities in videos are ideal exhibition for fashion corporations to sell their products, audiences do not always know where to buy fashion products in videos, which is a cross-domain problem called video-to-shop. In this paper, we propose a novel deep neural network, called Detect, Pick, and Retrieval Network (DPRNet), to break the gap between fashion products from videos and audiences. For the video side, we have modified the traditional object detector, which automatically picks out the best object proposals for every commodity in videos without duplication, to promote the performance of the video-to-shop task. For the fashion retrieval side, a simple but effective multi-task loss network obtains new state-of-the-art results on DeepFashion. Extensive experiments conducted on a new large-scale cross-domain video-to-shop dataset shows that DPRNet is efficient and outperforms the state-of-the-art methods on video-to-shop task.","doi":"10.24963/ijcai.2020/147","url":"https://ijcai.org/proceedings/2020/147","pdf":"https://ijcai.org/proceedings/2020/0147.pdf","markdown":"https://ijcai.org/proceedings/2020/147.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/147","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":147,"pages":{"first":1054,"last":1060},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Data Mining: Mining Text, Web, Social Media","Multidisciplinary Topics and Applications: Recommender Systems","Machine Learning: Recommender Systems"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24653/file_24653.m3u8","length_sec":301,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen et al., 2018] Kai Chen, Jiaqi Wang, Shuo Yang, Xingcheng Zhang, Yuanjun Xiong, Chen Change Loy, and Dahua Lin. Op- timizing video object detection via a scale-time lattice. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7814-7823, June 2018.","title":"Optimizing video object detection via a scale-time lattice","authors":"Kai Chen, Jiaqi Wang, Shuo Yang, Xingcheng Zhang, Yuanjun Xiong, Chen Change Loy, Dahua Lin","year":"2018","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cheng et al., 2017] Zhi-Qi Cheng, Xiao Wu, Yang Liu, and Xian- Sheng Hua. Video2shop: Exact matching clothes in videos to online shopping images. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4048-4056, July 2017.","title":"Video2shop: Exact matching clothes in videos to online shopping images","authors":"Zhi-Qi Cheng, Xiao Wu, Yang Liu, Xian-Sheng Hua","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chopra et al., 2019] Ayush Chopra, Abhishek Sinha, Hiresh Gupta, Mausoom Sarkar, Kumar Ayush, and Balaji Krishna- murthy. Powering robust fashion retrieval with information rich feature embeddings. In The IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR) Workshops, June 2019.","title":"Powering robust fashion retrieval with information rich feature embeddings","authors":"Ayush Chopra, Abhishek Sinha, Hiresh Gupta, Mausoom Sarkar, Kumar Ayush, Balaji Krishnamurthy","year":"2019","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dodds et al., 2018] Eric Dodds, Huy Nguyen, Simao Herdade, Jack Culpepper, Andrew Kae, and Pierre Garrigues. Learning embeddings for product visual search with triplet loss and online sampling. CoRR, abs/1810.04652, 2018.","title":"Learning embeddings for product visual search with triplet loss and online sampling","authors":"Eric Dodds, Huy Nguyen, Simao Herdade, Jack Culpepper, Andrew Kae, Pierre Garrigues","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gajic and Baldrich, 2018] Bojana Gajic and Ramon Baldrich. Cross-domain fashion image retrieval. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1869-1871, June 2018.","title":"Cross-domain fashion image retrieval","authors":"Baldrich Gajic, Bojana Gajic, Ramon Baldrich","year":"2018","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ge et al., 2019] Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xi- aoou Tang, and Ping Luo. Deepfashion2: A versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5337-5345, June 2019.","title":"Deepfashion2: A versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images","authors":"Yuying Ge, Ruimao Zhang, Xiaogang Wang, Xiaoou Tang, Ping Luo","year":"2019","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 580- 587, June 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik","year":"2014","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Girshick, 2015] Ross Girshick. Fast r-cnn. In The IEEE Inter- national Conference on Computer Vision (ICCV), pages 1440- 1448, December 2015.","title":"Fast r-cnn","authors":"Ross Girshick","year":"2015","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hadi Kiapour et al., 2015] M. Hadi Kiapour, Xufeng Han, Svet- lana Lazebnik, Alexander C. Berg, and Tamara L. Berg. Where to buy it: Matching street clothing photos in online shops. In The IEEE International Conference on Computer Vision (ICCV), pages 3343-3351, December 2015.","title":"Where to buy it: Matching street clothing photos in online shops","authors":"M Hadi Kiapour, Xufeng Han, Svetlana Lazebnik, Alexander C Berg, Tamara L Berg","year":"2015","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hermans et al., 2017] Alexander Hermans, Lucas Beyer, and Bas- tian Leibe. In defense of the triplet loss for person re- identification. CoRR, abs/1703.07737, 2017.","title":"In defense of the triplet loss for person reidentification","authors":"Alexander Hermans, Lucas Beyer, Bastian Leibe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2015] Junshi Huang, Rogerio S. Feris, Qiang Chen, and Shuicheng Yan. Cross-domain image retrieval with a dual attribute-aware ranking network. In The IEEE International Con- ference on Computer Vision (ICCV), pages 1062-1070, Decem- ber 2015.","title":"Cross-domain image retrieval with a dual attribute-aware ranking network","authors":"Junshi Huang, Rogerio S Feris, Qiang Chen, Shuicheng Yan","year":"2015","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kuang et al., 2019] Zhanghui Kuang, Yiming Gao, Guanbin Li, Ping Luo, Yimin Chen, Liang Lin, and Wayne Zhang. Fashion retrieval via graph reasoning networks on a similarity pyramid. In The IEEE International Conference on Computer Vision (ICCV), pages 3066-3075, October 2019.","title":"Fashion retrieval via graph reasoning networks on a similarity pyramid","authors":"Zhanghui Kuang, Yiming Gao, Guanbin Li, Ping Luo, Yimin Chen, Liang Lin, Wayne Zhang","year":"2019","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaim- ing He, and Piotr Dollar. Focal loss for dense object detection. In The IEEE International Conference on Computer Vision (ICCV), pages 2980-2988, Oct 2017.","title":"Focal loss for dense object detection","authors":"Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollar","year":"2017","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2016a] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexan- der C. Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37, 2016.","title":"Ssd: Single shot multibox detector","authors":"Liu","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016b] Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. Deepfashion: Powering robust clothes recog- nition and retrieval with rich annotations. In The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 1096-1104, June 2016.","title":"Deepfashion: Powering robust clothes recognition and retrieval with rich annotations","authors":"Liu","year":"2016","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Müllner, 2011] Daniel Müllner. Modern hierarchical, agglomera- tive clustering algorithms, 2011.","title":"Modern hierarchical, agglomerative clustering algorithms","authors":"Daniel Müllner","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Redmon and Farhadi, 2017] Joseph Redmon and Ali Farhadi. Yolo9000: Better, faster, stronger. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7263- 7271, July 2017.","title":"Yolo9000: Better, faster, stronger","authors":"Joseph Redmon, Ali Farhadi","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Redmon and Farhadi, 2018] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. CoRR, abs/1804.02767, 2018.","title":"Yolov3: An incremental improvement","authors":"Joseph Redmon, Ali Farhadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Redmon et al., 2016] Joseph Redmon, Santosh Divvala, Ross Gir- shick, and Ali Farhadi. You only look once: Unified, real-time object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779-788, June 2016.","title":"You only look once: Unified, real-time object detection","authors":"Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi","year":"2016","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems 28, pages 91-99. 2015.","title":"Faster r-cnn: Towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recogni- tion and clustering. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815-823, June 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2018] Yantao Shen, Tong Xiao, Hongsheng Li, Shuai Yi, and Xiaogang Wang. End-to-end deep kronecker-product matching for person re-identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6886- 6895, June 2018.","title":"End-to-end deep kronecker-product matching for person re-identification","authors":"Yantao Shen, Tong Xiao, Hongsheng Li, Shuai Yi, Xiaogang Wang","year":"2018","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 2818-2826, June 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Z. Wang, Y. Gu, Y. Zhang, J. Zhou, and X. Gu. Clothing retrieval with visual attention model. In 2017 IEEE Vi- sual Communications and Image Processing (VCIP), pages 1-4, 2017.","title":"Clothing retrieval with visual attention model","authors":"Z Wang, Y Gu, Y Zhang, J Zhou, X Gu","year":"2017","venue":"IEEE Visual Communications and Image Processing (VCIP)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xuan et al., 2018] Hong Xuan, Richard Souvenir, and Robert Pless. Deep randomized ensembles for metric learning. In The European Conference on Computer Vision (ECCV), pages 723- 734, September 2018.","title":"Deep randomized ensembles for metric learning","authors":"Xuan","year":"2018","venue":"The European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2018] Yanhao Zhang, Pan Pan, Yun Zheng, Kang Zhao, Yingya Zhang, Xiaofeng Ren, and Rong Jin. Visual search at alibaba. In Proceedings of the 24th ACM SIGKDD Interna- tional Conference on Knowledge Discovery; Data Mining, KDD '18, pages 993-1001, 2018.","title":"Visual search at alibaba","authors":"Zhang","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining, KDD '18","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu et al., 2017] Xizhou Zhu, Yuwen Xiong, Jifeng Dai, Lu Yuan, and Yichen Wei. Deep feature flow for video recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2349-2358, July 2017.","title":"Deep feature flow for video recognition","authors":"Xizhou Zhu, Yuwen Xiong, Jifeng Dai, Lu Yuan, Yichen Wei","year":"2017","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}