{"type":"ScholarlyArticle","title":"Fashion Style Generator","authors":[{"name":"Shuhui Jiang","slug":"shuhui-jiang","given":"Shuhui","family":"Jiang","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA"},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA; College of Computer and Information Science , Northeastern University , Boston , MA 02115 , USA"}],"abstract":"In this paper, we focus on a new problem: applying artificial intelligence to automatically generate fashion style images. Given a basic clothing image and a fashion style image (e.g., leopard print), we generate a clothing image with the certain style in real time with a neural fashion style generator. Fashion style generation is related to recent artistic style transfer works, but has its own challenges. The synthetic image should preserve the similar design as the basic clothing, and meanwhile blend the new style pattern on the clothing. Neither existing global nor patch based neural style transfer methods could well solve these challenges. In this paper, we propose an end-to-end feed-forward neural network which consists of a fashion style generator and a discriminator. The global and patch based style and content losses calculated by the discriminator alternatively back-propagate the generator network and optimize it. The global optimization stage preserves the clothing form and design and the local optimization stage preserves the detailed style pattern. Extensive experiments show that our method outperforms the state-of-the-arts.","doi":"10.24963/ijcai.2017/520","url":"https://ijcai.org/proceedings/2017/520","pdf":"https://ijcai.org/proceedings/2017/0520.pdf","markdown":"https://ijcai.org/proceedings/2017/520.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/520","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":520,"pages":{"first":3721,"last":3727},"keywords":["Robotics and Vision: Vision and Perception","Multidisciplinary Topics and Applications: Art and Music","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Chetlur et al., 2014] Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. cudnn: Ef- ficient primitives for deep learning. arXiv preprint arXiv:1410.0759, 2014.","title":"cudnn: Efficient primitives for deep learning","authors":"Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, Evan Shelhamer","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Collobert et al., 2011] Ronan Collobert, Koray Kavukcuoglu, and Clément Farabet. Torch7: A matlab- like environment for machine learning. In BigLearn, NIPS Workshop, number EPFL-CONF-192376, 2011.","title":"Torch7: A matlablike environment for machine learning","authors":"Ronan Collobert, Koray Kavukcuoglu, Clément Farabet","year":"2011","venue":"BigLearn, NIPS Workshop","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ding et al., 2016] Zhengming Ding, Ming Shao, and Yun Fu. Deep robust encoder through locality preserving low- rank dictionary. In Proceedings of ECCV, pages 567-582. Springer, 2016.","title":"Deep robust encoder through locality preserving lowrank dictionary","authors":"Zhengming Ding, Ming Shao, Yun Fu","year":"2016","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Efros and Freeman, 2001] Alexei A Efros and William T Freeman. Image quilting for texture synthesis and trans- fer. In Proceedings of the 28th annual conference on Com- puter graphics and interactive techniques, pages 341-346. ACM, 2001.","title":"Image quilting for texture synthesis and transfer","authors":"Alexei A Efros, William T Freeman","year":"2001","venue":"Proceedings of the 28th annual conference on Computer graphics and interactive techniques","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fu12 et al., 2017] Jingtian Fu12, Jia Jia, Yihui Ma, Fanhang Meng, and Huan Huang. A virtual personal fashion con- sultant: Learning from the personal preference of fashion. In Proceedings of AAAI. AAAI Press, 2017.","title":"A virtual personal fashion consultant: Learning from the personal preference of fashion","authors":"Fu12","year":"2017","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[FYihui Ma and Tong, 2017] Suping Zhou Jingtian Fu Yejun Liu FYihui Ma, Jia Jia and Zijian Tong. Towards better understanding the clothing fashion styles: A multi- modal deep learning approach. In Proceedings of AAAI. AAAI Press, 2017.","title":"Towards better understanding the clothing fashion styles: A multimodal deep learning approach","authors":"Fyihui Ma, Suping Tong, Jingtian Zhou, Yejun Fu, Jia Liu Fyihui Ma, Zijian Jia, Tong","year":"2017","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gatys et al., 2015] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015.","title":"A neural algorithm of artistic style","authors":"Leon A Gatys, Alexander S Ecker, Matthias Bethge","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gatys et al., 2016] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of CVPR, pages 2414- 2423, 2016.","title":"Image style transfer using convolutional neural networks","authors":"Leon A Gatys, Alexander S Ecker, Matthias Bethge","year":"2016","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Proceedings of NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hadi Kiapour et al., 2015] M Hadi Kiapour, Xufeng Han, Svetlana Lazebnik, Alexander C Berg, and Tamara L Berg. Where to buy it: Matching street clothing photos in online shops. In Proceedings of ICCV, pages 3343-3351, 2015.","title":"Where to buy it: Matching street clothing photos in online shops","authors":"Xufeng Hadi Kiapour, Svetlana Han, Alexander C Lazebnik, Tamara L Berg, Berg","year":"2015","venue":"Proceedings of ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jiang et al., 2016a] Shuhui Jiang, Ming Shao, Chengcheng Jia, and Yun Fu. Consensus style centralizing auto-encoder for weak style classification. In Proceedings of AAAI, pages 1223-1229. AAAI Press, 2016.","title":"Consensus style centralizing auto-encoder for weak style classification","authors":"Jiang","year":"2016","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2016b] Shuhui Jiang, Yue Wu, and Yun Fu. Deep bi-directional cross-triplet embedding for cross- domain clothing retrieval. In Proceedings of MM, pages 52-56. ACM, 2016.","title":"Deep bi-directional cross-triplet embedding for crossdomain clothing retrieval","authors":"Jiang","year":"2016","venue":"Proceedings of MM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Johnson et al., 2016] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. arXiv preprint arXiv:1603.08155, 2016.","title":"Perceptual losses for real-time style transfer and super-resolution","authors":"Justin Johnson, Alexandre Alahi, Li Fei-Fei","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kiapour et al., 2014] M Hadi Kiapour, Kota Yamaguchi, Alexander C Berg, and Tamara L Berg. Hipster wars: Discovering elements of fashion styles. In Proceedings of ECCV, pages 472-488. Springer, 2014.","title":"Hipster wars: Discovering elements of fashion styles","authors":"Kota Hadi Kiapour, Alexander C Yamaguchi, Tamara L Berg, Berg","year":"2014","venue":"Proceedings of ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Wand, 2016a] Chuan Li and Michael Wand. Com- bining markov random fields and convolutional neu- ral networks for image synthesis. arXiv preprint arXiv:1601.04589, 2016.","title":"Combining markov random fields and convolutional neural networks for image synthesis","authors":"Wand ; Li, Chuan Li, Michael Wand","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li and Wand, 2016b] Chuan Li and Michael Wand. Precomputed real-time texture synthesis with marko- vian generative adversarial networks. arXiv preprint arXiv:1604.04382, 2016.","title":"Precomputed real-time texture synthesis with markovian generative adversarial networks","authors":"Wand ; Li, Chuan Li, Michael Wand","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2016] J. Li, , T. Zhang, W. Luo, J. Yang, X.T. Yuan, and J. Zhang. Sparseness analysis in the per- training of deep neural networks. IEEE Transac- tions on Neural Networks and Learning Systems, DOI: 10.1109/TNNLS.2016.2541681, 2016.","title":"Sparseness analysis in the pertraining of deep neural networks","authors":"J Li, T Zhang, W Luo, J Yang, X T Yuan, J Zhang","year":"2016","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":"10.1109/tnnls.2016.2541681","url":null,"internal_url":null},{"position":20,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Simo-Serra and Ishikawa, 2016] Edgar Simo-Serra and Hi- roshi Ishikawa. Fashion style in 128 floats: joint ranking and classification using weak data for feature extraction. In Proceedings of CVPR, pages 298-307, 2016.","title":"Edgar Simo-Serra and Hiroshi Ishikawa. Fashion style in 128 floats: joint ranking and classification using weak data for feature extraction","authors":"Simo-Serra, Ishikawa","year":"2016","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ulyanov et al., 2016] Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. In Proceedings of ICML, 2016.","title":"Texture networks: Feed-forward synthesis of textures and stylized images","authors":"Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, Victor Lempitsky","year":"2016","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2016] Yue Wu, Jun Li, Yu Kong, and Yun Fu. Deep convolutional neural network with independent soft- max for large scale face recognition. In Proceedings of MM, pages 1063-1067. ACM, 2016.","title":"Deep convolutional neural network with independent softmax for large scale face recognition","authors":"Yue Wu, Jun Li, Yu Kong, Yun Fu","year":"2016","venue":"Proceedings of MM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yamaguchi et al., 2012] Kota Yamaguchi, M Hadi Kiapour, Luis E Ortiz, and Tamara L Berg. Parsing clothing in fash- ion photographs. In Proceedings of CVPR, pages 3570- 3577. IEEE, 2012.","title":"Parsing clothing in fashion photographs","authors":"Yamaguchi","year":"2012","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yamaguchi et al., 2013] Kota Yamaguchi, M Hadi Kiapour, and Tamara L Berg. Paper doll parsing: Retrieving similar styles to parse clothing items. In Proceedings of ICCV, pages 3519-3526, 2013.","title":"Paper doll parsing: Retrieving similar styles to parse clothing items","authors":"Yamaguchi","year":"2013","venue":"Proceedings of ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}