{"type":"ScholarlyArticle","title":"Controlling Neural Style Transfer with Deep Reinforcement Learning","authors":[{"name":"Chengming Feng","slug":"chengming-feng","given":"Chengming","family":"Feng","affiliation":"Chengdu University of Information Technology , China"},{"name":"Jing Hu","slug":"jing-hu","given":"Jing","family":"Hu","affiliation":"Chengdu University of Information Technology , China"},{"name":"Xin Wang","slug":"xin-wang","given":"Xin","family":"Wang","affiliation":"University at Buffalo , SUNY , USA"},{"name":"Shu Hu","slug":"shu-hu","given":"Shu","family":"Hu","affiliation":"Carnegie Mellon University , USA"},{"name":"Bin Zhu","slug":"bin-zhu","given":"Bin","family":"Zhu","affiliation":"Microsoft Research Asia , China"},{"name":"Xi Wu","slug":"xi-wu","given":"Xi","family":"Wu","affiliation":"Chengdu University of Information Technology , China"},{"name":"Hongtu Zhu","slug":"hongtu-zhu","given":"Hongtu","family":"Zhu","affiliation":"University of North Carolina at Chapel Hill , USA"},{"name":"Siwei Lyu","slug":"siwei-lyu","given":"Siwei","family":"Lyu","affiliation":"University at Buffalo , SUNY , USA"}],"abstract":"Controlling the degree of stylization in the Neural Style Transfer (NST) is a little tricky since it usually needs hand-engineering on hyper-parameters. In this paper, we propose the first deep Reinforcement Learning (RL) based architecture that splits one-step style transfer into a step-wise process for the NST task. Our RL-based method tends to preserve more details and structures of the content image in early steps, and synthesize more style patterns in later steps. It is a user-easily-controlled style-transfer method. Additionally, as our RL-based model performs the stylization progressively, it is lightweight and has lower computational complexity than existing one-step Deep Learning (DL) based models. Experimental results demonstrate the effectiveness and robustness of our method.","doi":"10.24963/ijcai.2023/12","url":"https://ijcai.org/proceedings/2023/12","pdf":"https://ijcai.org/proceedings/2023/0012.pdf","markdown":"https://ijcai.org/proceedings/2023/12.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/12","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":12,"pages":{"first":100,"last":108},"keywords":["Agent-based and Multi-agent Systems: MAS: Applications","Computer Vision: CV: Applications"],"videos":[],"references":[{"position":1,"raw":"[An et al., 2021] Jie An, Siyu Huang, Yibing Song, Dejing Dou, Wei Liu, and Jiebo Luo. Artflow: Unbiased image style transfer via reversible neural flows. In CVPR, pages 862-871, 2021.","title":"Artflow: Unbiased image style transfer via reversible neural flows","authors":"Jie An, Siyu Huang, Yibing Song, Dejing Dou, Wei Liu, Jiebo Luo","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Butler et al., 2012] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In ECCV, pages 611- 625. Springer, 2012.","title":"A naturalistic open source movie for optical flow evaluation","authors":"J Daniel, Jonas Butler, Garrett B Wulff, Michael J Stanley, Black","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, and Gang Hua. Stylebank: An explicit repre- sentation for neural image style transfer. In CVPR, 2017.","title":"Stylebank: An explicit representation for neural image style transfer","authors":"Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, Gang Hua","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2021] Haibo Chen, Zhizhong Wang, Huiming Zhang, Zhiwen Zuo, Ailin Li, Wei Xing, Dongming Lu, et al. Artistic style transfer with internal-external learning and contrastive learning. Advances in Neural Information Processing Systems, 34, 2021.","title":"Artistic style transfer with internal-external learning and contrastive learning","authors":"Haibo Chen, Zhizhong Wang, Huiming Zhang, Zhiwen Zuo, Ailin Li, Wei Xing, Dongming Lu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cheng et al., 2021] Jiaxin Cheng, Ayush Jaiswal, Yue Wu, Pradeep Natarajan, and Prem Natarajan. Style-aware nor- malized loss for improving arbitrary style transfer. In CVPR, 2021.","title":"Style-aware normalized loss for improving arbitrary style transfer","authors":"Jiaxin Cheng, Ayush Jaiswal, Yue Wu, Pradeep Natarajan, Prem Natarajan","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2021a] Yingying Deng, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, and Changsheng Xu. Arbitrary video style transfer via multi-channel corre- lation. AAAI, 2021.","title":"Arbitrary video style transfer via multi-channel correlation","authors":"Deng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2021b] Yingying Deng, Fan Tang, Xingjia Pan, Weiming Dong, Chongyang Ma, and Changsheng Xu. Stytrˆ2: Unbiased image style transfer with transformers. arXiv preprint arXiv:2105.14576, 2021.","title":"Stytrˆ2: Unbiased image style transfer with transformers","authors":"Deng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Donahue et al., 2017] Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, and Trevor Darrell. Long- term recurrent convolutional networks for visual recogni- tion and description. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 39(4):677-691, 2017.","title":"Longterm recurrent convolutional networks for visual recognition and description","authors":"Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, Trevor Darrell","year":"2017","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gatys et al., 2015a] Leon Gatys, Alexander S Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks. NeurIPS, 28:262-270, 2015.","title":"Texture synthesis using convolutional neural networks","authors":"Gatys","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gatys et al., 2015b] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015.","title":"A neural algorithm of artistic style","authors":"Gatys","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gu et al., 2017] Shixiang Gu, Ethan Holly, Timothy Lilli- crap, and Sergey Levine. Deep reinforcement learning for robotic manipulation with asynchronous off-policy up- dates. In 2017 IEEE ICRA, pages 3389-3396. IEEE, 2017.","title":"Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates","authors":"Gu","year":"2017","venue":"IEEE ICRA","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Haarnoja et al., 2017] Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies. In ICML, pages 1352- 1361. PMLR, 2017.","title":"Reinforcement learning with deep energy-based policies","authors":"Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, Sergey Levine","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Haarnoja et al., 2018] Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.","title":"Soft actor-critic algorithms and applications","authors":"Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2023] Jing Hu, Zhikun Shuai, Xin Wang, Shu Hu, Shanhui Sun, Siwei Lyu, and Xi Wu. Attention guided pol- icy optimization for 3d medical image registration. IEEE Access, 2023.","title":"Attention guided policy optimization for 3d medical image registration","authors":"Jing Hu, Zhikun Shuai, Xin Wang, Shu Hu, Shanhui Sun, Siwei Lyu, Xi Wu","year":"2023","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huang and Belongie, 2017] Xun Huang and Serge Be- longie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE ICCV, pages 1501-1510, 2017.","title":"Arbitrary style transfer in real-time with adaptive instance normalization","authors":"Belongie Huang","year":"2017","venue":"Proceedings of the IEEE ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Johnson et al., 2016] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV. Springer, 2016.","title":"Perceptual losses for real-time style transfer and super-resolution","authors":"Justin Johnson, Alexandre Alahi, Li Fei-Fei","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2017a] Yanghao Li, Naiyan Wang, Jiaying Liu, and Xiaodi Hou. Demystifying neural style transfer. arXiv preprint arXiv:1701.01036, 2017.","title":"Demystifying neural style transfer","authors":"Li","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/310"},{"position":20,"raw":"[Li et al., 2017b] Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang. Universal style transfer via feature transforms. Advances in neural infor- mation processing systems, 30, 2017.","title":"Universal style transfer via feature transforms","authors":"Li","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Li et al., 2019] Xueting Li, Sifei Liu, Jan Kautz, and Ming- Hsuan Yang. Learning linear transformations for fast image and video style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3809-3817, 2019.","title":"Learning linear transformations for fast image and video style transfer","authors":"Li","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lillicrap et al., 2015] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous con- trol with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.","title":"Continuous control with deep reinforcement learning","authors":"Jonathan J Timothy P Lillicrap, Alexander Hunt, Nicolas Pritzel, Tom Heess, Yuval Erez, David Tassa, Daan Silver, Wierstra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lin et al., 2021] Tianwei Lin, Zhuoqi Ma, Fu Li, Dongliang He, Xin Li, Errui Ding, Nannan Wang, Jie Li, and Xinbo Gao. Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer. In CVPR, pages 5141-5150, 2021.","title":"Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer","authors":"Tianwei Lin, Zhuoqi Ma, Fu Li, Dongliang He, Xin Li, Errui Ding, Nannan Wang, Jie Li, Xinbo Gao","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Liu et al., 2021] Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, and Errui Ding. Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6649-6658, 2021.","title":"Adaattn: Revisit attention mechanism in arbitrary neural style transfer","authors":"Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, Errui Ding","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Luo et al., 2021] Ziwei Luo, Jing Hu, Xin Wang, Siwei Lyu, Bin Kong, Youbing Yin, Qi Song, and Xi Wu. Stochastic actor-executor-critic for image-to-image translation. IJ- CAI, 2021.","title":"Stochastic actor-executor-critic for image-to-image translation","authors":"Ziwei Luo, Jing Hu, Xin Wang, Siwei Lyu, Bin Kong, Youbing Yin, Qi Song, Xi Wu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2021/382"},{"position":27,"raw":"[Mahendran and Vedaldi, 2015] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representa- tions by inverting them. In CVPR, pages 5188-5196, 2015.","title":"Understanding deep image representations by inverting them","authors":"Vedaldi Mahendran, Aravindh Mahendran, Andrea Vedaldi","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Mirowski et al., 2017] Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J Ballard, Andrea Ban- ino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, et al. Learning to navigate in complex envi- ronments. ICLR, 2017.","title":"Learning to navigate in complex environments","authors":"Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Park and Lee, 2019] Dae Young Park and Kwang Hee Lee. Arbitrary style transfer with style-attentional networks. In proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 5880-5888, 2019.","title":"Arbitrary style transfer with style-attentional networks","authors":"Lee ; ] Park, Young Dae, Kwang Park, Lee Hee","year":"2019","venue":"proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Phillips and Mackintosh, 2011] Fred Phillips and Brandy Mackintosh. Wiki art gallery, inc.: A case for critical thinking. Issues in Accounting Education, 26(3):593-608, 2011.","title":"Fred Phillips and Brandy Mackintosh. Wiki art gallery, inc.: A case for critical thinking","authors":"Mackintosh Phillips","year":"2011","venue":"Issues in Accounting Education","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Risser et al., 2017] Eric Risser, Pierre Wilmot, and Con- nelly Barnes. Stable and controllable neural texture syn- thesis and style transfer using histogram losses. arXiv preprint arXiv:1701.08893, 2017.","title":"Stable and controllable neural texture synthesis and style transfer using histogram losses","authors":"Risser","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sheng et al., 2018] Lu Sheng, Ziyi Lin, Jing Shao, and Xi- aogang Wang. Avatar-net: Multi-scale zero-shot style transfer by feature decoration. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8242-8250, 2018.","title":"Avatar-net: Multi-scale zero-shot style transfer by feature decoration","authors":"Lu Sheng, Ziyi Lin, Jing Shao, Xiaogang Wang","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Shi et al., 2015] Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural informa- tion processing systems, 28, 2015.","title":"Convolutional lstm network: A machine learning approach for precipitation nowcasting","authors":"Shi","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Tassa et al., 2018] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018.","title":"Deepmind control suite","authors":"Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego De Las, David Casas, Abbas Budden, Josh Abdolmaleki, Andrew Merel, Lefrancq","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ulyanov et al., 2016a] Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture net- works: Feed-forward synthesis of textures and stylized im- ages. In ICML, volume 1, page 4, 2016.","title":"Texture networks: Feed-forward synthesis of textures and stylized images","authors":"Ulyanov","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Ulyanov et al., 2016b] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.","title":"Instance normalization: The missing ingredient for fast stylization","authors":"Ulyanov","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2020] Wenjing Wang, Shuai Yang, Jizheng Xu, and Jiaying Liu. Consistent video style transfer via re- laxation and regularization. IEEE Transactions on Image Processing, 29:9125-9139, 2020.","title":"Consistent video style transfer via relaxation and regularization","authors":"Wenjing Wang, Shuai Yang, Jizheng Xu, Jiaying Liu","year":"2020","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xiang et al., 2022] Yanfei Xiang, Xin Wang, Shu Hu, Bin Zhu, Xiaomeng Huang, Xi Wu, and Siwei Lyu. Rmbench: Benchmarking deep reinforcement learning for robotic manipulator control. arXiv preprint arXiv:2210.11262, 2022.","title":"Rmbench: Benchmarking deep reinforcement learning for robotic manipulator control","authors":"Yanfei Xiang, Xin Wang, Shu Hu, Bin Zhu, Xiaomeng Huang, Xi Wu, Siwei Lyu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhao et al., 2019] Xujiang Zhao, Shu Hu, Jin-Hee Cho, and Feng Chen. Uncertainty-based decision making using deep reinforcement learning. In 2019 22th International Conference on Information Fusion (FUSION), pages 1-8. IEEE, 2019.","title":"Uncertainty-based decision making using deep reinforcement learning","authors":"Xujiang Zhao, Shu Hu, Jin-Hee Cho, Feng Chen","year":"2019","venue":"22th International Conference on Information Fusion (FUSION)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}