{"type":"ScholarlyArticle","title":"Utilizing Non-Parallel Text for Style Transfer by Making Partial Comparisons","authors":[{"name":"Di Yin","slug":"di-yin","given":"Di","family":"Yin","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"},{"name":"Shujian Huang","slug":"shujian-huang","given":"Shujian","family":"Huang","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"},{"name":"Xin-Yu Dai","slug":"xin-yu-dai","given":"Xin-Yu","family":"Dai","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"},{"name":"Jiajun Chen","slug":"jiajun-chen","given":"Jiajun","family":"Chen","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"}],"abstract":"Text style transfer aims to rephrase a given sentence into a different style without changing its original content. Since parallel corpora (i.e. sentence pairs with the same content but different styles) are usually unavailable, most previous works solely guide the transfer process with distributional information, i.e. using style-related classifiers or language models, which neglect the correspondence of instances, leading to poor transfer performance, especially for the content preservation. In this paper, we propose making partial comparisons to explicitly model the content and style correspondence of instances, respectively. To train the partial comparators, we propose methods to extract partial-parallel training instances automatically from the non-parallel data, and to further enhance the training process by using data augmentation. We perform experiments that compare our method to other existing approaches on two review datasets. Both automatic and manual evaluations show that our approach can significantly improve the performance of existing adversarial methods, and outperforms most state-of-the-art models. Our code and data will be available on Github.","doi":"10.24963/ijcai.2019/747","url":"https://ijcai.org/proceedings/2019/747","pdf":"https://ijcai.org/proceedings/2019/0747.pdf","markdown":"https://ijcai.org/proceedings/2019/747.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/747","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Natural Language Processing","track":"Main track","number":747,"pages":{"first":5379,"last":5386},"keywords":["Natural Language Processing: Natural Language Generation","Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In Proceedings of ICML, 2017.","title":"Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fu et al., 2018] Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. Style transfer in text: Explo- ration and evaluation. In Proceedings of AAAI conference on Artificial Intelligence, pages 663-670, 2018.","title":"Style transfer in text: Exploration and evaluation","authors":"Fu","year":"2018","venue":"Proceedings of AAAI conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in Neural Informa- tion Processing Systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hovy, 1987] Eduard Hovy. Generating natural language under pragmatic constraints. Journal of Pragmatics, 11(6):689-719, 1987.","title":"Generating natural language under pragmatic constraints","authors":"Eduard Hovy, Hovy","year":"1987","venue":"Journal of Pragmatics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2014] Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Proceedings of NIPS, pages 2042-2050, 2014.","title":"Convolutional neural network architectures for matching natural language sentences","authors":"Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen","year":"2014","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hu et al., 2017] Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward controlled generation of text. In International Conference on Ma- chine Learning, pages 1587-1596, 2017.","title":"Toward controlled generation of text","authors":"Hu","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jang et al., 2016] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lan and Xu, 2018] Wuwei Lan and Wei Xu. Neural net- work models for paraphrase identification, semantic tex- tural similarity, natural language inference and question answeing. In Proceedings of COLING, 2018.","title":"Neural network models for paraphrase identification, semantic textural similarity, natural language inference and question answeing","authors":"Lan, Xu ; ] Wuwei Lan, Wei Xu","year":"2018","venue":"Proceedings of COLING","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2018a] Juncen Li, Robin Jia, He He, and Percy Liang. Delete, retrieve, generate: A simple approach to sentiment and style transfer. In Proceedings of NAACL- HLT 2018, pages 1865-1874, 2018.","title":"Delete, retrieve, generate: A simple approach to sentiment and style transfer","authors":"Li","year":"2018","venue":"Proceedings of NAACL-HLT 2018","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018b] Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. Paraphrase generation with deep reinforcement learning. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing, pages 3865-3878, 2018.","title":"Paraphrase generation with deep reinforcement learning","authors":"Li","year":"2018","venue":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mathews et al., 2018] Alexander Mathews, Lexing Xie, and Xuming He. Semstyle: Learning to generate stylised im- age captions using unaligned text. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 8591-8600, 2018.","title":"Semstyle: Learning to generate stylised image captions using unaligned text","authors":"Alexander Mathews, Lexing Xie, Xuming He","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mirza and Osindero, 2014] Mehdi Mirza and Simon Osin- dero. Conditional generative adversarial net. In arXiv preprint arXiv:1411.1784, 2014.","title":"Conditional generative adversarial net","authors":"Mehdi Mirza, Simon Osindero","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Niu and Bansal, 2018] Tong Niu and Mohit Bansal. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:273-389, 2018.","title":"Polite dialogue generation without parallel data","authors":"Tong Niu, Mohit Bansal","year":"2018","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rao and Tetreault, 2018] Sudha Rao and Joel R. Tetreault. Dear sir or madam, may i introduce the gyafc dataset: Cor- pus, benchmarks and metrics for formality style transfer. In Proceedings of NAACL-HLT, pages 129-140, 2018.","title":"Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer","authors":"Sudha Rao, Joel R Tetreault","year":"2018","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2017] Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Informa- tion Processing Systems, pages 6833-6844, 2017.","title":"Style transfer from non-parallel text by cross-alignment","authors":"Shen","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approxi- mation. In Advances in neural information processing sys- tems, pages 1057-1063, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tieleman and Hinton, 2012] Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. In COURSERA: Neural Networks for Machine Learning, 2012.","title":"Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude","authors":"Hinton Tieleman, Tijmen Tieleman, Geoffrey Hinton","year":"2012","venue":"COURSERA: Neural Networks for Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2012] Wei Xu, Alan Ritter, Bill Dolan, Ralph Gr- ishman, and Colin Cherry. Paraphrasing for style. In Pro- ceedings of the 24th International Conference on Compu- tational Linguistics, page 2899-2914, 2012.","title":"Paraphrasing for style","authors":"Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, Colin Cherry","year":"2012","venue":"Proceedings of the 24th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2018] Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick. Unsupervised text style transfer using language models as discrimina- tors. In Advances in Neural Information Processing Sys- tems, pages 1-12, 2018.","title":"Unsupervised text style transfer using language models as discriminators","authors":"Yang","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of AAAI conference on Artificial Intelligence, pages 2852-2858, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"Proceedings of AAAI conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2018] Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. Generating informative and diverse conversational re- sponses via adversarial information maximization. In Ad- vances in Neural Information Processing Systems, 2018.","title":"Generating informative and diverse conversational responses via adversarial information maximization","authors":"Zhang","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}