{"type":"ScholarlyArticle","title":"Learning to Converse with Noisy Data: Generation with Calibration","authors":[{"name":"Mingyue Shang","slug":"mingyue-shang","given":"Mingyue","family":"Shang","affiliation":"Institute of Computer Science and Technology , Peking University , China"},{"name":"Zhenxin Fu","slug":"zhenxin-fu","given":"Zhenxin","family":"Fu","affiliation":"Institute of Computer Science and Technology , Peking University , China"},{"name":"Nanyun Peng","slug":"nanyun-peng","given":"Nanyun","family":"Peng","affiliation":"Information Science Institute , University of Southern California , USA"},{"name":"Yansong Feng","slug":"yansong-feng","given":"Yansong","family":"Feng","affiliation":"Institute of Computer Science and Technology , Peking University , China"},{"name":"Dongyan Zhao","slug":"dongyan-zhao","given":"Dongyan","family":"Zhao","affiliation":"Institute of Computer Science and Technology , Peking University , China; Beijing Institute of Big Data Research , China"},{"name":"Rui Yan","slug":"rui-yan","given":"Rui","family":"Yan","affiliation":"Institute of Computer Science and Technology , Peking University , China; Beijing Institute of Big Data Research , China"}],"abstract":"The availability of abundant conversational data on the Internet brought prosperity to the generation-based open domain conversation systems. In the training of the generation models, existing methods generally treat all the training data equivalently. However, the data crawled from the websites may contain many noises. Blindly training with the noisy data could harm the performance of the final generation model. In this paper, we propose a generation with calibration framework, that allows high- quality data to have more influences on the generation model and reduces the effect of noisy data. Specifically, for each instance in training set, we employ a calibration network to produce a quality score for it, then the score is used for the weighted update of the generation model parameters. Experiments show that the calibrated model outperforms baseline methods on both automatic evaluation metrics and human annotations.","doi":"10.24963/ijcai.2018/603","url":"https://ijcai.org/proceedings/2018/603","pdf":"https://ijcai.org/proceedings/2018/0603.pdf","markdown":"https://ijcai.org/proceedings/2018/603.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/603","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Natural Language Processing","track":"Main track","number":603,"pages":{"first":4338,"last":4344},"keywords":["Natural Language Processing: Natural Language Generation","Natural Language Processing: Dialogue"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hu et al., 2014] Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. Convolutional neural network architectures for matching natural language sentences. In Advances in neural information processing systems, pages 2042-2050, 2014.","title":"Convolutional neural network architectures for matching natural language sentences","authors":"Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Jiang and Zhai, 2007] Jing Jiang and ChengXiang Zhai. In- stance weighting for domain adaptation in nlp. In Pro- ceedings of the 45th annual meeting of the association of computational linguistics, pages 264-271, 2007.","title":"Instance weighting for domain adaptation in nlp","authors":"Zhai ; ] Jing Jiang, Chengxiang Jiang, Zhai","year":"2007","venue":"Proceedings of the 45th annual meeting of the association of computational linguistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2015] Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting ob- jective function for neural conversation models. arXiv preprint arXiv:1510.03055, 2015.","title":"A diversity-promoting objective function for neural conversation models","authors":"Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, Bill Dolan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2016] Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. A persona-based neural conversation model. arXiv preprint arXiv:1603.06155, 2016.","title":"A persona-based neural conversation model","authors":"Jiwei Li, Michel Galley, Chris Brockett, P Georgios, Jianfeng Spithourakis, Bill Gao, Dolan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lison and Bibauw, 2017] Pierre Lison and Serge Bibauw. Not all dialogues are created equal: Instance weighting for neural conversational models. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 384-394, 2017.","title":"Not all dialogues are created equal: Instance weighting for neural conversational models","authors":"Bibauw Lison, Pierre Lison, Serge Bibauw","year":"2017","venue":"Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2016] Chia-Wei Liu, Ryan Lowe, Iulian V Ser- ban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation met- rics for dialogue response generation. arXiv preprint arXiv:1603.08023, 2016.","title":"How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation","authors":"Chia-Wei Liu, Ryan Lowe, Michael Iulian V Serban, Laurent Noseworthy, Joelle Charlin, Pineau","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lowe et al., 2017] Ryan Lowe, Michael Noseworthy, Iu- lian V Serban, Nicolas Angelard-Gontier, Yoshua Ben- gio, and Joelle Pineau. Towards an automatic turing test: Learning to evaluate dialogue responses. arXiv preprint arXiv:1708.07149, 2017.","title":"Towards an automatic turing test: Learning to evaluate dialogue responses","authors":"Ryan Lowe, Michael Noseworthy, Nicolas Iulian V Serban, Yoshua Angelard-Gontier, Joelle Bengio, Pineau","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lu and Li, 2013] Zhengdong Lu and Hang Li. A deep ar- chitecture for matching short texts. In Advances in Neural Information Processing Systems, pages 1367-1375, 2013.","title":"A deep architecture for matching short texts","authors":"Li ; ] Zhengdong Lu, Hang Lu, Li","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mou et al., 2016] Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. arXiv preprint arXiv:1607.00970, 2016. [Rebbapragada and Brodley, 2007] Umaa Rebbapragada and Carla E Brodley. Class noise mitigation through instance weighting. In European Conference on Machine Learning, pages 708-715. Springer, 2007.","title":"Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation","authors":"Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, Zhi Jin","year":"2007","venue":"Umaa Rebbapragada and Carla E Brodley. Class noise mitigation through instance weighting. In European Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ritter et al., 2011] Alan Ritter, Colin Cherry, and William B Dolan. Data-driven response generation in social media. In Proceedings of the conference on empirical methods in natural language processing, pages 583-593. Association for Computational Linguistics, 2011.","title":"Data-driven response generation in social media","authors":"Alan Ritter, Colin Cherry, William B Dolan","year":"2011","venue":"Proceedings of the conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Serban et al., 2016] Iulian Vlad Serban, Alessandro Sor- doni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. Building end-to-end dialogue systems using gen- erative hierarchical neural network models. In AAAI, pages 3776-3784, 2016.","title":"Building end-to-end dialogue systems using generative hierarchical neural network models","authors":"Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, Joelle Pineau","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shang et al., 2015] Lifeng Shang, Zhengdong Lu, and Hang Li. Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364, 2015.","title":"Neural responding machine for short-text conversation","authors":"Lifeng Shang, Zhengdong Lu, Hang Li","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Song et al., 2017] Yiping Song, Zhiliang Tian, Dongyan Zhao, Ming Zhang, and Rui Yan. Diversifying neural con- versation model with maximal marginal relevance. In Pro- ceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), volume 2, pages 169-174, 2017.","title":"Diversifying neural conversation model with maximal marginal relevance","authors":"Yiping Song, Zhiliang Tian, Dongyan Zhao, Ming Zhang, Rui Yan","year":"2017","venue":"Proceedings of the Eighth International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sordoni et al., 2015] Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. A neural network approach to context-sensitive gen- eration of conversational responses. arXiv preprint arXiv:1506.06714, 2015.","title":"A neural network approach to context-sensitive generation of conversational responses","authors":"Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, Bill Dolan","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tao et al., 2017] Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. Ruber: An unsupervised method for auto- matic evaluation of open-domain dialog systems. arXiv preprint arXiv:1701.03079, 2017.","title":"Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems","authors":"Chongyang Tao, Lili Mou, Dongyan Zhao, Rui Yan","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vinyals and Le, 2015] Oriol Vinyals and Quoc Le. A neural conversational model. arXiv preprint arXiv:1506.05869, 2015.","title":"Oriol Vinyals and Quoc Le. A neural conversational model","authors":"Le Vinyals","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. Instance weighting for neural machine translation domain adaptation. In Proceed- ings of the 2017 Conference on Empirical Methods in Nat- ural Language Processing, pages 1482-1488, 2017.","title":"Instance weighting for neural machine translation domain adaptation","authors":"Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, Eiichiro Sumita","year":"2017","venue":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xing et al., 2017] Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. Topic aware neural response generation. In AAAI, pages 3351-3357, 2017.","title":"Topic aware neural response generation","authors":"Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2016] Rui Yan, Yiping Song, and Hua Wu. Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In Proceedings of the 39th International ACM SIGIR con- ference on Research and Development in Information Re- trieval, pages 55-64. ACM, 2016.","title":"Learning to respond with deep neural networks for retrieval-based human-computer conversation system","authors":"Rui Yan, Yiping Song, Hua Wu","year":"2016","venue":"Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}