Modeling Topical Relevance for Multi-Turn Dialogue Generation

1 Introduction

Multi-turn dialogue generation is widely used in many natural language processing (NLP) applications, such as customer services, mobile assistant and chatbots. Given a conversation history containing several contexts, a dialogue generation model is required to automatically output an appropriate response. Therefore, how to fully understand and utilize these contexts is important for designing a good multi-turn dialogue generation model.

Different from single-turn dialogue generation, people usually model the multi-turn dialogue generation in a hierarchical way. A typical example is the Hierarchical Recurrent Encoder-Decoder (HRED) model [Serban et al., 2016; Sordoni et al., 2015]. In the encoding phase, a recurrent neural network (RNN) based encoder is first used to encode each context as a sentence-level vector, and then a hierarchical RNN is utilized to encode these context vectors to a history representation. In the decoding process, another RNN decoder is conducted to generate the response based on the history representation. The parameters of both encoder and decoder are learned by maximizing the averaged likelihood of the training data. However, the desired response is usually only dependent on some relevant contexts, instead of all the contexts. Recently, some works have been proposed to model the relevant contexts by using some similarity measures. For example, Tian et al. [2017] calculates the cosine similarity of the sentence embedding between the current context and the history contexts as the attention weights, Xing et al. [2018] introduces the word and sentence level attention mechanisms to HRED, and Zhang et al. [2019] utilizes the sentences level self-attention mechanism to detect the relevant contexts. However, these similarities are defined on either word or sentence level, which cannot well tackle the topic drift problem in multi-turn dialogue generation.

∗This work was done when the first author was a Ph.D student at ICT, CAS.

†Corresponding Author.

Here we give an example conversation, as shown in Table 1. The contexts are of three different topics. The (context1,context2) pair talks about 'greeting', the (context3,context4) pair talks about 'low-price', and the (context5,...,response) pair talks about 'invoice'. In this case, using all the contexts indiscriminately will obviously introduce many noises to the decoding process, which will hurt the performance of the multi-turn dialogue generation model. If we use word level similarities to locate the relevant contexts, the current context and context4 in the example will be associated because 'send' and 'receive' are highly similar words, which is clearly wrong. If we use sentence level similarities to locate the relevant contexts, it may still involve the false relevant context4 into consideration.

We argue that context relevance should be computed at the topic level, to better tackle the topic drift problem in multiturn dialogue generation. From both linguistic and cognitive perspective, topic is the high level cluster of knowledge, which can describe the relationship of sentences in the context, and has an important role in human dialogue for directing focus of attention. In this paper, we propose a new model, namely STAR-BTM, to model the Short-text Topic-level Attention Relevance with Biterm Topic Model (BTM) [Yan et al., 2013]. Specifically, we first pre-train the BTM model on the whole training data, which split every customer-server pair in the context as a short document. Then, we use the BTM to get each sentence topic distribution and calculate the topic distribution similarity between the current context and each history context as the relevance attention. Finally, we utilize the relevance attention and the topic distribution to conduct the decoding process. The BTMmodel and the text generation model are jointly learned to improve their performances in this process.

In our experiments, we use two public datasets to evaluate our proposed models, i.e., Chinese customer services and English Ubuntu dialogue corpus. The experimental results show that STAR-BTM generates more informative and suitable responses than traditional HRED models and its attention variants, in terms of both metric-based evaluation and human evaluation. Besides, we have shown the relevant attention words, indicating that STAR-BTM obtains coherent results with human's understanding.

2 RelatedWork

Recently, multi-turn dialogue generation has gained more attention in both research community and industry, compared with the single-turn dialogue generation [Li et al., 2017; Mou et al., 2017; Zhang et al., 2018a; Zhang et al., 2018b]. One of the reasons is that it is closely related to the real application, such as chatbot and customer service. More importantly, multi-turn dialogue generation needs to consider more information and constraints [Chen et al., 2018; Zhang et al., 2018c; Zhang et al., 2019; Wu et al., 2017; Zhou et al., 2016], which brings more challenges for the researchers in this area. To better model the historical information, Serban et al. [Serban et al., 2016] propose the HRED model, which uses a hierarchical encoder-decoder framework to model all the contexts information. With the widespread use of HRED, more and more variant models have been proposed. For example, Serban et al. [Serban et al., 2017b; Serban et al., 2017a] propose Variable HRED (VHRED) and MrRNN which utilize the latent variables as intermediate states to generate diverse responses.

However, it is unreasonable to use all the contexts indiscriminately for the multi-turn dialogue generation task, since the responses are usually only associated with a portion of the previous contexts. Therefore, some researchers try to use the similarity measure to define the relevance of the context. Tian et al. [Tian et al., 2017] propose a weighted sequence (WSeq) attention model for HRED, which uses the cosine similarity as the attention weight to measure the correlation of the contexts. But this model only uses the unsupervised sentence level representation, which fails to capture some detailed semantic information. Recently, Xing et al. [Xing et al., 2018] introduced the traditional attention mechanism [Bahdanau et al., 2015] into HRED, named hierarchical recurrent attention network (HRAN). In this model, the weight of attention is calculated based on the current state, the sentence level representation and the word level representation. However, the word level attention may introduce some noisy relevant contexts. Shen et al. [Chen et al., 2018] propose to introduce the memory network into the VHRED model, so that the model can remember the context information. Theoretically, it can retrieve some relevant information from the memory in the decoding phase, however, it is not clearly whether and how the system accurately extracts the relevant contexts. Zhang et al. [Zhang et al., 2019] proposed to use the sentence level self-attention to model the long distance dependency of contexts, to detect the relevant context for the multi-turn dialogue generation. Though it has the ability to tackle the position bias problem, the sentence level self-attention is still limited in capturing the topic level relevant contexts.

The motivation of this paper is to detect the topic level attention relevance for multi-turn dialogue generation. It is a more proper way to deal with the topic draft problem, as compared with the traditional word or sentence level methods. Some previous works[Xing et al., 2017; Xing et al., 2018] have been proposed to use topic model in dialogue generation. They maily use the topic model to provide some topic related words for generation, while our work focuses on detecting the topic level relevant contexts.

3 STAR-BTM

In this section, we will describe our Short-text Topic Attention Relevance with Biterm Topic Model (STAR-BTM) in detail, with the architecture shown in Figure 1. STAR-BTM consists of three modules, i.e. , the pre-trained BTM model, topic level attention module and the joint learning decoder. Firstly, we pre-train the BTM model on the whole training data, to obtain the topic word distribution of each context. Secondly, the topic level attention is calculated as the similarity between the topic distributions of the current context and each history context. After that, the attention weights are multiplied with the hierarchical hidden state in HRED to obtain the history representation. Finally, the history representation and the topic distribution of the current context are concatenated to decode the response step by step.

From the architecture, we can see that STAR-BTM introduces the short text topic model into the HRED model, to incorporate the topic level relevant contexts to the decoding process. It is clear that the topic level distribution can provide more specific topic information than only using the word and sentence level representations. What is more, the topic model firstly 'sees' the whole data globally by the pre-training techniques, and is then fine-tuned by the joint learning technique with the generation model.

3.1 Pre-train BTMModule

We use the pre-trained BTM model on the whole training data to obtain the topic distribution. The pre-trained model on training data can be viewed as the background knowledge, which supplies additional information for the current dialogue session. Like human dialogue in reality, the background knowledge about potential topics will help to detect actual focus of attention model.

BTM [Yan et al., 2013] is a widely used topic model especially designed for short text, which is briefly introduced as follows. For each co-occurrence biterm b = (wi, wj ) of word wi and wj, the joint probability of b is written as:

where t stands for a topic.

To infer the topics in a document, BTM assumes that the topic proportions of a document equals to the expectation of the topic proportions of biterms generated from the document. Then we have,

where d is a document.

Both P(t|b) and P(b|d) can be calculated via Bayes�formula as follows.

where nd(b) is the frequency of the biterm b in the document d. The parameters inference is based on the Gibbs Sampling.

Now we introduce how we apply BTM in our work. Firstly, we split the whole training data D = {(C, Y ) = (c1, . . . , cN, Y )} to context pairs, i.e. D = {(c1, c2), (c3, c4), . . . , (cN, Y )}. In the training process, we treat each context pair as one document for BTM. This is reasonable because each pair can be viewed as a single-turn dialogue, and the input and output of a single-turn dialogue are usually of the same topic. After utilizing the Gibbs Sampling, we obtain the word distribution of each topic P(wi|t) and the topic distribution P(t). In the inferring process, given each sentence ci in D, the topic of ci is computed by P(ti) = argmaxt P(t|ci) in Equation 1.

The BTM model is more suitable for the dialogue generation task than the traditional topic models, such as Latent Dirichlet Allocation(LDA) model. That is because the dialogue has the characteristic of short text with omitted information, which makes LDA not reliable any more. BTM uses word co-occurrence as the core information to determine the topic. So it only depends on the semantic dominance of local co-occurrence information, breaks the document boundary, uses the information of the entire corpus instead of a single document to overcome the sparse problem in short text topic modeling.

3.2 Topic-level AttentionModule

We define the context data as C = {c1, . . . , cN}, and each sentence in C is defined as ci = {x(i) 1 , . . . , x(i) M }. Given the sentence ci as input, the RNN model first maps the input sequence ci to the fixed dimension vector hiM as follows:

where w(i) k is the word vector of x(i) k , h(i) k represents the hidden state vector of the RNN at time k, which combines w(i) k and h(i) k-1 . We obtained the state representation set of the contexts {h(1) , . . . , h(N) }.

Then we use a high-level RNN model to take the context state representation set {h(1) , . . . , h(N) } as input, and obtain the high-level context representation vector sk:

where h(k) is the vector representation of the k-th sentence, and sk represents the state vector of the high-level RNN at time k, which combines h(k) and sk-1. We obtained the output of the high-level RNN at each step: {s1, . . . , sN}.

Given the context data C = {c1, . . . , cN}, we obtained the topic for each sentence as T = {t1, . . . , tN} through the pretrained BTM model. We define attention weights as:

where E(ti, ci) is the sum of the word distribution for topic ti and the projected word distribution for context ci, which is defined as the product of the word distribution for topic ti and the one-hot representation of context ci.

Finally, we obtain the softmax attention weights α′i and the context vector SN as:

3.3 Joint Learning Decoder

We conduct another RNN as the decoder to generate the response Y = {y1, · · · , yM}. Given the context vector SN, the topic distribution of the current context DN and the previously generated word y1, · · · , yi-1, the decoder predicts the probability of the next word yi by converting the joint probability into a conditional probability through a chain rule in probability theory. We use the topic distribution of the current context DN in decoder for the reason that it could supply the topic information to generate more relevant response.

Given a set of training data D = {(C, T, Y )}, STAR-BTM assumes that the data is conditionally independent, and samples from the probability Pg, and uses the following negative log likelihood as a minimized objective function:

where C is the context, T is the topic distribution of C and Y is the real response.

4 Experiment

In this section, we conducted experiments on the Chinese customer service dataset and the English Ubuntu conversation dataset to verify the effectiveness of our proposed method.

4.1 Experimental Settings

We first introduce experimental settings, including datasets, baselines, parameter settings, and evaluation measures.

Datasets

We utilize two public multi-turn dialogue datasets in our experiments, which are widely used in the evaluation of multiturn dialogue generation task. The Chinese customer service dataset, named JDC, consists of 515,686 history-response pairs published by the JD contest 1 . We randomly divided the corpus into training, validation and testing, each contains 500,000, 7843, and 7843 pairs, respectively. The Ubuntu conversation dataset 2 is extracted from the Ubuntu Q&A forum, called Ubuntu [Lowe et al., 2015]. We utilize the official scripts for tokenizing, stemming and morphing, and remove the duplicates and sentence whose length is less than 5 or greater than 50. Finally, we obtain 3,980,000, 10,000, and 10,000 history-response pairs for training, validation and testing, respectively.

Baseline Methods and Parameter Settings

We used seven baseline methods for comparison, including the traditional Seq2Seq [Sutskever et al., 2014], HRED [Serban et al., 2016], VHRED [Serban et al., 2017b], Weighted Sequence with Concat (WSeq) [Tian et al., 2017], Hierarchical Recurrent Attention Network (HRAN) [Xing et al., 2018], Hierarchical Hidden Variational Memory Network (HVMN) [Chen et al., 2018] and Relevant Context with SelfAttention (ReCoSa) [Zhang et al., 2019]. To fairly compare the topic-level attention model with self-attention model, we extend our STAR-BTM to the ReCoSa scenario, named ReCoSa-BTM, where the topic embedding is concatenated with the sentence representation.

1 http://jddc.jd.com/

2 https://github.com/rkadlec/ubuntu-ranking-dataset-creator

For JDC, the Jieba tool is utilized for Chinese word segmentation, and its vocabulary size is set to 68,521. For Ubuntu, we set the vocabulary size to 15,000. To fairly compare our model with all baselines, the number of hidden nodes is all set to 512 and the batch size set to 32. The max length of sentence is set to 50 and the max number of dialogue turns is set to 15. The number of topics in BTM is set to 8. We use the Adam for gradient optimization in our experiments. The learning rate is set to 0.0001. We run all models on the Tesla K80 GPU with Tensorflow. 3

Evaluation Measures

We use both quantitative evaluation and human judgements in our experiments. Specifically, we use the traditional indicators, i.e., PPL and BLEU [Xing et al., 2017] to evaluate the quality of generated responses [Chen et al., 2018; Tian et al., 2017; Xing et al., 2018]. And we also use the distinct value [Li et al., 2016a; Li et al., 2016b] to evaluate the degree of diversity of generation responses. They are widely used in NLP and multi-turn dialogue generation tasks [Chen et al., 2018; Tian et al., 2017; Xing et al., 2018; Zhang et al., 2018c; Zhang et al., 2018a; Zhang et al., 2018b].

For human evaluation, given the 300 randomly sampled context and its generated responses from all the models, we invited three annotators (all CS majored students) to compare the STAR-BTM model with the baseline methods, e.g. win and loss, based on the coherence of the generated response with respect to the contexts. In particular, the win tag indicates that the response generated by STAR-BTM is more relevant than the baseline model. In order to compare the informativeness of the response generated by the models, we also require the annotators to label the informativeness of each model. If the response generated by STAR-BTM is more informative than the baseline, the annotator will label 1, otherwise label 0.

4.2 Experimental Results

Experimental results on two datasets are demonstrate below.

Metric-based Evaluation

The metric-based evaluation results are shown in Table 2. From the results, we can see that the models which detect the relevant contexts, such as WSeq, HRAN, HVMN and ReCoSa, are superior to the traditional HRED baseline models in terms of BLEU, PPL and distinct. This is mainly because these models further consider the attention of the relevant context information rather than all the contexts in the optimization process. HRAN introduces the traditional attention mechanism to learn the important context sentences. HVMN utilizes the memory network to remember the context information. ReCoSa uses the self-attention to detect the relevant contexts. But their effects are limited since they do not consider the topical level relevance. Our proposed STARBTM and ReCoSa-BTM have shown good results. Taking the BLEU value on the JDC dataset as an example, the BLEU value of the STAR-BTM and ReCoSa-BTM are 13.386 and JDC Dataset

3 https://github.com/zhanghainan/STAR-BTM

13.912, which are significantly better than that of HVMN and ReCoSa, i.e., 13.125 and 13.797. The distinct value of our model is also higher than other baseline models, indicting that our model can generate more diverse responses. We also conducted a significance test. The results show that the improvement of our model is significant in both Chinese and English datasets with p-value < 0.01. In summary, our STAR-BTM and ReCoSa-BTM model are able to generate higher quality and more diverse responses than the baselines.

Human Evaluation

The results of human evaluation are shown in Table 3. The percentage of win, loss, and informativeness(inform.), as compared with the baseline models, are given to evaluate the quality and the informativeness of the generated responses by STAR-BTM. From the experimental results, the percentage of win is greater than the loss, indicating that our STAR-BTM model is significantly better than the baseline methods. Taking JDC as an example, STAR-BTM obtains a preference gain (i.e., the win ratio minus the loss ratio) of 36.18 %, 23.4 %, 14.89 % and 4.63%, respectively, as compared with WSeq, HRAN, HVMN and ReCoSa. In addition, the percentage of informativeness is more than 50 percent, as compared with WSeq, HRAN, HVMN and ReCoSa, i.e.,66.99%�60.19%, 61.02% and 55.63%, respectively, showing that topic level information is effective for the multi-turn dialogue generation task and our STAR-BTM can generate interesting response with more information. The Kappa [Fleiss, 1971] value demonstrates the consistency of different annotators.

Case Study

To facilitate a better understanding of our model, we present some examples in Table 4, and show the top 10 words of each topic in the Table 5. From the results, we can see that why the topic level attention model performs better than the model only using the word and sentence level representation. Taking the example1 in Table 4 as an example, it easy to generate common responses by using only sentence level representation, such as 'What can I do for you?' and 'Yes'. However, our topic level attention model has the ability to generate more relevant and informative responses, such as 'Based on the submitted after-sales service form' and 'Yes, you need apply after-sales and select lack'. This is mainly because the topic level attention is able to associate some important information such as 'e�(send a new one for a replacement)' and '.�(after-sales)' by topic modeling, which are usually hard to be captured by traditional word or sentence level similarities. These results indicate the advantage of modeling topic level relevance.

We also show the top 10 words of each topic from the BTM model on the two dataset, as shown in Table 5. Take the JDC dataset as an example, from the results, we can see that BTM model distinguishes eight topics, i.e., 'M �(shipping), � h(invoice), � >(refund), .�(after-sale), ‹U(reminder), ��(low-price),:'(out-of-stock) and � "(thanks)'. For each topic, the top 10 words represent the core information of the topic. Take the example1 in the Table 4 as an example, since the 'e�(send a new one for a replacement)' and '.�(after-sales)' are the 15-th and second word in the same topic 4, respectively, the model can generate 'submitted after-sales service form' based on the topic level attention. In the example2, the current context is about the 'gateway' with topic 'network', so the topic distribution can supply some additional topic information, such as 'restart', 'dhcp' and 'router'. In a word, our STAR-BTM and ReCoSa-BTM model can supply the critical topic information to improve the informativeness of the generated response.

5 Conclusion

In this paper, we propose a new multi-turn dialogue generation model, namely STAR-BTM. The motivation comes from the fact that topic drift is a common phenomenon in multiturn dialogue. The existing models usually use word or sentence level similarities to detect the relevant contexts, which ignore the topic level relevance. Our core idea is to utilize topic models to detect the relevant context information and generate a suitable response accordingly. Specifically, STARBTM first pre-trains a Biterm Topic Model on the whole train- Example1

ing data, and then incorporate the topic level attention weights to the decoding process for generation.We conduct extensive experiments on both Chinese customer services dataset and English Ubuntu dialogue dataset. The experimental results show that our model significantly outperforms existing HRED models and its attention variants. Therefore, we obtain the conclusion that the topic-level information can be useful for improving the quality of multi-turn dialogue generation, by using proper topic model, such as BTM.

In future work, we plan to further investigate the proposed STAR-BTMmodel. For example, some personal information can be introduced to supply more relevant information for personalized modeling. In addition, some knowledges like concerned entities can be considered in the relevant contexts to further improve the quality of generated response.

Acknowledgments

This work was supported by Beijing Academy of Artificial Intelligence (BAAI) under Grants No. BAAI2019ZD0306, and BAAI2020ZJ0303, the National Natural Science Foundation of China (NSFC) under Grants No. 61722211, 61773362, 61872338, and 61906180, the Youth Innovation Promotion Association CAS under Grants No. 20144310, and 2016102, the National Key R&D Program of China under Grants No. 2016QY02D0405, the Lenovo-CAS Joint Lab Youth Scientist Project, and the Foundation and Frontier Research Key Program of Chongqing Science and Technology Commission (No. cstc2017jcyjBX0059), and the Tencent AI Lab Rhino-Bird Focused Research Program (No. JR202033).