TopicKA: Generating Commonsense Knowledge-Aware Dialogue Responses Towards the Recommended Topic Fact

1 Introduction

Commonsense knowledge plays a crucial role in our daily lives, which consists of a set of widespread and well-known facts: for example, "cats are animals". During conversations, humans can make commonsense inferences that frame their understandings, and help them in making responses [Zhou et al., 2018]. However, unlike human beings, machines can merely utilize the surface knowledge that appears in the given query [Ghazvininejad et al., 2018]. Consequently, traditional models tend to generate generic responses (e.g., "I don't know"), or they generate responses that violate the commonsense (e.g. "The earth is square"). With the maturation of the large-scale commonsense knowledge bases such as ConceptNet [Speer et al., 2017] and ATOMIC [Sap et al., 2019], commonsense knowledge has been successfully applied in the language understanding, generation, and many others.

∗Corresponding author

The feasibility of incorporating commonsense knowledge into dialogue systems has already been proved. The first attempt [Young et al., 2018] integrates commonsense knowledge into the retrieval-based dialogue systems. Subsequently, commonsense knowledge has been introduced into the generative models [Zhou et al., 2018]. Moreover, if we broaden our horizons to other types of knowledge-aware dialogue systems [Zhu et al., 2017; Liu et al., 2019], some of them can also be transferred to commonsense knowledge-aware approaches. However, the performance of previous methods is still far from satisfactory. When utilizing the commonsense knowledge, the decision-making with which knowledge facts should be used is usually an uncontrollable black-box process, which always leads to the following issues: (1) Knowledge facts irrelevant to the dialogue context may be used. (2) The generation process is neither controllable nor interpretable. (3) The diversity and the informativeness among the generated responses can not be guaranteed.

This paper proposes a novel Topic fact-based commonsense Knowledge-Aware model, TopicKA, for the singleturn open-domain dialogue generation. The most notable difference between our proposal and the previous knowledgeaware approaches is that the dialogue generation framework. As illustrated in Figure 1, instead of generating a response conditioned on the given query message only, as traditional approaches do, the generation in our approach is conditioned on the query message and a controllable and interpretable topic fact simultaneously. Topic facts refer to the commonsense facts that can semantically and logically connect the query message and the target response. Topic facts are recommended by the proposed Topic Fact Recommender, and they are represented as discrete categorical variables, in our approach. Topic facts will be continuously involved in the generation process to guide the direction of the generation. The advantages of using topic facts include the flexibility to control the direction of the generation, higher interpretability to explain the knowledge selection process, and the ability to generate multiple diverse responses.

Three challenges have subsequently emerged. The first is how to select appropriate topic facts. To recommend a topic fact for a query message, we design a recommendation network, Topic Fact Recommender, which recommends topic facts by ranking the preliminary knowledge facts retrieved by entity names. Meanwhile, inspired by the Teacher-Student framework [Hinton et al., 2015], we design a Boost Network that further uses the posterior knowledge as a teacher to improve the ranking performance. The next is how to utilize the recommended topic facts during the response generation. We first propose a diffusion mechanism to diffuse a topic fact, and then the diffused topic fact can guide the generation of the response by using the proposed TFAG (Topic Fact-Aware GRU). The last challenge is how to integrate the recommendation network into the generation network. The generation network accepts a discrete categorical topic fact as an input, which requires sampling a topic fact from the output of the recommendation network. The sampling process would make the integration non-differentiable. To overcome this, we propose two non-sampling integration schemes: Two-Stage Learning and Multi-Task Learning; and we propose two differentiable sampling approaches: one is inspired by [Zhao et al., 2017], and another one approximates such a sampling process by using Gumbel-Softmax [Jang et al., 2017]. Both the automatic evaluation and the human annotation on an open Chinese benchmark dataset indicate that the proposed TopicKA can outperform the SOTA approach CCM [Zhou et al., 2018] in terms of most metrics. In the case study, we verify the controllability, interpretability, and diversity.

Our contributions are: (1) We proposed a novel topic factbased commonsense knowledge-aware approach, TopicKA. (2) We proposed four schemes to integrate the generator and the recommender. (3) We collected and constructed a largescale Chinese commonsense knowledge graph. Experimental results indicate our TopicKA outperforms various kinds of baselines.

2 Approach

2.1 ProblemFormulation

Let X denote the query message, Y denote the target response, F = {f} denote a set of facts retrieved from the commonsense knowledge graph G, and ft denote the topic fact. Each fact is organized as a SVO (Subject, Verb, Object) triplet, namely, f = (e1, r, e2). Instead of modeling dialogue generation as P(Y |X), TopicKA generates a response conditioned on the query message and a recommended topic fact simultaneously; that is, P(Y |X, ft).

2.2 Context Encoder

Context Encoder Cθ(·) is a bi-directional GRU network [Cho et al., 2014], which encodes an utterance U into contextual intermediate representations Hu = (hu1 , . . . , huT). The forward GRU reads the U in the normal order; the backward GRU reads the U in the inversed order. At the time step t, the output hut is the concatenation of two directions' outputs.

2.3 Topic Fact Recommender

The topic fact ft is selected from the retrieved knowledge facts F = {f} given the prior context Hx . The probability that fi can be recommended as a topic fact is given by a probability distribution over the F :

where f is the embedding of f, qxi is an attentive context, and α(·) is an alignment function [Luong et al., 2015], which measures the relevance between the two inputs.

Inspired by the success of distillation learning (TeacherStudent framework) [Hinton et al., 2015], we design a Boost Network (teacher) to improve the ranking performance of the recommendation network (student), by applying posterior soft labels. The Boost Network has a network structure similar to that of the student network, but it additionally employs the posterior context Hy :

Two networks share the same Context Encoder and the same embedding. Importantly, two networks are jointly trained; therefore, the objective is to minimize:

where φ denotes the parameters of two networks, ˆ ft is the labeled topic fact, CE(·, ·) is the Cross-Entropy that is used to optimize two networks, and KL(·||·) is the Kullback-Leibler Divergence [Kullback and Leibler, 1951] to force the student to generate a distribution similar to the teacher's distribution.

2.4 Knowledge-Aware Dialogue Generation

Topic Fact Diffusion Network

A single fact is usually not enough to cover the knowledge of a topic. Therefore, we design a Topic Fact Diffusion Network to retrieve more relevant facts based on the ft. Given the context and a topic fact ft, the diffusion network outputs a probability distribution pz over the F, by using an attention function D with softmax:

where elu is an activation function [Clevert et al., 2016], pz[i] is regarded as the relevance of the i-th fact to the central topic fact ft, and the fused fz is the diffused topic fact embedding. To improve the diversity, the objective entity will not be considered during generating pz, Fsv/fc,sv is the corresponding ablation of F/ft. As the recommendation network, the diffusion network utilizes the posterior context during the traning , where pz is the prior one, ppo z is the posterior one. KL(ppo z ||pz) is subsequently adopted.

Topic Fact-Aware GRU

Intuitively, a straightforward approach, by which to use the introduced topic facts to control the generation, is directly attaching such contents to the input of GRU. However, this approach usually fails to generate a fluent and diverse response, because it lacks enough flexibility to control the degree of participation of the introduced topic facts. Inspired by [Arevalo et al., 2017; Yao et al., 2017], we propose a TFAG (Topic Fact-Aware GRU) to integrate the diffused topic fact flexibly. A TFAG unit consists of two different GRUs that accept different inputs, and a FusionUnit ( see [Arevalo et al., 2017] for the detail) to fuse two GRUs' internal states. At the time step t, the internal state st of TFAG can be updated as :

where the first standard GRU s works as most conventional GRUs do, which outputs its own state sct given the attention ct-1 over the query message X and the embedding of the last generated word yt-1. The next GRU is the knowledge-aware GRU k , which additionally accepts the diffused topic fact embedding fz, and the attention cft over the retrieved knowledge facts F as inputs. Subsequently, two states sct and skt can be fused by adopting the fusion gate ot.

Both the attention ct over the query message X, and the attention cft over the retrieved knowledge facts, can be computed by using the approach proposed by [Luong et al., 2015].

Response Generation

Following [Wu et al., 2020], the next target word yt can be generated by choosing a word from the fixed vocabulary V , adopting an entity ∈ F, or copying a word from the query message X; therefore, the corresponding probability distribution p(yt) is calculated as:

where pV (yt), pK(yt), pC(yt) are the distributions over the fixed vocabulary V , the retrieved knowledge facts F , and the query message X, respectively; zV t , zK t , zC t are three gates to control the contribution of three types of words. Such three distributions can be calculated by:

where MLP2 refers to a 2-layer MLP network. Subsequently, the objective of the generation network is to minimize:

where π denotes the parameters of the generation network, Lbow is the bag-of-words loss to improve the fluency and avoid KL annealing (see [Zhao et al., 2017]), Lmode (see [Zhou et al., 2018]) is the teach-force loss to help the generation network to become more accurate when selecting a target word from three types of words.

2.5 Integration Schemes

The generation network requires a discrete categorical topic fact as an input, which means we need to sample a topic fact from the distribution outputted by the recommendation network, i.e., f ∗ t = argmaxft pc(ft). Considering such sampling is a nondifferentiable process, we here propose following schemes to overcome this issue:

Two-Stage (TopicKAts)

We separately train the recommendation network and the generation network in two different stages. During the inference, the recommendation network is used firstly to predict topic facts, and then the generation network can generate responses with the previously predicted topic facts. The topic fact label for the training is constructed under the distant supervision assumption; for a fact fi = (e1, r, e2) ∈ F, if its subjective entity e1 can be matched in the query X, and its objective entity e2 can be matched in the response Y , then fi is distantly labeled as a topic fact.

Multi-Task (TopicKAmt)

Considering that two networks use the same Context Encoder, they can be jointly trained by sharing the parameters of Context Encoder θ. Except for the training, the distantly labeled topic facts and the two-stage inference are kept. The objective is LMT = α ∗ Lφ+θ + Lπ+θ, where α it s a hyper parameter to coordinate the training speed of two networks, in our experiment, α = 0.25.

Latent Variable (TopicKAlv)

Inspired by the CVAE-based approaches [Zhao et al., 2017], topic facts can be regarded as latent variables. Consequently, we can also optimize the corresponding lower bound:

where the exception Eft∼ptt [Lπ+θ,ft ] can be approximated by sampling N times ft and then calculating the average: 1N ∑ Lπ+θ .

Gumbel-Softmax (TopicKAgs)

Topic facts are categorical variables. Thus, we adopt the Gumbel-Softmax [Jang et al., 2017] to approximate the categorical sampling in a differentiable way. The GumbelSoftmax distribution is given by:

where ω = 0.1 is a temperature value; g = - log(- log(u)); and u is sampled from the distribution Uniform(0, 1).

3 Experiments

3.1 Settings

Dataset

Our approach is evaluated on an open Chinese benchmark dataset [Li and Yan, 2018], which is collected from the largest Chinese SNS (weibo.com). We collect the commonsense knowledge from the ConceptNet (conceptnet.io). We align the dialogues and the knowledge facts by names. For example, if there is a word 'cat' in the dialogue, then facts whose subject/object entity is 'cat' are aligned. After the alignment, the remaining aligned data are randomly divided into three sets: training set, validation set, and test set, which have 847K, 30K, and 30K pairs of dialogue, respectively. Besides, the crawled commonsense knowledge graph includes 27K entities, 26 relations, and 661K facts, respectively . The entity/relation embedding of the knowledge graph is learned by using TransE [Bordes et al., 2013]. The vocabulary size is set to 50,000. Other OOV words are replaced by unk. Our experimental resources are open released 1 .

Comparison Models

We select five approaches as our baselines. ATS2S : The Attentive Seq2Seq [Luong et al., 2015]. MMI: The bidi-MMI [Li et al., 2016a], which first uses p(y|x) to generate 10 responses for each query using beam-search, then applies the inversed model pinv(x|y) to re-rank. HGFU: HGFU uses cue words predicted by PMI to control the generation [Yao et al., 2017]. GenDS: GenDS is able to copy entities during the generation [Zhu et al., 2017]. CCM: The SOTA model in the commonsense knowledge-aware dialogue generation [Zhou et al., 2018]. In the experiments, CCM is evaluated with the official code, while other models are re-implemented by Tensorflow. Most hyper parameters are selected from the CCM: batch size is 50, word embedding dimension is 300, GRU dimension is 512, Adam with the initial learning rate 0.0001 is used to optimize, and the maximum epoch is limited to 20. For the TopicKAlv/gs, their parameters are initialized from the TopicKAmt for the faster convergence. Meanwhile, considering the uncertainty from random sampling, we have two sampling strategies for TopicKAlv/gs in the inference stage. The first strategy (TopicKAmax lv/gs ) selects the fact with the maximum probability. The second strategy employs the MMI (TopicKAmmi lv/gs ). We first generate 10 responses y1:10 with the sampled 10 facts, and then we select the best candidate via y∗ = argmaxyi pinv(x|yi).

1 https://github.com/pku-orangecat/IJCAI2020-TopicKA

Metric

The first type focuses on knowledge utilization. EntN is the number of generated entities per generation [Zhou et al., 2018], EntR is the ratio of the recalled entities. The second type reveals the relevance between the generation and the ground-truth. Following [Liu et al., 2016], we adopt two embedding based metrics, EmbedA and EmbedX , which consider the averaged embedding and the extreme value of each dimension, respectively. And word overlap-based ROUGE and BLEU1/2. To measure the diversity, we report the ratio of distinct uni/bi-grams (DIST1/2) in all generated words [Li et al., 2016a]. Lastly, we use the Entropy to measure the informativeness [Mou et al., 2016].

3.2 Experimental Results

The left block of Table 1 shows the automatic evaluation results. In short, TopicKAs are better than baselines in terms of most metrics. Although our TopicKAs (except TopicKAmt) outperform baselines in terms of BLEU2, the non-zero cases (only 16%) cannot support the significance. Our approach has outstanding advantages in generating knowledgeable (EntN/R), diverse (DIST1/2) and informative (Entropy) responses. It can be attributed to (1) the effective utilization of the knowledge; (2) the introduction of the topic fact; (3) the design of the network architecture. In the relevance part, our approach's advantage is also notable except on the BLEU2. The mediocre performance on the BLEU2 is that BLEU-N itself only simply measures the n-gram overlap without considering the semantic meaning. Meanwhile, researchers have shown the low relevance between the BLEU and the human annotation. [Liu et al., 2016].

The Difference among Schemes

In the first two non-sampling schemes, the multitask scheme TopicKAmt slightly outperforms TopicKAts, which shows that the recommendation network and the generation network can benefit from each other by sharing the Context Encoder. In the comparison of the next two sampling schemes, Gumbel-Softmax brings more diversity and informativeness, while Latent-Variable brings higher knowledge utilization rate and relevance. After applying the re-ranking, the overall performance of two schemes increases, indicating the effectiveness of re-ranking. Although the performance gap between the non-sampling schemes and the sampling schemes is not quite notable in the automatic evaluation, they are very different in actual usage. The advantage of sampling schemes is generating multiple different responses given the same query message. We will exhibit this in the case study.

Human Annotation

We invited three volunteers to annotate the generated responses. We randomly sampled 200 queries from the test set, and then conducted the pair-wise comparison (TopicKAmmi lv vs. baselines, 1000 comparisons in total). Volunteers were required to judge which response is better (ties should be avoided as possible), in terms of two metrics: (1) Appropriateness (the fluency and the relevance to the query); (2) Informativeness (how much knowledge can be provided). We counted the agreement among the volunteers. For the appropriateness, 2/3 agreement (i.e., the percentage of cases that at least 2 volunteers gave the same label) is 97%, and the 3/3 agreement is 57%. For the informativeness, 2/3 agreement is 98%, and the 3/3 agreement is 63%. As shown in the right block of Table 1, our approach is better than the baselines in terms of both metrics, which indicates the generated responses of TopicKA are more acceptable by humans. We noticed that the performance of GenDS and CCM on the human annotation is even worse than the results on the automatic evaluation. This is because the generated entity words are a little unnatural (i.e., the generation is not fluent).

Ablation Study

The ablation results have been reported in Table 2. Firstly, we focus on the ranking performance of the recommendation network using Hit@1(the ratio that the labeled topic fact is the first rank fact, higher is better) and MeanRank (the averaged rank of the labeled topic fact, lower is better), TopicKAmt is the baseline (w/o None). Comparing the joint training (w/o None) and the two-stage (w/o Multitask) training, it can be verified that two networks can promote each other. Meanwhile, the results also demonstrate that the teacher Boost network can improve the ranking performance (MeanRank, w/o Multitask vs. w/o Boost+Multitask). Secondly, we also ablate the techniques related to the topic fact, TopicKAts is the baseline (w/o None). After the ablation of the topic fact diffusion (w/o Diff), the performance sharply decreases, which indicates the diffusion mechanism can effectively supplement background knowledge. Next, we further remove the TFAG (w/o TFAG+Diff), the overall performance decreases too. It shows that TFAG can bring a more flexible way to unitize the topic facts. Finally, we completely remove the participation of topic facts (w/o Topic Fact, TFAG, and Diff are also removed). Without the condition of the topic fact, the model can be more freedom in utilizing the knowledge (EntN), but the generated response will be more general and boring (Dist2 and Entropy).

3.3 Case Study

Table 3 reports four cases. In case 1, we compare our approach (TopicKAmmi lv ) with baselines. It can be seen that all baseline models have generated responses that are irrelevant to the query, the response generated by our approach is not only relevant but also correctly shows the commonsense knowledge that the cloudy day is not hot. In case 2/3, we show that our approach can generate multiple diverse responses conditioned on the different recommended topic facts. Such generated responses are relevant to both query message and the corresponding recommended topic facts, which indicates two things: 1) TopicKA can control the direction by using different topic facts. 2) The motivation why TopicKA generates a specific response is some-

what interpretable, because the used topic fact has the humanreadable semantic meaning. Meanwhile, topic facts can be either explicitly used in the generated response (Case 2), or implicitly used (Case 3). The last case is a bad case, which indicates what we need to improve in the future. In our approach, if the recommended topic fact is distorted, then the generated response may go in a strange direction.

4 RelatedWork

Open-Domain Dialogue Generation

The success of the Seq2Seq learning [Sutskever et al., 2014] has motivated the investigation of dialogue systems. In Seq2Seq models, the given query message is first encoded into contextual vectors by an Encoder, and then a Decoder uses such vectors to generate a response. However, Seq2Seq models tend to generate meaningless responses. Namely, some high-frequency general patterns are easy to be generated, in spite of different queries. Researchers have tried to overcome this issue from multiple aspects; for example, replacing the conventional training objective[Li et al., 2016a]; improving the decoding process [Li et al., 2016b].

Knowledge-Aware Models

Unlike human beings, who can associate the background knowledge from the mind, traditional models can merely access very limited information from the plain text of a given query [Ghazvininejad et al., 2018]. To tackle this issue, researchers begin to introduce knowledge bases into dialogue systems. [Zhu et al., 2017] allows a model to deal with OOV knowledge entities. [Young et al., 2018] enhances dialogue retrieval with commonsense knowledge; [Liu et al., 2019] augments the structured knowledge base with unstructured knowledge bases; In our task, dialogue generation with commonsense knowledge; the SOTA approach [Zhou et al., 2018] designs graph attention mechanisms to utilize commonsense knowledge. Compared with such approaches, TopicKA can exhibit the central topic fact of a generated response, and TopicKA is controllable such that TopicKA can generate multiple diverse responses based on different topic facts.

Content-Introducing Generation

Cue words selected by the external Pointwise Mutual Information (PMI) are first used to control the generation [Mou et al., 2016] explicitly. Then the approach has been improved to a more flexible implicit way [Yao et al., 2017]. However, PMI-based approaches must inefficiently search the whole vocabulary space to find a cue word during the inference. Meanwhile, the searched cue words are usually irrelevant to the dialogue context. Unlike the previous two works, contents can also be sampled from a latent space. The latent space can be either a fixed dialogue action space [Zhao et al., 2018] or a fixed vocabulary space [Gao et al., 2019b; Gao et al., 2019a]. TopicKA is also different from such approaches. First, the content that is introduced in the TopicKA is the high-quality commonsense knowledge facts, which are more reasonable/reliable than actions or words. Next, unlike previous approaches, which sample contents from a limited content space, our content space is quite larger than theirs. The largest content space of previously mentioned approaches is R|V | (vocabulary space), but ours is up to R|E|×|E|×|R| , where |E|/|R| denotes the number of the entities/relations.

5 Conclusion and FutureWork

To bridge the gap of the knowledge between the machine and the human in the context of dialogue response generation, this paper proposes a novel flexible model TopicKA, which can utilize the commonsense knowledge. TopicKA can generate diverse and informative responses conditioned on an interpretable and a controllable topic fact. To recommend topic facts, we propose a teacher-student recommendation network. We propose four schemes to integrate the generation network and the recommendation. Meanwhile, we collect and construct a Chinese commonsense knowledge graph. Experimental results on both objective evaluation and subjective evaluation, indicating our TopicKA can significantly outperform the state-of-the-art CCM in terms of most metrics.

However, there is still much room for improvement in the future. First, the Topic Fact Recommender can be further improved. Second, although the commonsense knowledge can significantly enrich the background knowledge understanding/generating, it is still far away from the satisfactory. In the future, we will continue to introduce more types of knowledge. Lastly, we will continue to improve the diversity, interpretability, and controllability of knowledge-aware dialogue generation models.

Acknowledgments

This work is supported by the National Key R&D Program of China (Grant No. 2017YFB1002000), and PKU-Tencent Joint Innovation Research Program.