Formal Query Building with Query Structure Prediction for Complex Question Answering over Knowledge Base

1 Introduction

Knowledge Base Question Answering (KBQA) is an active research area where the goal is to provide crisp answers to natural language questions. An important direction in KBQA is answering via semantic parsing [Bao et al., 2016; Luo et al., 2018; Yih et al., 2015] to natural language questions, transforming the corresponding semantic components, including entities, relations and various constraints, into formal queries (e.g., SPARQL) and then executing queries over the knowledge base (KB) to retrieve answers. In general, a question can be regarded as correctly answered if its correct query has been built. For a complex question, whose query consists of multiple entities, relations, and constraints, how to select the correct semantic components of the query and combine them in an appropriate way is still a problem to be solved.

∗Contact Author

In recent research, most query building approaches [Luo et al., 2018; Maheshwari et al., 2019] are based on query ranking, i.e., scoring candidate queries by similarities with the question and outputting the top-scored one. Such rankingbased approaches leverage a query graph to represent formal queries and generate candidate query graphs by a statetransition strategy [Yih et al., 2015; Luo et al., 2018; Maheshwari et al., 2019]. This strategy first assumes that the length of the path between the topic entity 1 and the answer does not exceed two hops, and then generates the candidate queries by enumerating all possible structures meeting this assumption. Although the strategy can cover almost all correct query graphs in existing KBQA datasets, it has a significant drawback: a considerable number of noisy query graphs are produced. These noisy queries have incorrect structures but their components have high similarity to the question. In fact, existing query ranking models often make erroneous judgments when dealing with them. For example, Figure 1 shows one question that was answered incorrectly by the states-of-thearts ranking-based approach 2 . Below the question are two candidate query graphs generated by the above strategy. The model does not identify the structure (dotted box) of the correct query (blue box) so that it selects the noisy query (red box).

To avoid producing noisy queries, we propose abstract query graphs (AQG) to describe query structures and propose a new formal query building approach with query structure prediction. Specifically, our approach consists of two stages. In the first stage, we first leverage a neural network-based generative model to generate an AQG according to the question. The AQG is regarded as a predicted structure of the query graph. Then, we utilize the AQG as a constraint to generate the candidate query graphs whose structures match it. In the second stage, we perform candidate query ranking by an existing model, in order to evaluate the improvement brought by the AQG constraint. Following the previous approaches, our work focuses on handling the query graph of the tree structure. We perform comprehensive experiments on multiple QA datasets, and our proposed method consistently outperforms previous approaches on complex questions and produces competitive results on the dataset mainly made up of simple questions.

1 "topic entity" is proposed in [Yih et al., 2015], which refers to the root of the query graph(tree).

2 We reimplement the approach proposed in [Luo et al., 2018].

2 Preliminaries

2.1 Query Graph

The knowledge base (KB) can be regarded as a collection of subject-predicate-object triples 〈s, p, o〉, where s is an entity, p is a relation and o can be either a literal or an entity.

The query graph is defined in [Yih et al., 2015], which is a structured representation of natural language questions. In theory, it is a directed graph. However, in the existing datasets, it is often a tree structure with n vertices and n - 1 edges. Therefore, we focus on dealing with the queries of the tree structure in this paper, and the process to general graph structures is left for future work. The vertices of query graphs include KB entities, KB types, etc., and the edges consist of KB relations and some built-in properties. Table 1 shows a detail classification of the vertices (middle) and edges (bottom).

2.2 Abstract Query Graph

Following the work given in [Hu et al., 2018; Ding et al., 2019], we propose an abstract query graph to describe the query structure. An abstract query graph (AQG) is a tree, which consists of a combination of n labeled vertices v ∈ {"Ent", "Type", "Num", "Var"}, connected with n - 1 labeled, undirected edges e ∈ {"Rel", "Ord", "Cmp", "Cnt", "Isa"}. Here, all the labels in AQG correspond to the vertex classes and edge classes in query graphs (see Table 1). Intuitively, AQG reflects the topology and the component classes of query graphs and each query graph corresponds to a unique AQG (Figure 1). Therefore, if the correct AQG is predicted, it can be used as a constraint to avoid generating the noisy queries with wrong structures.

2.3 Grounding

The operation of utilizing AQG to generate candidate queries is called grounding. In our work, we perform grounding on an AQG g by a two-step method: first, we obtain an intermediate graph by replacing all the vertices (not variables) and built-in property edges of g with the candidate instances. The intermediate graph is denoted by g∗ . Then, we execute the queries 3 (SPARQL) corresponding to g∗ against the KB to retrieve all instance relations corresponding to each "Rel" edges of g∗ . After replacing the "Rel" edges of g∗ with the retrieved instance relations, the query graph is generated. Concretely, for each invariable vertex, its candidate instances are obtained from the existing linking results, including entities, type, number (time and ordinal). For each edge, if its class label is "Rel", its candidate instances retrieved by executing g∗ . Otherwise, it is a built-in property and its candidate instances are obtained by referring to Table 1. After all the combinations of the candidate instances are tried to replace vertices and edges, and their legitimacy is verified by the KB, grounding is over and the candidate query graphs corresponding to g are finally generated.

2.4 Graph Transformer

Since query structure is a tree or even a graph, traditional neural networks cannot effectively capture its structural information. In this paper, we introduce graph transformer [KoncelKedziorski et al., 2019], which is a graph neural network, to learn the vector representation of AQG. Its input is a multilabeled graph (e.g., AQG), which consists of three components: a sequence of vertices, a sequence of edges, and an adjacency matrix. Here, both sequences are unordered and each vertex or edge is represented by its corresponding class label vector, which is randomly initialized. For each vertex or edge in the graph, graph transformer exploits the multi-head attention mechanism [Vaswani et al., 2017] to aggregate the information of its neighbors (within several hops) to update its vector representation. Finally, the output includes three parts: a sequence of vertex vectors, a sequence of edge vectors, and a global vector of the entire graph, which is obtained by the weighted sum of all the vertices and edges.

3 Approach

3.1 Process Overview

For an input question q, we first assume that the linking results of entities, type, and number in q have been obtained by preprocessing, denoted by Rl. Then, the query graph gq corresponding to q is built by the following two stages:

• (a) Generating candidate query graphs. First, an AQG generator translates q into an AQG, denoted by g. Then, according to Rl, a query graph candidate set Cg is generated by performing grounding on g.

3 The queries are generated from g∗ and all the possible directions of "Rel" edges are taken into account. That is, if there are two "Rel" edges in the AQG, the number of the direction combinations is 4.

• (b) Ranking candidate query graphs. An existing query ranking model is employed as a black box to score each query graph in Cg and the top-scored one is selected as gq .

Our approach aims to explore how to predict query structures (AQG generation) and whether they can improve existing query ranking approaches. Consequently, in order for the experimental comparison, we keep the query ranking model consistent with [Luo et al., 2018]. To more intuitively show the value of AQG and shorten the training time of the ranking model, we only employ their basic version model which removes the dependency representation of the question and id representation of the path (refer to [Luo et al., 2018] for details). In the rest of this section, we will detail the process of AQG generation.

3.2 Abstract Query Graph Generation

We first introduce the proposed generative framework, and then detail the neural network-based AQG generator which follows this framework.

Generative Framework

An AQG is a tree, which can be regarded as a specific case of undirected graphs, denoted by g = (V, E). Here, V and E are the sets of labeled vertices and labeled edges, respectively. In our framework, the generative process of AQG can be described by a sequence of graphs G = {g0 , g1 , ..., gL}, where g0 is an empty graph and gL is the completed AQG. For each time step t ≥ 1, gt = f(gt-1 , ∗at ), where f denotes the operator and ∗at denotes several arguments according to f. In this paper, we define the following three types of graph-level operators:

• (a) addVertex. For a graph gt = (V,E), addVertex(gt , cv) represents adding a fresh vertex of label cv into gt . The result is a graph gt+1 = (V ∪ {vadd} ,E), where vadd denotes the added vertex.
• (b) selectVertex. For the graph gt+1 = (V ∪ {vadd} ,E) after addVertex, selectVertex(gt+1 , vadd, vslc) means selecting a vertex vslc ∈ V that will be connected to the fresh vertex vadd. The results is a graph gt+2 = (V ∪ {vadd} , E)4 .
• (c) addEdge. For the graph gt+2 = (V ∪ {vadd} ,E) after selectVertex, addEdge(gt+2 , vadd, vslc, ce) means adding a fresh edge of label ce into gt+2 to connect vadd with vslc. The result is a graph gt+3 = (V ∪ {vadd} ,E ∪ {eadd}), where eadd = (vslc, ce, vadd) is the added edge.

Successively performing the operations addVertex, selectVertex, and addEdge on a graph is called an iteration, which adds a new triple 〈vslc, ce, vadd〉 into the graph. In the proposed framework, we stipulate that the first operation is always addVertex, which converts the initial empty graph to a graph containing only one isolated vertex. Then, the AQG is generated by multiple iterations. Although the operator f is predetermined in each time step, different arguments ∗at result in generating different AQG. Here, the arguments vadd in selectVertex, vadd and vslc in addEdge can be obtained by the previous step, consequently, only the last argument of each operation need be determined, i.e., cv in addVertex, vslc in selectVertex and ce in addEdge.

4 Note that the structures of gt+1 and gt+2 are identical. Here, utilizing the two different identifiers aims to distinguish the time steps they belong to.

There are two reasons for predicting query structures by a grammar (graph-level operations) -based generation process rather than predefined structure templates: (1) First, compared to the templates, the graph-level operations are more underlying and therefore more flexible. (2) Second, and more importantly, the test query structure may not be visible during training. The generation process can help deal with this situation by learning the visible steps of the invisible complex query structures.

NN-based AQG Generator

The architecture of the proposed AQG generator is shown in Figure 2. It takes a question and an empty graph g0 as the inputs and then outputs a sequence of graphs G = {g1 , g2 , ..., gL}, where gL is the completed AQG. At each time step, the generator first predicts the argument for the operation and then generates a new graph by performing the operation on the previous graph. Intuitively, for each prediction, the generator needs to integrate the information of both the question and the previous graph. Therefore, the model consists of four components: a question encoder, a graph encoder, a decoder, and an executor.

Question Encoder. To capture the semantic information of the question, we apply a Bi-LSTM as the question encoder to get the vector representations of the question. First, we replace all entity (number) mentions (from linking results) used in the question q by tokens 〈e〉 (〈n〉), to enhance the generalization ability. Then, the question encoder works on the word embeddings of q and outputs its vector representations Γq = [γ 1 q , γ 2 q , ..., γ lq ], where l denotes the number of the words in q.

Graph Encoder. To capture the semantic information of the graph generated at the previous time step, we leverage a graph transformer as the graph encoder. At time step t, the graph encoder embeds the graph gt-1 and outputs a list of vertex vectors Γt-1 v = [γ 1 v , γ 2 v , ..., γ n v ], a list of edge vectors Γt-1 e = [γ 1 e , γ 2 e , ..., γm e ] and a vector ht-1 g ∈ Rd , where n and m respectively denote the numbers of the vertices and edges in gt-1 , and ht-1 g is the representation of gt-1 .

Decoder. We employ a Bi-LSTMas the decoder to translate the information from the question and the previous graph into a vector at each time step. This vector carries the semantic information of the predicted argument. Specifically, at time step t, htq ∈ Rd is the context vector from the question q, which is computed by

where · denotes the dot product operation, W ∈ Rd×d is the trainable parameter matrix, ht-1 g is the output from the graph encoder for gt-1 , and αt i is the weight of attention from gt-1 for the i-th word of the question. This attention mechanism is based on the following intuition: At time step t, the model should focus on some parts of q which do not carry the redundant information of gt-1 . i.e., avoid repeatedly adding the triple that has been added. Thereafter, the input to the decoder, denoted by htin ∈ Rd , is calculated by a skip connection: htin = ht-1 g +htq . Here, + denotes the element-wise addition. Although ht-1 g is already involved in calculating htq , the decoder still needs the direct information of gt-1 when making decisions, such as selectVertex. Therefore, we adopt the idea from Residual Networks [He et al., 2016], enhancing the propagation of ht-1 g by this skip connection. Finally, the decoder takes htin as the input and outputs the semantic vector of the predicted argument, denoted by htout ∈ Rd .

Executor. At each time step t, the executor first transforms the decoder's output htout into the argument ∗at. Then, it performs the operation f(gt-1 , ∗at ) to output a new graph gt . Corresponding to the types of the operator f, the executor contains three modules. For each module, we design a pointer network [Vinyals et al., 2015] for predicting the argument:

• Add VertexModule. The argument cv denotes the label of the added vertex, which is determined by

where βi ∈ Rd is the parameter vector of label i and Cv ={"Ent", "Type", "Num","Var", "End"}. Comparing to the AQG vertex classes, Cv has one more class label "End", which is used to terminate the process. Specifically, if "End" is selected at time step t, the generation process ends and the generator outputs gt .

• Select Vertex Module. vslc denotes the index of the selected vertex of gt-1 , which is determined by

where V t-1 denotes the vertex set of gt-1 , and γ jv ∈ Rd is the vector representation of the vertex j in gt-1 from the graph encoder.

• Add Edge Module. ce denotes the label of the added edge, which is determined by

where ρi ∈ Rd is the parameter vector of label i and Ce ={"Rel", "Ord", "Cmp", "Cnt", "Isa"}.

KB Constraint. AQG aims to be grounded to the KB to generate query graphs. Therefore, whether the AQG can be grounded in the knowledge base is also an important indicator to evaluate its correctness. To prevent the excessive expansion of the AQG, we utilize the KB to constrain its generation. Concretely, whenever the AQG completes an iteration, we ground it against the KB to check if there is at least one query graph that matches the AQG. If it exists, the generation process will continue, otherwise, the triple added by the latest iteration will be removed and the current graph is returned as the result. In addition, we employ a strategy to enhance the recognition of the type constraint: if there is no "Type" vertex in the generated AQG but the type linking result set of the question is not empty, we attach a type vertex to a variable vertex and obtain a new AQG. Then, the KB constraint is required to verify the legality of this new AQG.

3.3 Training

In the experiments, each training sample consists of a question q and its gold query graph g+ q . The AQG corresponding to g+ q is regarded as the gold AQG, denoted by g+.

To get the ground-truth for training the AQG generator, we traverse g+ to restore its generation process. In fact, for the same AQG, different traversal strategies and starting vertices result in different ground-truth. By experimental comparison, we adopt depth-first traversal. In addition, in a query graph, there is at least one variable vertex representing the answer 5 , denoted by ?v0. Therefore, we select the vertex in g+, which corresponds to ?v0, as the starting vertex of the traversal. Specifically, the ground-truth π is obtained by the following steps: Initially, π is an empty sequence. At the beginning of the traversal, the starting vertex vs is first visited and its label is added to π. Thereafter, whenever a fresh vertex v is visited from the vertex u along the edge e, the class label of v, the index of u, and the class label of e will be added to π in turn. When all vertices in g+ are visited, the last label

5 Other variables are hidden variables, like ?v1 in Figure 1.

"End" is added to π and the ground-truth π = {a1 , a2 , ..., aL} is obtained. Here, L is the step number of the generation process, and at is the ground true argument (last argument) of each operation. During training time, our model is optimized by maximizing the log-likelihood of the ground true argument sequences:

Here, Q is the set of the questions, and p(at |q, gt-1 ) is the predicted probability of the ground true argument at , including pav, psv and pae.

4 Experiments

4.1 Experimental Setup

Our models are trained and evaluated over the following three KBQA datasets:

LC-QuAD [Trivedi et al., 2017] is a gold standard complex question answering dataset over the DBpedia 04-2016 release, having 5,000 NLQ and SPARQL pairs. The dataset is split into 4,000 training and 1,000 testing questions 6 .

ComplexQuestions (CompQ) [Bao et al., 2016] is a question answering dataset over Freebase, which contains 2,100 complex questions collected from Bing search query log. The dataset is split into 1,300 training and 800 testing questions 7 .

WebQuestions (WebQ) [Berant et al., 2013] is a question answering dataset over Freebase, which contains 5,810 questions collected fromGoogle Suggest API. It is split into 3,778 training and 2,032 testing QA pairs 8 . Since more than 80% of the questions in this dataset are simple questions that contain only one fact, we use it as a benchmark for simple questions.

For LC-QuAD, we directly use its provided standard SPARQL as the gold query. However, ComplexQuestions and WebQuestions only contain question-answer pairs, so we search the candidate queries released by [Luo et al., 2018] and take the query with the highest F1-score as the gold query. To compare with previous methods[Luo et al., 2018; Maheshwari et al., 2019], we follow them to use the results of the state-of-the-art entity-linking tool S-MART [Yang and Chang, 2015] on CompQ and WebQ, and use the gold entities on LC-QuAD. For each dataset, we randomly select the 10% of the training set as the development set.

Implementation details. In our experiments, all word vectors are initialized with 300-d pretrained word embeddings using GloVe[Pennington et al., 2014]. The following hyperparameters are tuned on development sets: (1) Both the sizes of the hidden states are set to 256; (2) The layer number of Bi-LSTM is set to 1 and the layer number of graph Transformer is set to 3; (3) The learning rate is set to 2×10-4 ; (4) The number of the training epochs is set to 30. Our code are publicly available 9 .

6 https://figshare.com/projects/LC-QuAD/21812

7 https://github.com/JunweiBao/MulCQA/tree/

ComplexQuestions

8 https://nlp.stanford.edu/software/sempre/

9 https://github.com/Bahuia/AQGNet

4.2 End-to-End Results

We compared our approach with several existing KBQA methods. [Abujabal et al., 2017] performs query building by existing templates. [Yih et al., 2015; Bao et al., 2016; Yu et al., 2017] construct pipelines to generate query graphs. [Luo et al., 2018; Maheshwari et al., 2019] obtain stateof-the-art performance on CompQ and LC-QuAD by query ranking, respectively. [Hu et al., 2018] achieves a state-ofthe-art result on WebQ by state-transition over dependency parsing. The main difference between our method and theirs is that they perform state transition by enumerating predefined transition conditions, but we let our model learn to how to expand the AQG automatically at each step.

The experimental results are reported in Table 2. Our approach achieves state-of-the-art results on both LC-QuAD and CompQ, and ranks second only to [Hu et al., 2018] on WebQ. Although the performance on WebQ did not exceed [Hu et al., 2018], our approach achieves a result that closes to theirs on WebQ without any predefined transition conditions. From another perspective, our approach outperforms all the existing ranking-based approaches [Luo et al., 2018; Maheshwari et al., 2019] on the three datasets.

The performance of [Abujabal et al., 2017; Bao et al., 2016; Yu et al., 2017] is limited by the error propagation of the pipeline for ranking components. [Luo et al., 2018; Maheshwari et al., 2019] rank entire queries to joint the information of components, thus achieve better results. However, enumeration on query structures makes their performance suffer from noisy queries. Existing models lack sufficient representation capacity to balance structure and components. Our approach decouples the query ranking task and exploits a specific generative model to handle the information of structures, achieving better results.

4.3 Detailed Analysis

Contribution of AQG Constraint

In order to evaluate the improvement by AQG constraint, we kept our baseline ranking model unchanged but generated candidate queries by the staged-transition strategy (ST).

Table 3 shows the average numbers of candidate queries, precision, recall, and F1-score of the entire systems based on different candidate generation strategies. Intuitively, the size of the candidate set generated by AQG is much smaller than that generated by ST, especially on LC-QuAD. In addition, comparing to the ST-based strategy, the AQG-based strategy improves the performance of the entire system on all the datasets by increasing the average precision of answering.

To further analyze the impact of question complexity on the performance, we divided the questions in each dataset into four complexity levels according to the number of edges in the correct query graph. Here, level i indicates that the query graph contains i edges. We tested the average F1-scores when answering the questions of different levels. Figure 3(a),3(b) and 3(c) show the results. The improvements of the AQGbased strategy over the ST-based strategy increase with the complexity of questions. Furthermore, we found that the performance of both methods on CombQ and WebQ (using entity linking results) increases as the question is more complex, but it is not reflected in LC-QuAD (using gold entities). It hints to the fact that constraints in complex questions can provide more comprehensive information for entity linking.

Ablation Study on AQG Generator

To explore the contributions of various components of our AQG generator, we compared the following settings:

w/o attention. We replaced the attention mechanism with an average-pooling to get htq .

w/o skip connection. We removed the skip connection of ht-1 g and htq and only used htq as the input to the decoder.

w/o graph encoder. We removed the graph encoder, that is, at decoding time step t, we use ht-1 out to calculate htq and obtain htin by the element-wise addition of ht-1 out and htq .

w/o KB constraint. We removed the KB constraint that prevents the excessive expansion of AQG.

The middle of Table 4 shows the accuracy of AQG generation of different settings. By removing the attention mechanism, the accuracy declined approximately 2% on the three datasets. Removing the graph encoder results in a significant drop in performance, especially on LC-QuAD and CombQ.

It demonstrates that the structural information of the previous graph is essential. The performance drop by removing skip connection also illustrates this fact. The main reason for the smaller drop on WebQ can be that most questions in WebQ are simple questions whose true query only contains one edge. In this case, the model can also achieve good results only by the memory of the previous operation. Rmoving the KB constraint makes the performance approximately drop 10% over both the complex datasets, which reveals that KB is an effective tool for the query structure disambiguation.

Impact of the Ground-truth Construction Strategy

We also evaluate the accuracy of the AQG generator supervised by different ground-truth, which are constructed by following traversal strategies.

Breadth-first traversal. We replaced the depth-first traversal with the breadth-first traversal.

Random traversal. We adopted the random traversal. Specifically, once visiting a vertex v, randomly select an unvisited vertex connected to v as the next vertex to visit.

The bottom of Table 4 shows the results. The performance is worst when the ground-truth is constructed by random traversal. Depth-first traversal achieves better performance than breadth-first traversal on LC-QuAD and CompQ. By observation, we speculate it is because the facts of complex questions are typically reflected in the form of chains. Depth-first traversal can effectively preserve this chain structure so that it achieved better results. Three traversal strategies achieve close results on WebQ because three traversal strategies have the same results for simple query graphs.

Results on Varied Sizes of Training Data

We tested the performance of our approach with different sizes of training data. The results are shown in Figure 3(d),3(e) and 3(f). The model equipped with the attention mechanism, the graph encoder, the skip connection, and the KB constraint always maintains optimal performance with different sizes of training data. Furthermore, with less training data, the performance of our approach can still maintain certain stability. It is due to the fact that our model learns fine-grained generative steps instead of entire structures. In this way, even if there are some invisible test query structures, which do not appear in the training data, our model can still handle them by visible generative steps.

5 RelatedWork

Traditional semantic parsing based KBQA approaches [Berant and Liang, 2014; Fader et al., 2014; Reddy et al., 2017] aim to learn semantic parsers that translate natural language questions into a logical form expression. Recent researches proposed an alternate approach to semantic parsing by treating KBQA as a problem of semantic graph generation. [Yih et al., 2015] defines a sequence of stages for generating query graphs, each relying on a set of manually defined rules for how query conditions are added. [Yu et al., 2017] proposes a hierarchical representation model for the matching of question and KB relations. Current KBQA methods simplify the pipeline into query ranking. [Luo et al., 2018] apply a staged candidate generation strategy to generate candidate query graphs and then rank them by utilizing a semantic matching model. [Maheshwari et al., 2019] follows the candidate generation methods proposed in [Yih et al., 2015] and propose a novel self-attention based slot matching model. These approaches try to capture the structural information and the semantic information by a single model. The main difference between our approach and previous approaches is that we explicitly leverage the structural constraint (AQG) to narrow the search space of candidate queries, in order to release the burden of query ranking. In addition, to avoid using any predefined structured templates which limit the generalization ability of our approach, we propose a generative framework to generate AQG based on basic graph-level operations. To the best of our knowledge, this is the first attempt to predict the query structures without any pre-defined templates. Last but not least, to utilize the structural information of the graph to help the model make decision in the AQG generation process, we apply graph transformer[Koncel-Kedziorski et al., 2019], which is an existing graph neural network model, to learn the representation of the AQG. The most related work to ours is [Hu et al., 2018] and [Ding et al., 2019]. [Hu et al., 2018] proposes a semantic query graph to constrain candidate query generation. They perform state transition on the dependency tree by predefined conditions. In contrast, we make the model learn to perform the state transition without any conditions. [Ding et al., 2019] proposes to leverage the query substructure templates in the training data to construct complex queries. Different from [Ding et al., 2019], we try to predict the query structure from the perspective of graph generation without using templates.

6 Conclusion

In this paper, we presented our two-stage formal query building approach. Our approach can automatically predict the query structure and use it as a constraint to avoid generating noisy candidate queries, thereby improving the performance of the following query ranking. The experimental results showed that our approach achieved superior results than the existing methods on complex questions, and produced competitive results on other simple question based datasets. In future work, We will try to extend the AQG generation framework in order to deal with the query of general graph structures. In addition, we plan to apply reinforcement learning instead of supervised learning to train the AQG generator.

Acknowledgments

The work is partially supported by the Natural Science Foundation of China under grant No. 61502095, National Key Research and Development Program of China under grants (2018YFC0830200, 2017YFB1002801), the Natural Science Foundation of China grants (U1736204, 61602259), and the Judicial Big Data Research Centre, School of Law at Southeast University.