--- title: "An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension" authors: - "Xin Liu" - "Kai Liu" - "Xiang Li" - "Jinsong Su" - "Yubin Ge" - "Bin Wang" - "Jiebo Luo" doi: "10.24963/ijcai.2020/525" url: "https://ijcai.org/proceedings/2020/525" pdf: "https://ijcai.org/proceedings/2020/0525.pdf" venue: "Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence" year: 2020 pages: "3794-3800" track: "Main track" keywords: - "Natural Language Processing: Natural Language Processing" - "Natural Language Processing: Question Answering" --- # An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension Xin Liu, Kai Liu, Xiang Li, Jinsong Su, Yubin Ge, Bin Wang, Jiebo Luo *Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence*, pages 3794-3800. DOI: [10.24963/ijcai.2020/525](https://doi.org/10.24963/ijcai.2020/525) ## Abstract The lack of sufficient training data in many domains, poses a major challenge to the construction of domain-specific machine reading comprehension (MRC) models with satisfying performance. In this paper, we propose a novel iterative multi-source mutual knowledge transfer framework for MRC. As an extension of the conventional knowledge transfer with one-to-one correspondence, our framework focuses on the many-to-many mutual transfer, which involves synchronous executions of multiple many-to-one transfers in an iterative manner.Specifically, to update a target-domain MRC model, we first consider other domain-specific MRC models as individual teachers, and employ knowledge distillation to train a multi-domain MRC model, which is differentially required to fit the training data and match the outputs of these individual models according to their domain-level similarities to the target domain. After being initialized by the multi-domain MRC model, the target-domain MRC model is fine-tuned to match both its training data and the output of its previous best model simultaneously via knowledge distillation. Compared with previous approaches, our framework can continuously enhance all domain-specific MRC models by enabling each model to iteratively and differentially absorb the domain-shared knowledge from others. Experimental results and in-depth analyses on several benchmark datasets demonstrate the effectiveness of our framework. ## Full text ## 1 Introduction As one of the core abilities of artificial intelligence, Machine Reading Comprehension (MRC) attempts to enable machines to answer questions after reading a passage. Due to its valuable industry applications such as search engines, it has always been one of the research focuses in natural language processing [Seo et al., 2017; Hu et al., 2018; Yu et al., 2018; Devlin et al., 2019; Liu et al., 2020]. As the basis for MRC studies, many datasets such as SQUAD [Rajpurkar et al., 2016] and NEWSQA [Trischler et al., 2017] have been developed in recent years. However, some domain-specific datasets are limited, and thus they are unable to individually train a domain-specific MRC model with satisfying performance. Therefore, how to overcome the shortage of domainspecific training data has become one of the important research directions in MRC. ∗These authors contributed equally to this work. †Corresponding author To this end, one direction is to mix multiple domains of data and train a unified MRC model [Xu et al., 2019] and another line of work introduces silver data by automatically generating questions [Duan et al., 2017; Song et al., 2018] . However, such approaches ignore the differences between these domains and introduce noises, resulting in the underutilization of these data. Recently, researchers have explored several transfer learning methods for MRC. Min et al. [2017] and Chung et al. [2018] first introduced fine-tuning into MRC. Furthermore, Talmor and Berant [2019] investigated the effect of domain-level similarities on fine-tuning. Despite their successes, fine-tuning still has some drawbacks. First, it is a knowledge transfer method with one-to-one correspondence, which also cannot effectively utilize multiple source-domain data with different impacts. Second, the significant difference between the source-domain and the targetdomain often makes the one-pass transfer procedure of finetuning fail to fully exploit domain-shared knowledge. It can be said how to fully exploit the training data in different domains to construct domain-specific MRC models still remains unresolved. In this work, we propose a novel iterative multi-source mutual knowledge transfer framework for MRC. The intuition behind our method includes two aspects. First, each domain-specific MRC model can be enhanced by the training data and models of other domains, whose effects depend on their domain-level similarities to the target domain. Second, a process of iterative mutual reinforcement can achieve better knowledge transfer, where domain-shared knowledge among training data in different domains can be fully exploited to benefit all domain-specific MRC models. Figure 1: Overview of our framework for MRC. Di is the i-the domain training data, θ(k) i and φ(k) i represent the i-th domain-specific model and its corresponding multi-domain model at the k-th iteration, respectively. Solid arrows show the distillation based knowledge transfer from {Dj , θ(k) j }j̸=i to φ(k) i , where color intensities of solid arrows indicate different domain-level similarities of multiple training data to the i-th domain data. The dotted arrow pointing from φ(k) i to θ(k+1) i indicates initializing θ(k+1) i using φ(k) i . The dotted arrow departing from {Di, θ(k) i } to θ(k+1) i illustrates the distillation based knowledge transfer from {Di, θ(k) i } to θ(k+1) i . Please note that each domain-specific model has its own multi-domain model. ![Image](https://ijcai.org/proceedings/2020/fulltext/525/image_000000_376a6ceb9b14e76ae4176073ae7d0b442791477b3956580fe3fb48a70c17ffbe.png) Figure 1 shows our framework for MRC. Significantly different from previous approaches, the execution process of our framework involves synchronous executions of multiple many-to-one knowledge transfers in an iterative manner. More specifically, as shown in the blue dashed part of Figure 1, we conduct a many-to-one transfer involving two steps to update a target-domain MRC model: First, we apply knowledge distillation [Hinton et al., 2015; Tan et al., 2019; Clark et al., 2019] to establish a multi-domain MRC model, which is trained to simultaneously fit the training data and match the outputs of other domain-specific MRC models. Particularly, we introduce domain-level similarities to control the impacts from the other domains on the construction of this multi-domain MRC model. Second, after the initialization with the multi-domain MRC model, we employ knowledge distillation to fine-tune the target-domain MRC model, supervised by both its training data and the outputs of its previous best model. Compared with previous methods [Min et al., 2017; Chung et al., 2018; Talmor and Berant, 2019], in this work, we make the following contributions: (1) We extend the existing knowledge transfer with one-to-one correspondence into a new setting of many-to-many mutual knowledge transfer for MRC; (2) We exploit multi-source mutual knowledge transfer for MRC, based on knowledge distillation, to particularly leverage the similarity between each domain and the target domain for better knowledge transfer; and (3) We propose to perform multi-source mutual knowledge transfer in an iterative manner, so that domain-shared knowledge can be fully exploited to benefit all domain-specific MRC models. We conduct experiments on several commonly-used datasets. Experimental results and in-depth analyses show that our framework achieves significantly better performance than the dominant knowledge transfer approaches for MRC. ## 2 RelatedWork Transfer learning for MRC. To address the lack of largescale domain-specific training data in MRC, previous studies use upstream datasets to enhance the performance of MRC models, including word embeddings [Pennington et al., 2014] and language models [Devlin et al., 2019]. Meanwhile, more attempts have been made to explore transfer learning approaches for MRC, where fine-tuning is the most common method achieving satisfying results on several datasets [Min et al., 2017]. Particularly, Golub et al. [2017] introduced a two-stage synthesis network to produce the target-domain synthetic dataset, which can be used to fine-tune the MRC model. Chung et al. [2018] demonstrated that transfer learning is helpful even in unsupervised scenarios. Xu et al. [2019] explored a multi-task learning framework to exploit different domains of datasets for MRC, where two re-weighting strategies are investigated. Talmor and Berant [2019] performed a thorough empirical investigation of generalization and transfer over different domains of MRC datasets. Unlike these work, our framework iteratively transfers the knowledge among multiple MRC datasets, where each domain-specific MRC model and its training data can be iteratively and differentially exploited to enhance other models. To the best of our knowledge, such an approach has not been fully explored before in MRC. Knowledge distillation based NLP. Recently, knowledge distillation has been successfully applied into many tasks, such as model compression [Kim and Rush, 2016] and knowledge transfer [Zeng et al., 2019; Tan et al., 2019]. Similar to our first many-to-one knowledge distillation, Tan et al. [2019] presented a distillation-based approach to boost the performance of multilingual machine translation. Furlanello et al. [2018] proposed BAN that is a simple retraining procedure. During this procedure, after the teacher model coverges, they initialize a new student model and train it with the dual goals of predicting the correct labels and matching the output distribution of the teacher model. On this basis, Clark et al. [2019] extended BAN to BAM, which utilizes multiple task datasets simultaneously and thus can be considered as the multi-task version of BAN. The most related work to ours is [Clark et al., 2019]. However, our work significantly differs from this work in the following important aspects: (1) During the first step of knowledge distillation, Clark et al. [2019] applied knowledge distillation to transfer the knowledge of all single-task models to a unified multi-task model. By contrast, we distills all non-target single-domain models into a multi-domain model, which leverages domain-level similarities and thus is a domain-specific one, resulting in the effectiveness of the iterative execution of our framework. (2) During the second step of knowledge distillation, our teacher model is the current domain-specific model rather than the multi-task model adopted by [Clark et al., 2019]; (3) We focus on MRC rather than natural language inference in [Clark et al., 2019]. Note that experimental results reported in Table 1 verify the superiority of our framework over [Clark et al., 2019]. ## 3 BERTQA In this section, we briefly introduce BERTQA [Devlin et al., 2019] that is chosen as our basic MRC model. Figure 2 shows the architecture of BERTQA. Given a passage p = p1, p2...p|p| and a question q = q1, q2...q|q|, we first pack them into a single sequence s=[<CLS>, q, <SEP>, p, <SEP>], where <CLS> is the token used for classification, <SEP> is the token separating q and p, and the sequence length |s| = |p| + |q| + 3. For each token st, we construct its input representation as h(0) t = etok t + epos t + eseg t , where etok t , epos t , and eseg t are the token, position, and segment embeddings for st, respectively. Please note that all tokens of p share a same segment embedding, and all tokens of q use a same segment embedding. Such input representations are then fed into a BERT encoder with successive L Transformer [Vaswani et al., 2017] layers, each of which is composed of two sub-layers with the layer normalization mechanism: a multi-head self-attention and a position-wise fully connected feed-forward neural network. Finally, on the top of the hidden states h(L) t at the L-th layer, we use a linear layer with softmax functions to produce the probability of each token st to be the beginning or ending position of the answer span: Start/End Span Figure 2: The BERTQA model. ![Image](https://ijcai.org/proceedings/2020/fulltext/525/image_000001_e9b0ab2176b7c4b42f8a51c72aadfd9c717ac4f3ad9b370b41bb688756ecd308.png) whereWs∗ are related parameters. To train BERTQA, we define the training objective based on the log-likelihood of the true beginning and ending positions: where the quadruple qu = (p, q, b, e) denotes an example of the training data D, with b and e as the answer beginning and ending position, respectively, and θ is the set of model parameters. At inference time, the span (b,e) where a ≤ b with the maximum P1b · P2e is chosen as the predicted answer. ## 4 Our Framework In this section, we describe our framework in detail. As displayed in Figure 1, it is a multi-source multi-pass knowledge transfer process. To facilitate the understanding of our framework, we also summarize its training procedure in Algorithm 1. We first train all initial domain-specific MRC models {θ(0) i }Ni=1 using their own training data {Di}Ni=1 , respectively (Line 1). Then, we calculate the domain-level similarities between training corpora, denoted by a matrix S={si,j}, 1≤i, j≤N (Line 2), which can be subsequently exploited to distinguish the impacts of other domains on the construction of each multi-domain MRC model. Afterwards, we conduct K iterations of many-to-many knowledge transfer (Lines 3-13). Concretely, at the k-th iteration, we synchronously perform multiple many-to-one knowledge transfers from other domains to each target-domain model θ(k+1) i , 1≤j≤N (Lines 4-12). Such a transfer mainly consists of two substeps: (1) We construct a multi-domain MRC model φ(k) i using {Dj}j̸=i and {θ(k) j }j̸=i (Lines 5-6), and (2) We use φ(k) i , Di and θ(k) i to establish θ(k+1) i (Lines 7-11). Through this multi-pass procedure, different domain-specific MRC models can constantly absorb domain-shared knowledge to reinforce each other. Clearly, the calculation of domain-level similarities S and these aforementioned two substeps are the most important steps in our framework. We describe them below. Calculations of Domain-level Similarities S. To this end, for the training data Di of each domain, we first collect its all questions to form a large question document Qi, and then use the TF-IDF representation to represent the domain Qi. Finally, we directly define the domain-level similarity si,j between i-th and j-th domains as the cosine similarity based on their TF-IDF representations: si,j = cosine(TF-IDF(Qi),TF-IDF(Qj)). Then, we normalize this similarity as si,j = si,j ∑j′̸=i si,j′ . Please note that we only use the question words to calculate the domain-level similarities. This is because the passages of different domains may be similar. For example, SQUAD and HOTPOTQA are both constructed from Wikipedia. By contrast, the question words are able to better reflect the domainlevel differences. Substep 1: Constructing Multi-domain MRC Model φ(k) i . Inspired by [Tan et al., 2019; Clark et al., 2019], we employ knowledge distillation to construct multi-domain MRC model φ(k) i , where we train φ(k) i to fit the training data {Dj}j̸=i and match the outputs of individual domain-specific models {θj}j̸=i simultaneously according to their domainlevel similarities to Di. Specifically, for the i-th domain, we mix training corpora {Dj}j̸=i from other domains to form a mixed training corpus Mi, where we conduct up-sampling on {Dj}j̸=i to make all domains have the same amount of training instances. For a sampled training example qu = (p, q, b, e) from Dj, two kinds of errors are involved: 1. Negative likelihood L1(qu; φ(k) i ): it is used to enable φ(k) i to fit training data Dj. Formally, it can be formulated as where P1b (φ(k) i ) and P2e (φ(k) i ) are defined in Equation 1 and 2, respectively. 2. Prediction divergence L2(qu; θ(k) j ; φ(k) i ): it aims to keep the outputs of φki consistent with those of domainspecific MRC model θ(k) j , and is formally defined as themean squared error between outputs of θ(k) j and φ(k) i : Thus, the joint error for the training instance qu is Here we follow Clark et al. [2019] to linearly increase λmd from 0 to 1 throughout training. In this way, φ(k) i will absorb the guidance information from its teachers models {θ(k) j }j̸=i as much as possible during the early stage of training, and then rely more on the training data so it can learn to surpass its teachers. Finally, the objective function over Mi becomes Algorithm 1 Iterative Multi-source Mutual Knowledge Transfer Framework for MRC Input: Training corpora {Di}Ni=1 , validation sets {Dv i }Ni=1 , the maximal iteration number K Output: Domain-specific MRCmodels {θ(K) i }Ni=1 - 1: θ(0) i ←TrainModel(Di), i = 1, 2, ...,N - 2: Compute the similarity matrix S={si,j}, i, j = 1, 2, ...,N, where si,j indicates the domain-level similarity between Di and Dj // Step 1 - 3: for k = 0, 1, 2, ...,K-1 do - 4: for i = 1, 2, ..., N do - 5: Initialize the multi-domain MRC model φ(k) i - 6: φ(k) i ← TrainMultiDomainModel({Dj}j̸=i, {θ(k) j }j̸=i, S) //Step 2 - 7: Initialize θ(k+1) i with φ(k) i - 8: θ(k+1) i ←FineTuneModel(Di, θ(k) i ) // Step 3 - 9: if EvalModel(θ(k+1) i , Dv i ) < EvalModel(θ(k) i ,Dv i ) then - 10: θ(k+1) i ← θ(k) i - 11: end if - 12: end for - 13: end for Where Mi is the previously-mentioned mixed training corpus. Note that by introducing si,j, we can effectively differentiate the impacts of different domains on φ(k) i . Substep 2: Constructing Domain-specific MRC Model θ(k+1) i . During this process, we first initialize θ(k+1) i with φ(k) i , and then also apply knowledge distillation [Tan et al., 2019; Clark et al., 2019] to update θ(k+1) i using its previous model θ(k) i and training corpus Di. Similarly, the objective function of each training example qu fromDi involves two kinds of errors: 1. Negative likelihood L3(qu; θ(k+1) i ): it requires θ(k+1) i to fit the reference of Di: 2. Prediction divergence L4(qu; θ(k+1) i ): it makes the outputs of θ(k+1) i consistent with those of θ(k) i : Finally, the objective function over Di is defined as where where the hyper-parameter λds is adjusted in a similar way as λmd. In particular, we compare the performance of θ(k+1) i and θ(k) i to retain the current best model (Lines 9-11). As described before, we apply the above procedure to generate all domain-specific models {θ(k+1) i }Ni=1 in a parallel manner, which can effectively reduce the training time of our framework. Furthermore, we repeat this many-to-many knowledge transfer process, until the maximal number of iteration K is reached or all domain-specific models converge. ## 5 Experiments ## 5.1 Setup In our experiments, we use the following datasets: - SQUAD [Rajpurkar et al., 2016]. It is one of the most popular MRC datasets, which consists of more than 100K instances. Each example is a pair of context paragraph from Wikipedia and a question created by a human, and the answer is a span in the context. Particularly, we use SQUAD v1.0 in this work. - NEWSQA [Trischler et al., 2017]. It contains about 120K question-answer pairs, all of which are related to CNN articles. - TRIVIAQA [Joshi et al., 2017]. This dataset includes 95K question-answer pairs from Trivia Database, each of which is paired with a document collected by completing a web search of the question. - HOTPOTQA [Yang et al., 2018]. Crowdsourcing workers were shown pairs of related Wikipedia paragraphs and asked to author questions that require multi-hop reasoning over the paragraphs. There are two versions of HOTPOTQA, and we use the second version, where 10 paragraphs retrieved by an information retrieval (IR) system are given. - NQ [Kwiatkowski et al., 2019]. It is a question answering dataset where the numbers of its training, validation and test examples are 307K, 7.8K and 7.8K, respectively. Each example is comprised of a google.com query and a corresponding Wikipedia page. We adopt the F1 score as our evaluation metric. We sample 300 examples from the training data of each domains as our validation sets. Finally, we evaluate the performance of different models on the official validation sets. To enhance the performance of our MRC model, we initialize the parameters of the BERT encoding layer using the pretrained model officially released by Google∗ , and randomly initialize other trainable parameters. These models were pretrained on the concatenation of BooksCorpus (800M words) and Wikipedia (2,500M words) via joint modeling of tasks of masked language model and next sentence prediction. We use the uncased base model, which is case insensitive and contains 12 Transformer encoding blocks, each with 12 selfattention heads and 768 hidden units. Moreover, we use the Adam optimizer [Kingma and Ba, 2015] with a learning rate of 1.5×10-5 and a batch size of 12. ∗ https://github.com/google-research/bert | Model | SAD | NQA | HQA | NQ | TQA | AVE. | |---------|-------|-------|-------|-------|-------|--------| | Single | 88.30 | 65.25 | 76.17 | 77.00 | 68.42 | 75.03 | | Mix | 87.88 | 67.19 | 75.69 | 76.08 | 72.60 | 75.89 | | FT | 89.52 | 66.87 | 77.83 | 77.92 | 72.74 | 76.98 | | MS-FT | 88.83 | 65.97 | 76.34 | 77.28 | 70.14 | 75.71 | | MFT | 88.56 | 67.65 | 76.35 | 77.70 | 73.46 | 76.74 | | MS-MFT | 87.69 | 67.47 | 76.50 | 77.67 | 71.20 | 76.11 | | BAM | 89.50 | 68.00 | 77.45 | 77.79 | 72.83 | 77.11 | | IMM | 89.44 | 68.78 | 78.52 | 79.23 | 73.71 | 77.94 | Table 1: Experimental results on various test sets. SAD = SQUAD, NQA = NEWSQA, HQA = HOTPOTQA, and TRIVIAQA = TQA AVE. = average score. ## 5.2 Baseline Models We refer to our framework as IMM and compare it with the following baseline models: - Single. The BERTQA model trained on a single domain training corpus. - Mix. The BERTQAmodel trained on the mix of all training corpora. - Fine-tuning (FT). We first train an MRC model on the mix of non-target domain training corpora until convergence, and then update its parameters on the targetdomain training corpus. - Fine-tuning with the most similar training data (MSFT). A variant of FT, where we only exploit a sourcedomain data that is the most similar to the target-domain data when we using fine-tuning. - Mixed fine-tuning (MFT) [Chu et al., 2017]. Unlike FT, we resume training the MRC model on a mix of all training data. - Mixed fine-tuning with the most similar training data (MS-MFT). A variant of MFT, where only the sourcedomain data most similar to the target-domain data is leveraged via mixed fine-tuning. - Born-Again Multi-task Network (BAM). It is the multi-task version of Born-Again Network (BAN) [Clark et al., 2019]. Using the same many-to-one knowledge distillation procedure as BAM, we first construct a unified multi-domain model using all equally considered domain-specific models and training data, then further fine-train the domain-specific models on individual datasets. ## Effect of the Maximal Iteration Number K From Figure 1, we observe that the maximal iteration number K is an important hyper-parameter, which directly determines the amount of the transferred knowledge into domainspecific MRC models. To investigate the effect of K, we report the average performance of domain-specific MRC models using different Ks on the validation sets. Note that we only keep the best models at each iteration. Figure 3 illustrates the experimental results using different Ks. At first, the performance of almost all domain-specific MRC models has been improved with the increment of the iteration number. However, when the iteration number is greater than 3, all models tend to converge. Therefore, we directly used K=3 in all subsequent experiments. Figure 3: Effect of the iteration number (K) on the validation sets. ![Image](https://ijcai.org/proceedings/2020/fulltext/525/image_000002_46d817b24c5c89521f3e4de8403761d957df21c1c974fd74837ddb5c98e437a8.png) ## Overall Performance Table 1 shows the experimental results. Overall, the conventional approaches including FT, MS-FT, MFT, MS-MFT and BAM, achieve better performance than both Single and Mix, echoing the reported results in previous studies [Min et al., 2017; Chung et al., 2018; Xu et al., 2019]. Besides, under our framework, the performance of all domain-specific models is further significantly improved. Specifically, the average score of our framework is 77.94, which is +2.91, +2.05, +0.96, +2.23, +1.20, +1.83, +0.83 points higher than those of Single, Mix, FT, MS-FT, MFT, MS-MFT and BAM, respectively. In particular, in all datasets except SQUAD, our model achieves the best performance among all models. Even compared with BAM, our improvement is still remarkable. This result strongly demonstrates the effectiveness and generality of our framework. ## Robustness inMRC To inspect the impact of our framework on enhancing robustness, we follow Hu et al. [2018] to report the results on two adversarial SQUAD datasets, namely ADDONESENT and ADDSENT. From Table 2, we can observe that the improvement on adversarial data is much higher than the one on original squad dataset. This result is reasonable, because our framework can better absorb the knowledge of other domains, boosting its ability in dealing with the much more confusing answers in adversarial datasets. ## 5.3 Ablation Study The main highlights of our framework consist of the usages of domain-level similarities of training corpora to the targetdomain one, and two stages of distillation-based knowledge transfers. To evaluate their effects on our framework, we compared our framework with its following variants: (1) - SIMI. The variant of IMM without domain-level similarities, i.e., we directly set si,j of Equation 7 to 1; (2) -KD1. Table 2: Comparison of different approaches on two adversarial SQUAD datasets. | Model | SQUAD | ADDSENT | ADDONESENT | |---------|---------|-----------|--------------| | Single | 88.30 | 50.15 | 61.38 | | Mix | 87.88 | 53.15 | 64.04 | | FT | 89.52 | 53.05 | 63.91 | | MS-FT | 88.83 | 50.88 | 62.78 | | MFT | 88.89 | 53.29 | 64.23 | | MS-MFT | 87.69 | 51.03 | 62.94 | | BAM | 89.50 | 55.52 | 65.20 | | IMM | 89.44 | 56.49 | 66.20 | Table 3: Ablation study of our framework on various test sets. | Model | AVE. | |-----------|--------| | IMM | 77.94 | | -SIMI | 77.49 | | -KD1 | 77.68 | | -KD2 | 77.58 | | -KD1, KD2 | 77.34 | The variant of IMMwithout the distillation-based knowledge transfer from {θ(k) j }j̸=i to φ(k) i , where λmd of Equation 6 is directly set to 0; and (3) -KD2. The variant of IMMwithout the distillation-based knowledge transfer from φ(k) i to θ(k+1) i , where λds of Equation 11 is directly set to 0. From the experimental results shown in Table 3, we can observe that our framework achieves better performance than all its variants. These results show that domain-level similarities, two stages of distillation-based knowledge transfers are indeed beneficial for MRC. ## 6 Conclusion In this work, we have presented a novel iterative multi-source mutual knowledge transfer framework for MRC, which enables all domain-specific MRC models to constantly reinforce each other by iteratively and differentially absorbing the domain-shared knowledge from others. Experimental results and in-depth analyses on several MRC datasets strongly demonstrate the effectiveness of our framework. In the future, we will focus on how to effectively leverage the unlabeled data of different domains under our framework. Besides, we plan to apply our framework to other tasks, such as machine translation [Zeng et al., 2019]. Finally, inspired by studies of multi-domain NMT [Zeng et al., 2018; Su et al., 2019], we will explore a unified multi-domain model for MRC. ## Acknowledgements This work was supported by the National Natural Science Foundation of China (No. 61672440), Beijing Advanced Innovation Center for Language Resources (No. TYR17002), and the Scientific Research Project of National Language Committee of China (No. YB135-49). We also thank the reviewers for their insightful comments. ## References 1. [Chu et al., 2017] Chenhui Chu, Raj Dabre, and Sadao Kurohashi. An empirical comparison of domain adapta- tion methods for neural machine translation. In ACL, 2017. 2. [Chung et al., 2018] Yu-An Chung, Hung-yi Lee, and James R. Glass. Supervised and unsupervised transfer learning for question answering. In NAACL-HLT, 2018. 3. [Clark et al., 2019] Kevin Clark, Minh-Thang Luong, Ur- vashi Khandelwal, Christopher D. Manning, and Quoc V. Le. Bam! born-again multi-task networks for natural lan- guage understanding. In ACL, 2019. 4. [Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, 2019. 5. [Duan et al., 2017] Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. Question generation for question answering. In EMNLP, 2017. 6. [Furlanello et al., 2018] Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born-again neural net- works. In ICML, 2018. 7. [Golub et al., 2017] David Golub, Po-Sen Huang, Xiaodong He, and Li Deng. Two-stage synthesis networks for trans- fer learning in machine comprehension. In EMNLP, 2017. 8. [Hinton et al., 2015] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural net- work. CoRR, abs/1503.02531, 2015. 9. [Hu et al., 2018] Minghao Hu, Yuxing Peng, Furu Wei, Zhen Huang, Dongsheng Li, Nan Yang, and Ming Zhou. Attention-guided answer distillation for machine reading comprehension. In EMNLP, 2018. 10. [Joshi et al., 2017] Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading compre- hension. In ACL, 2017. 11. [Kim and Rush, 2016] Yoon Kim and Alexander M. Rush. Sequence-level knowledge distillation. In EMNLP, 2016. 12. [Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 13. [Kwiatkowski et al., 2019] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. TACL, 2019. 14. [Liu et al., 2020] Kai Liu, Xin Liu, An Yang, Jing Liu, Jin- song Su, Sujian Li, and Qiaoqiao She. A robust adversarial training approach to machine reading comprehension. In AAAI, 2020. 15. [Min et al., 2017] Sewon Min, Min Joon Seo, and Hannaneh Hajishirzi. Question answering through transfer learning from large fine-grained supervision data. In ACL, 2017. 16. [Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014. 17. [Rajpurkar et al., 2016] Pranav Rajpurkar, Jian Zhang, Kon- stantin Lopyrev, and Percy Liang. Squad: 100, 000+ ques- tions for machine comprehension of text. In EMNLP, 2016. 18. [Seo et al., 2017] Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. In ICLR, 2017. 19. [Song et al., 2018] Linfeng Song, Zhiguo Wang, Wael Hamza, Yue Zhang, and Daniel Gildea. Leveraging context information for natural question generation. In NAACL-HLT (Short Papers), 2018. 20. [Su et al., 2019] Jinsong Su, Jiali Zeng, Jun Xie, Huating Wen, Yongjing Yin, and Yang Liu. Exploring discrimina- tive word-level domain contexts for multi-domain neural machine translation. TPAMI, 2019. 21. [Talmor and Berant, 2019] Alon Talmor and Jonathan Be- rant. Multiqa: An empirical investigation of generalization and transfer in reading comprehension. In ACL, 2019. 22. [Tan et al., 2019] Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, and Tie-Yan Liu. Multilingual neural machine trans- lation with knowledge distillation. In ICLR, 2019. 23. [Trischler et al., 2017] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. Newsqa: A machine comprehension dataset. In ACL, 2017. 24. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017. 25. [Xu et al., 2019] Yichong Xu, Xiaodong Liu, Yelong Shen, Jingjing Liu, and Jianfeng Gao. Multi-task learning with sample re-weighting for machine reading comprehension. In NAACL-HLT, 2019. 26. [Yang et al., 2018] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdi- nov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, 2018. 27. [Yu et al., 2018] Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. Qanet: Combining local convolution with global self-attention for reading comprehension. In ICLR, 2018. 28. [Zeng et al., 2018] Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. Multi-domain neural machine translation with word-level domain context discrimination. In EMNLP, 2018. 29. [Zeng et al., 2019] Jiali Zeng, Yang Liu, jinsong su, yubing Ge, Yaojie Lu, Yongjing Yin, and jiebo luo. Iterative dual domain adaptation for neural machine translation. In EMNLP, 2019.