Consistent MetaReg: Alleviating Intra-task Discrepancy for Better Meta-knowledge

1 Introduction

Learning quickly is a kind of ability of human intelligence, e.g., children can recognize objects only from a few examples. However, this poses a great challenge to the existing deep learning models, which require large-scale training data to achieve promising performance. To tackle this problem, in recent years, meta-learning (i.e., learning to learn) has drawn increasing interest in the machine learning community [Finn et al., 2017; Rajeswaran et al., 2019]. The goal of these methods is to learn the meta-knowledge across tasks, which can help model learn fast or adapt quickly in new tasks.

For meta-learning approaches, the core issue is - How to learn effective meta-knowledge. Regarding that the training data is usually limited in few-shot learning, these few data cannot describe the real data distribution, which results in the intra-task data-distribution discrepancy between training data and test data in each task. However, most existing metalearning approaches usually ignore this discrepancy during the meta-knowledge learning phase. In this case, the learned inferior meta-knowledge might have a disadvantage to the final performance.

∗Corresponding author

To this end, we propose a new Consistent Metaregularization (CM) method, which could alleviate the intratask discrepancy in the meta-training course. The motivation of our method is illustrated in Fig. 1. Although the acrosstask meta-knowledge can help the base-learner trained by the training data fit well on the test data, the base-learner indeed cannot work well on the test data because of the discrepancy. Therefore, we put forward the Consistent MetaReg to alleviate the intra-task discrepancy to help base-learner work well on the test data in each task for learning better metaknowledge. Moreover, the proposed meta-regularization method can be easily integrated into existing gradient-based meta-learning models. Concretely, we obtain the base-learner for training data as the traditional meta-learning approaches. In particular, these traditional methods only utilize test data to update the meta-learner. Differently, our method considers that using test data to update both the meta-learner and baselearner, i.e., we reversely use the test data as training data and acquire a new base-learner model for test data. To migrate the data distribution discrepancy between training data and test data in each task, we expect that two base-learner models trained on training data and test data to be consistent. Therefore, we develop Consistent Meta-regularization to achieve this goal.

In this paper, we demonstrate the efficacy of the Consistent Meta-regularization from the theoretical and experimental perspectives, respectively. On the theoretical aspect, we prove that using the proposed method, the conventional gradient-based meta-learning models can indeed achieve lower regret bound than before. In addition, we also validate CM on three benchmark datasets, i.e., miniImageNet, tiered-ImageNet and office31. The experimental results show that our regularization method could learn more useful metaknowledge by migrating the intra-task discrepancy. Integrated with our Consistent Meta-regularization, three existing state-of-the-art optimization-based meta-learning approaches perform better on all datasets.

In sum, our contributions in this paper are listed as follows:

• To the best of our knowledge, our method is the first one to consider the intra-task discrepancy problem in the typical meta-learning models, which is seldom touched

es

Figure 1: Concept. We utilize the 2-way few-shot classification task as an example.

in previous works.

• We develop Consistent Meta-regularization to reduce the intra-task discrepancy, which can be easily introduced into the existing gradient-based meta-learning models.
• We demonstrate the meta-learning methods with the proposed CM can have lower regret bound in theory, and extensive experiments also highlight that adding CM into the conventional meta-learning frameworks can indeed achieve better performance.

2 RelatedWork

The topic of meta-learning (or learning to learn) was introduced and studied several decades ago [Schmidhuber, 1987; Bengio et al., 1992; Thrun and Pratt, 1998]. Early work mostly focused on learning how to dynamically adjust the inductive bias [Utgoff, 1986] or hypothesis space for a learning algorithm [Vilalta and Drissi, 2002]. In recent years, twolevel framework is popular in the present meta-learning approaches. The motivation of this framework is that metalevel (meta-learner) is used to learn meta-knowledge, which can help base-level (task-specific model or base-learner) learn fast or adapt quickly in a new task. The two-level framework can be broadly divided into three categories.

• Metric-based method. In these methods, a nonparametric similarity function is used as base-learner to evaluate the similarity between examples. The metalearner is trained to learn useful meta-knowledge in the predefined metric space, e.g., Euclidean distance based prototypical networks [Snell et al., 2017], cosine similarity based recurrence with attention mechanisms [Vinyals et al., 2016].
• Model-based method. The meta-learner is usually designed as a parameterized predictor to generate baselearner parameters. For example, Ravi et al . [Ravi and Larochelle, 2017] used recurrent neural network as meta-learner to direct the updating for base-learner. Munkhdalai et al . [Munkhdalai and Yu, 2017] designed
• a meta-learner, which uses loss gradients from baselearner to predict parameters for base-learner.
• Gradient-based method. Finn et al . [Finn et al., 2017] proposed model-agnostic meta learning (MAML) for deep models. MAML is similar to the initialization of deep networks. The meta-learner aims to learn a good initialization for all tasks, and the base-learner merely requires a few gradient steps from this initialization to achieve great performance. However, there are still many limitations for MAML. Some works [Li et al., 2017; Rajeswaran et al., 2019; Na et al., 2019] were developed to further improve it. Besides MAML, some gradient-based methods [Bertinetto et al., 2019; Lee et al., 2019] leveraged bilevel framework to optimize meta-learning algorithms.

Our work is the most related to gradient-based methods. Compared with metric-based methods, gradient-based methods can be broadly applied in many areas, such as reinforcement learning [Rakelly et al., 2019] and NLP [Xie et al., 2019]. Moreover, the gradient-based methods do not introduce additional parameters or require a particular learner architecture. These factors result in that gradient-based methods become a promising and hot research topic in metalearning recently. However, the previous gradient-based methods ignore the intra-task discrepancy, which hinders meta-learning models to effectively learn meta-knowledge. The proposed Consistent Meta-regularization method aims to overcome this limitation, which can be integrated into MAML or bilevel gradient-based methods and enhance their performance.

3 Gradient-basedMeta-learningMethod

In this section, we firstly introduce the problem formulation for meta-learning in the context of supervised learning. Then, two important kinds of gradient-based meta-learning approaches are enumerated.

3.1 Meta-Learning ProblemFormulation

The problem formulation of meta-learning follows [Amit and Meir, 2018]. We assume that all tasks share the sample space Z, hypothesis space H and loss function ℓ : H × Z → [0, 1]. N different tasks are sampled from an unknown task distribution τ as meta-training set {T1, · · · , Tn}. And each task Ti has an observed dataset Str i . The goal of the meta-learner is to extract meta-knowledge from the meta-training set so that learning a new task only needs few training data. We denote that the meta-knowledge P comes in the form of a distribution over hypotheses. During learning a new task, the base-learner uses the task-specific observed dataset Str and the meta-knowledge P to learn a base-learner Q(P, Str ) over H. The quality of meta-knowledge P is measured by the expected loss for learning new tasks, as defined by

If we assume the meta-knowledge P is sampled from the hyper-posterior Q(P), we can ideally obtain the best metaknowledge P ∗ via minimizing the following formula as

However, Eq. 2 is not computable. Recent meta-learning algorithms use the idea from [Vinyals et al., 2016]: "make the test and train conditions much match". The observed data Str i in each task Ti is viewed as training dataset. Besides Str i , a test dataset Sts i is also sampled from Ti. Then the algorithms learn good meta-knowledge by minimizing the empirical risk

3.2 MAML

Model-agnostic meta-learning (MAML) is a pretty significant gradient-based meta-learning method, which has been widely applied to many fields [Al-Shedivat et al., 2018; Javed and White, 2019]. The initialization of model is regarded as metaknowledge in MAML. Thus, the goal of MAML is to metalearn the initial model parameter θ to generalize over the task distribution τ . The empirical loss function in MAML is

where α is the stepsize. When encountering a new task Tj , the task-specific predictor θj can be easily obtained in a single (or a few) gradient step from the inital θ.

3.3 Bilevel Method

Recently, the bilevel gradient-based method has also attracted many attentions, which achieves the state-of-the-art performance in many computer vision tasks [Lee et al., 2019; Tian et al., 2019]. Bilevel meta-learning framework is proposed in [Franceschi et al., 2018], and its formulation is as

where function f : Λ → R is defined at λ ∈ Λ as

In Eq. , E : Rd → R is the outer objective. For every λ ∈ Λ, Lλ : Rd → R is the inner objective. The two-level metalearning algorithm can be easily formulated as a bilevel problem. We can use the outer objective to learn meta-knowledge, and the inner objective is utilized to learn base-learner. Then the empirical loss Eq. 3 can be rewritten as

In Eq. 7, θ and w are the meta-parameter and the base-learner parameter, respectively. L and Lbase denote the meta-loss and task-specific loss, respectively. And Alg is a base-learner algorithm, which computes task-specific parameter based on meta-parameter θ. In fact, MAML can be regarded a special case of bilevel gradient-based meta-learning method, which is analyzed in [Rajeswaran et al., 2019].

Moreover, since both MAML and bilevel gradient-based method can be optimized by gradient descent, it means these approaches can be easily implemented in existing deep learning frameworks (e.g., Pytorch and Tensorflow).

4 Our Method

4.1 Consistent Meta-regularization

As mentioned above, how to learn good meta-knowledge, which can generalize well over the task distribution τ , is very important for meta-learning algorithm. Although MAML or bilevel based methods can indeed learn meta-knowledge by minimizing Eq. 4 and Eq. 7, respectively, all of them ignore the intra-task discrepancy in the meta-training set (i.e., the data-distribution discrepancy between training data and test data in each task because few samples in the few-shot setting cannot describe the real data-distribution of a dataset), which is not conducive to learn good meta-knowledge.

Our proposed Consistent Meta-regularization (CM) method aims to mitigate the impact of the intra-task discrepancy on gradient-based meta-learning models. Specially, we firstly train the base-learner for the training data Str i in the task Ti.Then we reversely exploit the test data Sts i from the task Ti as training data, and train a new base-learner for the test data. For simplicity, two base-learners for training data Str i and Sts i in task Ti are denoted as M tr i and M ts i , respectively. Finally, we minimize the difference between M tr i and M ts i as a regularization which can be inserted into the traditional meta-loss to alleviate the intra-task discrepancy. Differently, the test data Sts i is just used to minimize the meta-loss in traditional gradient-based meta-learning.

In this paper, we directly use F-norm of the difference between parameters of M tr i and M ts i to measure the gap between the two models. And we can easily integrate the proposed regularization into existing gradient-based metalearning models (i.e., MAML and bilevel method). For example, we can define the Consistent Meta-regularization loss for bilevel method as

where δ is the regularization parameter. If we consider that the base-learner is a deep model [Finn et al., 2017], which contains K layers, the Consistent Meta-regularization loss can be rewritten as

where Lj is the j-th layer in base-learner. The Consistent Meta-regularization gradient-based meta-learning algorithm is summarized in Alg. 1.

Remark. The traditional gradient-based meta-learning approaches (GBML) can be considered as a bilevel problem. The meta-learner object is used to learn meta-knowledge, and the base learner utilizes the meta knowledge to learn a base model in the task-specific space. In the meta-training phase, these GBML approaches firstly optimize the base-learner, then utilize test dataset Sts i in each task Ti to minimize metaloss to learn cross-task meta-knowledge. However, if the base-learner learned by training data in task-specific space cannot generalize well on test data, the meta-loss computed by test data will contain two errors (i.e., true meta-knowledge

Algorithm 1 The proposed CM for meta supervised learning

Require: τ: distribution over tasks, regularization parameter δ, meta-learner step size η, gradient-based meta-learning algorithm: GBML.

Output: Meta-parameter θ

• 1: Randomly initialize θ.
• 2: while not converged do
• 3: Sample mini-batch of tasks {Ti}Bi=1 ∼ τ.
• 4: for each task Ti do
• 5: Sample N datapoints Str i = {xm, ym} from Ti.
• 6: Train a base-learner M tr i for Str i by GBML.
• 7: Sample K datapoints Sts i = {xk , yk } from Ti.
• 8: Train a base-learner M ts i for Sts i by GBML.
• 9: end for
• 10: Update meta-parameter with gradient descent:

error and model discrepancy error). Thus, it is difficult to obtain good meta-knowledge. Particularly, this problem will be worse when the training data and test data in each task have a large domain discrepancy. Our proposed regularization eliminates the intra-task discrepancy by pulling M tr i andM ts i (i.e., the models for training data and test data) closer in the taskspecific space. Therefore, based on our method, the metaloss is merely from the true meta-knowledge error, which can guide the meta-learner to extract better meta-knowledge.

4.2 Regret Bound for Consistent MetaReg

To further explain the efficacy of our method, a theoretical analysis is given in this part. Specifically, we prove our method can achieve a lower regret bound when compared with the method without using Consistent Metaregularization in the online convex optimization framework.

We use the same notation with Sec. 3.1, and the Sts i in each task Ti contains mi samples, i.e., Sts i = {(xi,j, yi,j), · · · , (xi,mi , yi,mi )}. In addition, we assume that the learner incurs the loss ˆ ℓi,j := ℓ(ˆ yi,j , yi,j ), when a sample (xi,j, yi,j) is revealed.

Definition 1. The prediction error of a task Ti is

The average error (empirical risk) of a meta-learning algorithm after N tasks is1N N∑ i=1 1mi mi∑ j=1 ˆ ℓi,j . (11)

Given meta-knowledge P , we already know the best predictor h∗i for task Ti. The regret of a task Ti is

Then, we introduce the definition of online convex programming problem, following [Zinkevich, 2003].

Definition 2. An online convex programming problem consists of a feasible set F ⊆ Rn and a infinite sequence {f1, f2, · · · }, where each ft : F → R is a convex function.

At each time step t, an online player selects a vector xt ∈ F. After the vector is selected, it receives the cost function ft.

Lemma 1. [Hazan et al., 2007] If we use gradient descent to select vector at each step, i.e., xt = ΠF (xt-1 - ηt∇ft-1(xt-1)). Here, ΠF denotes the projection onto nearest point in F, ηt > 0 is the step size.

where ‖∇f‖ ≤ G and ∇2 f ⪰ σI. G and σ are positive constants.

Next, we use Lemma 1 to prove Theorem 1.

Theorem1. If we use parameter θ in a convex set Θ to learn meta-knowledge, and ℓ : Θ → R is a convex function. θ is optimized by gradient descent. And the within-task algorithm has a regret bound Ri(θ) ≤ β(θ,mi) for any θ, then

Proof.

Theorem 1 shows that the regret bound of average error is related with h∗i . As mentioned in Sec. 3.1, the traditional meta-learning algorithm uses Str i to train the task-specific model M tr i for task Ti. However, the intra-task discrepancy causes that the model trained by Str i is far from h∗i . Since h∗i denotes the best predictor for Sts i in the test set, our proposed Consistent Meta-regularization supposes that M ts i (i.e., the model trained by the test set Sts i ) is close to h∗i . Thus, our method aims to pull the M tr i closer to M ts i , which can reduce the gap between M tr i and h∗i to achieve lower regret bound.

5 Experiments

In this part, we first describe the implementation details. Then, we present results on three benchmark datasets for the few-shot classification task, including two derivatives of ImageNet [Russakovsky et al., 2015] (i.e., miniImageNet [Vinyals et al., 2016], tieredImageNet [Ren et al., 2018]) and office31 [Saenko et al., 2010]. Particularly, office31 involves the larger intra-task data-distribution discrepancy when compared with miniImageNet and tieredImageNet. Finally, we analyze the impact of our method on learning speed, and the parameter sensitivity of regularization parameter δ.

5.1 Implement Details

We use the same architecture as the embedding model used in [Vinyals et al., 2016], which has 4 modules with a 3 × 3 convolutions and 64 filters, followed by batch normalization, a ReLU nonlinearity, and 2 × 2 max-pooling in all experiments. And all the images are resized to 84×84. Adamwith learning rate 0.001 is used as optimizer. During the metatraining process, we sample 5 training (support) samples and 5 test (query) samples as Str i and Sts i in each task Ti, respectively. The number of test example for each task is set as 10 during the meta-testing phase. All methods are trained on a single NVIDIA 1080 Ti.

We insert the proposed CMmethod into three state-of-theart gradient-based meta-learning approaches, i.e., MAML, R2-D2 [Bertinetto et al., 2019] and MetaOptNet-SVM [Lee et al., 2019] to validate its effectiveness. Particularly, all tricks in R2-D2 and MetaOptNet-SVM aiming to improve the performance are not used in our experiments, such as learnable scale parameter to adjust the prediction score 1 .

5.2 Experiments on ImageNet Derivatives

MiniImageNet is a standard benchmark for few-shot image classification, consisting of 100 randomly chosen classes from ILSVRC-2012 [Russakovsky et al., 2015]. These classes are randomly split into 64, 16 and 20 classes for meta-training, meta-validation, and meta-testing respectively. Each class contains 600 images. The tieredImageNet benchmark is a larger subset of ILSVRC-2012, composed of 608 classes grouped into 34 high-level categories. These are divided into 20 categories for meta-training, 6 categories for meta-validation, and 8 categories for meta-testing. This corresponds to 351, 97 and 160 classes for meta-training, metavalidation, and meta-testing respectively.

We perform the pre-processing for the two datasets following [Lee et al., 2019]. Because of the slow convergence rate of MAML, we trained MAML 60,000 iterations on miniImageNet and tieredImagNet. The other methods are trained by

1 Source code: https://github.com/P1nzhuo/Consistent-MetaReg

30,000 and 50,000 iterations on miniImageNet and tieredImageNet, respectively. The meta batch-size is set as 4 and 8 for MAML and the other methods. The parameter δ is set as 1 for MAML and 5 for MetaOptNet-SVM and R2-D2.

Results. Tab. 1 reports the results on the 5-way miniImageNet and tieredImageNet. All reported results are average performances with 95% confidence interval over 2000 tasks randomly sampled from meta-testing split. OptNet denotes metaOptNet-SVM, ProNet is the Prototypical Networks. And MAML-CM, etc. represent the methods integrated with our Consistent Meta-regularization. The methods are separated into two groups: optimization-based (O) and metric-based (M). As seen, by mitigating the intra-task discrepancy, our proposed CM can improve the average performance of MAML, OptNet and R2D2, 0.30% (0.33%), 1.10% (1.90%) and 2.66% (2.59%) on miniImageNet (tieredImageNet), respectively. Particularly, ProNet achieves the best performance, which is not very surprised due to the superiority of metric-based methods for few-shot image classification on ImageNet derivatives [Chen et al., 2019]. However, metric-based meta-learning approach is not convenient to be extended to other fields, e.g., NLP, semantic segmentation and object detection, because it is very difficult to design an appropriate metric method.

5.3 Experiments on Office31

Office-31 is a standard benchmark for visual domain adaptation, consisting of 4,652 images in 31 classes collected from three domains: Amazon (A), which contains images downloaded from amazon.com, Webcam (W) and DSLR (D), which contain images taken by a web camera and a digital SLR camera, respectively, in an office environment. And we randomly divide office31 into three splits which contain 20, 5 and 6 classes as meta-training, meta-validation, and metatesting, respectively. For each task Ti, we sample training dataset Str i from two domains, which are randomly sampled from A, W and D, and test dataset Sts i is sampled from the rest of domain. MAML is trained in the same setting with Sec. 5.2. And the other models are trained by 30,000 tasks. The meta batch-size is set as 4 for all the methods. As for regularization parameter, we use the same set with Sec. 5.2.

Results. The results on office31 are shown in Tab. 2. All reported results are average performances with 95% confidence interval over 1000 tasks randomly sampled from meta-testing split. Because the training data Str i and test data Sts i in each task Ti are from different domains. There is a large intra-task data-distribution discrepancy on this dataset. Our CMmethod can boost the average performance of MAML, OptNet, and R2D2 by 3.14%, 3.37% and 2.72% on office31. Moreover, our R2D2-CM model achieves the best performance. Compared with the results on ImageNet derivatives, ProNet cannot handle this situation of a large data-distribution discrepancy due to its limitation. As for optimization-based methods, the data-distribution discrepancy impact greatly on MAML, and we can find similar results in [Finn and Levine, 2018]. The proposed regularization can better promote optimization-based methods in this case, i.e., there is a large intra-task difference.

5.4 Learning Efficiency Analysis

In this section, we reveal that with our proposed metaregularization, the traditional gradient-based meta-learning approach can converge faster and achieve better performance.

We report the results of R2D2 and R2D2-CM on metatesting split of miniImageNet and office31 at different training epochs, respectively, as shown in Fig. 2. Shaded region denotes 95% confidence interval. When conducting the comparison between R2D2 and R2D2-CM (ours) on 5-way, 5shot classification task on two datasets, we can observe that our method outperforms the R2D2 at the beginning of training (about 6th epoch), and maintain higher performance until convergence. In general, our method can help model converge faster and achieve better performance. Because office31 is smaller than miniImageNet (the number of images is just 7.7% of miniImageNet), with training epoch increasing, meta-overfitting happens in both two models.

5.5 Parameter Sensitivity Analysis

In order to evaluate the influence of meta-regularization parameter on our method, we train our R2D2-CM with different values of regularization parameter δ on office31. And the performance of these models on meta-testing split is in Fig. 3. Shaded region denotes 95% confidence interval.

Our method outperforms the traditional R2D2 approach (regularization parameter = 0) with different values of regularization parameter. And the performance of our method doesn't change much from δ = 6 to δ = 10. It can confirm that our model has a lager robust interval on δ.

6 Conclusion

In this paper, we consider the intra-task discrepancy issue in the traditional meta-learning models, which is usually ignored in previous works. To handle this issue, we introduce Consistent Meta-regularization to alleviate the discrepancy for gradient-based meta-learning approaches. Moreover, we prove that with our proposed meta-regularization, the traditional meta-learning approach can achieve lower regret bound in the convex setting. Furthermore, extensive experiments on three datasets reveal the superiority of our method in the nonconvex setting. Particularly, in this paper, we follow recent meta-learning approaches, which use deep models as metalearner in the non-convex setting. In the future work, we will further give the theoretical analysis in the non-convex setting.

Acknowledgments

The work is supported by Science and Technology Innovation 2030-"New Generation Artificial Intelligence" Major Project (No. 2018AAA0100905) and NSFC (No. 1673203).