{"type":"ScholarlyArticle","title":"Hacking Task Confounder in Meta-Learning","authors":[{"name":"Jingyao Wang","slug":"jingyao-wang","given":"Jingyao","family":"Wang","affiliation":"Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences {wangjingyao23 ,"},{"name":"Yi Ren","slug":"yi-ren","given":"Yi","family":"Ren","affiliation":"Institute of Software Chinese Academy of Sciences"},{"name":"Zeen Song","slug":"zeen-song","given":"Zeen","family":"Song","affiliation":"Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences {wangjingyao23 ,"},{"name":"Jianqi Zhang","slug":"jianqi-zhang","given":"Jianqi","family":"Zhang","affiliation":"Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences {wangjingyao23 ,"},{"name":"Changwen Zheng","slug":"changwen-zheng","given":"Changwen","family":"Zheng","affiliation":"Institute of Software Chinese Academy of Sciences"},{"name":"Wenwen Qiang","slug":"wenwen-qiang","given":"Wenwen","family":"Qiang","affiliation":"Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences {wangjingyao23 ,"}],"abstract":"Meta-learning enables rapid generalization to new tasks by learning knowledge from various tasks. It is intuitively assumed that as the training progresses, a model will acquire richer knowledge, leading to better generalization performance. However, our experiments reveal an unexpected result: there is negative knowledge transfer between tasks, affecting generalization performance. To explain this phenomenon, we conduct Structural Causal Models (SCMs) for causal analysis. Our investigation uncovers the presence of spurious correlations between task-specific causal factors and labels in meta-learning. Furthermore, the confounding factors differ across different batches. We refer to these confounding factors as ``Task Confounders\". Based on these findings, we propose a plug-and-play Meta-learning Causal Representation Learner (MetaCRL) to eliminate task confounders. It encodes decoupled generating factors from multiple tasks and utilizes an invariant-based bi-level optimization mechanism to ensure their causality for meta-learning. Extensive experiments on various benchmark datasets demonstrate that our work achieves state-of-the-art (SOTA) performance. The code is provided in https://github.com/WangJingyao07/MetaCRL.","doi":"10.24963/ijcai.2024/560","url":"https://ijcai.org/proceedings/2024/560","pdf":"https://ijcai.org/proceedings/2024/0560.pdf","markdown":"https://ijcai.org/proceedings/2024/560.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/560","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":560,"pages":{"first":5064,"last":5072},"keywords":["Machine Learning: ML: Few-shot learning","Machine Learning: ML: Meta-learning","Machine Learning: ML: Causality","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"],"videos":[],"references":[{"position":1,"raw":"[Abdollahzadeh et al., 2021] Milad Abdollahzadeh, Touba Malekzadeh, and Ngai-Man Man Cheung. Revisit multi- modal meta-learning through the lens of multi-task learn- ing. Advances in Neural Information Processing Systems, 34:14632-14644, 2021.","title":"Revisit multimodal meta-learning through the lens of multi-task learning","authors":"Milad Abdollahzadeh, Touba Malekzadeh, Ngai-Man Man Cheung","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arjovsky et al., 2019] Martín Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. CoRR, abs/1907.02893, 2019.","title":"Invariant risk minimization","authors":"Martín Arjovsky, Léon Bottou, Ishaan Gulrajani, David Lopez-Paz","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2019] Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, Sébastien Lachapelle, Olexa Bi- laniuk, Anirudh Goyal, and Christopher Pal. A meta- transfer objective for learning to disentangle causal mech- anisms. arXiv preprint arXiv:1901.10912, 2019.","title":"A metatransfer objective for learning to disentangle causal mechanisms","authors":"Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, Sébastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, Christopher Pal","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Jiaxin Chen, Li-Ming Zhan, Xiao-Ming Wu, and Fu-lai Chung. Variational metric scaling for metric-based meta-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3478-3485, 2020.","title":"Variational metric scaling for metric-based meta-learning","authors":"Jiaxin Chen, Li-Ming Zhan, Xiao-Ming Wu, Fu-Lai Chung","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fifty et al., 2020] Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Mea- suring and harnessing transference in multi-task learning. arXiv preprint arXiv:2010.15413, 2020.","title":"Measuring and harnessing transference in multi-task learning","authors":"Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126-1135. PMLR, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2024] Huijie Guo, Ying Ba, Jie Hu, Lingyu Si, Wenwen Qiang, and Lei Shi. Self-supervised representa- tion learning with meta comprehensive regularization. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 1959-1967, 2024.","title":"Self-supervised representation learning with meta comprehensive regularization","authors":"Huijie Guo, Ying Ba, Jie Hu, Lingyu Si, Wenwen Qiang, Lei Shi","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2022] Ziniu Hu, Zhe Zhao, Xinyang Yi, Tian- sheng Yao, Lichan Hong, Yizhou Sun, and Ed Chi. Im- proving multi-task generalization via regularizing spurious correlation. Advances in Neural Information Processing Systems, 35:11450-11466, 2022.","title":"Improving multi-task generalization via regularizing spurious correlation","authors":"Ziniu Hu, Zhe Zhao, Xinyang Yi, Tiansheng Yao, Lichan Hong, Yizhou Sun, Ed Chi","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Islam et al., 2020] Md Amirul Islam, Sen Jia, and Neil DB Bruce. How much position information do convo- lutional neural networks encode? arXiv preprint arXiv:2001.08248, 2020.","title":"How much position information do convolutional neural networks encode","authors":"Amirul Md, Sen Islam, Neil Db Jia, Bruce","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jamal and Qi, 2019] Muhammad Abdullah Jamal and Guo- Jun Qi. Task agnostic meta-learning for few-shot learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11719- 11727, 2019.","title":"Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta-learning for few-shot learning","authors":"Qi Jamal","year":"2019","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jensen and Shen, 2004] Richard Jensen and Qiang Shen. Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches. IEEE Transactions on knowledge and data engineering, 16(12):1457-1471, 2004.","title":"Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches","authors":"Shen Jensen, Richard Jensen, Qiang Shen","year":"2004","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang et al., 2022] Yinjie Jiang, Zhengyu Chen, Kun Kuang, Luotian Yuan, Xinhai Ye, Zhihua Wang, Fei Wu, and Ying Wei. The role of deconfounding in meta-learning. In Inter- national Conference on Machine Learning, pages 10161- 10176. PMLR, 2022.","title":"The role of deconfounding in meta-learning","authors":"Yinjie Jiang, Zhengyu Chen, Kun Kuang, Luotian Yuan, Xinhai Ye, Zhihua Wang, Fei Wu, Ying Wei","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Koyama and Yamaguchi, 2020] Masanori Koyama and Shoichiro Yamaguchi. When is invariance useful in an out-of-distribution generalization problem? arXiv preprint arXiv:2008.01883, 2020.","title":"When is invariance useful in an out-of-distribution generalization problem","authors":"Yamaguchi Koyama, Masanori Koyama, Shoichiro Yamaguchi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lee and Choi, 2018] Yoonho Lee and Seungjin Choi. Gradient-based meta-learning with learned layerwise met- ric and subspace. In International Conference on Machine Learning, pages 2927-2936. PMLR, 2018.","title":"Yoonho Lee and Seungjin Choi. Gradient-based meta-learning with learned layerwise metric and subspace","authors":"Choi Lee","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2020] Hae Beom Lee, Taewook Nam, Eunho Yang, and Sung Ju Hwang. Meta dropout: Learning to perturb latent features for generalization. 2020.","title":"Meta dropout: Learning to perturb latent features for generalization","authors":"Hae Beom, Lee, Taewook Nam, Eunho Yang, Sung Ju Hwang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2017] Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835, 2017.","title":"Meta-sgd: Learning to learn quickly for few-shot learning","authors":"Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2023] Ximan Li, Weihong Deng, Shan Li, and Yong Li. Compound expression recognition in-the-wild with au-assisted meta multi-task learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 5734-5743, 2023.","title":"Compound expression recognition in-the-wild with au-assisted meta multi-task learning","authors":"Ximan Li, Weihong Deng, Shan Li, Yong Li","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Martin et al., 2019] Eric J Martin, Valery R Polyakov, Xiang-Wei Zhu, Li Tian, Prasenjit Mukherjee, and Xin Liu. All-assay-max2 pqsar: activity predictions as ac- curate as four-concentration ic50s for 8558 novartis as- says. Journal of chemical information and modeling, 59(10):4450-4459, 2019.","title":"All-assay-max2 pqsar: activity predictions as accurate as four-concentration ic50s for 8558 novartis assays","authors":"Eric J Martin, Valery R Polyakov, Xiang-Wei Zhu, Li Tian, Prasenjit Mukherjee, Xin Liu","year":"2019","venue":"Journal of chemical information and modeling","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nichol and Schulman, 2018] Alex Nichol and John Schul- man. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2(3):4, 2018.","title":"Reptile: a scalable metalearning algorithm","authors":"Alex Nichol, John Schulman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nogueira et al., 2022] Ana Rita Nogueira, Andrea Pugnana, Salvatore Ruggieri, Dino Pedreschi, and João Gama. Methods and tools for causal discovery and causal infer- ence. Wiley interdisciplinary reviews: data mining and knowledge discovery, 12(2):e1449, 2022.","title":"Methods and tools for causal discovery and causal inference","authors":"Ana Rita Nogueira, Andrea Pugnana, Salvatore Ruggieri, Dino Pedreschi, João Gama","year":"2022","venue":"Wiley interdisciplinary reviews: data mining and knowledge discovery","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Oh et al., 2020] Jaehoon Oh, Hyungjun Yoo, ChangHwan Kim, and Se-Young Yun. Boil: Towards represen- tation change for few-shot learning. arXiv preprint arXiv:2008.08882, 2020.","title":"Boil: Towards representation change for few-shot learning","authors":"Jaehoon Oh, Hyungjun Yoo, Changhwan Kim, Se-Young Yun","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Qiang et al., 2023] Wenwen Qiang, Jiangmeng Li, Bing Su, Jianlong Fu, Hui Xiong, and Ji-Rong Wen. Meta attention-generation network for cross-granularity few- shot learning. International Journal of Computer Vision, 131(5):1211-1233, 2023.","title":"Meta attention-generation network for cross-granularity fewshot learning","authors":"Wenwen Qiang, Jiangmeng Li, Bing Su, Jianlong Fu, Hui Xiong, Ji-Rong Wen","year":"2023","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Raghu et al., 2019] Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157, 2019.","title":"Rapid learning or feature reuse? towards understanding the effectiveness of maml","authors":"Aniruddh Raghu, Maithra Raghu, Samy Bengio, Oriol Vinyals","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rajendran et al., 2020] Janarthanan Rajendran, Alexander Irpan, and Eric Jang. Meta-learning requires meta- augmentation. Advances in Neural Information Process- ing Systems, 33:5705-5715, 2020.","title":"Meta-learning requires metaaugmentation","authors":"Rajendran","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rivolli et al., 2022] Adriano Rivolli, Luís PF Garcia, Car- los Soares, Joaquin Vanschoren, and André CPLF de Car- valho. Meta-features for meta-learning. Knowledge-Based Systems, 240:108101, 2022.","title":"Meta-features for meta-learning","authors":"Adriano Rivolli, P F Luís, Carlos Garcia, Joaquin Soares, André Cplf De Vanschoren, Carvalho","year":"2022","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schrum et al., 2022] Mariah L Schrum, Erin Hedlund- Botti, Nina Moorman, and Matthew C Gombolay. Mind meld: Personalized meta-learning for robot-centric imita- tion learning. In 2022 17th ACM/IEEE International Con- ference on Human-Robot Interaction (HRI), pages 157- 165. IEEE, 2022.","title":"Mind meld: Personalized meta-learning for robot-centric imitation learning","authors":"Mariah L Schrum, Erin Hedlund-Botti, Nina Moorman, Matthew C Gombolay","year":"2022","venue":"17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Snell et al., 2017] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.","title":"Prototypical networks for few-shot learning","authors":"Jake Snell, Kevin Swersky, Richard Zemel","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sung et al., 2018] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learn- ing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1199-1208, 2018.","title":"Learning to compare: Relation network for few-shot learning","authors":"Sung","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Suter et al., 2019] Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, and Stefan Bauer. Robustly disentan- gled causal mechanisms: Validating deep representations for interventional robustness. In International Conference on Machine Learning, pages 6056-6065. PMLR, 2019.","title":"Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness","authors":"Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, Stefan Bauer","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ton et al., 2021] Jean-Franc ¸ois Ton, Dino Sejdinovic, and Kenji Fukumizu. Meta learning for causal direction. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pages 9897-9905, 2021.","title":"Meta learning for causal direction","authors":"Ton","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Timo- thy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information pro- cessing systems, 29, 2016.","title":"Matching networks for one shot learning","authors":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2021] Haoxiang Wang, Han Zhao, and Bo Li. Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation. In Interna- tional conference on machine learning, pages 10991- 11002. PMLR, 2021.","title":"Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation","authors":"Wang","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2023] Jingyao Wang, Chuyuan Zhang, Ye Ding, and Yuxuan Yang. Awesome-meta+: Meta- learning research and learning platform. arXiv preprint arXiv:2304.12921, 2023.","title":"Awesome-meta+: Metalearning research and learning platform","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xiang et al., 2014] Yu Xiang, Roozbeh Mottaghi, and Sil- vio Savarese. Beyond pascal: A benchmark for 3d object detection in the wild. In IEEE winter conference on appli- cations of computer vision, pages 75-82. IEEE, 2014.","title":"Beyond pascal: A benchmark for 3d object detection in the wild","authors":"Yu Xiang, Roozbeh Mottaghi, Silvio Savarese","year":"2014","venue":"IEEE winter conference on applications of computer vision","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2021] Xu Yang, Hanwang Zhang, and Jianfei Cai. Deconfounded image captioning: A causal retrospect. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 2021.","title":"Deconfounded image captioning: A causal retrospect","authors":"Xu Yang, Hanwang Zhang, Jianfei Cai","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yao et al., 2021] Huaxiu Yao, Long-Kai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou, Junzhou Huang, et al. Improving generalization in meta-learning via task augmentation. In International conference on machine learning, pages 11887-11897. PMLR, 2021.","title":"Improving generalization in meta-learning via task augmentation","authors":"Yao","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yin et al., 2019] Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn. Meta-learning without memorization. arXiv preprint arXiv:1912.03820, 2019.","title":"Meta-learning without memorization","authors":"Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, Chelsea Finn","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yue et al., 2020] Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua. Interventional few-shot learn- ing. Advances in neural information processing systems, 33:2734-2746, 2020.","title":"Interventional few-shot learning","authors":"Yue","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2020] Dong Zhang, Hanwang Zhang, Jin- hui Tang, Xian-Sheng Hua, and Qianru Sun. Causal intervention for weakly-supervised semantic segmenta- tion. Advances in Neural Information Processing Systems, 33:655-666, 2020.","title":"Causal intervention for weakly-supervised semantic segmentation","authors":"Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, Qianru Sun","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zimmermann et al., 2021] Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In International Conference on Machine Learn- ing, pages 12979-12990. PMLR, 2021.","title":"Contrastive learning inverts the data generating process","authors":"Yash Roland S Zimmermann, Steffen Sharma, Matthias Schneider, Wieland Bethge, Brendel","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}