{"type":"ScholarlyArticle","title":"Continual Learning with Pre-Trained Models: A Survey","authors":[{"name":"Da-Wei Zhou","slug":"da-wei-zhou","given":"Da-Wei","family":"Zhou","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University; School of Artificial Intelligence , Nanjing University"},{"name":"Hai-Long Sun","slug":"hai-long-sun","given":"Hai-Long","family":"Sun","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University; School of Artificial Intelligence , Nanjing University"},{"name":"Jingyi Ning","slug":"jingyi-ning","given":"Jingyi","family":"Ning","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University"},{"name":"Han-Jia Ye","slug":"han-jia-ye","given":"Han-Jia","family":"Ye","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University; School of Artificial Intelligence , Nanjing University"},{"name":"De-Chuan Zhan","slug":"de-chuan-zhan","given":"De-Chuan","family":"Zhan","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University; School of Artificial Intelligence , Nanjing University"}],"abstract":"Nowadays, real-world applications often face streaming data, which requires the learning system to absorb new knowledge as data evolves. Continual Learning (CL) aims to achieve this goal and meanwhile overcome the catastrophic forgetting of former knowledge when learning new ones. Typical CL methods build the model from scratch to grow with incoming data. However, the advent of the pre-trained model (PTM) era has sparked immense research interest, particularly in leveraging PTMs' robust representational capabilities. This paper presents a comprehensive survey of the latest advancements in PTM-based CL. We categorize existing methodologies into three distinct groups, providing a comparative analysis of their similarities, differences, and respective advantages and disadvantages. Additionally, we offer an empirical study contrasting various state-of-the-art methods to highlight concerns regarding fairness in comparisons. The source code to reproduce these evaluations is available at: https://github.com/sun-hailong/LAMDA-PILOT","doi":"10.24963/ijcai.2024/924","url":"https://ijcai.org/proceedings/2024/924","pdf":"https://ijcai.org/proceedings/2024/0924.pdf","markdown":"https://ijcai.org/proceedings/2024/924.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/924","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":924,"pages":{"first":8363,"last":8371},"keywords":["Machine Learning: ML: Classification","Machine Learning: ML: Multi-task and transfer learning","Machine Learning: ML: Incremental learning"],"videos":[],"references":[{"position":1,"raw":"[Ahrens et al., 2023] K. Ahrens, H. H. Lehmann, J. H. Lee, and S. Wermter. Read between the layers: Leveraging intra-layer rep- resentations for rehearsal-free continual learning with pre-trained models. arXiv preprint arXiv:2312.08888, 2023.","title":"Read between the layers: Leveraging intra-layer representations for rehearsal-free continual learning with pre-trained models","authors":"K Ahrens, H H Lehmann, J H Lee, S Wermter","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barbu et al., 2019] A. Barbu, D. Mayo, J. Alverio, W. Luo, C. Wang, D. Gutfreund, J. Tenenbaum, and B. Katz. Object- net: A large-scale bias-controlled dataset for pushing the limits of object recognition models. NeurIPS, 2019.","title":"Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models","authors":"A Barbu, D Mayo, J Alverio, W Luo, C Wang, D Gutfreund, J Tenenbaum, B Katz","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2023] B. Cao, Q. Tang, H. Lin, X. Han, J. Chen, T. Wang, and L. Sun. Retentive or forgetful? diving into the knowledge memorizing mechanism of language models. arXiv preprint arXiv:2305.09144, 2023.","title":"Retentive or forgetful? diving into the knowledge memorizing mechanism of language models","authors":"B Cao, Q Tang, H Lin, X Han, J Chen, T Wang, L Sun","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022] S. Chen, C. Ge, Z. Tong, J. Wang, Y. Song, J. Wang, and P. Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. NeurIPS, 35:16664-16678, 2022.","title":"Adaptformer: Adapting vision transformers for scalable visual recognition","authors":"S Chen, C Ge, Z Tong, J Wang, Y Song, J Wang, P Luo","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023] H. Chen, Z. Wu, X. Han, M. Jia, and Y.-G. Jiang. Promptfusion: Decoupling stability and plasticity for continual learning. arXiv preprint arXiv:2303.07223, 2023.","title":"Promptfusion: Decoupling stability and plasticity for continual learning","authors":"H Chen, Z Wu, X Han, M Jia, Y.-G Jiang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[De Lange et al., 2021] M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. TPAMI, 44(7):3366-3385, 2021.","title":"A continual learning survey: Defying forgetting in classification tasks","authors":"M De Lange, R Aljundi, M Masana, S Parisot, X Jia, A Leonardis, G Slabaugh, T Tuytelaars","year":"2021","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deng et al., 2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"J Deng, W Dong, R Socher, L.-J Li, K Li, L Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2020] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"A Dosovitskiy, L Beyer, A Kolesnikov, D Weissenborn, X Zhai, T Unterthiner, M Dehghani, M Minderer, G Heigold, S Gelly","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Floridi and Chiriatti, 2020] L. Floridi and M. Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4):681-694, 2020.","title":"Gpt-3: Its nature, scope, limits, and consequences","authors":"L Floridi, M Chiriatti","year":"2020","venue":"Minds and Machines","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fortin and Chaib-draa, 2022] M. P. Fortin and B. Chaib-draa. Con- tinual semantic segmentation leveraging image-level labels and rehearsal. In IJCAI, pages 1268-1275, 2022.","title":"Continual semantic segmentation leveraging image-level labels and rehearsal","authors":"M P Fortin, B Chaib-Draa","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/177"},{"position":11,"raw":"[Gan et al., 2023] Y. Gan, Y. Bai, Y. Lou, X. Ma, R. Zhang, N. Shi, and L. Luo. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In AAAI, volume 37, pages 7595-7603, 2023.","title":"Decorate the newcomers: Visual domain prompt for continual test time adaptation","authors":"Y Gan, Y Bai, Y Lou, X Ma, R Zhang, N Shi, L Luo","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2023] Q. Gao, C. Zhao, Y. Sun, T. Xi, G. Zhang, B. Ghanem, and J. Zhang. A unified continual learning frame- work with general parameter-efficient tuning. In ICCV, pages 11483-11493, October 2023.","title":"A unified continual learning framework with general parameter-efficient tuning","authors":"Q Gao, C Zhao, Y Sun, T Xi, G Zhang, B Ghanem, J Zhang","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gunasekara et al., 2023] N. Gunasekara, B. Pfahringer, H. M. Gomes, and A. Bifet. Survey on online streaming continual learn- ing. In IJCAI, pages 6628-6637, 2023.","title":"Survey on online streaming continual learning","authors":"N Gunasekara, B Pfahringer, H M Gomes, A Bifet","year":"2023","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2023/743"},{"position":14,"raw":"[Guo et al., 2024] H. Guo, F. Zhu, W. Liu, X.-Y. Zhang, and C.-L. Liu. Federated class-incremental learning with prototype guided transformer. arXiv preprint arXiv:2401.02094, 2024.","title":"Federated class-incremental learning with prototype guided transformer","authors":"H Guo, F Zhu, W Liu, X.-Y Zhang, C.-L Liu","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[He et al., 2016] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"K He, X Zhang, S Ren, J Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hendrycks et al., 2021a] D. Hendrycks, S. Basart, N. Mu, S. Kada- vath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al. The many faces of robustness: A critical analysis of out-of- distribution generalization. In ICCV, pages 8340-8349, 2021.","title":"The many faces of robustness: A critical analysis of out-ofdistribution generalization","authors":"Hendrycks","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hendrycks et al., 2021b] D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. In CVPR, pages 15262-15271, 2021.","title":"Natural adversarial examples","authors":"Hendrycks","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Janson et al., 2022] P. Janson, W. Zhang, R. Aljundi, and M. El- hoseiny. A simple baseline that questions the use of pretrained- models in continual learning. arXiv preprint arXiv:2210.04428, 2022.","title":"A simple baseline that questions the use of pretrainedmodels in continual learning","authors":"P Janson, W Zhang, R Aljundi, M Elhoseiny","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jia et al., 2022] M. Jia, L. Tang, B. Chen, C. Cardie, S. J. Belongie, B. Hariharan, and S. Lim. Visual prompt tuning. In ECCV, pages 709-727. Springer, 2022.","title":"Visual prompt tuning","authors":"M Jia, L Tang, B Chen, C Cardie, S J Belongie, B Hariharan, S Lim","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jung et al., 2023] D. Jung, D. Han, J. Bang, and H. Song. Generat- ing instance-level prompts for rehearsal-free continual learning. In ICCV, pages 11847-11857, 2023.","title":"Generating instance-level prompts for rehearsal-free continual learning","authors":"D Jung, D Han, J Bang, H Song","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Khan et al., 2023] M. G. Z. A. Khan, M. F. Naeem, L. Van Gool, D. Stricker, F. Tombari, and M. Z. Afzal. Introducing language guidance in prompt-based continual learning. In ICCV, pages 11463-11473, 2023.","title":"Introducing language guidance in prompt-based continual learning","authors":"M G Z A Khan, M F Naeem, L Van Gool, D Stricker, F Tombari, M Z","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Khattak et al., 2023] M. U. Khattak, S. T. Wasim, M. Naseer, S. Khan, M.-H. Yang, and F. S. Khan. Self-regulating prompts: Foundational model adaptation without forgetting. In ICCV, pages 15190-15200, 2023.","title":"Self-regulating prompts: Foundational model adaptation without forgetting","authors":"M U Khattak, S T Wasim, M Naseer, S Khan, M.-H Yang, F S Khan","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Krizhevsky et al., 2009] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. Technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"A Krizhevsky, G Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2022] J. Li, Z. Ji, G. Wang, Q. Wang, and F. Gao. Learning from students: Online contrastive distillation network for general continual learning. In IJCAI, pages 3215-3221, 2022.","title":"Learning from students: Online contrastive distillation network for general continual learning","authors":"J Li, Z Ji, G Wang, Q Wang, F Gao","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/446"},{"position":25,"raw":"[Liu et al., 2022] M. Liu, S. Chang, and L. Huang. Incremental prompting: Episodic memory prompt for lifelong event detection. In COLING, pages 2157-2165, 2022.","title":"Incremental prompting: Episodic memory prompt for lifelong event detection","authors":"M Liu, S Chang, L Huang","year":"2022","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liu et al., 2023] M. Liu, W. Yang, Y. Hu, and J. Liu. Dual prompt learning for continual rain removal from single images. In IJCAI, pages 7215-7223, 2023.","title":"Dual prompt learning for continual rain removal from single images","authors":"M Liu, W Yang, Y Hu, J Liu","year":"2023","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2023/849"},{"position":27,"raw":"[Marouf et al., 2023] I. E. Marouf, S. Roy, E. Tartaglione, and S. Lathuilière. Weighted ensemble models are strong continual learners. arXiv preprint arXiv:2312.08977, 2023.","title":"Weighted ensemble models are strong continual learners","authors":"Marouf","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Masana et al., 2023] M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer. Class-incremental learn- ing: Survey and performance evaluation on image classification. TPAMI, 45(05):5513-5533, 2023.","title":"Class-incremental learning: Survey and performance evaluation on image classification","authors":"M Masana, X Liu, B Twardowski, M Menta, A D Bagdanov, J Van De Weijer","year":"2023","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[McCloskey and Cohen, 1989] M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequen- tial learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.","title":"Catastrophic interference in connectionist networks: The sequential learning problem","authors":"M Mccloskey, N J Cohen","year":"1989","venue":"Psychology of learning and motivation","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[McDonnell et al., 2023] M. D. McDonnell, D. Gong, A. Parveneh, E. Abbasnejad, and A. v. d. Hengel. Ranpac: Random projections and pre-trained models for continual learning. arXiv preprint arXiv:2307.02251, 2023.","title":"Ranpac: Random projections and pre-trained models for continual learning","authors":"M D Mcdonnell, D Gong, A Parveneh, E Abbasnejad, A V D Hengel","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Moon et al., 2023] J.-Y. Moon, K.-H. Park, J. U. Kim, and G.-M. Park. Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning. In ICCV, pages 11731-11741, 2023.","title":"Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning","authors":"J.-Y Moon, K.-H Park, J U Kim, G.-M Park","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Moon, 1996] T. K. Moon. The expectation-maximization algorithm. IEEE Signal processing magazine, 13(6):47-60, 1996.","title":"The expectation-maximization algorithm","authors":"T K Moon","year":"1996","venue":"IEEE Signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Nicolas et al., 2024] J. Nicolas, F. Chiaroni, I. Ziko, O. Ahmad, C. Desrosiers, and J. Dolz. Mop-clip: A mixture of prompt-tuned clip models for domain incremental learning. In WACV, pages 1762-1772, 2024.","title":"Mop-clip: A mixture of prompt-tuned clip models for domain incremental learning","authors":"J Nicolas, F Chiaroni, I Ziko, O Ahmad, C Desrosiers, J Dolz","year":"2024","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Prabhu et al., 2023] A. Prabhu, H. A. Al Kader Hammoud, P. K. Dokania, P. H. Torr, S.-N. Lim, B. Ghanem, and A. Bibi. Com- putationally budgeted continual learning: What does matter? In CVPR, pages 3698-3707, 2023.","title":"Computationally budgeted continual learning: What does matter?","authors":"A Prabhu, H A Al Kader Hammoud, P K Dokania, P H Torr, S.-N Lim, B Ghanem, A Bibi","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Radford et al., 2021] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"A Radford, J W Kim, C Hallacy, A Ramesh, G Goh, S Agarwal, G Sastry, A Askell, P Mishkin, J Clark","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Razdaibiedina et al., 2023] A. Razdaibiedina, Y. Mao, R. Hou, M. Khabsa, M. Lewis, and A. Almahairi. Progressive prompts: Continual learning for language models. In ICLR, 2023.","title":"Progressive prompts: Continual learning for language models","authors":"A Razdaibiedina, Y Mao, R Hou, M Khabsa, M Lewis, A Almahairi","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Shibata et al., 2021] T. Shibata, G. Irie, D. Ikami, and Y. Mitsuzumi. Learning with selective forgetting. In IJCAI, pages 989-996, 2021.","title":"Learning with selective forgetting","authors":"T Shibata, G Irie, D Ikami, Y Mitsuzumi","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/137"},{"position":38,"raw":"[Smith et al., 2023] J. S. Smith, L. Karlinsky, V. Gutta, P. Cascante- Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In CVPR, pages 11909- 11919, 2023.","title":"Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning","authors":"J S Smith, L Karlinsky, V Gutta, P Cascante-Bonilla, D Kim, A Arbelle, R Panda, R Feris, Z Kira","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Steiner et al., 2021] A. Steiner, A. Kolesnikov, X. Zhai, R. Wight- man, J. Uszkoreit, and L. Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2106.10270, 2021.","title":"How to train your vit? data, augmentation, and regularization in vision transformers","authors":"A Steiner, A Kolesnikov, X Zhai, R Wightman, J Uszkoreit, L Beyer","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Sun et al., 2023] L. Sun, J. Ye, H. Peng, F. Wang, and S. Y. Philip. Self-supervised continual graph learning in adaptive riemannian spaces. In AAAI, volume 37, pages 4633-4642, 2023.","title":"Self-supervised continual graph learning in adaptive riemannian spaces","authors":"L Sun, J Ye, H Peng, F Wang, S Y Philip","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Tang et al., 2023] Y.-M. Tang, Y.-X. Peng, and W.-S. Zheng. When prompt-based incremental learning does not meet strong pretrain- ing. In ICCV, pages 1706-1716, 2023.","title":"When prompt-based incremental learning does not meet strong pretraining","authors":"Y.-M Tang, Y.-X Peng, W.-S Zheng","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[van de Ven et al., 2022] G. M. van de Ven, T. Tuytelaars, and A. S. Tolias. Three types of incremental learning. Nature Machine Intelligence, pages 1-13, 2022.","title":"Three types of incremental learning","authors":"G M Van De Ven, T Tuytelaars, A S Tolias","year":"2022","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Vaswani et al., 2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkor- eit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In NIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A N Gomez, Ł Kaiser, I Polosukhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Villa et al., 2023] A. Villa, J. L. Alcázar, M. Alfarra, K. Alhamoud, J. Hurtado, F. C. Heilbron, A. Soto, and B. Ghanem. Pivot: Prompt- ing for video continual learning. In CVPR, pages 24214-24223, 2023.","title":"Pivot: Prompting for video continual learning","authors":"A Villa, J L Alcázar, M Alfarra, K Alhamoud, J Hurtado, F C Heilbron, A Soto, B Ghanem","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wah et al., 2011] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Techni- cal Report CNS-TR-2011-001, California Institute of Technology, 2011.","title":"The Caltech-UCSD Birds-200-2011 Dataset","authors":"C Wah, S Branson, P Welinder, P Perona, S Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wang et al., 2022a] Y. Wang, Z. Huang, and X. Hong. S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning. NeurIPS, 35:5682-5695, 2022.","title":"S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning","authors":"Wang","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wang et al., 2022b] Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y. Lee, X. Ren, G. Su, V. Perot, J. Dy, et al. Dual- prompt: Complementary prompting for rehearsal-free continual learning. arXiv preprint arXiv:2204.04799, 2022.","title":"Dualprompt: Complementary prompting for rehearsal-free continual learning","authors":"Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Wang et al., 2022c] Z. Wang, Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, and T. Pfister. Learn- ing to prompt for continual learning. In CVPR, pages 139-149, 2022.","title":"Learning to prompt for continual learning","authors":"Wang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Wang et al., 2023a] L. Wang, J. Xie, X. Zhang, M. Huang, H. Su, and J. Zhu. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality. arXiv preprint arXiv:2310.07234, 2023.","title":"Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Wang et al., 2023b] R. Wang, X. Duan, G. Kang, J. Liu, S. Lin, S. Xu, J. Lü, and B. Zhang. Attriclip: A non-incremental learner for incremental knowledge learning. In CVPR, pages 3654-3663, 2023.","title":"Attriclip: A non-incremental learner for incremental knowledge learning","authors":"Wang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Wang et al., 2023c] X. Wang, T. Chen, Q. Ge, H. Xia, R. Bao, R. Zheng, Q. Zhang, T. Gui, and X.-J. Huang. Orthogonal sub- space learning for language model continual learning. In Findings of EMNLP, pages 10658-10671, 2023.","title":"Orthogonal subspace learning for language model continual learning","authors":"Wang","year":"2023","venue":"Findings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Wang et al., 2023d] Y. Wang, Z. Ma, Z. Huang, Y. Wang, Z. Su, and X. Hong. Isolation and impartial aggregation: A paradigm of incremental learning without interference. In AAAI, volume 37, pages 10209-10217, 2023.","title":"Isolation and impartial aggregation: A paradigm of incremental learning without interference","authors":"Wang","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Wiwatcharakoses and Berrar, 2019] C. Wiwatcharakoses and D. Berrar. Self-organizing incremental neural networks for continual learning. In IJCAI, pages 6476-6477, 2019.","title":"Self-organizing incremental neural networks for continual learning","authors":"C Wiwatcharakoses, D Berrar","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/927"},{"position":54,"raw":"[Yadav et al., 2023] P. Yadav, Q. Sun, H. Ding, X. Li, D. Zhang, M. Tan, P. Bhatia, X. Ma, R. Nallapati, M. K. Ramanathan, et al. Exploring continual learning for code generation models. In ACL, pages 782-792, 2023.","title":"Exploring continual learning for code generation models","authors":"P Yadav, Q Sun, H Ding, X Li, D Zhang, M Tan, P Bhatia, X Ma, R Nallapati, M K Ramanathan","year":"2023","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zhai et al., 2019] X. Zhai, J. Puigcerver, A. Kolesnikov, P. Ruyssen, C. Riquelme, M. Lucic, J. Djolonga, A. S. Pinto, M. Neumann, A. Dosovitskiy, et al. A large-scale study of representation learn- ing with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867, 2019.","title":"A large-scale study of representation learning with the visual task adaptation benchmark","authors":"Zhai","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Zhang et al., 2022] Y. Zhang, Z. Yin, J. Shao, and Z. Liu. Bench- marking omni-vision representation through the lens of visual realms. In ECCV, pages 594-611. Springer, 2022.","title":"Benchmarking omni-vision representation through the lens of visual realms","authors":"Y Zhang, Z Yin, J Shao, Z Liu","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Zhang et al., 2023] G. Zhang, L. Wang, G. Kang, L. Chen, and Y. Wei. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In ICCV, pages 19148-19158, October 2023.","title":"Slca: Slow learner with classifier alignment for continual learning on a pre-trained model","authors":"G Zhang, L Wang, G Kang, L Chen, Y Wei","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Zhang et al., 2024] N. Zhang, Y. Yao, B. Tian, P. Wang, S. Deng, M. Wang, Z. Xi, S. Mao, J. Zhang, Y. Ni, et al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286, 2024.","title":"A comprehensive study of knowledge editing for large language models","authors":"N Zhang, Y Yao, B Tian, P Wang, S Deng, M Wang, Z Xi, S Mao, J Zhang, Y Ni","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Zheng et al., 2023a] J. Zheng, S. Qiu, and Q. Ma. Learn or recall? revisiting incremental learning with pre-trained language models. arXiv preprint arXiv:2312.07887, 2023.","title":"Learn or recall? revisiting incremental learning with pre-trained language models","authors":"Zheng","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Zheng et al., 2023b] Z. Zheng, M. Ma, K. Wang, Z. Qin, X. Yue, and Y. You. Preventing zero-shot transfer degradation in continual learning of vision-language models. In ICCV, pages 19125-19136, October 2023.","title":"Preventing zero-shot transfer degradation in continual learning of vision-language models","authors":"Zheng","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Zhou et al., 2023a] D.-W. Zhou, Q.-W. Wang, Z.-H. Qi, H.-J. Ye, D.-C. Zhan, and Z. Liu. Deep class-incremental learning: A survey. arXiv preprint arXiv:2302.03648, 2023.","title":"Deep class-incremental learning: A survey","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Zhou et al., 2023b] D.-W. Zhou, H.-J. Ye, D.-C. Zhan, and Z. Liu. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. arXiv preprint arXiv:2303.07338, 2023.","title":"Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Zhou et al., 2023c] D.-W. Zhou, Y. Zhang, J. Ning, H.-J. Ye, D.-C. Zhan, and Z. Liu. Learning without forgetting for vision-language models. arXiv preprint arXiv:2305.19270, 2023.","title":"Learning without forgetting for vision-language models","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Zhou et al., 2024] D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D.-C. Zhan. Expandable subspace ensemble for pre-trained model-based class- incremental learning. In CVPR, 2024.","title":"Expandable subspace ensemble for pre-trained model-based classincremental learning","authors":"D.-W Zhou, H.-L Sun, H.-J Ye, D.-C Zhan","year":"2024","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Zhu et al., 2021] F. Zhu, X.-Y. Zhang, C. Wang, F. Yin, and C.-L. Liu. Prototype augmentation and self-supervision for incremental learning. In CVPR, pages 5871-5880, 2021.","title":"Prototype augmentation and self-supervision for incremental learning","authors":"F Zhu, X.-Y Zhang, C Wang, F Yin, C.-L Liu","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}