A Survey of Joint Online-Offline Fine-tuning for Large Language Models

Zhen, Taihang; Yang, Guang; Li, Chenzhang; Yan, Nuo; Zhou, Shilong; Liu, Guangyu; Tang, Xiaotong; Huo, Jing; Wang, Boyan; Feng, Junlan; Zhang, Yuyao

doi:10.24963/ijcai.2026/899

A Survey of Joint Online-Offline Fine-tuning for Large Language Models

Taihang Zhen, Guang Yang, Chenzhang Li, Nuo Yan, Shilong Zhou, Guangyu Liu, Xiaotong Tang, Jing Huo, Boyan Wang, Junlan Feng, Yuyao Zhang

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Survey Track. Pages 8106-8115.

https://doi.org/10.24963/ijcai.2026/899

Keywords

Machine Learning: Supervised Learning
Natural Language Processing: Language models
Machine Learning: Reinforcement learning

Abstract

Post-training for Large Language Models (LLMs) can be mainly categorized into offline Supervised Fine-Tuning (SFT) for knowledge acquisition and online Reinforcement Fine-Tuning (RFT) for adaptive refinement. Current state-of-the-art approaches typically employ a sequential cold-start pipeline (SFT-Then-RFT). However, we argue that this disjoint transition imposes an “alignment tax", leading to catastrophic forgetting and reward hacking during the unregularized exploration phase. In this work, we advocate for Joint Online-Offline Fine-Tuning as a superior paradigm that breaks the convention of restricting offline data to SFT and online data to RFT. By integrating full offline response generation with online rollouts—particularly within the realm of Reinforcement Learning with Verifiable Rewards (RLVR)—this approach mitigates the limitations of isolated training phases. We provide the first comprehensive survey focusing specifically on the synchronization of data provenance. We introduce a novel taxonomy for related works, analyze their theoretical advantages in balancing stability with plasticity, and outline a roadmap for next-generation post-training frameworks.

Taihang Zhen, Guang Yang, Chenzhang Li, Nuo Yan, Shilong Zhou, Guangyu Liu, Xiaotong Tang, Jing Huo, Boyan Wang, Junlan Feng, and Yuyao Zhang. 2026. A Survey of Joint Online-Offline Fine-tuning for Large Language Models. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 8106–8115. https://doi.org/10.24963/ijcai.2026/899

@inproceedings{ijcai2026p899,
  title     = {A Survey of Joint Online-Offline Fine-tuning for Large Language Models},
  author    = {Zhen, Taihang and Yang, Guang and Li, Chenzhang and Yan, Nuo and Zhou, Shilong and Liu, Guangyu and Tang, Xiaotong and Huo, Jing and Wang, Boyan and Feng, Junlan and Zhang, Yuyao},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {8106--8115},
  year      = {2026},
  month     = {8},
  note      = {Survey Track},
  doi       = {10.24963/ijcai.2026/899},
  url       = {https://doi.org/10.24963/ijcai.2026/899},
}

TY  - CPAPER
TI  - A Survey of Joint Online-Offline Fine-tuning for Large Language Models
AU  - Zhen, Taihang
AU  - Yang, Guang
AU  - Li, Chenzhang
AU  - Yan, Nuo
AU  - Zhou, Shilong
AU  - Liu, Guangyu
AU  - Tang, Xiaotong
AU  - Huo, Jing
AU  - Wang, Boyan
AU  - Feng, Junlan
AU  - Zhang, Yuyao
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 8106
EP  - 8115
DO  - 10.24963/ijcai.2026/899
UR  - https://doi.org/10.24963/ijcai.2026/899
ER  -