---
title: "Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing"
authors:
  - "Xinyu Wang"
  - "Sicheng Lyu"
  - "Yu Gu"
  - "Jerry Huang"
  - "Peng Lu"
  - "Yufei Cui"
  - "Xiao-Wen Chang"
doi: "10.24963/ijcai.2026/265"
url: "https://ijcai.org/proceedings/2026/265"
pdf: "https://ijcai.org/proceedings/2026/0265.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "2382-2390"
track: "Main Track"
keywords:
  - "Knowledge Representation and Reasoning: Applications"
  - "Machine Learning: Generative models"
  - "Constraint Satisfaction and Optimization: Constraint optimization problems"
  - "Natural Language Processing: Language models"
---

# Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing

Xinyu Wang, Sicheng Lyu, Yu Gu, Jerry Huang, Peng Lu, Yufei Cui, Xiao-Wen Chang

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 2382-2390. DOI: [10.24963/ijcai.2026/265](https://doi.org/10.24963/ijcai.2026/265)

## Abstract

Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a plasticity–stability dilemma: repeated locate-then-edit “hard writes” can accumulate interference over time, while rigid preservation constraints may protect only explicitly constrained directions, allowing past edits or unconstrained behaviors to deviate.
We propose RLSEdit, a recursive least-squares editor for long sequential editing. RLSEdit formulates editing as an online quadratic optimization with soft constraints, minimizing a cumulative key-value fitting objective together with two regularizers that control deviation from the pre-trained weights and from a designated anchor mapping. This objective admits an efficient Woodbury-based online recursion, with per-edit cost independent of history length and scaling only with the current edit size. We further provide deviation bounds and an asymptotic characterization of the adherence–preservation trade-off in the many-edits regime.
Experiments on CounterFact and ZsRE across multiple model families show stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability, while retaining early edits and preserving general capabilities on GLUE and held-out reasoning/code benchmarks.
Code will be at here.