{"type":"ScholarlyArticle","title":"Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing","authors":[{"name":"Xinyu Wang","slug":"xinyu-wang","given":"Xinyu","family":"Wang","affiliation":""},{"name":"Sicheng Lyu","slug":"sicheng-lyu","given":"Sicheng","family":"Lyu","affiliation":""},{"name":"Yu Gu","slug":"yu-gu","given":"Yu","family":"Gu","affiliation":""},{"name":"Jerry Huang","slug":"jerry-huang","given":"Jerry","family":"Huang","affiliation":""},{"name":"Peng Lu","slug":"peng-lu","given":"Peng","family":"Lu","affiliation":""},{"name":"Yufei Cui","slug":"yufei-cui","given":"Yufei","family":"Cui","affiliation":""},{"name":"Xiao-Wen Chang","slug":"xiao-wen-chang","given":"Xiao-Wen","family":"Chang","affiliation":""}],"abstract":"Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a plasticity–stability dilemma: repeated locate-then-edit “hard writes” can accumulate interference over time, while rigid preservation constraints may protect only explicitly constrained directions, allowing past edits or unconstrained behaviors to deviate.\r\nWe propose RLSEdit, a recursive least-squares editor for long sequential editing. RLSEdit formulates editing as an online quadratic optimization with soft constraints, minimizing a cumulative key-value fitting objective together with two regularizers that control deviation from the pre-trained weights and from a designated anchor mapping. This objective admits an efficient Woodbury-based online recursion, with per-edit cost independent of history length and scaling only with the current edit size. We further provide deviation bounds and an asymptotic characterization of the adherence–preservation trade-off in the many-edits regime.\r\nExperiments on CounterFact and ZsRE across multiple model families show stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability, while retaining early edits and preserving general capabilities on GLUE and held-out reasoning/code benchmarks.\r\nCode will be at here.","doi":"10.24963/ijcai.2026/265","url":"https://ijcai.org/proceedings/2026/265","pdf":"https://ijcai.org/proceedings/2026/0265.pdf","markdown":"https://ijcai.org/proceedings/2026/265.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/265","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Constraint Satisfaction and Optimization","track":"Main Track","number":265,"pages":{"first":2382,"last":2390},"keywords":["Knowledge Representation and Reasoning: Applications","Machine Learning: Generative models","Constraint Satisfaction and Optimization: Constraint optimization problems","Natural Language Processing: Language models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}