{"type":"ScholarlyArticle","title":"Streamlining Long-Chain Reasoning via Differentiable Hierarchical Fusion","authors":[{"name":"Chuangen Gao","slug":"chuangen-gao","given":"Chuangen","family":"Gao","affiliation":""},{"name":"Wenlun Zhang","slug":"wenlun-zhang","given":"Wenlun","family":"Zhang","affiliation":""},{"name":"Shang Wang","slug":"shang-wang","given":"Shang","family":"Wang","affiliation":""},{"name":"Shuyang Gu","slug":"shuyang-gu","given":"Shuyang","family":"Gu","affiliation":""}],"abstract":"While large-scale reasoning models have achieved remarkable performance gains by scaling test-time computation through extended Chain-of-Thought (CoT) sequences, their practical utility is severely constrained by protracted inference lengths and high computational latency. In this paper, we present Differentiable Hierarchical Fusion (DHF), a novel framework that merges reasoning models with efficient base models via differentiable optimization to produce concise, accurate outputs.  We introduce a dual-factor adaptive weighting mechanism to capture intra-block (Attention vs. MLP) variance and inter-block (shallow vs. deep layers) importance hierarchies, thereby addressing key limitations of static merging heuristics. Specifically, DHF optimizes fusion coefficients using gradient descent on a loss function that jointly minimizes cross-entropy (for accuracy) and response length (for conciseness). Furthermore, we enhance an outlier-aware initialization strategy to seed coefficients based on activation density and construct multi-calibration datasets to improve the model’s generalization ability. Comprehensive evaluations on Qwen and LLaMA models across six reasoning benchmarks show that DHF reduces the average response length by 55% while boosting accuracy by 2.8%–6% compared to existing state-of-the-art merging methods.","doi":"10.24963/ijcai.2026/635","url":"https://ijcai.org/proceedings/2026/635","pdf":"https://ijcai.org/proceedings/2026/0635.pdf","markdown":"https://ijcai.org/proceedings/2026/635.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/635","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Natural Language Processing","track":"Main Track","number":635,"pages":{"first":5702,"last":5710},"keywords":["Natural Language Processing: Resources and evaluation","Natural Language Processing: Language models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}