{"type":"ScholarlyArticle","title":"Rethinking Thinking Steps in Overthinking: Towards More Effective Reasoning","authors":[{"name":"Dezhi Zhao","slug":"dezhi-zhao","given":"Dezhi","family":"Zhao","affiliation":""},{"name":"Xin Liu","slug":"xin-liu","given":"Xin","family":"Liu","affiliation":""},{"name":"Xiaocheng Feng","slug":"xiaocheng-feng","given":"Xiaocheng","family":"Feng","affiliation":""},{"name":"Hui Wang","slug":"hui-wang","given":"Hui","family":"Wang","affiliation":""},{"name":"Bing Qin","slug":"bing-qin","given":"Bing","family":"Qin","affiliation":""}],"abstract":"Understanding overthinking in large reasoning models (LRMs) is crucial for interpretability as well as reasoning efficiency and effectiveness. However, existing approaches primarily adopt coarse-grained reasoning strategies, such as truncating Chains-of-Thought or switching reasoning modes, which reduce verbosity but are insufficient to actively guide reasoning toward more effective trajectories.\r\nTo address these issues, we propose a training-free, interpretable framework that selects thinking words via attention heads to guide LRMs toward more effective reasoning. Specifically, we categorize thinking steps into effective and redundant states, identify the attention head that best discriminates between them as the Thinking Partition Head to construct an Effective Thinking Representation Space, and compute the Information Gain Ratio (IGR) between candidate thinking words and this space to select the word that steers reasoning toward a more effective direction.\r\nExtensive experiments on mathematical and scientific reasoning benchmarks, including AIME24, AMC23, MATH-500, GSM8K, and GPQA-D, show that our method consistently outperforms the strong baseline DEER, achieving average improvements of 1.2–1.3% in accuracy and 2.5–4.5% in compression rate. Compared to vanilla baselines, our approach yields larger gains of 2.6–6.3% in accuracy while reducing token usage by 22–43%.","doi":"10.24963/ijcai.2026/678","url":"https://ijcai.org/proceedings/2026/678","pdf":"https://ijcai.org/proceedings/2026/0678.pdf","markdown":"https://ijcai.org/proceedings/2026/678.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/678","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Natural Language Processing","track":"Main Track","number":678,"pages":{"first":6091,"last":6100},"keywords":["Natural Language Processing: Interpretability and analysis of models for NLP"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}