{"type":"ScholarlyArticle","title":"RIVS: Mitigating Hallucination in Large Vision-Language Models via Representation Intervention on Visual Grounding Shift","authors":[{"name":"Xuanyu Yin","slug":"xuanyu-yin","given":"Xuanyu","family":"Yin","affiliation":""},{"name":"Xiaoye Qu","slug":"xiaoye-qu","given":"Xiaoye","family":"Qu","affiliation":""},{"name":"Wei Wei","slug":"wei-wei","given":"Wei","family":"Wei","affiliation":""}],"abstract":"Large Vision-Language Models (LVLMs) demonstrate powerful generative capabilities yet remain prone to object hallucinations. Most existing methods mitigate this issue through training or decoding strategies, but provide limited exploration of how hallucinations arise from internal representations during generation. \r\nIn this work, we study hallucination from the perspective of dynamic representation shift during generation and propose Representation Intervention based on Visual Grounding Shift (RIVS). Specifically, we first design an adaptive threshold-based method to identify visual attention drop points within the generation, finding that hallucinated tokens usually occur with decay of visual attention. Based on this observation, we construct a hallucination-related subspace from representation differences around these points on a small calibration set, without constructing contrast samples or extra supervision. \r\nDuring inference, we leverage the resulting hallucination-related subspace to perform an online projection-based intervention on intermediate hidden states to suppress the hallucination-related directions, mitigating hallucinations while preserving language quality.\r\nOur RIVS is training-free and computationally efficient. Experiments on four hallucination and two reasoning datasets demonstrate that RIVS consistently reduces hallucinations in both long and short sequence generation tasks.","doi":"10.24963/ijcai.2026/94","url":"https://ijcai.org/proceedings/2026/94","pdf":"https://ijcai.org/proceedings/2026/0094.pdf","markdown":"https://ijcai.org/proceedings/2026/94.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/94","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI Ethics, Trust, Fairnes","track":"Main Track","number":94,"pages":{"first":835,"last":843},"keywords":["AI Ethics, Trust, Fairnes: Trustworthy AI","AI Ethics, Trust, Fairnes: Other","AI Ethics, Trust, Fairnes: Bias"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}