---
title: "TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues"
authors:
  - "Junhui She"
  - "Fei Wang"
  - "Kun Li"
  - "Yiqi Nie"
  - "Yuxin Liu"
  - "Zhangling Duan"
  - "Xun Yang"
doi: "10.24963/ijcai.2026/177"
url: "https://ijcai.org/proceedings/2026/177"
pdf: "https://ijcai.org/proceedings/2026/0177.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "1586-1594"
track: "Main Track"
keywords:
  - "Humans and AI: Human-computer interaction"
  - "Computer Vision: Scene analysis and understanding"
  - "Machine Learning: Multi-modal learning"
---

# TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues

Junhui She, Fei Wang, Kun Li, Yiqi Nie, Yuxin Liu, Zhangling Duan, Xun Yang

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 1586-1594. DOI: [10.24963/ijcai.2026/177](https://doi.org/10.24963/ijcai.2026/177)

## Abstract

Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation. All contents are available at: https://github.com/idremo/TextGaze-IJCAI2026.