Open-Vocabulary Object 6D Pose Estimation via Modulated Textual Semantics

Abstract

Estimating the 6D pose of novel objects without CAD models or video sequences remains a challenging problem. Recent works explore text-driven approaches to address this challenge in an open-vocabulary manner. However, these methods typically treat text embeddings as static priors, which lack the flexibility to adapt to specific visual scenes, thereby limiting reliable cross-view correspondences. In this paper, we propose a framework that modulates static textual embeddings into adaptable semantic guidance for RGB-D open-vocabulary 6D pose estimation. Specifically, we introduce a Text Feature Modulation (TFM) module that uses learnable queries to transform fixed text embeddings into adaptive semantic representations. These representations are explicitly calibrated by a Foreground-Grounded Semantic Alignment (FGSA) strategy, which suppresses background distractions via contrastive learning. Furthermore, we design a Semantic-Anchored Cross-view Fusion (SACF) module that aggregates image features from the reference and query views using modulated semantics as an invariant anchor. Finally, we predict object segmentation and establish semantic-aware correspondences, leveraging modulated textual semantics to filter mismatches for robust 6D pose estimation. Extensive experiments on standard benchmarks demonstrate that our method achieves superior performance.