S³-TIR: Coupled Spatial-Spectral Splatting for Thermal Infrared Novel View Synthesis

1 Introduction

High-fidelity Novel View Synthesis is a fundamental problem in computer vision and graphics that enables applications ranging from digital twins to autonomous navigation. Given the recent progress of explicit neural representations, 3D Gaussian Splatting (3DGS) [Kerbl et al., 2023] has become a dominant standard for synthesizing photorealistic RGB scenes in real-time. However, real-world sensing goes beyond visible light. Thermal Infrared (TIR) imaging offers unique capabilities for safety-critical scenarios[Shi et al., 2023; Shi et al., 2024; Hu , 2025; Bao et al., 2025], ranging from drone-based fire detection [Wang et al., 2025b; Chen et al., 2004; Lin et al., 2all; Lin et al., 2026] to medical diagnostics [Ring and Ammer, 2012; Ma et al., 2023], by characterizing thermodynamic states in total darkness and penetrating atmospheric obscurant [Wang et al., 2023].

∗Corresponding author.

A straightforward idea is to directly extend standard 3DGS to thermal data. However, such methods often fail to produce structurally stable results [Song et al., 2024; Kheradmand et al., 2024; Liu et al., 2025a]. Smooth thermal fields may appear noisy, sharp material edges may blur, and highfrequency details often exhibit severe jitter artifacts in novel views. The key reason behind this limitation lies in the intricate nature of thermal radiation. Treating thermal images as simple color textures without capturing the underlying physics leads to a fundamental representation mismatch.

This paper aims to explicitly model the 3D distribution of thermal signals by analyzing the physical mechanisms underlying their appearance. Our approach is premised on the fundamental observation of the structural duality of thermal radiance (see Fig. 1(a)), wherein the frequency characteristics of a 3D point constitute an intrinsic, view-independent physical property governed by two opposing processes. First, heat conduction [Hahn and ¨Ozisik, 2012; Widder, 1976; Bao et al., 2023], governed by Fourier's law [Narasimhan, 1999; Hahn and ¨Ozisik, 2012], drives thermal energy to diffuse continuously across object surfaces. This process mathematically approximates a low-pass filter, yielding a dominant diffusive smoothness. Second, Material Discontinuities, such as abrupt variations in emissivity or geometric occlusions, manifest as extremely sharp, step-like signal jumps [Widder, 1976]. These jumps introduce structural sharpness and sparse high-frequency details. However, reconciling these conflicting attributes poses a significant challenge due to decomposition ambiguity, where a sharp thermal edge can be mathematically interpreted as either a spatially sharp envelope or a high-frequency internal texture. Standard direct rendering masks this distinction, leading the network to overfit view-dependent artifacts rather than inferring the true physical structure. By addressing these factors, we bridge the gap between thermal physics and 3D representation within a unified framework.

Building upon this insight, we introduce S3 -TIR, a physics-inspired framework designed to reconcile the coexistence of diffusive smoothness and structural sharpness. To address this, we first propose a unified generalized-exponential gabor primitive. By explicitly coupling a generalizedexponential spatial support with an internal spectral modulation field, we introduce a learnable shape parameter β acting as a physical truncation controller. This mechanism enables the primitive to adaptively transition from diffusive gaussian falloffs to sharp box-like boundaries, strictly confining high-frequency textures within the geometric envelope to prevent signal leakage. Crucially, to ensure a physically plausible decomposition, we implement a frequencyaware deferred splatting pipeline. Instead of directly rendering the final color, we independently rasterize intrinsic physical attributes into observable frequency g-buffers, including the low-frequency energy, high-frequency energy, and spatial sharpness maps. Leveraging these intermediate representations, we enforce a multi-view frequency consistency loss that anchors the frequency distribution to the 3d geometry across perspectives. This effectively resolves the decomposition ambiguity and prevents high-frequency jittering. Quantitative and qualitative results on benchmarks like rgbt-scenes and tinsd demonstrate that S3 -TIR achieves state-of-the-art performance, notably reducing lpips by 42.5% and improving psnr by 3.2% on the challenging rgbt-scenes dataset, while yielding structurally stable thermal sequences free from physically implausible artifacts.

In summary, our main contributions are as follows:

1. Coupled Spatial-Spectral Representation: We propose a unified generalized-exponential Gabor primitive that physically truncates internal spectral modulation at geometric boundaries, reconciling diffusive smoothness with structural sharpness.
2. Frequency-Aware Ambiguity Resolution: We introduce a deferred splatting pipeline with a multi-view frequency consistency loss, which anchors rasterized physical attributes to the 3D structure and resolves decomposition ambiguity.
3. Superior Performance: S3 -TIR achieves state-of-theart performance on multiple benchmarks, setting a new standard for structurally stable thermal synthesis free from jittering artifacts.

2 RelatedWork

2.1 Physics-Informed Thermal Rendering

Early thermal 3D reconstruction heavily relied on fusing thermal imagery with auxiliary depth sensors or RGB-D systems to retrieve geometric clues from textureless regions [Rangel et al., 2014; Vidas and Moghadam, 2013; Zhao et al., 2017]. With the paradigm shift to neural rendering, implicit representations first adapted the NeRF framework to the thermal domain to enable continuous scene representation [Hassan et al., 2024; Ye et al., 2024], followed by recent explicit 3D Gaussian Splatting (3DGS) approaches that incorporate constraints for atmospheric transmission [Chen et al., 2024; Lu et al., ] or dynamic temporal evolution [Yang et al., 2025; Liu et al., 2025b; Liu et al., 2024]. Beyond synthesis, emerging research aims to ground vision models in physical laws, utilizing heat conduction [Wang et al., 2025c] or thermodynamic modeling [Chen et al., 2025; Wang et al., 2026] to enhance consistency. However, unlike visible spectrum methods that leverage physics for inverse rendering [Liang et al., 2024; Gao et al., 2024; Jiang et al., 2024], explicitly inverting intrinsic thermal parameters remains a highly ill-posed problem, especially when accurate ground-truth supervision is unavailable. This limitation motivates our approach to focus on developing a more expressive representation capable of capturing the unique duality of thermal radiation, which is defined by both diffusive smoothness and structural sharpness, without relying on intractable inverse physics solving.

2.2 Frequency-Aware and Shape-Adaptive 3DGS

To overcome the inherent low-pass nature of standard 3DGS [Kerbl et al., 2023], two distinct research directions have emerged. Frequency-aware methods introduce Fourier regularization [Zhang et al., 2024a; Zeng et al., 2025; Ren et al., 2025] or replace Gaussians with Wavelet and Gabormodulated primitives [Zhang et al., 2025a; Zhao et al., 2025a; Zhou et al., 2025; Wurster et al., 2024; Zhang et al., 2025b] to capture high-frequency textures. However, these methods typically treat frequency as a texture attribute independent of geometry, lacking the mechanism to spatially truncate signals at sharp boundaries, which often results in high-frequency energy spilling into empty space. Conversely, shape-adaptive approaches employ generalized-exponential falloffs [Hamdi et al., 2024] or learnable deformations and ensembling strategies [Di et al., 2024; Zhang et al., 2024b; Li et al., 2025; Huang et al., 2025; Zhao et al., 2025b] to fit sharp geometric silhouettes. Other works enforce strict geometric constraints by flattening Gaussians into 2D disks [Huang et al., 2024; Yu et al., 2024]. While effective for boundaries and surfaces, these methods oversimplify internal content (e.g., assuming homochromatic interiors) and fail to capture complex thermal gradients driven by heat diffusion [Wang et al., 2025a], rendering them insufficient for physically consistent thermal modeling. Our S3 -TIR bridges this gap by explicitly coupling adaptive spatial truncation with internal spectral modulation, thereby reconciling the conflict between geometric sharpness and diffusive smoothness within a unified primitive.

3 Method

We present S3 -TIR, a physics-inspired framework designed to reconstruct high-fidelity thermal radiance fields from posed TIR images by explicitly reconciling the physical coexistence of diffusive smoothness and structural sharpness. We represent the scene using a set of unified primitives P = {Pn}, where each primitive Pn = (Gn,Fn) is decomposed into a spatial geometry state Gn and a spectral content state Fn.

Specifically, the geometry state Gn = {µn,Σn, βn} defines the spatial support via a Generalized-Exponential formulation, where µn denotes the 3D center position, Σn represents the anisotropic covariance matrix determining scale and rotation, and βn serves as a shape parameter to control the steepness of boundary sharpness. Complementarily, the spectral state Fn = {ωlow,n, ωhigh,n, fn, ϕn} parameterizes the internal thermal distribution via Gabor modulation, consisting of a base thermal intensity ωlow,n, a modulation amplitude ωhigh,n, a spatial frequency vector fn, and a phase shift ϕn. The complete optimization pipeline is illustrated in Fig. 2.

3.1 Coupled Spatial-Spectral Primitive

The Gap in Thermal Representation. Thermal scenes inherently combine macroscopic geometry with fine-scale textures. Standard 3DGS, limited by its Gaussian low-pass nature, models heat diffusion well but struggles to capture material discontinuities. While recent variants like GES or Gabor wavelets improve shape or texture respectively, they treat spatial geometry and frequency content in isolation. Specifically, shape-focused methods lack the spectral capacity for textures, while frequency-focused methods fail to truncate signals at sharp boundaries. To bridge this, we propose a unified primitive that models the thermal field as a spatially-constrained spectral signal, enforcing a direct coupling between geometric sharpness and frequency distribution.

Primitive Formulation. We represent the scene using a set of primitives P = {Pi}Ni=1 . Departing from traditional methods that assume a fixed Gaussian falloff, we formulate each primitive P(x) as the coupled product of an Adaptive Spatial Support S and a Spectral Modulation Field F:

1. Adaptive Spatial Support (S). This component defines the effective physical domain of the primitive. We employ a Generalized-Exponential formulation to govern the spatial falloff:

Here, the shape parameter β ∈ [2,∞) serves as a physical truncation controller:

• Diffusion Mode (β ≈ 2): The support approximates a standard Gaussian, effectively approximating smooth thermal diffusion where intensity naturally decays across surfaces.
• Discontinuity Mode (β → ∞): The support converges to a box-like function. Crucially, this mode spatially truncates the internal spectral content at geometric boundaries, preventing signal leakage. This allows a single primitive to represent a textured step function-a capability absent in fixed-Gaussian methods.
• Spectral Modulation Field (F). Within the spatial support defined by S, we model intricate thermal variations using a phase-aware frequency formulation:

By explicitly parameterizing the 3D frequency vector fj, this term captures anisotropic thermal textures (e.g., directional heat dissipation patterns) that cannot be adequately represented by spherical harmonics alone.

3.2 Frequency-Aware Deferred Rendering

The Frequency Decoupling Ambiguity. Existing frequency-aware 3DGS methods typically modulate Gaussian primitives with high-frequency signals, operating under the implicit assumption that geometric boundaries and internal textures are naturally separable. We contend that this assumption breaks down at material discontinuities. Due to the intrinsic smooth falloff of standard primitives with β = 2, these methods unavoidably suffer from Signal Leakage as illustrated in Fig. 3. This introduces a fundamental ambiguity in the solver, making it an ill-posed problem to distinguish between the physically sharp spatial envelope of Geometric Truncation and the artificially decaying texture of Spectral Overfitting. Consequently, rather than recovering the underlying scene structure, the model compensates for this ambiguity by generating spurious high-frequency artifacts that degrade structural fidelity.

Frequency-Aware G-Buffer Generation. To address this ambiguity, we propose a deferred rendering pipeline that explicitly decouples thermal signals. Drawing inspiration from Spatial Coordinate 𝑥

(a) Physical Thermal Signal

(b) Texture-Dominant (Ambiguity)

Figure 3: Mechanism of Spatial-Spectral Coupling. (a) Physical thermal signal characterized by sharp transitions. (b) Standard 3DGS (β = 2): the smooth Gaussian fall-off causes signal leakage, leading to texture overfitting. (c) S3 -TIR (β = 20): the sharp spatial support effectively captures geometry while maintaining smooth and consistent texture. (d) 2D comparison: our Ssharp physically anchors Gabor modulations within geometric boundaries to prevent spectral leakage and reconstruction artifacts.

inverse rendering frameworks that separate material properties (e.g., albedo, normal) via deferred buffers [Liang et al., 2024; Gao et al., 2024], we here repurpose the rasterization engine to independently project the intrinsic spatial and spectral attributes of primitives into observable 2D feature maps, termed Frequency G-Buffers. Formally, this process accumulates specific physical properties for each pixel u via alphablending:

1. Low-Frequency Energy Map (Ψlow): Isolates the base thermal intensity ωlow to recover the smooth diffusive background;
2. High-Frequency Energy Map (Ψhigh): Extracts the Gabor modulation energy ∑ |ωj,high| to reveal internal structural details;
3. Spatial Sharpness Map (Ψβ): Projects the shape parameter β to expose geometric discontinuity boundaries.

To efficiently support this multi-channel rasterization while ensuring valid coverage of the Generalized-Exponential support, we employ a Bounding Gaussian Strategy, utilizing the standard Gaussian (β = 2) as a geometric proxy for sorting and tile-culling.

Multi-view Frequency Consistency. The pivotal advantage of this deferred architecture is the ability to constrain the 2D G-Buffers to produce multi-view consistent frequency maps. Drawing inspiration from Multi-View Stereo (MVS) constraints, we enforce that for a surface point visible from both viewpoint vi and vj, its projected frequency attributes on maps Ψi and Ψj must be identical. Specifically, we warp the G-Buffers from a source view vj to a reference view vi using the rendered depth Di and scene geometry:

where K, R, T denote intrinsic and extrinsic parameters. We then impose a consistency loss on the physical attribute maps (specifically Ψhigh and Ψβ):

By anchoring frequency properties to the 3D geometry, this constraint eliminates ambiguity, ensuring that structural sharpness and textural details remain geometrically consistent across all perspectives.

3.3 Physics-inspired Hybrid Optimization

To ensure a structurally consistent decomposition of spatial support and spectral content, we employ a hybrid optimization strategy. The total objective function is formulated as:

where Lrender comprises standard ℓ1 and D-SSIM losses, and Lconsist enforces the multi-view frequency consistency. To further regularize the intrinsic attributes, we impose two physics-informed priors on the intermediate G-Buffers.

1) Thermodynamic Sparsity (Lsparse). Since heat conduction acts as a natural low-pass filter, high-frequency energy in thermal fields is inherently sparse, concentrating primarily at material discontinuities. We enforce this via an ℓ1-sparsity penalty on the High-Frequency Energy Map Ψhigh:

This prior encourages the model to explain the scene using the low-frequency base ωlow and adaptive spatial support β, reserving spectral modulation for necessary textural details. 2) Geometric Continuity (Lsmooth). Material boundaries, parameterized by the spatial sharpness β, typically exhibit piecewise smoothness rather than random noise. To enforce clean geometric transitions, we apply an isotropic Total Variation (TV) regularizer on the Spatial Sharpness Map Ψβ:

where ∆ denotes spatial gradients. This term suppresses artifacts in the shape parameter while preserving sharp physical discontinuities. Jointly, these constraints force the primitives to learn a structured decomposition by handling smooth regions via diffusion and edges via geometric truncation, thereby eliminating high-frequency jittering.

4 Experiments

4.1 Experimental Setup

Datasets. To ensure robustness, we evaluate on three distinct benchmarks. RGBT-Scenes provides geometrically complex handheld data with over 1, 000 aligned pairs. TI-NSD offers large-scale diversity (6, 664 images) across indoor, outdoor, and UAV scenarios with significant thermal variance. Finally, NTR serves as a challenging testbed for dynamic environments, featuring four UAV-captured night scenes with calibrated poses.

Implementation Details. Our framework is implemented in PyTorch based on official 3DGS CUDA kernels. We train for 30, 000 iterations on a single NVIDIA RTX 3090 GPU. The spatial sharpness β is initialized to 2 and optimized with a learning rate of 0.001. We set the Gabor frequency bands M = 4. Loss weights are empirically set to λc = 0.01, λs = 0.005, and λg = 0.01. A 3, 000-iteration geometric warmup with standard 3DGS is applied before enabling spectral modulation. Input images are normalized to [0, 1]. Baselines. We benchmark against three categories: (1) Standard 3DGS [Kerbl et al., 2023]; (2) Thermal-Specific Methods, including Thermal3D-GS [Chen et al., 2024], ThermalGaussian [Ye et al., 2024], and NTR-Gaussian [Yang et al., 2025]; and (3) Shape & Frequency-Aware Methods, comprising GES [Hamdi et al., 2024], 3D-HGS [Li et al., 2025], and

WIPES [Zhang et al., 2025a].

Metrics. Following the evaluation protocols established in previous works [Kerbl et al., 2023; Chen et al., 2024; Yang et al., 2025], we report PSNR, SSIM, and LPIPS to quantitatively evaluate the reconstruction quality.

4.2 Comparisons

As detailed in Table 1, S3 -TIR achieves consistent state-ofthe-art performance across the RGBT-Scenes, TI-NSD, and NTR benchmarks, underscoring the robustness of our framework in diverse thermal environments. Most notably, on the RGBT-Scenes dataset-characterized by complex geometric and thermal distributions-our method delivers a substantial 42.5% reduction in LPIPS (0.186 → 0.107) and a 3.2% improvement in PSNR compared to the second-best baseline, ThermalGaussian. This quantitative superiority is visually corroborated in Fig. 5, particularly in scenarios with significant emissivity variations, such as heated pipes and pedestrian silhouettes. While standard 3DGS and its thermal variants often yield over-smoothed artifacts, recent specialized methods exhibit distinct trade-offs: shape-aware approaches like GES delineate sharp edges but lose internal gradient details, whereas frequency-aware methods like WIPES capture textures but suffer from boundary signal leakage. In contrast, S3 -TIR achieves an optimal balance by leveraging generalized-exponential spatial support for precise physical truncation and enforcing multi-view frequency consistency. Consequently, our approach effectively mitigates nonphysical high-frequency noise in background regions while accurately reconstructing the structural duality of diffusive smoothness and sharp structural discontinuities.

4.3 Ablation and Analysis

Contribution of Individual Components. We quantitatively validate the contribution of each component in Table 2. Specifically, removing the shape control mechanism (i.e., fixing β ≡ 2) results in a marked decline in SSIM, confirming that the generalized-exponential formulation is indispensable for modeling sharp thermal discontinuities. Similarly, disabling spectral modulation (f = 0) leads to over-smoothed reconstructions, evidenced by elevated LPIPS scores. This indicates that the model fails to capture fine-grained thermal textures arising from complex heat distributions without this component. In terms of optimization strategies, the most substantial performance drop occurs upon the removal of the multi-view frequency consistency loss (Lconsist). Absent this constraint, the optimization becomes susceptible to shape-texture ambiguity, resulting in view-dependent highfrequency jittering artifacts. Finally, the thermodynamic sparsity prior (Lsparse) proves essential for noise suppression, ensuring that high-frequency modulation is selectively applied only where supported by physical evidence.

Effect of Spectral Modulation Bandwidth (M). The parameter M controls the spectral capacity for encoding internal thermal textures. As shown in Table 3, low bandwidths (e.g., M = 1, 2) constrain the representation to lowfrequency diffusion, failing to capture fine-grained emissivity variations and resulting in over-smoothed artifacts. Conversely, excessive bands (e.g., M = 8) yield diminishing returns. Due to the inherent low Signal-to-Noise Ratio (SNR) of thermal sensors, over-parameterization tends to overfit sensor noise rather than physical structures. Empirically, M = 4 strikes the optimal balance, preserving sharp material textures while implicitly suppressing noise via band-limitation.

Analysis of Decoupled Attributes. To validate the physical fidelity of S3 -TIR, we visualize the learned intermediate G-Buffers in Fig. 4. As shown in Fig. 4(b), the LowFrequency Map Ψlow captures a smooth, continuous distribution, characterizing macroscopic heat diffusion. In contrast, the High-Frequency Map Ψhigh (Fig. 4c) exhibits structural sparsity, selectively encoding details arising from material heterogeneity (e.g., windows). Crucially, the Spatial Sharpness Map Ψβ (Fig. 4d) strictly delineates geometric silhouettes. High β values align with object boundaries, confirming that our Generalized-Exponential support enforces physical truncation to prevent signal leakage. This clear separation substantiates that our representation successfully resolves the ambiguity between geometry and texture.

Validation of Structural Duality To validate our Coupled Spatial-Spectral representation, we analyze the distribution of the shape parameter β. As shown in Fig. 6, thermal scenes exhibit a distinct structural duality. Spatially (Fig. 6a), high β values concentrate at material discontinuities to enforce boundary truncation, whereas low β values dominate the diffusive background. Quantitatively, Fig. 6b confirms a structured sparsity: a heavy-tailed distribution where most primitives follow Gaussian diffusion (β ≈ 2), while a sparse subset requires generalized-exponential truncation (β ≫ 2). This validates our learnable shape parameter. In contrast, standard fixed kernels fail to capture this duality, causing the oversmoothed edges observed in baselines.

5 Conclusion

We present S3 -TIR, a physics-inspired framework that resolves the inherent duality of thermal infrared novel view synthesis. By coupling adaptive spatial support with internal spectral modulation via a unified generalized-exponential Gabor primitive, our method confines high-frequency textures within geometric boundaries while preserving diffusive smoothness. Our frequency-aware deferred splatting pipeline further resolves decomposition ambiguity by enforcing multiview consistency on decoupled frequency attributes. Extensive experiments demonstrate that S3 -TIR achieves state-ofthe-art performance, generating structurally stable thermal sequences free from high-frequency jittering.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62372382).