Veta-GS:View-dependent deformable 3D Gaussian Splatting for thermal infrared Novel-view Synthesis

T L D R :

We introduce Veta-GS, which leverages a view-dependent deformation field and a Thermal Feature Extractor (TFE) to precisely capture subtle thermal variations and preserve robustness.

Abstract

Recently, 3D Gaussian Splatting (3DGS) based on Thermal Infrared (TIR) imaging has gained attention in novel-view synthesis, showing real-time rendering. However, novel-view synthesis with thermal infrared images suffers from trans- mission effects, emissivity, and low resolution, leading to floaters and blur effects in rendered images. To address these problems, we introduce Veta-GS, which leverages a view- dependent deformation field and a Thermal Feature Extractor (TFE) to precisely capture subtle thermal variations and pre- serve robustness. Specifically, we design view-dependent deformation field that leverages camera position and viewing direction, which capture thermal variations. Furthermore, we introduce the Thermal Feature Extractor (TFE) and MonoS- SIM loss, which consider edge, frequency, and appearance to preserve robustness. Extensive experiments on the TI-NSD benchmark confirm the effectiveness of our method and the impressive improvement in reconstruction quality.

Overview of Veta-GS pipeline.

Overview of Veta-GS pipeline. We utilize camera position x, viewing direction v, time t, and 3D Gaussian’s position μ with positional encoding as input to deform 3D gaussians, by obtaining the offset (δμ, δr, δs). Further we introduce Thermal Feature Extractor (TFE) and MonoSSIM loss which focus on edge, frequency, and appearance to show robustness rendering.