DivCon-NeRF: Diverse and Consistent Ray Augmentation for Few-Shot NeRF

1 Introduction

Neural Radiance Field (NeRF) [Mildenhall et al., 2020], a neural network-based method, has significantly advanced the performance of novel view synthesis. However, NeRF-based models face the common deep learning challenge of performance degradation in few-shot scenarios. A limited number of training rays with sparse viewpoints causes overfitting. Therefore, the need for numerous images from different angles presents a substantial challenge for the practical application of NeRF-based models. To address the few-shot view synthesis problem, three main approaches are actively researched: prior-, framework-, and regularization-based methods. Each approach takes a different direction and is relatively orthogonal. Prior-based methods [Yu et al., 2021; Chen et al., 2021] leverage 3D knowledge through pre-training on large-scale datasets, but they incur high computational costs. Meanwhile, framework-based methods [Zhu et al., 2024a; Zhong et al., 2024] modify or extend model architectures, which can increase model complexity.

∗The corresponding author.

Our proposed method is a regularization-based approach that uses explicit regularizers [Xu et al., 2024; Yang et al., 2023] or auxiliary training sources [Wang et al., 2023]. In particular, ray augmentation generates additional rays from a limited training set, enabling the model to learn from both original and augmented rays, thereby reducing overfitting. We define two key properties for ray augmentation: ray diversity, the range of viewpoints from which augmented rays are generated, and ray consistency, the requirement that each augmented ray converges to the same surface point as its original. Existing methods [Seo et al., 2023b; Seo et al., 2023a] sample rays only near the originals to avoid occlusion. However, as shown in Fig. 1, these approaches suffer from significant floaters owing to limited viewpoint variation (low diversity) and training on inconsistent rays obstructed by nearby obstacles and complex surfaces (low consistency). Furthermore, they introduce geometric and color distortions.

To address these limitations, we propose DivCon-NeRF, which enhances diversity while preserving consistency. In general data augmentation, various studies [Cubuk et al., 2019; Cubuk et al., 2020; Xie et al., 2020; Lee et al., 2024; Hendrycks et al., 2020] have focused on diversifying data while maintaining consistency. Based on this observation, we hypothesize that enhancing ray diversity while preserving consistency is critical for improving performance in few-shot view synthesis. To achieve this, DivCon-NeRF consists of surface-sphere and inner-sphere augmentations. As shown in Fig. 1, our method effectively reduces floaters and appearance distortions compared to existing approaches.

We first define a virtual 3D sphere centered at the inferred surface point, with its radius set to the distance between the camera and that point. We cast rays from both the surface and interior of the sphere toward the inferred surface point. In surface-sphere augmentation, a consistency mask is generated by comparing the order of high-probability surface points between the original and augmented rays, allowing the model to filter out inconsistent rays easily without relying on precise rendered depth. We propose a ray consistency loss that aligns surface points by comparing the similarity of blending weight distributions, applying a temperature factor to assign more weight to regions with higher surface probabilities. Additionally, we introduce a positional constraint to the bottleneck feature loss to account for the relative positions of points. For greater diversity, inner-sphere augmentation utilizes randomized angles and distances, providing a broader range of viewpoints.

Our DivCon-NeRF effectively increases diversity while maintaining consistency, resulting in improved rendering quality compared to that of recent NeRF-based methods. These results support our hypothesis that both diversity and consistency are essential for enhancing rendering performance. Our main contributions are as follows:

• We experimentally demonstrate the importance of ray diversity and consistency in augmentation, analyzing their effects in both object-centric scenes and those with diverse depth ranges.
• We propose DivCon-NeRF, a novel ray augmentation method that simultaneously enhances diversity and preserves consistency.
• DivCon-NeRF significantly reduces floaters and appearance distortions, outperforming recent few-shot NeRFs on the Blender, LLFF, and DTU datasets.
• Our method is compatible with other regularization- and framework-based methods, demonstrating strong generalizability.

2 RelatedWork

2.1 Neural Scene Representations

Neural scene representations have become a prominent method for encoding 3D scenes, achieving impressive results in tasks such as novel view synthesis [Mildenhall et al., 2020; Barron et al., 2022; Mildenhall et al., 2022; M¨uller et al., 2022] and 3D scene generation [Jain et al., 2022; Poole et al., 2022; Lin et al., 2023]. These methods typically employ neural networks to model scene properties continuously, rather than relying on traditional discretized representations, such as meshes [Kanazawa et al., 2018; Liao et al., 2018; Pan et al., 2019], voxels [Gadelha et al., 2017; Xie et al., 2019; Yu et al., 2022], or point clouds [Achlioptas et al., 2018; Thomas et al., 2019]. Neural scene representations enable highly detailed and photorealistic scene reconstructions [Park et al., 2021; Martin-Brualla et al., 2021]. Among the various approaches, NeRF [Mildenhall et al., 2020] has gained attention for its ability to synthesize novel views. However, NeRF-based models experience a significant performance drop when trained on sparse image data [Zhong et al., 2024; Xu et al., 2024], posing a critical challenge in practical applications.

2.2 Few-Shot View Synthesis

Various studies have aimed to address few-shot view synthesis in NeRF-based models. Prior-based methods [Chen et al., 2021; Yu et al., 2021] pre-train models on large-scale datasets to provide prior knowledge of 3D scenes. However, these methods incur substantial costs due to their reliance on largescale datasets, and performance can degrade when the pretraining and target data distributions differ.

Framework-based methods modify network structures with models such as Transformers [Vaswani et al., 2017] and CNNs [Krizhevsky et al., 2012] and use these modified models during rendering. mi-MLP [Zhu et al., 2024a] incorporates input embeddings into each MLP layer for flexible learning, while CVT-xRF [Zhong et al., 2024] employs an invoxel Transformer to refine radiance properties within voxels. However, these methods increase model complexity and limit integration with other framework-based methods. Additionally, appearance distortions persist due to sparse viewpoints (e.g., CVT-xRF; see Fig. 1).

Regularization-based methods optimize each scene using regularization terms [Deng et al., 2022] or auxiliary training sources [Wang et al., 2023]. FreeNeRF [Yang et al., 2023] implements a frequency domain to mitigate artifacts. AR-NeRF [Xu et al., 2024] adapts rendering loss to match the frequency progression of positional encoding. However, data scarcity remains unresolved, as these methods do not directly increase the number of training rays, leading to several floaters (e.g., FreeNeRF; see Fig. 1).

Additionally, recent few-shot 3D Gaussian Splatting (3DGS) methods [Zhang et al., 2024; Li et al., 2024; Zhu et al., 2024b] have demonstrated promising performance in general few-shot scenarios. However, they encounter challenges in specific scenarios, such as background-free object rendering, which is crucial for practical applications like ecommerce and digital advertising.

2.3 Ray Augmentation for Sparse Images

Ray augmentation is a regularization-based approach that generates augmented rays from the originals. This approach [Chen et al., 2022b; Seo et al., 2023b; Truong et al., 2023] effectively addresses the shortage of training rays by directly increasing their quantity. InfoNeRF [Kim et al., 2022] reduces ray entropy to improve consistency across close views, while FlipNeRF [Seo et al., 2023a] employs flipped reflection rays to improve 3D geometry estimation.

FrugalNeRF [Lin et al., 2025] enforces novel view consistency by jointly optimizing poses and voxel features. However, these methods, which augment rays only near the originals, struggle with complex surfaces and nearby obstacles, causing inconsistencies. Furthermore, these methods generate augmented rays from limited viewpoints. GeCoNeRF [Kwak et al., 2023] applies a depth-based mask to every pixel of a warped patch rather than directly to a ray. This mask relies on image warping, which causes interpolation issues and fails to maintain equal distances from surface points to both the original and pseudo cameras for each pixel, resulting in coarse filtering. Therefore, GeCoNeRF depends on pseudo-views near the training views. These limitations in existing methods lead to pronounced floaters and appearance distortions due to low diversity and inadequate consistency. In contrast, our proposed method enhances both diversity and consistency, effectively addressing these limitations.

3 Preliminaries: NeRF

NeRF [Mildenhall et al., 2020] utilizes MLPs to synthesize novel views of a scene from densely sampled images. The NeRF network F maps 3D coordinates X = (x, y, z) and viewing direction d to color c and volume density σ:

NeRF casts a single ray into 3D space for each pixel in the image. Because the ray is continuous, points along the ray are sampled. These sampled points determine the final pixel color C through volume rendering. The ray is expressed as r(t) = O + td, where O is the camera origin. The predicted pixel color C(r) is calculated as follows:

where Ti = exp (- ∑i-1 j=1 σj δj ), and δi = ti+1 - ti. Blending weight wi represents the contribution of the color at the i-th point to the pixel color C(r). The NeRF network F is optimized using a mean square error (MSE) loss to ensure that the predicted pixel color C(r) matches the ground truth color Cgt(r) of the training rays:

where R represents a set of rays. However, NeRF requires dense training views and performs poorly with sparse inputs.

4 Method

We propose DivCon-NeRF, a ray augmentation method that simultaneously enhances ray diversity and preserves ray consistency. In contrast to depth-based image warping methods [Kwak et al., 2023; Chen et al., 2022a] that require a rendering for augmentation and suffer from interpolation issues, DivCon-NeRF augments rays directly in 3D space. This method combines surface-sphere augmentation (Sec. 4.1), which generates rays on a sphere, and inner-sphere augmentation (Sec. 4.2), which introduces random distance variations within the sphere, both directed toward the inferred surface point of the original ray. We introduce a consistency mask and three losses tailored to these augmentations. The overall scheme of DivCon-NeRF is depicted in Fig. 2.

4.1 Surface-Sphere Augmentation

In contrast to the typical approach [Mildenhall et al., 2020], which positions the virtual sphere at the center of the object or 3D scene to determine camera positions, we center the virtual sphere on the predicted surface point Ps that the original ray hits. When the original and augmented rays cast from the sphere intersect at the same surface point, the augmented ray is unobstructed. We verify the consistency of the augmented rays by comparing the surface points hit by both rays.

For surface-sphere augmentation, the sphere's center is set at Ps, the most likely surface point predicted by the model: Ps = O + tsd, where s = argmaxi (wi). Here, wi denotes the blending weights of the original ray r. The radius is defined as ∥O-Ps∥. Consequently, the augmented ray's camera coordinates O′ are given by:

where (θ, ϕ) are spherical coordinates, with θ ∼ U[0, π] and ϕ ∼ U [0, 2π).

During training, the model progressively improves its prediction of Ps, refining the accuracy of the virtual sphere. Notably, regardless of the point on the sphere from which an augmented ray is cast, all rays share the same distance R (= ts∥d∥) and converge toward Ps. This property of surfacesphere augmentation allows for the use of a consistency mask by estimating the index of the peak blending weight along the ray, eliminating the need for precise depth prediction.

Consistency Mask

To generate the consistency mask for augmented rays without constraining angle and distance, we consider all points on the virtual sphere as potential positions for O′ . In each iteration, a point on the sphere is randomly selected as O′ for each original ray r. The coordinates of the sampled points along the surface-sphere augmented ray r′ are computed as r′ (t) = O′ +td′ , where d′ denotes the augmented ray's camera viewing direction:

The magnitude of d′ is set equal to that of d to match the distribution of the original training data. The sampled points along r′ are then fed into the model F. The point with the highest blending weight is identified as the most probable surface point along the augmented ray: Ps′ = O′ + ts′d′ , where s′ = argmaxi (w′i ), with the blending weight w′i derived from r′ . Note that this process occurs during coarse sampling, where points are sampled at equal intervals. Because ∥Ps -O∥ = ∥Ps -O′ ∥ and δ = δ′ , where δ and δ′ represent the distances between adjacent samples on the original and augmented rays, respectively, Ps′ of the consistent ray should be identical to Ps. To exclude occluded rays from training, we define the following consistency mask:

where s and s′ denote the indices of the maximum blending weights along the rays r and r′ , respectively. We introduce the parameter ϵ as an error tolerance, given that this process occurs during coarse sampling. If the difference between the indices exceeds ϵ, the augmented ray is filtered out, indicating that the ray reaches a different surface point.

Accurately estimating wi and w′i at every point is challenging for a model during training. For example, GeCoNeRF [Kwak et al., 2023], which relies on depth for masking warped patches, employs an additional depth smoothness loss to improve rendered depth accuracy. In contrast, determining the point with the highest blending weight via the argmax function is more straightforward and effective, resulting in fewer floaters and better performance. This claim is empirically validated in Sec. 5.3 and the supplement.

Ray Consistency Loss

In addition to the consistency mask, we introduce a ray consistency loss to further align the surface points of r and r′ , enhancing overall consistency. We apply a Kullback-Leibler (KL) divergence loss between the blending weight distributions of the original and augmented rays, using a temperaturescaled softmax. The ray consistency loss for each ray is defined as:

where

The temperature T sharpens these softmax distributions, emphasizing their peak regions. As KL divergence primarily penalizes mismatches in high-probability regions, it encourages the peak locations of these distributions to align during training. As this loss functions as a regularizer, it allows flexibility in low-probability regions. Note that augmented rays far from the originals may not exhibit strong global similarity but should exhibit consistent peak locations near the surface.

In the LLFF dataset [Mildenhall et al., 2019], which contains many objects within a scene, augmented rays farther from the original rays tend to have more dissimilar overall distributions. For these augmented rays located beyond a certain range, we apply clipping to wi and w′i by setting them to 0 when i > s, and then obtain PT (i) and QT (i). Finally, the overall ray consistency loss LRC for a set of rays R is defined as:

In contrast to InfoNeRF [Kim et al., 2022], which only applies information regularization to augmented rays with minimal viewpoint variations, our approach leverages temperature scaling and clipping to enforce a strong consistency constraint near the surface, utilizing augmented rays across all ranges for LRC. Additional comparison experiments are detailed in the supplementary material.

Positional Bottleneck Feature Loss

We introduce a positional constraint to the bottleneck feature loss as proposed alongside FlipNeRF [Seo et al., 2023a]. In contrast to FlipNeRF, which fails to maintain a consistent positional relationship among sampled points due to the varying normal vector magnitudes, we utilize sampled points from the original and augmented rays that are equidistant from the predicted surface point. The positional bottleneck feature loss is computed using Jensen-Shannon divergence as follows:

where ∥Ps - Xi∥ = ∥Ps - X′i ∥, ψ denotes the softmax function, and h(·) maps the input to the last layer's feature space. This constraint effectively reduces geometric discrepancies between Xi and X′i for reliable feature matching, as demonstrated in Sec. 5.3. Therefore, we obtain lPBF through the coarse sampling of surface-sphere augmentation, excluding inner-sphere augmentation. The overall positional bottleneck feature loss LPBF is defined as:

(10)

4.2 Inner-Sphere Augmentation

In addition to surface-sphere augmentation, inner-sphere augmentation randomly samples the distance R within the sphere in each iteration. This further enhances diversity and improves the model's robustness to variations in the camera's distance from the surface. When a pseudo camera is positioned farther from the surface point than the original camera, a single pixel represents a larger area, covering both the ground truth and its neighboring pixels, making it more difficult to maintain color consistency (see supplementary material for further analysis). Therefore, the inner-sphere augmented ray's camera coordinates O′′ are given by:

(11)

where r ∼ U(0, 1], and θ and ϕ are common random variables shared with surface-sphere augmentation, resulting in the same viewing direction as d′ . Therefore, we apply the same consistency mask, which detects obstacles within the sphere along the line containing both augmented rays. Although this may filter out a few consistent rays, it significantly reduces the likelihood of training on inconsistent rays.

Instead of using MSE loss, we extend the NLL loss from MixNeRF [Seo et al., 2023b] to our inner-sphere augmented ray r′′ , enabling the model to maintain color consistency across diverse viewpoints:

(12)

where Cgt is the ground truth color of r, and p(Cgt | r′′ ) = ∑Ni=1 πiF(Cgt | r′′ ). Here, πi = w′′ i ∑Nm=1 w′′m , w′′ i denotes the blending weights of r′′ , and F denotes a Laplacian distribution centered at the predicted color C(r′′ ). Although the rays converge at the same surface point, the consistent augmented ray does not necessarily have the same ground truth FlipNeRF

FreeNeRF

Ours Ground Truth

FreeNeRF

LLFF 2pt

(a) Blender (4-view)

FrugalNeRF

Ours Ground Truth

Dtu 2pt

(b) LLFF (3-view)

(c) DTU (3-view)

Figure 3: Qualitative comparisons on the Blender, LLFF, and DTU datasets. Red boxes indicate severe floaters, and green boxes indicate significant visual distortions. Our method exhibits fewer floaters and visual distortions than other NeRF-based methods across all three datasets.

color as the original ray due to view-dependent effects. As the ground truth for r′′ is unavailable, we utilize the predicted color C(r′′ ) from coarse sampling and a probability-based NLL loss to handle view-dependent variations while maintaining color consistency. This allows the augmented ray to produce a visually consistent color that adapts naturally across views.

4.3 Training Loss

The overall training loss is defined as follows, with λ terms representing balancing weights for the respective losses:

Here, LMSE is calculated using the original ray, LNLL is the NLL loss of the original ray, introduced with MixNeRF [Seo et al., 2023b], and LUE refers to the uncertainty-aware emptiness loss introduced with FlipNeRF [Seo et al., 2023a], stabilizing the blending weights to prevent fluctuations. The supplementary material provides more details of both losses.

5 Experiments

5.1 Datasets andMetrics

We evaluate our method against recent NeRF-based methods without external generative priors on the Blender [Mildenhall et al., 2020], LLFF [Mildenhall et al., 2019], and DTU [Jensen et al., 2014] datasets. For reference, we report the results of recent few-shot 3DGS methods on Blender; their performance on the LLFF and DTU datasets is reported in the supplementary material. The Blender dataset comprises 8 synthetic, background-free, object-centric scenes, while LLFF consists of 8 forward-facing real-world scenes.

For the DTU dataset, we use 15 scenes following the protocol established by RegNeRF [Niemeyer et al., 2022].

We report PSNR, SSIM [Wang et al., 2004], LPIPS [Zhang et al., 2018], and the average error (AVGE) calculated as the geometric mean of MSE = 10-PSNR/10 , √1-SSIM, and LPIPS, following Niemeyer et al. (2022). LPIPS is evaluated using AlexNet [Krizhevsky et al., 2012], following Zhong et al. [2024]. Additional experimental details and extended results with more input views across Blender, LLFF, and DTU are provided in the supplement.

5.2 Comparisons

Blender

Our method significantly outperforms all compared NeRFbased approaches on the Blender dataset, as shown in Tab. 1. For a fair comparison, we follow the experimental protocols of MixNeRF [Seo et al., 2023b] (no mark) and DietNeRF [Jain et al., 2021] (marked by ∗ ). In both cases, our method consistently achieves the highest performance by a substantial margin. As illustrated in Fig. 3a, it effectively reduces floaters and appearance distortions compared to existing methods, thereby reinforcing the importance of consistency and diversity, as hypothesized. For completeness, we include the results of recent few-shot 3DGS methods [Zhu et al., 2024b; Li et al., 2024] at the bottom of Tab. 1; these perform notably worse on background-free, object-centric scenes (see Sec. 5.4 and the supplementary material).

LLFF

As shown in Tab. 2, our method achieves a substantial performance improvement over other approaches, demonstrating its effectiveness across both synthetic and real-world datasets. As shown in Fig. 3b, our method more effectively prevents geometric distortions compared with other methods owing to enhanced consistency, better preserving the continuity of straight lines on wall surfaces.

DTU

To demonstrate the generalizability of our method, we integrate it with the regularization-based FlipNeRF [Seo et al., 2023a] and the framework-based CVT-xRF [Zhong et al., 2024]. As shown in Tab. 3, our DivCon-Flip outperforms FlipNeRF in all metrics, and our DivCon-CVT achieves the best performance. Integrating our method with CVT-xRF substantially improves PSNR, SSIM, and AVGE while closely maintaining LPIPS. This suggests that our method is architecture-agnostic and effectively complements framework-based methods. As illustrated in Fig. 3c, our method yields cleaner renderings than other methods.

5.3 Analysis Ray Diversity

We evaluate the impact of ray diversity on rendering quality by progressively expanding the ranges of θ, ϕ, and r. As shown in Fig. 4, low diversity results in severe floaters (red boxes), while increased diversity notably reduces these artifacts, indicating the importance of augmented rays from diverse viewpoints. In particular, when the angular range is narrow, more floaters and pronounced visual distortions are observed (green boxes). However, employing a full viewing angle without the consistency mask can lead to challenges in maintaining ray consistency.

Absence of Consistency Mask

To analyze the importance of consistency, we evaluate our method without the consistency mask. In object-centric scenes, as shown in Fig. 5a, reduced consistency leads to significant visual distortions. The corresponding depth map reveals empty regions on the object surfaces, indicating that the rays fail to locate the surface. This suggests that inconsistent rays are used during training, increasing uncertainty in depth Figure 4: Analysis of diversity for angle and radius on Blender. The insets in the second row show depth maps. Best viewed when enlarged.

w/o consistency mask w/ consistency mask

(a) Blender (4-view)

w/o consistency mask w/ consistency mask

(b) LLFF (3-view)

Figure 5: Comparison of rendered RGB and depth images with and without the consistency mask.

PSNR ↑ SSIM ↑ LPIPS ↓

DivCon-NeRF (w/ argmax) 20.46 0.662 0.221

DivCon-NeRF (w/ depth) 19.25 0.607 0.289

estimation and resulting in unreliable depth predictions. This highlights the crucial role of consistency in ray augmentation for preventing visual distortions. As shown in Fig. 5b, diversity alone cannot eliminate many floaters in the LLFF scene with varying depths. In this case, consistency masks effectively reduce floaters and appearance distortions. Therefore, our model demonstrates robust performance across both synthetic and real-world scenes by enhancing diversity and consistency.

Limitations of Using Rendered Depth

We compare the performance of our consistency mask when using the argmax function versus rendered depth. Using rendered depth results in lower performance, as shown in Tab. 4. This suggests, that because the depth calculation depends on accurate blending weights for all points, the argmax-based masking is more effective for maintaining consistency.

(3) ✓ ✓ ✓

(4) ✓ ✓ ✓ ✓

22.08 0.839 0.129

20.69 0.820 0.168

(5) ✓ ✓

✓ ✓ 20.73 0.817 0.149

Table 5: Ablation study. Base includes LNLL and LUE. We high-

light the effect of the newly introduced loss terms.

Ablation Study

Tab. 5 shows the impact of each component in our method. In (2), adding only LRC to align surface points shows a notable performance improvement, except for SSIM. As shown in (3), incorporating LPBF significantly surpasses the baseline across all metrics. Adding LMNLL of inner-sphere augmented rays achieves the ideal performance in (4), underscoring the importance of enhancing diversity while preserving consistency.

In (5), LBF represents the bottleneck feature loss proposed with FlipNeRF [Seo et al., 2023a]. Substituting LPBF with LBF results in substantial overall performance degradation, indicating the importance of incorporating relative positional information for reliable feature matching.

5.4 Effectiveness in Background-Free Scenes

Our method is highly effective in background-free, objectcentric scenes. In contrast to recent few-shot 3DGS methods [Zhu et al., 2024b; Li et al., 2024] which suffer from significant distortions and floaters (Fig. 6, Tab. 1), our approach produces cleaner renderings with no floaters. This makes it especially suitable for applications, such as e-commerce and digital advertising, where accurate multi-view rendering of isolated products is essential for flexible scene composition. Additional analysis on Blender and other scene types is provided in the supplementary material.

6 Conclusion

In this paper, we present DivCon-NeRF, which effectively reduces floaters and visual distortions in NeRF-based fewshot view synthesis. Our consistency mask effectively filters out inconsistent augmented rays, making us the first to utilize pseudo views from all 360-degree directions based on a virtual sphere. Our method simultaneously enhances diversity and preserves consistency, with experimental results confirming its effectiveness in both object-centric scenes and real-world environments. Our results demonstrate that ray diversity and consistency are crucial for few-shot NeRF.

Acknowledgments

This work was supported by the Korean Government through the grants from IITP (RS-2021-II211343 and RS-202525442338) and KOCCA (RS-2024-00398320).