OrienDiffusion: Taming Diffusion Towards Fine-grained Portrait Matting

1 Introduction

Portrait matting [Chuang et al., 2001] [Sun et al., 2004] [Chen et al., 2013] [Levin et al., 2007], a technique estimating pixel-wise opacity to extract human foregrounds from complex backgrounds, is fundamental to image editing, online conferencing, live streaming, etc. While significant progress has been made in this field [Wang et al., 2024b], fine-grained portrait matting remains a core challenge in real-world scenarios, especially for thinstructured hair strands and small ambiguous-color regions in foreground-background transition zones, where contextual information is confusing. More critically, fine-grained alpha matte annotations are notoriously difficult to obtain [Liu et al., 2025], resulting in unreliable supervision that impedes the training of alpha matting models. As shown in Figure 1, the hair strands in the ground-truth data suffer from structural discontinuity, and small background regions are mislabeled as foreground.

∗Corresponding author.

Most existing portrait matting approaches adopt discriminative models based on Convolutional Neural Networks (CNNs) or Transformers [Yao et al., 2024] to directly regress alpha mattes from input images. By leveraging multi-scale features [Chen et al., 2018] [Sun et al., 2021] and global attention mechanisms [Li and Lu, 2020], these methods can effectively capture the overall characteristics of portraits for matting; however, they often struggle with thin hair strands and small regions with ambiguous colors or context.

Recently, generative diffusion models [Chen et al., 2023] [Fu et al., 2024] [Ke et al., 2024] [Zhou et al., 2024] have been introduced into the matting task [Li et al., 2024] [Wang et al., 2024a], providing a new modeling perspective for alpha estimation. By formulating matting as a gradual denoising-driven stochastic generation process, diffusion models can progressively recover high-frequency details through coarse-to-fine reconstruction. Existing works either fine-tune large-scale pre-trained generative models such as Stable Diffusion [Rombach et al., 2022] or train lightweight diffusion denoising networks from scratch [Hu et al., 2024]. Although generative matting methods exhibit certain advantages over traditional discriminative approaches in structural generation, the inherent randomness of the diffusion sampling process makes it difficult to precisely constrain the evolution trajectory of alpha values, particularly under imperfect ground-truth supervision. Therefore, when dealing with hair regions or small regions with ambiguity, the denoising process is prone to structural drift and detail degradation.

To address these issues, we propose a novel generative portrait matting framework termed OrienDiffusion, which is built on a lightweight diffusion model trained from scratch. Departing from sole reliance on ground-truth supervision, we emphasize structural priors during the generation process. Specifically, we construct a Hair Orientation Field (HOF) and develop a HOF injection strategy to incorporate the field as an explicit geometric prior for guiding the diffusion denoising process. This prior constrains alpha matte generation toward structurally consistent and visually natural hair patterns, even with inaccurate ground-truth annotations. Furthermore, for pixels with ambiguous color and contextual information, excessive feature abstraction undermines local structural discriminability, especially under imprecise supervision conditions. To mitigate this, we design a Pixel Color-anchored Local Structure Embedding (PCLSE) method that models spatial opacity transitions as local intensity patterns derived from original pixel colors within neighborhoods. This method furnishes stable local structural constraints for alpha matte generation in such ambiguous regions. As shown in Figure 1, even with noisy ground-truth data, our proposed model yields results with coherent hair structures and correctly classifies small background regions with ambiguous colors and context.

The contributions of this paper are as follows:

• We propose OrienDiffusion, a novel generative portrait matting framework based on a lightweight diffusion model. It integrates hair orientation continuity and local structural patterns as dual explicit structural priors to constrain the alpha matte generation trajectory, addressing the core limitations of diffusion models in finegrained portrait matting.
• We introduce Hair Orientation Field guidance into the diffusion denoising process, which effectively ensures structural coherency in challenging hair strand regions by leveraging directional geometric priors.
• We design a Pixel Color-anchored Local Structure Embedding method that incorporates local structural cues to capture fine-grained pixel relationships, thereby improving alpha estimation accuracy in color- and contextambiguous regions.
• Extensive experiments validate that OrienDiffusion achieves state-of-the-art (SOTA) performance on portrait matting tasks, exhibiting superiority particularly in handling challenging hair strands and color/contextambiguous pixels.

2 RelatedWork

2.1 Deep Portrait Matting

Discriminative deep learning-based methods [Sengupta et al., 2020] [Zhong and Zharkov, 2024] [Yu et al., 2021a] [Sun et al., 2022] have long dominated portrait matting, aiming to directly regress alpha mattes from RGB images in an end-toend manner while maintaining semantic consistency and fine structural details.

Existing methods can be divided into auxiliary-based and auxiliary-free approaches. Auxiliary-based methods such as MGMatting [Yu et al., 2021b], GCA [Li and Lu, 2020] and IndexNet [Lu et al., 2019] leverage explicit priors to constrain ambiguous regions, demonstrating strong capability in recovering fine structures when accurate auxiliary are provided. However, their performance is highly sensitive to auxiliary quality [Kim et al., 2025], limiting practical applicability.

To improve automation, auxiliary-free methods including [Shen et al., 2016], MODNet [Ke et al., 2022] and P3M [Li et al., 2021] take only RGB images as input and compensate for missing priors using semantic cues and multi-scale context aggregation. These methods typically combine global semantic parsing with local detail refinement to balance structural integrity and boundary accuracy, but still struggle in colorambiguous or structurally complex regions.

A fundamental limitation of discriminative approaches lies in formulating matting as one-shot regression, without explicit mechanisms for iterative refinement or uncertaintyaware correction. Consequently, predictions tend to be spatially smoothed to reduce regression variance, leading to blurred transitions, broken hair strands, and limited robustness to complex unseen scenes.

2.2 Generative Image Matting

Generative models reformulate portrait matting by explicitly modeling the uncertainty of alpha mattes, offering a fundamentally different perspective from discriminative regression. Early GAN-based approaches [Lutz et al., 2018] [Xu et al., 2022] introduced adversarial supervision to regularize alpha distributions, but their limited ability to synthesize finegrained structures and reliance on carefully balanced objectives hinder robustness in complex scenes.

Recently, diffusion models [Yu et al., 2025] have emerged as a powerful generative paradigm for portrait matting, leveraging iterative denoising to capture high-frequency details and complex alpha distributions. By treating the alpha matte as a samplable distribution rather than a direct regression target, diffusion-based methods demonstrate improved robustness in challenging backgrounds and boundary regions. However, existing approaches [Huang et al., 2025] [Li et al., 2024] either rely on large pre-trained diffusion models [Rombach et al., 2022] with prohibitive computational cost or adopt lightweight diffusion frameworks [Xu et al., 2025] [Hu et al., 2024] that treat denoising as post-hoc refinement with fixed trajectories. As a result, the generation process lacks explicit structural controllability [Liu et al., 2024], particularly in hair regions where directional consistency and local continuity are critical [Zhang et al., 2023] [Ye et al., 2023], leading to detail degradation during diffusion.

3 Method

3.1 Overview

We formulate portrait matting as a diffusion-based alpha generation problem, where the target alpha matte is progressively refined through a conditional denoising process. Specifically, given the ground-truth alpha matte x0, the forward diffusion process gradually corrupts it by adding Gaussian noise [Ho et al., 2020], producing a sequence of latent variables:

where t ∈ {1, . . . , T} denotes the diffusion timestep, ¯αt = ∏ti=1 (1-βi), βi is a predefined noise schedule, and I denotes the identity matrix. During the reverse process, the denoising process is conditioned on the input image I, Trimap Tri, and additional structural priors C derived from I. Accordingly, we learn a conditional reverse denoising process:

which progressively recovers the clean alpha matte from noisy samples.

As illustrated in Figure 2, firstly, a Hair Orientation Field is computed from I and injected into the diffusion process to provide geometry guidance for hair regions (Section 3.2). This orientation prior enforces directional continuity during denoising, guiding the model toward structurally coherent hair alpha generation. Secondly, for regions with severe color or context ambiguity, we design a Pixel Color-anchored Local Structure Embedding module (Section 3.3), which captures pixel-wise constraints based on local neighborhood comparisons. These priors are incorporated into the denoising network at each diffusion step, explicitly constraining the denoising trajectory and leading to more accurate and structurally consistent alpha matte estimation.

3.2 Hair Orientation Field Computation and Injection

Hair-Orientation Field Computation. Hair regions in portrait matting exhibit strong strand-level directional continuity. Explicitly modeling such directional cues provides valuable structural guidance for alpha estimation, helping preserve spatial coherence and maintain consistent hair patterns during the matting process.

We therefore propose a HOF-guided denoising strategy, where a continuous orientation field is estimated following [Tan et al., 2020] [Pan et al., 2025] and injected as structural prior to constrain the diffusion trajectory. The HOF enforces spatially consistent feature evolution along hair growth directions, enabling more coherent strand structures and improved matting of fine hair details.

Specifically, the construction of the HOF first relies on accurate hair region localization. In practical applications, portrait images are often not standard half-body photos, but are frequently accompanied by complex poses, occlusions, and accessories such as hats and hair ornaments, making it difficult to obtain a reliable hair mask using pre-trained portrait or hair segmentation models. To improve hair region extraction capabilities in complex scenes, we introduce the segmentation extraction capabilities of the SAM3 [Carion et al., 2025]. Potential hair regions are coarsely located using text prompts (such as "hair of the people") to obtain a hair mask for selecting hair regions in the orientation field. For portraits that do not contain hair regions (e.g., wearing a hat or with no visible hair), we set their hair mask to a zero matrix to avoid introducing incorrect structural priors and ensure that the model maintains higher sensitivity to real hair samples during training.

The angle values of the HOF are determined based on the response of each pixel to different orientation filters, as illustrated in Figure 3. Specifically, we first convert image I to a grayscale image Ig. Subsequently, we use a set of differential Gaussian directional filters {Kθ}Nθ=1 with N = 32, where the corresponding orientations are uniformly sampled from [0, π), to compute the directional responses of the image:

where ⊗ denotes the convolution operator and Rθ(p) represents the response at pixel p under direction θ. To suppress non-structural noise, only positive responses are retained. The dominant orientation at pixel p is determined by the filter that yields the maximum response:

Meanwhile, the corresponding maximum response is used as the orientation confidence:

Since the above procedure produces discrete orientation labels and hair orientation exhibits periodic symmetry (i.e., θ and θ +π are equivalent), we map the discrete orientations to continuous two-dimensional direction vectors:

The direction vector field is then Gaussian smoothed to suppress local noise and enhance overall direction consistency. Only the direction vectors within the hair mask are retained, and encoded with the confidence wp into the feature vector.

Multi-level HOF Injection. We construct a HOF O to encode the continuous growth directions of hair strands. To effectively leverage this structural prior in diffusion-based matting, we introduce a multi-level HOF injection mechanism inspired by [Huang and Belongie, 2017] [Park et al., 2019] [Zhang et al., 2025] that aligns the orientation guidance with the hierarchical feature representations of the U-Net denoising network.

Specifically, given the HOF O, we employ a conditional encoder to extract a set of multi-scale orientation features F lo (l = 1, 2, 3) through progressive downsampling. These features are spatially aligned with the corresponding layers of the U-Net backbone, enabling orientation-aware constraints to be imposed throughout the diffusion denoising process. Low-level features capture fine-grained strand directions to stabilize local structures, while high-level features encode global hairstyle layouts to enforce long-range directional consistency.

To inject orientation cues into the diffusion network, we adopt a feature modulation scheme. For each U-Net feature F lu and its corresponding orientation feature F lo , a mapping network conditioned on F lo predicts a scale γ l and a bias βl , and the modulated feature is computed as

Here, ⊙ denotes element-wise multiplication, where γ l enhances structure-aware responses in hair regions and βl provides directional guidance without disrupting semantic representations. The residual form 1 + γ l ensures stable feature propagation during training.

By injecting orientation priors across multiple scales in a controlled manner, the diffusion model preserves its denoising capability while being explicitly guided toward structurally coherent hair alpha generation. This multi-level HOF injection enables more continuous and visually consistent hair structures in complex portrait scenes.

3.3 Pixel Color-anchored Local Structure Embedding

In matting regions where foreground and background colors are highly similar, alpha estimation based on color cues becomes unreliable. Although existing methods enhance discrimination using high-level Transformer features [Hu et al., 2024], alpha regression in portrait matting still fundamentally depends on relative intensity variations and local mixing relationships. Such fine-grained structural cues are often weakened by patch-based modeling and progressive semantic abstraction in Vision Transformers, and this issue is further aggravated by imperfect supervision, leading to blurred transition boundaries.

To mitigate this problem, we propose a Pixel Coloranchored Local Structure Embedding module that constrains the solution space using pixel colors as anchors while explicitly reinforcing local structural consistency. When color anchors become ambiguous, reliable alpha estimation must rely more on structural relationships within local neighborhoods. From the linear compositing model I = αF + (1 - α)B, the image gradient can be written as

When F and B are indistinguishable in color space, the contrast term (F -B)∇α diminishes, causing the image gradient ∇I to be dominated by foreground and background textures, making color differences insufficient for inferring alpha variations. Nevertheless, local texture and structural patterns still implicitly encode the spatial transition of pixel mixing, motivating us to constrain alpha prediction from a structural perspective. Specifically, we introduce Local Binary Pattern (LBP) structural cues, which capture relative intensity variations within local neighborhoods and are less sensitive to absolute color similarity. For a sampling direction ϕ on a circle of radius r, the difference response can be approximated as D(r,ϕ)I(x) ≈ I(x+ ruϕ) - I(x), uϕ = (cosϕ, sinϕ) (9) where uϕ denotes the unit vector along direction ϕ. This response reflects local intensity transitions and remains sensitive to alpha changes across unknown regions even under low-contrast conditions.

Since standard LBP is non-differentiable and scalefixed [Pietik¨ainen, 2010] [Ahonen et al., 2004], we design a continuous and learnable variant suitable for end-to-end training. For a gray scale image G, it is formulated as

where Gc and Gk denote the center and neighboring pixels at scale r, σ(·) is the Sigmoid function, and τ is a learnable temperature parameter controlling the smoothness of binarization. Multi-scale LBP responses are averaged to obtain the fused structural representation.

To further enhance structural representation, we introduce a shallow convolutional branch to learn complementary local patterns and concatenate them with the LBP responses to construct the final structural features. These structural features are then adaptively fused with semantic features via attention weighting and injected into the diffusion bottleneck through lightweight adapters.

4 Experiments

4.1 Implementation Details

Datasets and Evaluation Metric. The P3M-10K dataset [Li et al., 2021] is one of the most widely used portrait matting datasets, containing 9,421 high-resolution portrait images and alpha mattes. The HPM-17K dataset, introduced by MTGINet [Yang et al., 2025], contains approximately 17,000 half-body images and mattes. Both datasets cover a wide range of human poses and diverse real-world scenes. We conduct training and evaluation on both datasets to assess the performance of our method.

We evaluate our model using commonly adopted matting metrics [Rhemann et al., 2009] including Sum of Absolute Differences (SAD), Mean Squared Error (MSE), Gradient loss (Grad), and Connectivity loss (Conn). In regions where the ground-truth hair strands are inaccurate, we focus on generating visually coherent and naturally synthesized alpha mattes.

Training Details. Our conditional denoising network UNet consists of downsampling and upsampling convolutional blocks. Our model was trained for 500,000 steps on a single NVIDIA 4090 GPU with a batch size of 2 within 48 hours, and has 819 GFLOPs. For the loss function, we employ pixellevel loss composed of MSE, L1, gradient, and Laplacian losses followed [Hu et al., 2024]. During inference, our model requires approximately 1 second to process a single image.

4.2 Ablation Study

Effect of Modules. Table 1 shows that the basic generative framework alone performs poorly across metrics. Adding HOF consistently improves performance, indicating that explicit structural constraints enhance alpha continuity. However, orientation cues alone are insufficient for distinguishing regions with similar colors. Incorporating PCLSE further boosts performance, and even alone yields significant gains, demonstrating its effectiveness in improving alpha estimation in such ambiguous regions.

Figure 4 shows the results of the visualization ablation experiment. It can be seen that when the HOF injection module is removed (third column), the structural continuity of the hair region decreases significantly and local blurring occurs. After removing PCLSE (second column), blurring and misjudgment are more likely to occur at the foreground-background boundary. In contrast, the complete model OrienDiffusion maintains a more stable and natural alpha transition in complex boundaries, verifying the effectiveness of each module in detail restoration and structural consistency.

Effect of Hyperparameter β. As shown in Table 2, we analyze the effect of the bias term β used in the HOF injection. The results indicate that the model achieves consistently better performance across all metrics when β is used jointly with γ, compared to using either term alone. This suggests that β, serving as an orientation-guided compensation term, can effectively introduce structural priors when added to the weighted features especially around boundary regions.

Effect of Injection Mechanisms. We compare our injection method with two strategies: ControlNet-style zero convolution and feature concatenation. As given in Table 3, our injection method performs well. Specifically, we perform fine-grained modulation on hair-region features while preserving feature coherence. In contrast, ControlNet-style conditioning and feature concatenation lack fine-grained modulation and may disrupt structural consistency across the modulation process.

4.3 Comparative Experiments

Comparison on HPM-17K. We conduct comprehensive quantitative and qualitative comparisons with representative portrait matting methods, including MODNet [Ke et al., 2022], P3MNet [Li et al., 2021], MTGINet [Yang et al., 2025], and DiffMatte [Hu et al., 2024], covering both discriminative approaches and generative diffusion-based models. The quantitative results are summarized in Table 4. Our method achieves the best or second-best performance across the SAD, MSE, and Conn metrics, consistently outperforming existing approaches. These results demonstrate that the proposed OrienDiffusion is more effective in recovering continuous and natural alpha distributions, particularly in regions with fine-grained hair structures and severe foregroundbackground ambiguity.

It is worth noting that although MTGINet [Yang et al., 2025] reports relatively low errors on certain metrics such as Grad, its visual results still fail to achieve hair-level precision. As illustrated in the first row of Figure 5, structural discontinuities and hair breakage remain evident in fine-grained regions, indicating that its results struggle to preserve subtle hair structures. In contrast, despite potential discrepancies in quantitative evaluation, OrienDiffusion consistently produces more coherent and visually plausible alpha mattes, especially in delicate hair boundaries.

Comparison on P3M-10K. The P3M-10Kdataset presents a more challenging evaluation setting, containing a larger proportion of full-body portraits captured in complex real-world scenes. P3M-500-NP, which consists of normal portraits without occlusions, and P3M-500-P, where facial regions are partially occluded by masks. Our method outperforms the discriminative matting models in terms of quantitative metrics, indicating a higher level of foreground-background separation quality. We further compare the model complexity of different approaches. As reported in Table 5, SDMatte [Huang et al., 2025] achieves state-of-the-art quantitative performance but requires approximately 30× more parameters than our method. Its distilled variant, LiteSDMatte, still contains nearly 20× parameters while yielding inferior performance compared to OrienDiffusion.

Due to the generative nature of diffusion models, OrienDiffusion produces richer alpha details in high-frequency regions (e.g. hair). However, imperfect ground-truth annotations often fail to capture such fine structures, biasing quantitative evaluation; thus, metric gaps do not fully reflect detail recovery. As shown in the second and third rows of Figure 5, our method yields more natural and continuous alpha mattes on complex backgrounds, reduces hair breakage, and remains robust in challenging cases with similar foreground-background colors.

In addition, OrienDiffusion can be naturally extended to matting animal fur. We extract the unknown region of the animal trimap to extract the corresponding orientation field to guide inference. As shown in Figure 6, owing to the shared filamentary structures between human hair and animal fur, the model preserves directional consistency and produces smooth, coherent transitions in fine fur regions. Moreover, it maintains stable boundaries under complex backgrounds and varying illumination, demonstrating strong generalization capability beyond human hair.

5 Conclusion

In summary, we proposed OrienDiffusion, a generative portrait matting framework that incorporates explicit structural priors to address two long-standing challenges in portrait matting: structural discontinuity in fine hair regions and severe color ambiguity between foreground and background. By formulating matting as a conditional diffusion process, our method benefits from iterative error correction, which is particularly effective for recovering delicate and spatially coherent alpha structures. Specifically, we introduced HOF as geometric guidance to enforce spatial continuity of hair strands during denoising, and designed a PCLSE module with LBP-based structural constraints to enhance robustness in regions with similar foreground and background colors. Extensive experiments demonstrate that OrienDiffusion shows promising generalization to furry objects such as animals.

Nevertheless, like most existing matting methods, our method still relies on imperfect ground-truth alpha mattes for supervision, which may bias training toward numerical fidelity over perceptual quality, especially in fine hair regions. Moreover, the high computational cost (819 GFLOPs) limits the inference speed and practical deployment. Future work will explore human-preference-aware quality assessment or perceptual feedback as auxiliary supervision signals, and lightweight designs to improve both quality and efficiency.

Acknowledgments

This work was supported by the Beijing Natural Science Foundation (No. 4252029) and the National Natural Science Foundation of China (No. 62176010).