EKFEdit: Extended Kalman Filter for Training-Free Flow-Based Image Editing

1 Introduction

Rectified Flow (RF) models [Liu et al., 2022; Lipman et al., 2022; Albergo and Vanden-Eijnden, 2022] have recently emerged as a compelling alternative to Diffusion Models [Ho et al., 2020; Rombach et al., 2022] for text-to-image (T2I) generation. By constructing straight transport paths between noise and data distributions, RF models achieve superior generation quality with significantly fewer sampling steps. Building on pretrained large-scale RF models [Labs, 2024; Esser et al., 2024; Stability AI, 2024], training-free image editing aims to modify visual content via text guidance while preserving the structural fidelity of the source image.

∗Corresponding author.

Previous editing approaches typically follow an inversionbased paradigm, which first inverts the source image into a noise representation and then reconstructs it under the guidance of a target prompt. This pipeline faces a fundamental limitation: the inversion process inevitably introduces discretization error. Since the exact velocity field at previous timestep is unknown during inversion, local approximations must be used. In the absence of an effective feedback mechanism, these errors are not rectified but instead accumulate along the trajectory. Consequently, this leads to background distortion and loss of details in the output.

To address reconstruction degradation caused by inversion, recent research has explored inversion-free approaches, such as FlowEdit [Kulikov et al., 2025]. Instead of explicit inversion, these methods construct a direct transport path mapping the source distribution to the target distribution. By coupling velocity fields, they attempt to transport the image latent directly. Without accumulated inversion errors, inversion-free methods typically perform better at preserving unedited details. However, they suffer from trajectory instability and dynamic oscillation. Lacking a structural anchor to guide the generation process, the open-loop editing path becomes susceptible to layout collapse or semantic drift, with results often tending to be either under-edited or over-edited.

Thus, a critical trade-off persists in existing work. Inversion-based methods struggle with the loss of background details due to cumulative inversion errors. On the other hand, inversion-free approaches excel at preserving local details, but often suffer from structural inconsistency and semantic misalignment. This raises a core question:

Can we leverage the detail preservation of inversion-free dynamics while stabilizing their trajectories to ensure robust structural and semantic alignment?

To tackle this challenge, we propose Extended Kalman Filter Editing (EKFEdit), a novel framework that reformulates the image editing process as an optimal state estimation problem. We view the inversion-free trajectory as a noisy system dynamic and employ an Extended Kalman Filter (EKF) to stabilize it using outputs from auxiliary algorithms as observations. EKFEdit effectively corrects the deviations of the inversion-free path to realize its potential for high-fidelity editing while ensuring structural consistency. To ensure computational feasibility, we further implement a gradient-free Jacobian Diagonal Approximation, which enables efficient state estimation in high-dimensional latent space. Building on this approximation, we instantiate our framework with two variants, EKFEdit-RF and EKFEdit-DNA using RFSolver [Wang et al., 2024] and DNAEdit [Xie et al., 2025] as external observers, respectively.

In summary, our contributions are:

• We propose EKFEdit, a novel framework to interpret flow-based editing as an optimal state estimation problem. We instantiate it into two variants, EKFEdit-RF and EKFEdit-DNA, which achieve state-of-the-art performance by unifying the strengths of different editing methods.
• We develop an efficient, gradient-free EKF implementation using Jacobian Diagonal Approximation, minimizing the computational overhead for high-dimensional models.
• We further incorporate a Trajectory Alignment technique and dynamic noise scheduling into EKFEdit implementation, which effectively reduce ghosting artifacts and improve the editing performance.

2 RelatedWork

2.1 Inversion-Based Image Editing

To address reconstruction degradation, existing inversionbased methods focus on refining inversion accuracy. RFSolver [Wang et al., 2024] and FireFlow [Deng et al., 2024] adopt high-order solvers for accurate velocity field estimation to alleviate inversion errors. FTEdit [Xu et al., 2025] elevates the inversion precision via iterative average velocity. Other works [Zhu et al., 2025b; Avrahami et al., 2025; Xu et al., 2024; Yan et al., 2025] boost background consistency through complex model-specific attention mechanisms.

By contrast, DNAEdit [Xie et al., 2025] directly estimates initial noise in the noise space but still suffers from minor initial errors that amplify progressively during generation.

2.2 Inversion-Free Image Editing

To fully avoid inherent flaws of the inversion process, FlowEdit [Kulikov et al., 2025] pioneered constructing an Ordinary Differential Equation (ODE) that directly maps source and target distributions, without requiring explicit estimation of the initial noise. However, for complex scenarios, FlowEdit experiences editing trajectory drift due to inconsistent velocity directions, often causing source structure damage (over-edited) or editing failure (under-edited). To mitigate this, TweezeEdit [Mao et al., 2025] strengthens source preservation via regularization and SplitFlow [Yoon et al., 2025] decomposes prompts and flow fields and aggregates outputs to alleviate the under-edited versus over-edited tradeoff.

2.3 Control Theory in Generative Processes

Control theory has been effectively applied to model generative processes, with validated use in diffusion models [Berner et al., 2022; Koo et al., 2024; Rout et al., 2024b; Zhu et al., 2025a]. While previous flow-based control methods [Rout et al., 2024a; Zhou et al., 2025] focusing on optimizing the inversion process via linear quadratic regulators, FlowAlign [Kim et al., 2025] extends optimal control to inversion-free editing, enforcing trajectory smoothness to enhance source consistency.

While Kalman-Edit [Chi et al., 2025] introduces Kalman filtering to flow-based editing, it ignores time-varying covariance by employing a simple identity matrix as the state transition matrix and fundamentally acts as a regulator that tracks a static, pre-computed reference trajectory. This reliance on a decoupled reference fails to adapt to real-time deviations, often introducing artifacts when the editing direction conflicts with the reference.

In contrast, EKFEdit reformulates editing as an optimal state estimation problem within a nonlinear stochastic system. By leveraging the EKF to linearize system dynamics, our method enables real-time adaptive correction of trajectory drift, fundamentally diverging from the rigid trajectory tracking of the prior work.

3 Method

3.1 Preliminaries

Rectified Flow Models. Rectified Flow (RF) [Lipman et al., 2022; Liu et al., 2022] learns a transport map between distributions π0 and π1 via an ODE on t ∈ [0, 1]:

where vθ is a neural velocity field. To ensure stable training and efficient sampling, RF enforces straight-line trajectories via linear interpolation zt = (1 - t)z0 + tz1.

Inversion-Based Image Editing. These methods seek to recover a latent noise zT that reconstructs the source image.

However, the discrete inversion usually approximates the velocity at the unknown state zk using the known state zk+1:

This approximation introduces local truncation errors that accumulate along the trajectory. Consequently, the inverted noise diverges from the standard Gaussian (zT /∈ N(0, I)), leading to background distortion and detail loss in the final output.

Inversion-Free Editing. Leveraging the linear property of Rectified Flow, inversion-free methods, such as FlowEdit [Kulikov et al., 2025], bypass latent inversion by constructing a transition path directly in the image space. The latent in the direct path, Z FE t , is approximated via parallelogram law:

This trajectory evolves under an ODE driven by the relative velocity difference:

Here, the pseudo-source ˆ Z src t is linearly interpolated from the known source Xsrc , while the unknown pseudo-target is approximated using the current state: ˆ Z tar t ≈ Z FE t + ˆ Z src t - Xsrc . Iteratively updating Z FE t using this velocity difference gradually maps the source image toward the target. This process is shown in Figure 2 (a).

Extended Kalman Filter. The Extended Kalman Filter (EKF) is a recursive estimator for nonlinear dynamical systems, which propagates uncertainty by locally linearizing the transition and observation functions. Consider a discrete-time nonlinear system:

where xk denotes the system state, zk is the observation, and f(·), h(·) are the nonlinear transition and observation function, respectively.

EKF linearizes these functions via first-order Taylor expansion, computing Jacobian matrices Fk = ∂f ∂x |ˆxk-1 and Hk = ∂h ∂x | ˆx-k . Here, Pk denotes the posterior state estimation error covariance, P-k denotes its predicted counterpart, while Qk and Rk denote the process and observation noise covariances, respectively. State estimation proceeds recursively: Prediction:

Update:

3.2 Overall Framework of EKFEdit

Existing flow-based image editing methods operate fundamentally in an open-loop manner. Without feedback to actively correct deviations, inversion-based approaches accumulate errors over the trajectory. Inversion-free methods assume that the coupled velocity fields would produce the expected output, but do not account for the discrepancy between actual and expected outputs. In practice, the lack of explicit constraints often leads to trajectory drift and makes it hard to reach the optimal balance between text adherence and structural preservation.

To address this limitation, we argue that image editing should not be viewed as a rigid trajectory generation task, but rather as an optimal state estimation problem within a stochastic dynamic system. Under this perspective, the inversion-free trajectory can serve as a noisy prior of the system's state, while outputs from auxiliary editing algorithms provide external observations.

Based on this insight, we propose EKFEdit, a unified framework that employs the Extended Kalman Filter to dynamically fuse these two information sources. As illustrated in Figure 2 (b), EKFEdit adaptively weights the predicted state with external observations via Kalman gain. This method transforms the editing process into an estimation system that dynamically corrects trajectory drift and oscillations of the inversion-free prior. Moreover, it avoids reliance on background information embedded with accumulated errors unlike inversion-based methods.

Since generative flow models denoise from t = 1 to t = 0, we apply EKF on the reverse timeline, where each iteration estimates the transition from tk to tk-1 through two steps:

• Prediction Step: This step predicts a prior state transition using the inversion-free trajectory as the system dynamics.
• Update Step: This rectifies the predicted state by fusing it with external observations via the Kalman gain, dynamically injecting semantic guidance to correct trajectory drifts.

3.3 State Space Modeling on Latent Space

We define the high-dimensional latent representation of the image as system state xk, which is sequentially estimated by fusing predictions with observations.

Prediction with Inversion-Free Dynamics. Following the approach in FlowEdit [Kulikov et al., 2025], we treat the coupling velocity field associated with the direct path from source image distribution to target image distribution as the system's state transition function. The prediction step aims to estimate the prior state ˆx-k-1 at timestep tk-1 based on the state at tk, which is defined as:

where ∆t = tk-1-tk, and v∆ represents the relative velocity difference:v∆ = vθ(ˆztar k , ctar ; tk) - vθ(ˆzsrc k , csrc ; tk), (9)

where ˆztar k denotes the intermediate state of the noise-to-target trajectory estimated via the parallelogram law: ˆztar k = ˆxk + ˆzsrc k - xN, and ˆzsrc k is obtained by forward process as ˆzsrc k = (1 - t)xN + tNt, where Nt is a random Gaussian noise.

The predicted error covariance P-k-1 follows Eq. (6), with the transition Jacobian Fk = ∂f ∂x |ˆxk = I + ∆t ∂v∆ ∂x |ˆxk .

Update using Independent Observation Fusion. To correct drift in predicted trajectories, we introduce an independent observer module Mobs, which produces a target-aware latent observation at the same timestep as the state to be estimated. Given its internal state xobs k , the current EKF state ˆxk, and the source-target prompts, the observer outputs

The observation lies in the same latent space as xk-1 and serves as an alternative estimate carrying the target semantics.

We define the observation function as direct observation, given by zk-1 = Ixk-1 + νk-1. State update is completed through Kalman gain Kk-1:

It indicates that the posterior state is a weighted sum of prediction and observation, with weights dynamically determined by K. The final edited state is equivalent to the last posterior state ˆx0.

3.4 Jacobian Diagonal Approximation

Diagonal Assumption via Locality. The state dimension in diffusion models is typically very large (d ∼ 104 -105 ), making the construction and propagation of a full-rank Jacobian computationally prohibitive. Fortunately, recent studies [Kamb and Ganguli, 2025; Lukoianov et al., 2025] show that diffusion denoisers exhibit strong spatial locality: each output element is primarily sensitive to a limited neighborhood of the input. This property suggests that the dominant local dependency can be captured by diagonal or near-diagonal Jacobian components. We empirically verify this observation in Figure 3 on two representative T2I backbones, SD3.5 [Stability AI, 2024] and Flux [Labs, 2024]. The token-level Jacobian energy maps show that gradient energy is concentrated around aligned input-output tokens in both models. Motivated by this locality, we assume approximate independence between state dimensions within minute timesteps and estimate only the diagonal entries of Fk.

Gradient-Free Approximation. We estimate the diagonal of the transition Jacobian with a finite-difference approximation, avoiding memory-intensive backpropagation:

In practice, we set u = vθ(ˆztar k , ctar ; tk), use a small ϵ, and clip the estimated values for stability. This requires only one additional inference step.

Tensorized Implementation. Consistent with the diagonal assumption, we model P, Q, R as diagonal tensors matching the shape of x. This reformulation converts matrix multiplications into element-wise operations, significantly boosting inference efficiency by reducing space complexity from O(d2 ) to O(d).

3.5 Instantiation of EKFEdit

A simplified overview of the EKFEdit procedure is provided in Algorithm 1. Following previous work [Kulikov et al., 2025; Xie et al., 2025], we keep the optional start step Ts to control when to perform editing.

Algorithm 1 EKFEdit Framework

Input: Source image Isrc, Source/Target text csrc, ctar, Number of sampling steps N, Start Step Ts, Process/Observation Noise Q, R, Pinit

Output: Itar

• 1: zsrc ←VAE(Isrc);
• 2: ˆxN ← zsrc; zobs N+1 ← zsrc; PN ←Pinit;
• 3: Init xobs N according to specific Observer.
• 4: for k = N -Ts down to 1 do
• 5: // Prediction Step
• 6: ˆx-k-1 ← fflow(ˆxk, zsrc,∆t) Eq. (8)
• 7: Fk ←DiagJac(ˆxk, ϵ) Eq. (12)
• 8: P-k-1 ←FkPkFTk +Qk
• 9: // Update Step
• 10: zobs k-1 ← Mobs(xobs k , ˆxk, csrc , ctar , tk) // varies with the observer implementation
• 11: Kk-1 ←P-k-1 (P-k-1 +Rk-1)-1
• 12: ˆxk-1 ← ˆx-k-1 +Kk-1(zobs k-1 - ˆx-k-1 )

13: Pk-1 ← (I -Kk-1)P-k-1

• 14: xobs k-1 ← Align(xobs k , ˆxk-1) Eq. (14)
• 15: end for
• 16: return VAEdec(ˆx0)

Dynamic Noise Schedule. Following prior observations that velocity estimation is less reliable at high noise levels [Kulikov et al., 2025; Xie et al., 2025], we use timevarying diagonal covariances:

This schedule assigns larger process uncertainty in early steps and gradually reduces the reliance on external observations in later steps, mitigating accumulated observation errors.

Trajectory Alignment. Direct fusion of the predicted state ˆxk and the external observation can lead to ghosting artifacts when their underlying velocity fields conflict. Inspired by DNAEdit, we align the internal state of observer xobs t towards the posterior state ˆxt to ensure geometric consistency before each prediction loop:

where vtar t denotes the target velocity estimated by the external observer at time t, and η ∈ [0, 1] is a balancing factor. This constraint forces the observation path xobs to geometrically converge towards the predicted trajectory ˆx, mitigating directional conflicts.

EKFEdit-RF. We first implement EKFEdit with the RFSolver [Wang et al., 2024] as the observer. During the RFSolver inversion stage, we cache the source velocity vsrc t along the inversion trajectory, and then compute the target velocity vtar t from the aligned observer state during the EKFEdit loop. All velocity fields are estimated by the Second-order Solver within RF-Solver, and the attention injection mechanism is retained in the update of zedit to enhance structure preservation. The observation is constructed as:

EKFEdit-DNA. We further adapt the framework to DNAEdit [Xie et al., 2025]. In this variant, the observer first performs Direct Noise Alignment to obtain a noisealigned trajectory and cache the DNA correction terms. During the EKFEdit loop, the target velocity vtar t is computed on the DNA-specific residual-corrected latent, while the cached DNA correction serves as the source-side term vsrc t . EKFEditDNA therefore inherits the same observation construction Eq. (15) and alignment Eq. (14), but obtains its observation from DNAEdit-specific noise alignment rather than RFSolver inversion. Please refer to the project page for details.

4 Experiments

4.1 Implementation and ComparedMethods

Implementation Details. We employ the pretrained SD3.5-medium [Stability AI, 2024] as the base T2I model. For our methods, we set the sampling steps to N = 50 and start step Ts = 17 with hyperparameters η = 0.8, λ1 = 0.1, and λ2 = 1.0. For all compared methods, we adopt the default settings in their official implementations. Image resolution is fixed at 512 × 512. Classifier-Free Guidance (CFG) [Ho and Salimans, 2022] is important for balancing editing effects, where higher values enhance image-text alignment but often compromise background preservation. To ensure a fair comparison, we group experiments by different CFG settings to fairly evaluate background preservation under similar text adherence levels.

Compared Methods. We benchmark against current advanced training-free flow-based methods, covering inversionbased RF-Inversion [Rout et al., 2024a], RF-Solver [Wang et al., 2024] and DNAEdit [Xie et al., 2025], as well as inversion-free FlowEdit [Kulikov et al., 2025] and FlowAlign [Kim et al., 2025].

4.2 Qualitative Comparison

Figure 4 shows the visual comparison between our method and competitive methods. In general, our methods achieve better structural and background preservation while accurately executing the target edits. In contrast, FlowEdit shows clear structural deviations, such as altering the object pose in the bear case. RF-Solver fails to effectively preserve background details, while DNAEdit exhibits editing failures, resulting in either failure to generate the expected content (white dog, cat → husky, lion) or severe grid-like artifacts (cat, crown → rabbit, hat). The failure case of DNAEdit is partially inherited by EKFEdit-DNA, reflecting that EKFEdit cannot fully reject arbitrary observation errors. Nevertheless, EKFEdit is not a pure observer-following scheme: the Kalman gain downweights observations in stable regions with low predicted uncertainty, but may still preserve observer influence when errors overlap with regions requiring substantial semantic changes. Similarly, in the photo →watercolor case, the orientation of the woman's face remains consistent with the corresponding observer methods, empirically validating that observations provide significant semantic guidance within the EKFEdit framework.

photo of a woman → water color of a woman steak by a side of salad→ quiche by a side of salad

Table 1: Quantitative comparison on FlowEdit dataset. Red ( ), blue ( ), yellow ( ) represent the top 3 performers.

4.3 Quantitative Comparison

Dataset. We conduct quantitative evaluations on the FlowEdit dataset [Kulikov et al., 2025], which consists of 76 diverse real images. Each image is paired with a source prompt and several handcrafted target prompts, forming 280 text-image pairs in total. Additionally, we utilize PIEBench [Ju et al., 2023], which contains 700 synthetic and natural images with short paired prompts.

Metrics. To quantitatively evaluate editing quality, we adopt metrics consistent with existing benchmarks: MSE, PSNR [Gonzalez, 2009], SSIM [Wang et al., 2004] and LPIPS [Zhang et al., 2018] for background preservation; Structure Distance [Ju et al., 2023] for structural integrity; and CLIP Similarity Score (CLIP) [Wu et al., 2021] for semantic adherence between edited images and target text.

Results. Table 1 reports the quantitative performance of our method and baselines on the FlowEdit dataset, as well as the average rank of each method across all metrics. All methods within each CFG group yield comparable CLIP similarity scores 1 for fair comparison. Despite achieving the highest CLIP scores, FlowEdit consistently obtains the lower PSNR, which stems from the absence of trajectory constraints and consequent semantic drift in non-edited regions. Other methods fail to achieve top-2 performance across all metrics. In contrast, our method achieves top-3 CLIP scores while outperforming all baselines on all four background preservation metrics and Structure Distance, fully demonstrating the robust performance of our method. Quantitative performance on PIE-Bench is presented in Table 2. In this table, CLIPW refers to the CLIP similarity between the whole image and the target text, and CLIPE corresponds to the CLIP similarity calculated on the masked edited area.

1 FlowAlign failed to achieve a higher CLIP score even when tuning the CFG scale across a wide range of values.

4.4 Ablation Study

Effectiveness of Kalman Fusion. Table 3 validates the superiority of EKF fusion. Unlike Static Fusion (ˆxk = ˆx-k + 0.5(zobs k - ˆx-k )) which yields suboptimal results using fixed weights, our method adaptively fuses prediction and observation with Kalman gain, achieving superior performance in semantic alignment and structural preservation.

Effectiveness of Key Components. Table 4 validates the efficacy of key components in EKFEdit: Jacobian Diagonal Approximation, Dynamic Noise Schedule and Trajectory Alignment. First, regarding linearization, replacing our Jacobian Diagonal Approximation with an Identity Matrix leads to performance degradation, confirming the necessity of capturing local non-linearities. Second, improvements across metrics confirm the effectiveness of our Dynamic Noise Schedule. Notably, the Reversed Schedule, which increases the process noise and reduces observation noise sequentially, results in structural degradation, indicating that the estimator should prioritize prior dynamics over observations in later stages. Finally, Trajectory Alignment effectively mitigates ghosting artifacts caused by conflicting velocity fields between the prior and observer, as visualized in Figure 5.

5 Conclusion

In this paper, we introduce EKFEdit, a unified framework that reformulates training-free flow-based image editing as a stochastic state estimation problem. By dynamically fusing inversion-free priors with external observations via the Extended Kalman Filter, our approach effectively balances the trade-off between text adherence and structural preservation. We further enable efficient high-dimensional inference through a gradient-free Jacobian diagonal approximation. To demonstrate the framework's versatility, we present two concrete instantiations, EKFEdit-RF and EKFEdit-DNA. Extensive experiments on standard benchmarks confirm that both variants consistently improve the balance between semantic alignment and structural preservation.

A current limitation of EKFEdit is its additional inference overhead from joint prediction-observation execution, which we aim to reduce via parallelization in future work.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62495084 and 62473307).