SirenFNO: Efficient and Full Frequency Learning of Fourier Neural Operators

1 Introduction

Operator learning aims to approximate mappings between infinite-dimensional function spaces, with applications across scientific computing and physical simulation. Recently, neural operators (NOs) [Lu et al., 2021; Li et al., 2021; Kovachki et al., 2023] have been used to efficiently solve families of partial differential equations (PDEs), including weather forecasting [Pathak et al., 2022], engineering design [Liu et al., 2023], forward/inverse problems [Raissi and Karniadakis, 2018; Raissi et al., 2019].

The Fourier Neural Operator (FNO) [Li et al., 2021] is a class of NOs that has proven highly effective and efficient for learning PDE solution operators. Specifically, FNO computes the kernel integral in the frequency domain by applying the Fast Fourier Transform (FFT), performs a learned convolution there, and then maps back to the spatial domain via the inverse FFT (IFFT). In practice, instead of learning the convolution weights for every frequency exhaustively, FNO performs frequency truncation with a chosen threshold to retain only low-frequency modes and discard high-frequency components. This yields certain computational savings, fewer parameters, and near-discretization invariance. However, the use of frequency truncation may fail to preserve the equivalence between the continuous and discrete representations of neural operators [Gao et al., 2025], and hinder the model's ability to capture high-frequency details in certain PDEs with strong high-frequency oscillations [Xiao et al., 2024].

To address the aforementioned critical limitations that frequency truncation imposes on the FNO, we propose to parameterize the Fourier kernel via a hypernetwork. This hypernetwork performs mode-wise kernel weight generation, dynamically producing kernel parameters for each frequency mode without any explicit truncation, thereby enabling learning across arbitrary discretization resolutions. More importantly, as the spectral kernel learns a separate weight for each Fourier mode, the number of learnable kernel parameters in the plain FNO is determined by the modes retained after truncation, and without truncation it scales with the grid resolution. In contrast, in our approach, the number of parameters solely depends on the hypernetwork architecture and remains fixed across different resolutions.

Specifically, we use SIREN [Sitzmann et al., 2020] as the hypernetwork to generate the Fourier kernel coefficients. Unlike conventional alternatives such as MLPs, which exhibit spectral bias toward low-frequency modes [Rahaman et al., 2019; Qin et al., 2024; Zhi-Qin John Xu et al., 2020], SIREN's sinusoidal activations efficiently capture both highand low-frequency information. Therefore, SIREN provides a smooth and continuous parameterization that is agnostic to any specific discretization and less susceptible to spectral bias, making it a well-suited hypernetwork for the FNO integral kernel. Furthermore, SIREN efficiently generates Fourier kernel coefficients with a relatively small number of learnable parameters, and we further improve its efficiency by incorporating functional tensor decompositions [Vemuri et al., 2024; Vemuri et al., 2025].

The main contributions of this work are as follows:

• We propose the Siren-FNO framework, which utilizes

(b) Trained on 32 × 32 and tested on 64 × 64

Figure 1: Zero-Shot Super-Resolution on the Darcy Flow for SirenFNO trained on 32 × 32 resolution.

SIREN as a hypernetwork to generate convolution kernel weights for all frequency modes, avoiding spectral truncation, while maintaining computational and parameter efficiency.

• We extend the Siren-FNO framework with functional tensor decompositions of the Fourier integral kernel, specifically Canonical Polyadic (CP), Tensor-Train (TT), and Tucker decompositions.
• We demonstrate the effectiveness and resolutionagnostic behavior of the proposed methods across multiple PDE benchmarks.

2 RelatedWorks

Neural Operators. Neural operators learn mappings between continuous function spaces and are often used for solving PDEs. For practical implementation, these continuous functions are generally represented on discretized grids. DeepONet [Lu et al., 2021] employs two sub-networks: a branch network that encodes input function values at fixed sensor points and a trunk network learns the mapping between query locations to a set of basis functions. The outputs from these two networks are then combined to approximate operators, with guarantees from the universal approximation theorem. Laplace Neural Operator (LNO) [Cao et al., 2024] uses the Laplace transform to decompose the input space and introduces Laplace layers to better learn nonperiodic signals in the Laplace domain. Moreover, attention mechanisms have also been explored for operator learning [Cao, 2021]. Operator Transformer (OFormer) [Cao, 2021] is a transformer-based neural operator framework that employs an input encoder to embed function samples and a query encoder to project arbitrary output locations. General Neural Operator Transformer (GNOT) [Hao et al., 2023] introduces a heterogeneous attention operator to process diverse input function types. Our work focuses on the Fourier neural operators [Li et al., 2021].

Fourier Neural Operators. The FNO [Li et al., 2021] defines its convolution kernel in the Fourier space and uses Fast Fourier Transform (FFT) to learn the operator efficiently. Several works seek to further enhance the efficiency of the vanilla FNO. Factorized FNO (F-FNO) [Tran et al., 2023] deepens the FNO architecture to capture long-range dependencies and factorizes layers to learn each feature dimension efficiently with fewer parameters. Adaptive FNO (AFNO) [Guibas et al., 2021] utilizes MLP layers with learnable weights adaptively shared across frequency modes to reduce computational complexity. Beyond efficiency, motivated by the low-frequency bias of FNOs, a number of studies aim to advance its high-frequency learning ability. U-FNO [Wen et al., 2022] incorporates a U-Net pathway to each Fourier layer to enhance high-frequency learning. Amortized FNO (AMFNO) [Xiao et al., 2024] introduces a learnable amortized parameterization of the Fourier-kernel integral that captures high-frequency information without explicit truncation.

Dynamic Hypernetworks. Hypernetworks (hypernets) are neural networks that generate learnable weights of a primary neural network. Hypernets can be categorized by their weight-generation strategy as either static or dynamic [Chauhan et al., 2024]. A hypernet is static if its conditioning inputs and outputs have fixed dimensions for a predetermined target architecture. In contrast, dynamic hypernets are used to generate model weights for a primary network with a dynamic architecture, which means that the structure of the primary neural network could change or be unknown during the training and inference stages. Dynamic hypernets have been leveraged for shape reconstruction [Littwin and Wolf, 2019], automatic network pruning [Li et al., 2020], or efficient architecture search [Peng et al., 2020]. In this work, we adopt a dynamic hypernetwork to enable parameterization generalization across discretizations.

Tensor Decomposition for Neural Operators. For operator learning, tensor decomposition is often used to separate PDE dimensions or variables for improving parameter efficiency. TDMD-DeepONet [Chen et al., 2025] integrates tensor-train and dynamic mode decompositions into DeepONet for solving time-dependent PDEs. TensorGRaD [Loeschcke et al., 2025] directly performs Tucker decomposition on the gradient tensor. F-FNO [Tran et al., 2023] factorizes the Fourier transforms with separable spectral layers along the problem dimensions. D-FNO [Li and Ye, 2025] decomposes latent features into a sum of rank-1 tensor products, and substitutes the 3-D FFT by multiple 1-D FFTs, improving efficiency on 3-D PDE tasks. MG-TFNO [Kossaifi et al., 2023] employs tensor decomposition in a multi-grid setting to separate FNOS Fourier convolution weights for greater parameter efficiency.

3 Preliminaries

This section describes the setting of the Fourier neural operator and its frequency truncation in detail.

3.1 Fourier Neural Operators

For an input function space A = A(D;Rda ) and a target function space U = U(D;Rdu ) on a bounded domain D ⊂ Rd , operator learning aims to learn a neural operator Gθ with parameters θ ∈ Rp to approximate the ground-truth mapping G : A → U from a set of observed function samples {ai, ui}Ni=1 . For a loss function L, we learn the neural operator by solving the problem,

Neural operator (NO) takes the following composition form,

where Q : A(D,Rda ) → U(D,Rdv ) is the lifting operator and P : U(D, Rdv ) → U(D, Rdu ) is the projection operator, and each NO layer Ll for l = 1, 2, . . . , L is defined as,

where σ is nonlinear activation, W ∈ Rdv×dv is a learnable weight matrix, and κ is the integration kernel.

Fourier Neural Operator (FNO) [Li et al., 2021] proposes a Fourier integral operator by employing convolution operation in the Fourier domain, satisfying translation invariance that κϕ(x, x′ , a(x), a(x′ )) = κϕ(x - x′ ). Applying the convolution theorem, the Fourier integral kernel is then defined as ∫D κϕ(x - x′ )dx′ = F -1 (F(κϕ) · F(vl ))(x), ∀x ∈ D Then, FNO parameterizes the periodic kernel function κ in the Fourier domain by Rϕ(k) ∈ Cdv×dv such that Rϕ(k) = F(κ)(k) for k ∈ Zd .

3.2 Frequency Truncation

Low-frequency components generally have larger magnitudes than high-frequency components in many PDE problems [Boffetta and Ecke, 2012; George et al., 2024; Jing et al., 2025]. To prioritize the learning of low-frequency modes and improve parameter efficiency, FNO truncates the Fourier spectrum by discarding high-frequency components with a predetermined number of retained frequency modes kmax = |{k ∈ Zd : |kj| ≤ kmax,j for j = 1, . . . , d}|. In this case, we have learnable kernel parameters Rϕ ∈ Ckmax×dv×dv , and the size of complex-valued tensor Rϕ can be greatly reduced owing to frequency truncation. Then, FNO performs zeropadding on discarded modes and applies an IFFT to recover the solution field in the physical domain.

By parameterizing only the retained low-frequency components, FNO achieves superior efficiency and captures dominant components well. However, FNO exhibits Fourier parameterization bias or spectral bias due to its inability to capture non-dominant frequencies effectively [Qin et al., 2024]. Moreover, the predetermined kmax imposes inductive bias and may degrade model performance [George et al., 2024].

4 The ProposedMethod

In this work, we propose SirenFNO, a novel framework that uses SIREN [Sitzmann et al., 2020] to generate kernel coefficients mode by mode for all discrete Fourier components, eliminating the need for truncation. A key advantage of SirenFNO is that the number of learnable parameters is fixed by the SIREN architecture and does not depend on the grid resolution. Thereby, our proposed SirenFNO efficiently represents both lower- and higher-frequency details across varying discretizations.

4.1 Parameterization of Fourier Kernel via SIREN

SIREN [Sitzmann et al., 2020] employs sinusoidal activations to to learn continuous implicit representations. In our parameterization, each discrete Fourier mode is indexed by a (normalized) frequency coordinate ξk ∈ [-1, 1]d for k ∈ Zd . A SIREN network Φθ : [-1, 1]d → Cdv×dv maps ξk to the corresponding complex Fourier kernel coefficient matrix. For a batch of modes K at a given resolution, stacking inputs yields outputs in C|K|×dv×dv the set size |K| is adaptively determined by the discretization. Because all frequency coordinates are normalized to [-1, 1], the network is insensitive to changes in grid resolution. Specifically, for each k, the coefficients are produced by a SIREN stack,

where EB : Rd → R2m is a learnable random Fourier feature (RFF) embedding of the input frequency coordinates, defined as

where m is the number of RFF frequencies, and B is a learnable parameter initialized as B(0) = σG, with Gij i.i.d. ∼ N(0, 1) and σ > 0.

Then, we have ϕ(1) : R2m →Rh , and ϕ(l) : Rh →Rh for l = 2, . . . L. Each SIREN layer ϕ(l) in the above (5) takes the following form,

where a factor w is used in each SIREN layer to accelerate convergence and span multiple periods of the sine basis, enabling our parameterization method to efficiently represent high-frequency information. Finally, O : Rh → Cdv×dv is an MLP layer.

Now, we replace the discreted kernel coefficients Rϕ(k) ∈ Cdv×dv in (4) with coefficients Φθ(ξk) ∈ Cdv×dv generated by the above SIREN parameterization, our SirenFNO layer is formulated as,

In contrast to FNO, which directly maintains and optimizes a large discretization-dependent tensor of Fourier kernel coefficients, our SIREN parameterization learns a continuous mapping from frequency coordinates to kernel coefficients. This eliminates the need to store all coefficients explicitly, and they are generated on demand for each frequency mode instead. As such, SirenFNO can efficiently learn the Fourier integral operator over all frequency modes without frequency truncation, and the number of learnable parameters is independent of the discretization resolution, determined solely by the SIREN architecture. Figure 2 illustrates the process of parameter generation by our kernel parameterization.

4.2 Functional Tensor Decomposition for SirenFNO

For SirenFNO without truncation, we construct input coordinate points for SIREN block Φθ : [-1, 1]d → Cdv×dv on a full d-dimensional domain, and the SIREN block is shared across all resolution grids with N points per axis. Therefore, for a d-dimensional PDE discretized on a K = N d grids , the construction of input coordinates for SIREN takes up O(N d × d) memory, which requires excessive memory usage, especially for finer grids and high-dimensional PDE applications. To alleviate this, we further extend SirenFNO by leveraging functional tensor decompositions.

In this work, we employ three tensor decomposition schemes, Canonic-Polyadic (CP), Tensor-Train (TT), and Tucker, within SirenFNO. In the functional tensor variants, the kernel is parameterized by factorizing the SIREN trunk into d branch sub-networks, one per geometric dimension. Instead of sampling a full N d grid, each branch processes N coordinates for its dimension, yielding a factor vector/matrix; the full tensor of Fourier-kernel coefficients is then reconstructed via tensor (outer) products. This reduces the sampling requirement from N d points to N × d and the memory footprint from exponential to linear in d, i.e. O(Nd).

CP-SirenFNO. With CP rank R ≪ mini∈[d] Ni, the kernel tensor Φθ ∈ CN1×···×Nd is parameterized as a sum of R rank-1 outer products. SirenFNO realizes this by using d SIREN branches to produce factor matrices A(i) ∈ RNi×R for i = 1, . . . , d. Let a(i) r := A(i) (:, r) ∈ RNi denote the r-th column. The reconstruction is

where ⊗ denotes the outer (tensor) product. This reduces storage from ∏di=1 Ni to O(R ∑di=1 Ni ) while enabling ondemand coefficient generation.

TT-SirenFNO. With TT ranks R0 = Rd = 1 and R1, . . . ,Rd-1, SirenFNO uses d SIREN branches to produce cores G(i) ∈ CRi-1×Ni×Ri for i = 1, . . . , d. Let g(i) ri-1,ri ∈ CNi denote the mode-2 fiber of G(i) with TT indices (ri-1, ri) fixed. The kernel tensor Φθ ∈ CN1×···×Nd is reconstructed as

This representation reduces storage from ∏di=1 Ni to O( ∑di=1 NiRi-1Ri ).

Tucker-SirenFNO. Given Tucker ranks R1, . . . ,Rd, SirenFNO employs d SIREN branches to produce factor matrices A(i) ∈ CNi×Ri for i = 1, . . . , d, together with a learnable core C ∈ CR1×···×Rd . The kernel tensor Φθ ∈ CN1×···×Nd is reconstructed via the Tucker product

where ×i means mode-i tensor-matrix product.

For SirenFNO with functional tensor decompositions, these factor matrices {A1 , . . . ,Ad } are generated and parameterized using a number of d small SIREN networks {Φ1θ1 , . . . , Φdθd } accordingly. The reconstructed tensor is then used as Fourier kernel coefficients in the FNO. Figure 3 illustrates an example of SIREN parameterization with functional tensor decompositions.

4.3 Model Architecture

Our SirenFNO follows the standard FNO architecture,

where Q is the lifting layer to transform input data α to hidden state z, and P is the projection layer to project hidden state z to output prediction u. Furthermore, intermediary layers Ll for l = 1, . . . ,L are our SirenFNO layers introduced in (8). Now, we use KΦ(v(l) (z)) := F -1 (Φ(l) θ (ξk) · F(v(l) )(k))(z) to denote our SirenFNO parameterization kernel, we have

In this work, we leverage the same model architecture as AMFNO [Xiao et al., 2024; Tran et al., 2023], using residual connections and replacing linear mapping W l vl (z) such that

It is worth noting that each SirenFNO operator layer includes additional nonlinearities after the residual connection, and the last operator layer omits the activation after the residual connection.

5 Experiments

This section describes our experimental setups and results. All experiments are conducted on the same machine equipped with an NVIDIA GeForce RTX 5090. Our Code is available at https://github.com/pengqingshi/SirenFNO.

5.1 Benchmarks and Baselines

We compare our proposed models against FNO [Li et al., 2021] and its several well-known variants, including UFNO in [Wen et al., 2022], Tensorized FNO with CP decomposition (TFNO-CP) [Kossaifi et al., 2023], and AM-FNO with MLPs parameterization [Xiao et al., 2024], and UNO in [Rahman et al., 2023].

Table 1 lists the benchmark datasets used in our experiments. Darcy and Navier-Stokes datasets are obtained from the NeuralOperator library [Kossaifi et al., 2024; Kovachki et al., 2023]; and 1D Burgers 1 , 1D CFD 2 , 2D CFD 3 , and reaction-diffusion 4 are sourced from PDEBench [Takamoto et al., 2024].

5.2 Implementation Details

The ℓ2 relative error is utilized for training and evaluation. Given data pairs {ai, ui}Ni=1 and a model Gθ : A → U, the ℓ2 relative error is computed as,

For each experiment, we conduct 500 training epochs using AdamW optimizer with a batch size of 32, a learning rate of 1e-3 with the cosine annealing schedule and a weight decay of 1e-4. We set 4 layers with 32 hidden channels for FNO and its variants by default, including our SirenFNO. The hidden channels for lifting and projection layers are set to 64 by default. For FNO frequency truncation, we retain modes of mx = my = 32 for Navier-Stokes and 2D CFD, and mx = my = 16 for the Darcy flow. We learn full frequency for FNO on Burgers, 1D CFD, and reaction-diffusion. Please note that our proposed SirenFNO and its variant forms do not perform frequency truncation. Additionally, We evaluate time-dependent PDEs in an one-step autoregressive scheme, where we perform 10 rollout steps with previous 10 states for Burgers, reaction-diffusion, and 1D CFD; and we conduct 5 rollout steps with previous 5 states for 2D CFD.

5.3 Main Results

Table 2 reports the ℓ2 relative test set error for each PDE benchmark, with all models trained and tested at identical discretization resolutions. Table 3 lists the number of learnable parameters in thousands for every neural operator in each experiment. For each problem, we boldface the lowest ℓ2 relative test error and the smallest parameter count.

As shown in Table 2, our SirenFNO and its functional decomposition variants consistently surpass the vanilla FNO across all PDE benchmarks. Moreover, as indicated in Table 3, they achieve remarkably greater parameter efficiency without relying on frequency truncation.

For the 2D Darcy flow problem, our SirenFNO reduces the ℓ2 relative test error from 0.0730 to 0.0351 (by 52%) and from 0.0626 to 0.0236 (by 62%) for discretizations of 32 × 32 and 128 × 128 respectively, with an approximately 4× reduction in model parameters, compared to FNO. Moreover, CP, TT, and Tucker variants maintain comparable performance while further reducing the number of parameters. For solving the 2D Navier-Stokes equation, our SirenFNO obtains a ℓ2 relative error of 0.0351, compared to that of 0.0561 for FNO; while using about 8 times fewer parameters. Notably, our TT-SirenFNO achieves the best overall performance on the Navier-Stokes benchmark, with a further reduction in parameters by 21 times compared to FNO. On the 1D Burgers benchmark, SirenFNO yields the lowest error, which is 51% lower than that of FNO, with about 14 times fewer parameters. CP-SirenFNO also outperforms FNO with roughly 60 times fewer parameters, and the TT and Tucker variants similarly improve over FNO with about 50 and 57 times fewer parameters. For 1D CFD, SirenFNO reduces the

1 1D_Burgers_Sols_Nu0.001.hdf5

2 1D_CFD_Rand_Eta0.01_Zeta0.01_periodic_

Train.hdf5

3 2D_CFD_Rand_M0.1_Eta0.01_Zeta0.01_ periodic_128_Train.hdf5

4 ReacDiff_Nu0.5_Rho1.0.hdf5

ℓ2 error by 18% lower) while using 14 times fewer parameters than FNO, and CP-SirenFNO is the best-performing model with only 57.7 thousands parameters. Similarly, for 2D CFD and reaction-diffusion, SirenFNO performs the best among all, while using significantly lower parameters than standard FNO. The improvement gains can be attributed to the ability of SIREN parameterization to capture high- and low-frequency information, enabling full-frequency learning irrespective of discretizations.

5.4 Ablation Study

As shown in Table 2, our proposed methods adopt the enhanced architecture used in AM-FNO (MLP). SirenFNO consistently outperforms all baselines while using significantly fewer parameters to learn full-frequency information, demonstrating both the effectiveness and efficiency of our model.

For the ablation study, we further isolate the effect of architectural changes by keeping the operator architecture identical to the standard FNO implementation. According to Table 2, under a strictly controlled setting, SirenFNO † results indicate that our SIREN parameterization yields consistent performance gains across all PDE benchmarks. Performance gains can be purely attributed to our kernel parameterization, suggesting that our methods mitigate the spectral bias of FNO and enhance its high-frequency learning capability while achieving superior parameter efficiency. Furthermore, our SirenFNO delivers further improvements, indicating a complementary and synergic effect by employing the improved operator architecture.

5.5 Zero-Shot Super-Resolution

A key advantage of FNO is resolution invariance, which enables a model trained on a coarse grid to generalize well to a finer grid without retraining. To preserve resolution-invariant generalization for our SirenFNO, we adopt frequency truncation with modes mx = my = 12. Table 4 reports the zero-shot super-resolution performance on 2D Darcy flow, and Figure 1 visualizes the predictions. All models are trained on a discretization of 32 × 32 and then evaluated on 32 × 32 and 64 × 64 grids. Our SirenFNO consistently outperforms FNO and maintains strong performance across resolutions.

6 Conclusion

In this work, we propose SirenFNO to perform full-frequency learning and mitigate spectral bias without truncating Fourier modes, we also consider functional tensor decompositions with CP, TT, and Tucker variants to further improve the parameter and learning efficiency. As suggested by our experiments across various PDE benchmarks, our SirenFNO and its variants surpass FNO in terms of ℓ2 relative test error, using approximately 4 to 73 times fewer parameters without truncation. Overall, our proposed SirenFNO provides a more accurate and efficient alternative to standard FNO architectures for operator learning on PDE benchmarks.