FedGLoRA: Grassmann-Manifold Federated Learning via Dual LoRA for Large EEG Models

1 Introduction

Electroencephalography (EEG) is a non-invasive modality for recording brain activity and supports cognitive neuroscience, neurological disorder analysis, and brain-computer interfaces (BCIs) [Zhou et al., 2025]. However, real-world EEG datasets are often small and subject-specific, making robust generalization across individuals and recording conditions challenging [He and Wu, 2019; Chen and Yu, 2026]. Recently, Large EEG Models (LEMs) have addressed this limitation by self-supervised pretraining on large, diverse EEG corpora, learning transferable spatiotemporal representations that improve performance across downstream tasks and domains [Jiang et al., 2024; Wang et al., 2024; Wang et al., 2025a; Liu et al., 2025; Tang et al., 2026].

However, as LEMs move toward real-world deployment, we observe a phenomenon: the distribution and scale of EEG datasets co-evolve with the research community. This evolution is driven by the evolving EEG research system, manifested as systematic shifts in research objectives, experimental paradigms, data-collection conditions, and devices [Xu et al., 2020]. As EEG research progresses from controlled, small-scale laboratory validation to real-world deployment, the field broadens from relatively homogeneous paradigms to diverse in-the-wild scenarios, prioritizing robustness and generalization while exacerbating cross-paradigm distribution shift [Jayaram and Barachant, 2018]. Meanwhile, as more institutions participate and adoption grows, data collection scales unevenly across sites, resulting in pronounced scale imbalance [Luo et al., 2021].

Formally, task evolution is evident in benchmark task formulations for MI and emotion recognition. In MI benchmarks, task formulations have evolved from simple left-right binary classification to finer-grained multi-class protocols [Zhang et al., 2025]. Moreover, the labels are no longer tied solely to hand laterality, whereas in many paradigms they denote distinct unilateral action semantics and execution dynamics, such as finger-movement imagery or rhythmic grasping. Consequently, MI datasets can encode different action meanings, inducing systematic distribution shifts [Tangermann et al., 2012; Cho et al., 2017; Schalk et al., 2004; Dreyer et al., 2023]. In emotion recognition, protocols have moved from coarse valence categories to finer affective states, and elicitation paradigms have diversified, with video and game experiments inducing distinct EEG dynamics [Katsigiannis and Ramzan, 2017; Zheng et al., 2018; Chen et al., 2023; Alakus et al., 2020]. These trends show that EEG benchmark distributions and scale are nonstationary, so LEMs pretrained on historical data can lose effectiveness as distributions drift [Gama et al., 2014]. Therefore, sustaining the utility of LEMs requires adaptation on an enlarging corpus of newly collected EEG datasets [Bian et al., 2025]. However, repeatedly training LEMs is resourceintensive, motivating the central question we study:

How can LEMs be updated efficiently on an enlarging corpus of newly collected EEG datasets under persistent domain shifts?

To solve this problem, LEMs must encounter heterogeneous, multi-site data distributions and align and integrate cross-domain updates into its representation space [Rieke et al., 2020; Fu et al., 2024]. However, relying solely on local fine-tuning is insufficient: it lacks multi-source diversity, readily overfits site-specific artifacts, and can induce representational drift that weakens cross-site transfer [Wang et al., 2020]. Centralized joint fine-tuning could aggregate multi-source information, but it is often infeasible for multicenter EEG which is subject to strict privacy and regulatory constraints [Mongardi and Pinoli, 2024]. Therefore, federated learning (FL) offers a practical alternative: each site optimizes on private data and shares only model updates for server-side aggregation, enabling collaborative continual fine-tuning without exposing data [Guo et al., 2021].

While FL has achieved promising results in many domains, directly applying standard FL to federated fine-tuning of LEMs is often ineffective, due to EEG's distinctive properties [Xu et al., 2020; Cooray et al., 2025].

1. Destructive backbone drift. Direct full-parameter aggregation is a widely used fine-tuning strategy, but under cross-site domain shift it can be severely harmful for LEMs. Under cross-site heterogeneity, conflicting client gradients may drive uncontrolled backbone shifts that erase pretrained knowledge, causing negative transfer and degraded generalization [Huo et al., 2026; Wang et al., 2025b].
2. Pronounced multi-source heterogeneity. Federated EEG exhibits strong cross-site heterogeneity, leading to a highly non-IID regime [Baykara et al., 2025]. As a result, local updates are often misaligned, inducing negative transfer that harms site-wise and overall generalization.
3. Severe scale imbalance across sites. Federated EEG often shows extreme data imbalance: due to differences in hardware devices, trial protocols, and recruitment cost, some sites contribute only a few subjects. This lets high-volume sites dominate aggregation, biasing the global model toward majority patterns and underfitting site-specific shifts, degrading performance at minority centers.

Motivated by these challenges, we propose FedGLoRA, a federated fine-tuning framework for LEMs that enables cross-site joint adaptation under privacy constraints. FedGLoRA continually integrates knowledge from diverse EEG domains while preserving site-specific behavior. For Challenge 1, we freeze the pretrained backbone and communicate only LoRA updates, using Dual LoRA with a shared global branch for transfer and private LoRA on device for local shifts. For Challenge 2, we aggregate global updates in a shared low-dimensional Grassmann subspace and apply a Grassmann proximal regularizer to reduce misalignment and optimization conflicts. For Challenge 3, we use bidirectional knowledge distillation to transfer shared knowledge to local models while feeding back residual site-specific cues to the global model, improving robustness under non-independent and identically distributed data and imbalance. Across three different LEMs on multi-center motor imagery and emotion recognition, FedGLoRA consistently outperforms federated baselines and yields more stable multi-task gains.

Our main contributions include:

• First Systematic Study of Federated Fine-tuning for LEMs. We present the first systematic investigation of federated fine-tuning for LEMs on multi-site EEG, identifying key cross-site deployment challenges and outlining a practical privacy-preserving adaptation route.
• FedGLoRA: A New FL Paradigm. We propose FedGLoRA, a new FL paradigm that aggregates LoRA updates on Grassmann subspaces to improve robustness under scale imbalance and non-IID heterogeneity.
• Improved Performance on Diverse EEG Tasks. Experiments on motor imagery and emotion recognition achieve improved performance, with FedGLoRA outperforming strong federated baselines.

2 RelatedWork

2.1 Large EEGModels (LEMs)

LEMs for brain decoding are typically self-supervised pretrained on large EEG corpora to learn transferable spatiotemporal representations [Wu et al., 2024], which improves downstream performance [Jiang et al., 2024; Wang et al., 2024; Wang et al., 2025a]. However, deployment faces substantial distribution shift from evolving acquisition conditions, so pretrained LEMs often require adaptation to new populations and settings [Gama et al., 2014]. Although centralized multi-domain fine-tuning can reduce this drift, it is rarely feasible because raw EEG is privacy-sensitive [Liu et al., 2024a]. We therefore propose a federated framework that enables cross-site fine-tuning without sharing raw data, improving generalization under privacy constraints.

2.2 Federated Learning for FoundationModels

Pretraining followed by downstream fine-tuning is the dominant foundation-model paradigm, but effective adaptation often needs large, diverse data [Zhou et al., 2025]. Because downstream datasets are siloed across institutions and cannot be centralized under privacy constraints, federated learning enables joint fine-tuning while keeping raw data local [McMahan et al., 2017; Zhou et al., 2025]. Although recent work studies federated adaptation of large language models [Qi et al., 2024; Yang et al., 2025], EEG poses distinct challenges as structured non-IID shifts from sites cause update misalignment, further exacerbated by cross-site scale imbalance [Zhang et al., 2022]. This motivates EEG federated designs that couple shared transfer with site-specific adaptation.

2.3 Federated Learning for EEG

EEG varies across sites, so reliable performance needs diverse cohorts [Liu et al., 2024a]. Privacy and compliance rules prevent centralizing multicenter EEG, so federated learning enables joint training without sharing raw signals [Wang et al., 2020]. Studies on motor imagery and emotion recognition identify two bottlenecks as non-IID heterogeneity and scale imbalance [Rieke et al., 2020]. FedBS and FLEEG mitigate them via normalization, sharpness optimization, and hierarchical personalization [Liu et al., 2024b; Jia et al., 2024]. Yet most work targets task models, leaving federated adaptation of LEMs underexplored [Kuruppu et al., 2026]. We study federated fine-tuning of LEMs and propose a framework for heterogeneity [Bian et al., 2025].

3 Preliminaries

Federated Learning For LEMs. Given a multi-center EEG network with N sites, each site i owns a private dataset D(i) = (X(i) , Y (i) , t(i) ), where X(i) ∈ Rni×Ti×Ci denotes EEG segments with center-dependent channel layouts and acquisition settings, Y (i) denotes supervision signals whose label spaces may vary across tasks, and t(i) is a paradigm indicator. Due to privacy and regulatory constraints, raw EEG cannot be pooled across sites. Let f(X | t; Θ) be a pretrained LEM with parameters Θ. Federated learning for LEMs aims to perform cross-site joint fine-tuning by collaboratively optimizing a set of trainable parameters ϕ on EEG data:

(1)

where αi is the aggregation weight and ℓ(·, ·) is the task loss. This formulation provides a unified view of federated finetuning for LEMs; in practice, however, the multi-center EEG setting induces pronounced heterogeneity and scale imbalance across sites, which motivates the designs in our method.

4 Method

4.1 Framework Overview

In this subsection, we present our federated learning framework as shown in Figure 2, which iterates four stages. (1) Site Upload: prioritize privacy and locality, sharing only nonsensitive updates. (2) Server Update: robustly aggregate heterogeneous clients to recover shared structure and mitigate imbalance. (3) Site Update: integrate global knowledge while stabilizing training and preserving personalization. (4) Inference: deploy a site-composed model combining shared and local adaptations on downstreams. We next detail the design and methods in each stage.

4.2 Dual Low-rank Adaptation

Intuition: In the site-upload step, motivated by the central tension in full-parameter fine-tuning and highly heterogeneous cross-site EEG, models must capture both transferable shared representations and site-specific personalization.

Therefore, we introduce a dual-LoRA structure that explicitly disentangles a communicable global LoRA from a locally updated personalized LoRA. Formally, for client k, we parameterize any LoRA-injected weight matrix W (k) ∈ Rdout×din as the sum of the frozen pretrained weight W0 and the low-rank increments from a global and a personalized LoRA branch as:

where W0 denotes the shared pretrained backbone parameters and is kept frozen during training, ∆W (k) g = B(k) g A(k) g and ∆W (k) p = B(k) p A(k) p , and s is the LoRA scaling factor. Here, ∆W (k) g is the communicable global update used for serverside aggregation, while ∆W (k) p is the client-private personalized update kept local to preserve site-specific adaptation.

Based on the Dual LoRA setting, we partition the parameters in federated training into three parts: the frozen pretrained backbone parameters θ0, the global LoRA parameters on site k, denoted by ϕk = (A(k) g ,B(k) g ), and the personalized LoRA parameters ψk = (A(k) p ,B(k) p ). At communication round t, the server broadcasts the shared global LoRA parameters ϕt to all sites. After receiving ϕt , site k initializes (ϕk, ψk) from (ϕt , ψtk ) and jointly optimizes them on Dk:

where ℓ(·) denotes the supervised loss function. Notably, ϕt denotes the global parameters broadcast by the server at round t, while ϕt+1 k is the site-updated global branch after local optimization on site k and aggregated to obtain ϕt+1 for the next round.

4.3 FedGLoRA

Intuition: On the server, global LoRA updates from clients must be aggregated effectively, but FedAvg is ill-suited for LoRA as site-specific factors can be misaligned, causing cancellation and rank inflation, and the result is not guaranteed to remain a rank-r LoRA update.

Based on these considerations, we propose FedGLoRA, which operates at the subspace level, aligning and aggregating the low-rank subspaces induced by sites on the Grassmann manifold, thereby preserving the structure while improving the expressiveness of the global representation.

GrassmannManifold. We first briefly introduce theGrassmann manifold. Given n and r, the Grassmann manifold G(n, r) is defined as the set of all r-dimensional linear subspaces of Rn :

Numerically, a subspace point U ∈ G(n, r) can be represented by any orthonormal basis U ∈ Rn×r such that U = col(U). Since col(U) = col(UQ) for any Q ∈ O(r), the Grassmann manifold characterizes only the subspace direction and is invariant to basis rotations.

These properties naturally align with the intrinsic nonuniqueness of LoRA factorization, since for any invertible matrix R ∈ Rr×r ,

Therefore, during aggregation, we lift each site update to the low-rank subspace and perform alignment on the Grassmann manifold to eliminate basis misalignment caused by LoRA reparameterization.

Grassmann Subspace Representation of LoRA. We then explain how to lift each site's Global LoRA update to a subspace representation on the Grassmann manifold. For models with multiple LoRA-injected layers, the following decomposition and aggregation are applied independently to each layer. For site k, we denote its Global LoRA update in the current round as ∆W (k) g ∈ Rdout×din . Rather than aggregating directly in the Euclidean coordinates, we extract the basisinvariant subspace structure. Concretely, we compute a rankr orthogonal factorization of ∆W (k) g as:

where Uk ∈ Rdout×r and Vk ∈ Rdin×r satisfy U ⊤ k Uk = Ir and V ⊤ k Vk = Ir. This induces rank-r output and input subspaces, Uk = col(Uk) ∈ G(dout, r) and Vk = col(Vk) ∈ G(din, r), respectively. From this, we derive the coefficient matrix Ck:

The coefficient matrix characterizes the magnitude and coupling of the update under the chosen subspace coordinates. Note that (Uk, Vk) determine the subspaces but not a unique basis. Therefore, (Uk, Vk) provides a geometric representation that is invariant to basis rotations and serves as the object for subsequent alignment and aggregation.

Grassmann Subspace Distance. To quantify subspace deviation, we use the projection distance. For two output-side bases Um and Un, we define

The input-side distance is defined analogously. For a candidate global subspace (U, V ) and client k, the alignment discrepancy is

Aggregation Algorithm. We now describe federated aggregation in FedGLoRA. In each round, site k lifts its global LoRA increment ∆W (k) g to a Grassmann subspace representation (Uk, Vk,Ck). The server then estimates a shared rank-r global LoRA (U, V,C) that aligns client updates at the subspace level to reduce optimization conflicts and negative transfer, while faithfully reconstructing their low-rank increments within the shared subspace. To this end, we formulate aggregation as a weighted joint estimation problem on the Grassmann manifold:

where wk = 1/N is a uniform site weight in our implementation. The first term aligns the subspaces (U, V ) with (Uk, Vk) on the Grassmann manifold, removing LoRA reparameterization ambiguity and reducing misalignment-induced cancellation. The second term preserves the dominant energy of client increments within the shared subspace, yielding a rankr global update by construction.

4.4 Grassmann Prox and BiKD

Corresponding to the Site Update stage, after the server broadcasts the aggregated global LoRA, each client performs local optimization to update its global and personalized LoRA. In this stage, our goal is to strengthen transferable global representations while balancing site-specific personalization. To this end, we introduce Grassmann Manifold Prox and global-local bidirectional distillation.

Grassmann Prox

Under strong non-IID heterogeneity, local updates can drift to disparate directions, which degrades subspace alignment and destabilizes training.

Inspired by FedProx, we regularize client optimization by constraining the global LoRA subspaces to stay close to the server initialization. Specifically, at round t, client k receives reference bases (Um, Vm) from the server; after local updates induce new bases (Un, Vn), we define

This Grassmann proximal regularization limits subspace drift across rounds, improving convergence consistency.

Bidirectional Knowledge Distillation (BiKD)

During local training, each client maintains a global and a personalized LoRA. The key challenge is balancing crosssite transfer and local personalization: direct aggregation can let global knowledge dominate personalization or allow sitespecific bias to contaminate the shared representation, degrading generalization and local adaptation.

Therefore, we introduce bidirectional knowledge distillation (BiKD) to learn transferable global semantics while retaining personalized information each round. For each LoRA-injected layer, write the global LoRA increment as ∆Wg = UgCgV ⊤ g and use its output-side basis Ug to define Pg = UgU ⊤ g . Let ˜ Up be the output-side basis of the personalized update; orthonormalizing (I - Pg) ˜ Up gives Up and Pp = UpU ⊤ p . Thus, Pg and Pp act on output-side hidden activations, capturing site-shared and site-specific structure.

Global Semantic Alignment. First, during local updating at client k, we treat the Global-LoRA model as teacher fG and the model with both global and personalized LoRA as student fP . Let hG(x), hP (x) ∈ Rdout be their output-side hidden activations at this layer; fG and fP still denote the full predictors. We distill only on U g , aligning shared semantics while leaving the private subspace unconstrained as:

where Pg projects hidden representations onto the shared Grassmann subspace. This directional loss is robust to magnitude variation and suppresses drift.

Personalized Semantic Feedback. One-way distillation can over-transfer the student toward the global mode and weaken personalized LoRA on local private structures. We therefore feed the student's residual in U p back to the teacher, allowing recurring useful private patterns to influence the global branch. Let zpm(x) = gcls(Pphm(x)) and ppm(x; τ) = softmax(z pm(x)/τ ) for m ∈ {P, G} and τ > 0. We align the teacher to the detached student private-channel distribution

Here, sg[·] denotes stop-gradient; the projection is applied to hidden representations, while the softmax converts privatechannel logits into valid predictive distributions.

Overall Local Training Objective. At round t, site k optimizes its Global LoRA and Personalized LoRA by minimizing the task loss augmented with Grassmann Prox and BiKD:

where λ1, λ2, λ3 > 0. LT→S and LS→T are defined in Eqs. (12) and (13), respectively, and L(k) prox is given in Eq. (11).

5 Experiments and Results

5.1 Experimental Setup

Datasets. To comprehensively assess our method across diverse EEG application domains, we evaluate on two downstream tasks: motor imagery and emotion recognition. Specifically, for motor imagery, we use four widely adopted benchmarks including BCIC-IV-2a [Tangermann et al., 2012], Cho2017 [Cho et al., 2017], PhysioNet-MI [Schalk et al., 2004], and Dreyer2023 [Dreyer et al., 2023].

For emotion recognition, we use four representative datasets: DREAMER [Katsigiannis and Ramzan, 2017], SEED-IV [Zheng et al., 2018], FACED [Chen et al., 2023], and GAMEEMO [Alakus et al., 2020]. For each dataset, we adopt a cross-subject protocol. We randomly split subjects into disjoint train/validation/test sets with an 8:1:1 ratio, assigning all trials from each subject to a single split.

Large EEGModels (LEMs). We consider three representative LEMs: LaBraM [Jiang et al., 2024], EEGPT [Wang et al., 2024], and CBraMod [Wang et al., 2025a]. For each model, we initialize from the publicly released pretrained checkpoint and follow the corresponding default fine-tuning configuration to ensure fair backbone comparison.

Implementation. Throughout all experiments, we apply a unified preprocessing pipeline to standardize EEG and fix the federated-learning hyperparameters to a single default setting across all models and datasets.

5.2 Cross-site generalization performance

To assess whether the cross-site domain shift hypothesized above manifests in practice, we conduct a cross-site generalization evaluation. Specifically, we train the model using data from a single site and then evaluate it on all remaining sites. As shown in Fig. 3, off-diagonal accuracies are consistently lower than the diagonal, indicating substantial performance drops under cross-site transfer for both motor imagery and emotion recognition. This pronounced gap suggests a strong distribution mismatch across sites. Therefore, we can conclude that EEG data in cross-hospital settings exhibit objectively significant site heterogeneity and domain shift. This observation provides key motivation for our subsequent federated learning setting: under privacy constraints, collaborative training must explicitly model and mitigate cross-site heterogeneity to improve transferability across hospitals.

5.3 Comparison with FL Baselines

As summarized in Table 1, we compare FedGLoRA with FedAvg [McMahan et al., 2017], FedProx [Li et al., 2020], SCAFFOLD [Karimireddy et al., 2020], FedKD [Wu et al., 2022], FedRoD [Chen and Chao, 2021], and FDLoRA [Qi et al., 2024] on three LEMs. Reported numbers are final local test accuracies aggregated at the dataset level. FedGLoRA is consistently strong across baselines, achieving the best performance and improving the within-task-group average accuracy for both motor imagery and emotion recognition. Notably, it also outperforms single-site full fine-tuning, indicating that cross-site collaboration provides more effective information than isolated local updates. FDLoRA achieves the second-highest performance overall. FedAvg offers limited gains, while FedProx and SCAFFOLD improve performance in some cases but are not consistently effective across models and task groups. These results support our motivation: under cross-site non-IID and imbalance, naive aggregation suffers from update misalignment and drift, whereas FedGLoRA's subspace alignment yields transferable global updates.

5.4 Effectiveness of LoRA for Fine-tuning LEMs

To demonstrate the effectiveness of LoRA for fine-tuning LEMs, we compare LoRA with full fine-tuning on three LEMs across eight datasets. We further vary the LoRA rank r ∈ {2, 4, 8, 16} to assess accuracy under different parameter budgets. As shown in Table 2, LoRA achieves performance comparable to full fine-tuning on most datasets. These results confirm that LoRA provides an accuracy-preserving adaptation strategy with substantially fewer trainable parameters. This supports FedGLoRA: small site updates learn useful EEG cues without moving the full backbone during training. We thus use r = 8 to balance accuracy and efficiency.

5.5 Ablation Study

As shown in Table 3, we conduct ablation studies of FedGLoRA. Dual LoRA is essential: using only the personalized branch causes a large accuracy drop, while using only the global branch improves performance but remains inferior, confirming complementary shared transfer and local adaptation. Removing FedGLoRA yields the largest degradation, highlighting the importance of subspace alignment under non-IID update misalignment. Grassmann-Prox improves performance by limiting subspace drift. Bidirectional distillation further boosts accuracy: removing BiKD degrades results, and either one-way variant is weaker than the bidirectional design, indicating that both global-to-local alignment and local-to-global feedback are needed. Overall, FedGLoRA drives the primary accuracy gains, while GrassmannProx and BiKD provide additional improvements.

Table 3: Ablation study on (a) Motor Imagery (MI) and (b) Emotion Recognition. Metrics are accuracy (%).

6 Conclusion

In this work, we observe that EEG dataset co-evolve with the research community, inducing cross-site drift and fragmented deployments that require LEM adaptation. We formulate federated fine-tuning around two key challenges: multi-source heterogeneity and scale imbalance. To address them, we propose FedGLoRA, a Dual LoRA framework that separates global transfer from local personalization, aligns global updates on basis-invariant Grassmann subspaces, and introduces Grassmann Prox and BiKD. Across motor imagery and emotion recognition with three LEMs, FedGLoRA consistently outperforms strong federated baselines, and ablations confirm the complementary effects of its components. FedGLoRA offers a route for privacy-preserving cross-site LEM adaptation.

Acknowledgments

This work is supported by the Youth Science Fund Project of National Natural Science Foundation of China (Grant No.62306317), and the Open Research Fund of the Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education (Grant No.SCCI2025YB01), and Sponsored by Beijing Nova Program (Grant No.20250484804), and Young Elite Scientists Sponsorship Program of the Beijing High Innovation Plan (Grant No.20250912).