SPOTR: Spatio-temporal Pooling One-Token Reconstruction for Universal Physiological Signal Self-supervised Learning

1 Introduction

Physiological signals such as electroencephalography (EEG), electrocardiography (ECG), and photoplethysmography

* Corresponding authors.

(PPG) are typically recorded as multi-channel, hightemporal-resolution waveforms, providing essential evidence for characterizing individual physiological states and health variations. Deep learning models have achieved remarkable progress in automated biosignal analysis [Wang et al., 2024; Fan et al., 2025] and have been widely applied to sleep analysis [Eldele et al., 2021; Thapa et al., 2024; Hu et al., 2025], EEG-based seizure detection [Jing et al., 2023; Hogan et al., 2025; Chen et al., 2026], ECG-driven arrhythmia detection [Guhdar et al., 2025; Rahmani et al., 2025], and PPG-based hypertension detection [Lin et al., 2025; El-Dahshan et al., 2024]. Despite their effectiveness, supervised methods often rely on large-scale datasets with high-quality annotations and expert review, which are costly and difficult to scale. To reduce reliance on expensive expert annotations, self-supervised learning (SSL) has made substantial advances leveraging unlabeled biosignals. Current SSL techniques for physiological signals can be broadly grouped into two categories: contrastive learning and masked reconstruction. Contrastive SSL (C-SSL) learns discriminative representations by bringing positives closer while separating negatives [Yang et al., 2023; Pillai et al., 2025; Saha et al., 2025]. Masked reconstruction SSL (MR-SSL), in contrast, learns representations by predicting masked signal segments (or tokens) from unmasked ones [Jiang et al., 2024; Wang et al., 2025; Jin et al., 2025; Na et al., 2024].

C-SSL: Performance Is Sensitive to View Design. Recent C-SSL methods mainly fall into two approaches: augmentation-based contrastive learning (e.g., BIOT [Yang et al., 2023]) and domain-guided contrastive learning (e.g., PaPaGei [Pillai et al., 2025] and Pulse-PPG [Saha et al., 2025]). The former constructs multiple views via augmentations to encourage invariance in the learned representation, as shown in Fig. 1(a). The latter defines positive and negative pairs using task-specific or domain-specific rules and learns representations by pulling positives together while separating negatives, as shown in Fig. 1(b). While effective in specific settings, the learned invariances are highly dependent on pretraining design choices, and even slight changes in augmentation strength, view semantics, or pairing criteria can alter the supervision signal and induce unstable transfer behavior. In practice, this makes C-SSL harder to reliably generalize across heterogeneous datasets, where no single augmentation or pairing rule consistently preserves clinically relevant structures.

MR-SSL: Masked Reconstruction Is Prone to Shortcut Learning. Current MR-SSL for biosignals includes reconstructing raw signals (e.g., ST-MEM [Na et al., 2024], CBraMod [Wang et al., 2025], and CSBrain [Zhou et al., 2025]) and reconstructing discrete tokens (e.g., LaBraM [Jiang et al., 2024] and HeartLang [Jin et al., 2025]). Their difference lies in whether the reconstruction target is continuous waveforms (Fig. 1(c)) or tokenized representations (Fig. 1(d)). However, under the mask-then-reconstruct setup, models can often exploit temporal continuity and cross-channel redundancy to recover missing content, creating shortcuts that bypass compact global representation learning. Consequently, the reconstruction objective can be weakly aligned with learning task-relevant, modality-stable structures, allowing pretraining loss to drop quickly while downstream performance improves only marginally.

Taken together, these limitations leave key gaps in generalization for building foundation models. As a result, pretrained representations are often weak under linear probing, a lightweight adaptation protocol that better matches real-world medical deployment where labeled data and compute are limited. In addition, existing architectures are frequently computationally heavy. As summarized in Table 1, most Transformer-based SSL models process a flattened spatiotemporal sequence with length on the order of C×N, where C denotes the number of channels and N the number of temporal tokens per channel, leading to high attention cost and memory usage. Finally, most previous studies remain modality-specific in practice, typically tailoring pretraining and evaluation to a single signal type rather than learning a generalizable representation across EEG, ECG, and PPG within one universal model.

Our Approach. To address these gaps, we propose SPOTR (Spatio-temporal Pooling One-Token Reconstruction), a universal SSL framework built around a compress-reconstruct pretraining scheme with a single-token global bottleneck. SPOTR compresses each waveform into a single-token representation and reconstructs the signal conditioned only on this representation, suppressing shortcut learning and encouraging globally organized, generalizable features that benefit linear probing. To improve efficiency, SPOTR introduces an efficient spatiotemporal compaction module (ST Compactor) that reduces the token sequence length from C × N to C+N before sequence modeling, substantially lowering computation while preserving spatiotemporal structure. Finally, by adopting a modality-agnostic objective and architecture, we enable unified pretraining across multiple physiological signal types, targeting a universal foundation model that can generalize across modalities and downstream datasets.

We summarize our main contributions as follows:

1. We introduce SPOTR, a universal SSL framework that learns generalizable representations across physiological signal modalities via a compress-reconstruct pretraining scheme.
2. We design an efficient spatiotemporal compaction architecture that shortens the encoder token sequence from C ×N to C+N, improving training and inference efficiency without sacrificing representation quality.
3. We demonstrate consistent gains across diverse downstream tasks and modalities, supporting SPOTR as a universal pretraining framework for physiological signals.

2 Method

We present SPOTR (Spatio-temporal Pooling One-Token Reconstruction), a compress-reconstruct pretraining framework that introduces a single-token global bottleneck for physiological signals. SPOTR comprises an ST Compactor S for spatiotemporal token compaction, a Latent Aggregator A for single-token global fusion, and a Latent Renderer G for bottleneck-conditioned signal reconstruction (Fig. 2).

2.1 SPOTR Training Objective

Physiological signals often exhibit substantial redundancy across both time and channel dimensions. Under masked reconstruction, a decoder may still leverage visible local tokens or neighboring context to predict masked regions, so the reconstruction loss can be dominated by local correlations and common-mode structure-yielding shortcut learning that bypass global representation learning [He et al., 2021; Madan et al., 2023; Wei et al., 2023].

To address this issue and promote a more compact latent representation, SPOTR adopts a compress-reconstruct pretraining scheme with an explicit information bottleneck at both the encoder and decoder. The input is first compressed into two short sequences, then aggregated into a single class token as the global representation; reconstruction is conditioned only on this global token.

Let the physiological signal be x ∈ RB×C×L, where B is the batch size, C is the number of channels, and L is the number of sample points. We denote the ST Compactor, Latent Aggregator, and Latent Renderer by S(·), A(·), and G(·), respectively. With patch size P , we define the number of temporal tokens as N = ⌈ L P ⌉.

(i) Spatiotemporal Compaction. The ST Compactor compresses the input into temporal tokens and spatial tokens, and uses learnable 2D positional encodings PEA2D to mark the relative position of each channel-time token:

where DA is the hidden dimension of the Latent Aggregator, zt i is the i-th temporal token, and zs j is the j-th spatial token.

(ii) Single-token Global Aggregation. We add a learnable class token zcls and take the first output position as the global representation:

Here hcls serves as the only information bottleneck in SPOTR, responsible for capturing global structure across both time and channels. With this design, the reconstruction is no longer conditioned on a large set of visible local tokens; instead, it relies solely on a fixed-size global representation, imposing a stronger constraint on representation compression and information organization.

(iii) Mask-token Sequence and Conditional Rendering. To reconstruct the signal, we build a mask-token sequence aligned with the target spatiotemporal grid. Let zM ∈ R1×1×DG be a learnable mask token, and let R(·) be a repeat operator that expands a single token to B ×C ×N positions and flattens them into a sequence:

where DG is the hidden dimension of the Latent Renderer and PEG2D is the decoder-side 2D positional encoding. The renderer conditions on hcls and outputs the reconstruction:

(iv) ReconstructionLoss. Pretraining is optimized byminimizing the reconstruction error, and the objective can be formulated as the following empirical risk minimization problem:

where pdata is the pretraining data distribution and ˆx is the reconstructed signal produced end-to-end by the ST Compactor, Latent Aggregator, and Latent Renderer. By minimizing the reconstruction loss under a single-token information bottleneck, this objective drives the model to learn global representations that are both compact and reconstructive. Unlike contrastive learning, which relies on constructing positive and negative pairs to form training signals, the reconstruction objective uses the input itself as supervision, thereby avoiding the uncertainty introduced by negative-sample selection [Wei et al., 2023].

2.2 ST Compactor: Compressing Spatiotemporal Structure

The ST Compactor S(·) consists of a channel-independent convolutional patch embedding layer C followed by two orthogonal attention-based aggregation paths. The input signal x ∈ RB×C×L is first segmented into temporal patches of length P and projected into patch tokens:

Two attention operations are then applied:

(i) Spatial Token Aggregation. For each channel j, temporal self-attention is applied over its N patches, followed by temporal pooling:

(ii) Temporal Token Aggregation. For each temporal token i, channel-wise self-attention aggregates information across channels, followed by spatial pooling:

This design reduces the subsequent sequence length from C × N to C + N, preserving spatiotemporal structure while significantly lowering computation.

2.3 Latent Aggregator: Aggregating into a Single Token

The Latent Aggregator A(·) is stacked Transformer encoder blocks that jointly encode Z and aggregate it into the class token, yielding hcls. By using the class token as the sole global representation throughout the network, A enforces a persistent single-token bottleneck, encouraging compact and transferable representations.

We adopt a pre-norm design with RMSNorm for training stability. The feed-forward network uses SwiGLU:

where ⊙ denotes the Hadamard product and Wg, W1, W2 are learnable projection matrices. This design effectively enhances the model's ability to conduct feature selection and knowledge encoding under limited parameters.

2.4 Latent Renderer: Rendering Signals from a Single Token Bottleneck

The Latent Renderer G reconstructs the full physiological signal by conditioning on the highly compressed global latent hcls and progressively "rendering" the mask-token sequence ZM into the signal domain. Concretely, G consists of an input projection Pin ∈ RDA×DG , an output projection Pout ∈ RDG×P , and a stack of Transformer decoder blocks. Here, Pin maps the conditioning vector to the decoder hidden space, while Pout regresses each token representation to a length-P local segment, which is then reassembled to form the reconstructed signal.

Different from standard Transformer decoders that apply self-attention before cross-attention, we adopt a crossattention-first update order, so that each position reads from the global condition hcls prior to any intra-sequence interaction. Specifically, let the input to the k-th decoder block be Q(k-1) ∈ RB×(CN)×DG , ˜h = Pin(hcls) ∈ RB×1×DG . We first inject the global condition via cross-attention:

Next, we perform self-attention over the mask-token sequence to enable information exchange across positions:

Finally, the resulting features are passed through a feedforward network with a residual connection:

After stacking K decoder blocks, we obtain Q(K) . The final reconstruction is produced by projecting each position back to a length-P segment and reshaping tokens into the original spatiotemporal layout:

This design forces all local reconstructions to be mediated by the same global condition hcls, thereby preventing the mask tokens from completing reconstruction solely via local correlations.

3 Experimental Setup

We pretrain and evaluate models under the SPOTR framework on multi-modal physiological signal datasets, and refer to the resulting family of models as SPOTR.

3.1 Baselines

As summarized in Table 1, we compare SPOTR with representative SSL baselines. For each modality, we include (i) a general-purpose time-series SSL foundation model, MOMENT, and (ii) domain-specific SSL models. Specifically, for EEG/iEEG we use LaBraM, CBraMod, and CSBrain; for ECG we use ST-MEM and HeartLang; and for PPG we use PaPaGei and Pulse-PPG. We do not include BIOT [Yang et al., 2023] in our comparison because, although it claims applicability to multiple physiological modalities, only pretrained EEG weights are publicly available. More details are provided in the Appendix.

3.2 Datasets

Pretraining Datasets

We curate a large-scale pretraining dataset covering 4 physiological modalities, including scalp EEG, intracranial EEG (iEEG), ECG, and PPG. Overall, the pretraining data collection contains 20 datasets, including 6 EEG datasets [Miltiadous et al., 2023; Kemp et al., 2000; Quan et al., 1997; Shoeb, 2009; Van Dijk et al., 2022; Khan et al., 2022], 2 iEEG datasets [Li et al., 2021; Burrello et al., 2019], 8 ECG datasets [Gow et al., 2023; Zheng et al., 2022; Liu et al.,

2022; Kalyakulina et al., 2020; Bousseljot et al., 1995; Tihonenko et al., 2008; Alday et al., 2020; Ribeiro et al., 2021], 3 PPG datasets [Mehrgardt et al., 2022; Nemcova et al., 2021; Garde et al., 2014], and one multimodal dataset [Lee et al., 2022]. In total, it provides over 17 million signal samples from more than 450,000 subjects, spanning diverse acquisition settings. More details are provided in the Appendix.

Downstream Datasets

As shown in Table 2, we conduct a comprehensive evaluation of SPOTR on 17 downstream datasets spanning multiple physiological signal modalities, including iEEG, EEG, ECG, and PPG. For all the downstream datasets, we resampled the signals to 200 Hz and applied a 50/60 Hz notch filter (depending on the acquisition region) to suppress power-line interference. More details are provided in the Appendix.

3.3 Implementation Details

All experiments were conducted on a Linux server with an NVIDIA A100 80GB GPU, using PyTorch 2.0.1 and CUDA 12.2. For pretraining, we used AdamW with β = (0.9, 0.999), weight decay 0.1, and a peak learning rate of 2×10-4 . Each dataset was trained with a per-step batch size of 256 and gradient accumulation of 20, resulting in an effective batch size of 5120. We pretrained for 3 epochs with a learning-rate schedule including a 10% warm-up; bfloat16 precision was used for computational efficiency.

For downstream evaluation, we used subject-independent train/validation/test splits for all tasks and report the mean and standard deviation of test-set performance over 5 independent runs with different random seeds. We kept the optimizer as AdamW with the same β and weight decay, but used a constant learning rate of 2 × 10-4 and a batch size of 128. All downstream runs were trained for up to 200 epochs with early stopping based on validation AUC (patience = 20 epochs). Unless otherwise specified, downstream training was performed in float32 for stable optimization.

4 Results

We evaluate three adaptation protocols: (i) Linear probing, where the pretrained backbone is frozen and only a single linear classification head is trained (Fig. 3(a)); (ii) Few-shot classification, where we construct a support set by randomly sampling K ∈ 1, 2, 4, 8 labeled examples per class from the training set, freeze the backbone, and train only the same linear head (Fig. 3(b)); and (iii) Full fine-tuning, where we train the backbone and the classification head under the same downstream configuration (Fig. 3(c)). In all the experiments, we use AUC as the evaluation metric.

EEG

ECG

4.1 Linear Probing

Table 3 summarizes linear-probing performance, which directly reflects the quality of pretrained representations under lightweight adaptation.

On iEEG, SPOTR achieves the best performance with an average AUC of 88.22%, surpassing the strongest iEEG/EEG SSL baseline by over 20%. On EEG, SPOTR is consistently superior across the five datasets, reaching an average AUC of 88.10% and exceeding EEG-specific SSL baselines. On ECG, SPOTR again delivers the strongest results, achieving an average AUC of 87.39% and improving over the best ECG-specific SSL baseline by 17.86%. On PPG, SPOTR attains the highest average AUC of 64.14%, outperforming PPG-specific baselines (PaPaGei and Pulse-PPG) on most datasets. Since MOMENT is evaluated on all datasets, we further summarize results over all 17 datasets in Table 3. SPOTR achieves an overall mean AUC of 80.86%, exceeding MOMENT by 19.11%.

Across four physiological modalities, SPOTR consistently surpasses both the general time-series baseline and modalityspecific SSL models in the linear-probing setting, supporting its practicality as a universal and lightweight-transfer pretraining framework for physiological signals.

4.2 Few-shot Classification

As shown in Fig. 4, under 1/2/4/8-shot settings, SPOTR achieves the best or near-best AUC on four representative datasets, each corresponding to a different modality: ISRUC (EEG), Mayo (iEEG), CPSC2018 (ECG), and PPG-Dalia

(PPG). Across all modalities, SPOTR exhibits a stable and consistent improvement as more labeled examples are provided, indicating that it can make good use of very limited labeled data. Moreover, the relatively tight performance dispersion across runs suggests robust adaptation under limited supervision. Overall, these results support SPOTR for practical few-shot scenarios, where annotation is scarce but reliable adaptation is still needed.

4.3 Full Fine-tuning

As shown in Table 4, under the full-parameter fine-tuning setting, SPOTR remains competitive across all four modalities. It achieves the best average performance on iEEG (90.25%) and EEG (89.61%), and shows the clearest advantage on ECG with the highest mean score (92.70%). On PPG, SPOTR is overall comparable to the strongest PPG-specific baseline, with only a marginal difference in average performance. Overall, even when end-to-end fine-tuning reduces the gaps among pretraining paradigms, SPOTR still provides robust transfer across modalities and datasets.

4.4 Efficiency

In the efficiency evaluation, we compare SPOTR against the general time-series foundation model MOMENT across 17 downstream datasets, reporting the mean results over four metrics: Latency (p50/p95), Throughput, and Peak GPU Memory. As shown in Table 5, SPOTR achieves substantially better inference efficiency. Specifically, the average p50 and p95 latency are reduced from 32.61 ms and 35.48 ms to 7.15 ms and 7.69 ms, corresponding to 78.1% and 78.3% lower latency, respectively. Meanwhile, throughput increases from 45.66 to 139.84 samples/s, yielding a 206.3% improvement. SPOTR also reduces peak GPU memory consumption from 532.09 MB to 256.39 MB (51.8% lower). These results demonstrate that SPOTR delivers markedly faster and more memory-efficient inference, highlighting its practicality for resource-constrained deployment. More details are provided in the Appendix.

4.5 ScalingModel Size

As shown in Fig. 5, increasing the model size from SPOTRSmall (3.69M parameters) to SPOTR-Medium (13.93M) and SPOTR-Base (62.47M) leads to a clear scaling trend in pretraining. Larger models drop the loss faster early on and settle at a lower value, suggesting stronger capacity and easier optimization. The same pattern appears in linear probing. SPOTR-Base gives the best AUC on iEEG, EEG, ECG, and PPG, followed by SPOTR-Medium and then SPOTR-Small. Overall, scaling up the model improves both reconstruction training and representation transfer across signal types, which supports further scaling of model and data.

5 Conclusion

In this paper, we presented SPOTR, a universal selfsupervised learning framework for physiological signals that learns generalizable representations across modalities via a compress-reconstruct pretraining scheme. By compressing each input into one representation and reconstructing the signal only from this bottleneck, SPOTR discourages shortcut learning and promotes compact global representations. SPOTR further improves efficiency by shortening the token sequence before modeling. Extensive experiments demonstrate consistent gains in cross-modality generalization, with particularly strong improvements under linear probing, supporting lightweight and practical adaptation in real-world clinical scenarios, while reducing inference latency and peak GPU memory compared with a representative general-purpose time-series foundation model.

Acknowledgements

This work has been supported by Guangdong S&T Program (2025B0101140001, 2026B0101070006), Guangdong Provincial Key Laboratory of In-Memory Computing Chips (2024B1212020002), Shenzhen Science and Technology Program (ZDCY20250901103401002, JCYJ20241202125907011), and Beijing Natural Science Foundation (L234026, L257010). This work has been supported by the New Cornerstone Science Foundation and Financial Support for Outstanding scientific and technological innovation Talents Training Fund in Shenzhen.