Semantics-Guided Representation Learning for Long-Tailed ECG Classification

1 Introduction

In recent years, the rising mortality of cardiovascular diseases has underscored the importance of early intervention and timely treatment. Electrocardiogram (ECG) is widely used in clinical practice for cardiac disease screening and evaluation [Ansari et al., 2017]. However, the interpretation of ECG by physicians may be influenced by factors such as clinical experience and fatigue, which can lead to missed diagnoses and misdiagnoses. With the rapid advancement of deep learning, computer-aided diagnosis systems have emerged to automatically analyze ECG data and assist clinicians in identifying pathological abnormalities, thereby improving diagnostic efficiency and reducing human-related errors. Nevertheless, this technology still faces several challenges.

∗ The corresponding author.

(1) Existing long-tailed solutions mainly operate in the signal domain, lacking semantic guidance. Real-world ECG data exhibit a long-tailed distribution, in which common diseases (head classes) have abundant samples, whereas rare diseases (tail classes) suffer from severe sample scarcity, as shown in Figure 1(a). This imbalance causes models to be dominated by head classes during feature learning, resulting in poor discrimination of tail classes. To mitigate this issue, prior studies have explored data resampling methods [Sharma and Sunkaria, 2020; Xia et al., 2023; Li et al., 2022], class-specific loss reweighting [Hammad et al., 2022; Romdhane and Pr, 2020], and modifications to model architectures and training procedures [Jiang et al., 2024; Chen et al., 2024]. However, these approaches primarily operate within the signal feature space through empirical data-level adjustments, without explicitly addressing the structural organization of the representation space. As a result, when tailclass samples are extremely scarce, purely data-driven learning often fails to produce discriminative representations for rare categories (Figure 1(b)). Therefore, the long-tailed problem cannot be attributed solely to data scarcity or biased optimization objectives. Rather, it fundamentally arises from the structural imbalance of the learned representation space under class-imbalanced distributions. From this perspective, alleviating long-tailed imbalance requires introducing additional structural constraints to guide representation organization. Semantic space offers a promising solution by modeling semantic relationships among classes, offering tail classes additional semantic priors and structural support (Figure 1(c)).

In the vision-language domain, semantic modeling approaches have achieved significant progress. Traditional vision-language models (e.g., CLIP [Radford et al., 2021]) jointly learn representations of natural language and visual features, mapping language and visual concepts to a shared semantic space. However, such semantic embedding models are not directly transferable to the ECG domain, as ECG signals exhibit strong temporal characteristics and are governed by complex physiological mechanisms, resulting in semantic representations that differ from those of visual image.

(2) ECG-semantic alignment approaches are constrained by coarse-grained label prompts, failing to characterize the fine-grained ECG features within the same category. Recent studies have attempted to integrate diagnostic label text and ECG signals into a shared semantic space to enhance representation [Liu et al., 2024; Ye et al., 2025; Seki et al., 2025; Chen et al., 2025]. These approaches typically rely on clinical diagnostic labels as semantic prompts. However, ECG signals within the same label category often exhibit morphological diversity. For instance, although both patients may be diagnosed with left ventricular hypertrophy (LVH), those with long-standing hypertension often present pronounced voltage increases with ST-T strain patterns, whereas patients with aortic stenosis tend to exhibit axis deviation and atypical repolarization abnormalities. In such cases, relying solely on the static textual description "LVH" results in coarse-grained semantic prompts that fail to characterize fine-grained ECG waveform features. This deviation between semantic constraints and ECG representations limits the effectiveness of semantic guidance. Therefore, we argue that constructing a learnable text prompt strategy capable of adaptively calibrating this deviation based on ECG signal features is essential.

To address these challenges, we propose a SemanticsGuided Representation Learning (SGRL) framework. Inspired by the image-text alignment paradigm in CLIP, we first pretrain an ECG encoder to align ECG embeddings with their corresponding label prompts. Subsequently, to transform coarse-grained label prompts into fine-grained semantic representations that better reflect ECG characteristics, we introduce learnable parameters into the label prompts and calibrate them using ECG features to generate adaptive label semantic embeddings. These adaptive semantic embeddings are fused with ECG embeddings via multi-head attention to construct class semantic anchors. We further suppress inter-class semantic correlations to enhance anchor separability, thereby significantly enhancing the recognition performance of individual classes, particularly for tail classes. The contributions of our work can be summarized as follows:

• We propose the SGRL framework, which leverages label semantic priors to guide feature learning and correct feature bias under long-tailed distributions, offering a novel
• semantics-based solution for long-tailed ECG classification.
• A Label-Semantic Prompt Adaptation (LSPA) strategy is proposed. By incorporating learnable parameters calibrated by ECG into label prompts under a semantic consistency constraint, it generates adaptive label semantic embeddings that better characterize fine-grained ECG features.
• A Semantic-Guided Representation Decorrelation (SGRD) is constructed, which integrates ECG embedding with adaptive label semantic embeddings via multi-head attention, and separates class semantic anchors through label-wise decorrelation, improving discrimination for tail classes.
• Extensive experiments are conducted on multiple public long-tailed datasets and performed comparisons with baseline methods, and the experimental results demonstrate the superiority of the proposed approach.

2 RelatedWork

Long-Tailed ECG Classification. To alleviate the longtailed distribution problem in ECG-based cardiac disease classification, some studies have introduced specialized loss functions to mitigate data distribution bias. For example, Hammad et al. [Hammad et al., 2022] and Romdhane et al. [Romdhane and Pr, 2020] employ focal loss to mitigate class imbalance, enabling deep CNN models to emphasize hard and minority samples and achieve improved performance on underrepresented classes. However, these lossbased approaches do not introduce additional supervisory information, making it difficult to learn robust and discriminative representations for extremely rare tail classes. Several studies have employed data generation methods to augment datasets by synthesizing samples of rare classes. Sharma and Sunkaria [Sharma and Sunkaria, 2020] adopt adaptive synthetic oversampling to improve myocardial infarction detection, while Xia et al. [Xia et al., 2023] and Li et al. [Li et al., 2022] utilize GAN-based frameworks to augment ECG data and alleviate long-tailed distributions. Despite their effectiveness, such methods often suffer from overfitting and limited generalization when tail-class samples are severely scarce.

Multimodal ECG-Text Representation Learning. In the field of ECG-based cardiac disease diagnosis, several studies have attempted to incorporate textual prompt information to construct multimodal models, thereby enhancing the performance of ECG-based disease classification. Liu et al. [Liu et al., 2024] align ECG signals with clinical text reports to enable zero-shot diagnosis. Ye et al. [Ye et al., 2025] propose a dual-adapter framework to fuse medical time-series signals and clinical text from complementary perspectives. Seki et al. [Seki et al., 2025] investigate multimodal large language models for ECG understanding in zero-shot visual question answering settings. Chen et al. [Chen et al., 2025] integrate ECG signals with patient static information encoded as textual prompts for hierarchical disease classification. However, the textual prompts adopted in these approaches follow a fixed paradigm. Given the substantial inter-patient variability within the same diagnosis, such fix prompting can limit model generalization.

3 Methodology

In this section, we introduce the proposed SGRL method, whose overall framework is illustrated in Figure 2. First, we pretrain an ECG encoder that can produce semantic-aligned ECG embeddings. Next, we employ LSPA to generate adaptive label semantic embeddings for ECG signals. We then apply SGRD to fuse the ECG embeddings with adaptive label semantic embeddings and separate class semantic anchors. Finally, SGRL integrates the ECG classification logits and anchor-based semantic matching scores for classification, and is jointly optimized using multiple loss functions.

3.1 ECGEncoder Pretraining

During the pretraining stage, we aim to learn an ECG encoder that produces ECG embeddings aligned with the label semantic embedding. Given the ECG embedding ei of the ith sample produced by the ECG encoder and the corresponding label semantic embedding si generated by the LLM, we adopt the symmetric InfoNCE loss used in CLIP to align the ECG and semantic embeddings:

where B denotes the batch size, and τ is is the temperature coefficient. The pretrained ECG encoder serves as the backbone network in the subsequent stages, providing initial ECG embeddings for subsequent LSPA and SGRD.

3.2 Label-Semantic Prompt Adaptation

Label Semantic Prompt Construction. We first define an initial label semantic prompts for each category. Unlike traditional approaches that only use the disease name as the label semantic prompt, we augment each category prompt by incorporating synonyms of the corresponding disease. In addition, depending on the specific characteristics of each cardiac disease category, we selectively include the full name or abbreviation, as well as the names of related subtypes or supertypes of cardiac diseases.

We tokenize the initial label semantic prompts to obtain their corresponding initial prompt representations, denoted as V I = [vI1 , ..., vIC ], V I ∈ RC×d , where C denotes the number of categories, d denotes the feature dimensionality of each category, and vI i represents the initial prompt representation of the i-th category.

Learnable Prompt Parameterization and Calibration. Considering that initial label semantic prompts are insufficient to represent the fine-grained semantic characteristics of ECG signals, we introduce a set of learnable prompt parameters to supplement the semantic information:

where d′ denotes the optimization dimension for each category, and pi represents the learnable parameter associated with the i-th category.

We sample a matrix Θ ∈ RC×d′ from a standard normal distribution Θ ∼ N(0, σ2I), where σ2 is set to 1 in this study and I is the identity matrix. We then map the matrix Θ into the vocabulary index space to obtain the initialization of the learnable prompt parameter matrix P :

where V denotes the vocabulary size, ⌊⌉ denotes rounding to the nearest integer.

To ensure that the learnable prompt parameters can be effectively calibrated based on the ECG signals, we employ a lightweight MLP (Multilayer Perceptron) to map the ECG embedding z0 extracted by the pretrained ECG encoder into a semantic calibration vector:

where δ(·) denotes the ReLU activation function, W1, W2 are learnable weight matrices. Subsequently, a residual modulation mechanism is applied to adjust the learnable prompt parameters P :

where ⊙ denotes the element-wise multiplication with broadcasting, and ˜ P = [˜ p1, ..., ˜ pC] represents the ECG-calibrated learnable prompt parameters.

For each category i, we concatenate the initial prompt representation vI i with its ECG-calibrated learnable prompt parameter ˜ pi, thereby obtaining the optimized prompt representation:

Accordingly, the optimized prompt representation for all categories can be expressed as:

Semantic Consistency Constraint. To prevent the optimized prompt representation from drifting excessively away from the original medical prior during optimization, we introduce a semantic consistency constraint to stabilize the semantic refinement process. Specifically, we feed the optimized prompt representations V O and the initial prompt representations V I into frozen LLM to produce the corresponding semantic embeddings:

where SO and SI denote the optimized and initial semantic embeddings, respectively. Based on these embeddings, we define Semantic Consistency (SC) loss to constrain the deviation between SO and SI :

where C denotes the number of labels, and sOi , sIi represent the optimized and initial semantic embeddings of the i-th category, respectively.

3.3 Semantic-Guided Representation Decorrelation

Based on the optimized semantic embeddings, we adopt SGRD to fuse them with ECG embeddings via multi-head attention, producing a semantic-enhanced ECG embedding and ECG-integrated class semantic anchors. We further introduce Label-wise Decorrelation to reduce semantic overlap among anchors. These class semantic anchors serve as classmatching anchors for computing similarity with the semanticenhanced ECG embedding.

Specifically, given the optimized semantic embeddings SO , we treat each class-wise semantic embedding in SO as a semantic token. Given the ECG embedding z0 extracted by the pretrained ECG encoder, we treat it as the ECG token. These tokens are concatenated to form a unified sequence representation:

where z0 denotes the original ECG token, and zi(i = 1, ..., C) denotes the semantic token corresponding to the i-th class. Subsequently we feed Z into a Transformer encoder to facilitate effective interaction between ECG token and semantic tokens:

where e′ 0 is the semantic-enhanced ECG embedding, and e′ i(i = 1, 2, ..., C) denotes the ECG-integrated class semantic anchor (class semantic anchors for brevity) for the i-th class. By leveraging the self-attention mechanism, each semantic token selectively focuses on label-relevant ECG features, producing class semantic anchors. Meanwhile, the ECG token is enhanced by aggregating semantic information from all class contexts, thereby obtaining the semanticenhanced ECG embedding.

To mitigate semantic overlap among class semantic anchors e′ i across different classes, we introduce a Label-wise Decorrelation (LD) loss to suppress shared inter-class components and enhance class separability:

where C ′ denotes the number of classes that appear in the current training mini-batch, e′ i, e′ j ∈ RB×d are the class semantic anchors for the i-th and j-th classes, B is the batch size, d is the feature dimension, and Corr(e′ i, e′ j) denotes the Pearson correlation coefficient between e′ i and e′ j. Minimizing LLD encourages anchors from different classes to be decorrelated, yielding a more discriminative class-specific semantic constraint space for ECG representations.

To identify which the semantic constraint space the semantic-enhanced ECG embedding belongs to, we compute the similarity between the semantic-enhanced ECG embedding e′ 0 and the class semantic anchors e′ i, thereby obtaining class-wise matching scores:

where πi indicates the degree of matching between the ECG and the i-th diagnostic category in the semantic space.

Meanwhile, to preserve the discriminative information in the original ECG signal and facilitate stable gradient propagation, we employ a residual connection to fuse the original ECG token z0 with the semantic-enhanced ECG embedding e′ 0:

where Eout is the ECG output embedding obtained by residual fusion.

3.4 Classification and Training Optimization

To obtain the final multi-label classification results, we integrate the classifier logits derived from the fused ECG embedding Eout with the class-wise semantic matching scores:

where W and b are the weight and bias of the classification layer, π = [π1, . . . , πC] is the semantic matching similarity vector, and ˆ Y denotes the predicted probability over all classes. This integration enables both the ECG feature information and the semantic information from the label matching contribute to the final classification.

We adopt the Class-Balanced Binary Cross-Entropy (CBBCE) loss to measure the discrepancy between the predicted probabilities and the ground-truth labels in a multi-label setting. Given a training set with N samples and C classes, let yij ∈ {0, 1} denote the ground-truth label indicating whether the i-th sample belongs to the j-th class, and let ˆ yij ∈ [0, 1] denote the corresponding predicted probability. The CB-BCE loss is defined as:

where ωj represents the class-balanced weight associated with the j-th class, which is introduced to enhance the contribution of minority classes during training.

Finally, we combine LCB-BCE with LLD and LSC to jointly optimize the LSPA and SGRD:

where α and β are hyperparameters that balance the contributions of different loss terms.

4 Experiments

4.1 Experimental Setup

Pre-training Dataset. We pre-train our model on the MIMIC-IV-ECG dataset [Gow et al., 2023], which contains 800,035 paired ECG-report samples from 161,352 subjects. Each sample consists of a 10-second 12-lead ECG signal and its corresponding machine-generated diagnostic report. The original signals are sampled at 500 Hz and uniformly downsampled to 100 Hz for computational efficiency.

Downstream Datasets. PTB-XL [Wagner et al., 2020]: It comprises 21,837 12-lead ECG recordings collected from 18,885 patients. All signals are sampled at 500 Hz with a fixed duration of 10 seconds. In this work, we adopt the finest-grained set of 44 diagnostic labels to formulate a multilabel classification task.

SPH [Liu et al., 2022]: It is a publicly available 12lead multi-label ECG dataset, consisting of 25,770 recordings from 24,666 patients. The signals are sampled at 500 Hz, with recording durations ranging from 10 to 60 seconds. To ensure a consistent input length, recordings longer than 10 seconds are temporally truncated. The final task is constructed using 44 finest-grained diagnostic categories, resulting in a longtailed multi-label classification setting.

PhysioNet 2020 [Alday et al., 2020]: It contains 17,497 12-lead ECG recordings, all sampled at 500 Hz with a uniform duration of 10 seconds. We utilize the 26 diagnostic categories provided by the dataset to evaluate model performance on a long-tailed multi-label classification task.

Furthermore, to assess the degree of class imbalance in each dataset, we compute the normalized entropy (NE). A smaller NE value indicates a more skewed class distribution.

Implementation Details. All methods presented in this paper are implemented using the PyTorch 2.6.0 framework and trained and evaluated with Python 3.11. All experiments are conducted on a server equipped with an Intel(R) Xeon(R) Gold 6230 CPU (2.10 GHz), 251 GB of RAM, and four Tesla V100-PCIE-32GB GPUs. To improve the statistical stability of the results, five-fold cross-validation is employed, and the final performance is reported as the average over five runs. During model training, the learning rate is set to 1 × 10-4 , and the batch size is set to 256. The Adam optimizer is used for all experiments. In addition, all ECG signals originally sampled at 500 Hz are downsampled to 100 Hz.

Evaluation Metric. To comprehensively evaluate model performance while accounting for class imbalance, we adopt four metrics: Accuracy (Acc), macro-averaged F1 score (F1), macro-averaged area under the receiver operating characteristic curve (AUC), and Tail-AUC. Specifically, Tail-AUC is computed over tail classes whose sample sizes account for the lowest 1% of the overall class distribution.

4.2 Comparison with Baseline Methods

We compare SGRL with diverse baselines. ASGX, MSDNN, DenseNet, ML-ResNet, MLC-CNN, Autoformer, PatchTST, and FEDformer are conventional ECG classification models. Time-LLM is a time-text fusion method. DeepSMOTE and ADASYN are data augmentation approaches, while ASL is a loss-based reweighting strategy. In addition, ECGAD and ECGFounder are transfer-learning-based methods.

Table 1 summarizes the performance comparison between SGRL and baselines on three datasets. Overall, SGRL demonstrates stable and consistent advantages across longtailed datasets. On the relatively balanced PhysioNet 2020 dataset, conventional imbalance-handling methods, such as ASL,ADYSN and ECGAD, remain competitive. However, on the more severely imbalanced PTB-XL and SPH datasets, SGRL exhibits substantially larger improvements, outperforming the best competing methods by 6.88% and 9.86% in F1 score, respectively. Moreover, SGRL also achieves clear gains in Tail-AUC, exceeding second-best methods by about 3.35% on PTB-XL and 5.09% on SPH. These results indicate that SGRL effectively enhances inter-class separability by imposing semantic-space constraints, enabling robust discrimination across datasets with varying imbalance severities.

Table 2: Tail-AUC results of the LSPA ablation study.

4.3 Ablation Studies

Effectiveness of LSPA. To validate the effectiveness of LSPA, we individually remove the semantic consistency loss LSC, the ECG-calibrated learnable prompt parameters ˜ P , the semantic calibration vector γ, and the entire LSPA , and evaluate their impacts on model performance. As observed from the results in Table 2, all variants exhibit clear performance degradation, confirming that LSPA is critical to SGRL. Among all ablation variants, excluding w/o LSPA, removing ˜ P results in the most significant performance degradation. This indicates that the adaptive semantic label prompts provide more informative semantic constraints, which is beneficial for improving the recognition of rare tail classes.

Effectiveness of SGRD. To assess the effectiveness of the SGRD module, we respectively remove the label-wise decorrelation loss LLD, the class-wise matching scores π, and the entire SGRD. As summarized in Table 3 all variants lead to clear performance drops, confirming the importance of SGRD within SGRL. Among all ablation variants, excluding w/o SGRD, removing π lead to most performance degradation, indicating that class-wise matching in the separated semantic space is crucial for effective recognition of tail classes.

4.4 Discussion

Impact of Different ECG Encoders. To evaluate the impact of ECG encoder choices on model performance, we compare five representative architectures: ResNet1d [He et al., 2016], DenseNet [Jahmunah et al., 2023], MS-CNN [Wang et al., 2020], ECGTransformer [El-Ghaish and Eldele, 2024], and RNN [Salloum and Kuo, 2017]. As shown in Table 4, incorporating the SGRL framework consistently improves the performance of all encoders. Among them, MSCNN achieves the best results, benefiting from its multi-scale convolutional design that captures discriminative ECG features across diverse temporal scales. Accordingly, MS-CNN is adopted as the backbone encoder in all experiments.

Impact of Different LLMs. To investigate the impact of using different LLMs as text encoders on the results, we conduct a comparative analysis of three representative models: BERT, Llama 3, and GPT-2. The experimental results are shown in Figure 3. It can be observed that BERT achieves the best performance on the highly imbalanced PTB-XL and SPH datasets. This advantage can be attributed to its bidirectional attention mechanism, which enables more comprehensive capture of contextual semantic information and provides greater robustness on datasets with skewed class distributions. Accordingly, considering overall performance, BERT is adopted as the text encoder for all experiments.

Impact of Hyperparameter α. To study the impact of the weighting coefficient α for the label-wise decorrelation loss, we vary alpha from 0.6 to 1.5, with results shown in Figure 4(a). The model achieves optimal performance at α = 0.9.

Bold denotes the optimal results. Grey represents the results of training using the SGRL framework.

Figure 4: Hyperparameter analysis on PTB-XL.

Smaller values fail to sufficiently decorrelate class semantic anchors, whereas larger values overemphasize the decorrelation constraint and interfere with discriminative feature learning. Thus, α = 0.9 provides the best balance between regularization strength and the primary classification objective.

Impact of Hyperparameter β. To investigate the impact of the hyperparameter β on model performance, we vary its value uniformly within [0, 2], with the results shown in Figure 4(b). β controls the strength of semantic consistency during training, with larger values enforcing stronger alignment between the optimized and initial semantics. The model achieves optimal performance at β = 0.5, indicating that a moderate semantic consistency constraint is most effective, while overly weak or strong constraints lead to performance degradation. Accordingly, β is set to 0.5 in all experiments.

Impact of Hyperparameter ω. To investigate the impact of the class-balanced weight ω on model performance, we vary its value from 1 to 6 , with the results shown in Figure 4(c). As ω increases, the Tail-AUC consistently improves, indicating that assigning larger weights to rare classes is beneficial for enhancing tail-class performance. However, when ω exceeds 3, the overall accuracy drops sharply, as excessive emphasis on rare classes compromises the recognition of head classes and leads to degraded overall performance. Based on this trade-off, we set ω = 3 in all experiments.

Impact of the Learnable Parameter Dimensionality d′ . To analyze the impact of the learnable parameter dimensionality d′ in the LSPA module, we evaluate four settings: 128, 256, 512, and 1024. As shown in Figure 4(d), the best performance is achieved at d′ = 256, while both smaller and larger values lead to performance degradation. This indicates that setting the dimensionality to 256 effectively supplements the original prompt information while avoiding the introduction of excessive redundant features. Accordingly, d′ is fixed to 256 in all experiments.

Visualization. We employ Grad-CAM [Selvaraju et al., 2017] to visualize the feature weights of ECG signals from a patient with third-degree atrioventricular block (3AVB, a rare class) using SGRL and ECGAD. In this 3AVB case, the P waves and QRS complexes lack a fixed conduction relationship, exhibiting independent rhythmic patterns (known as atrioventricular dissociation). As illustrated in Figure 5, SGRL concurrently focus on abnormal rhythmic features of P waves and QRS complexes. In contrast, ECGAD primarily focuses on the QRS complex, difficult to identify 3AVB.

5 Conclusion

This study proposes SGRL for long-tailed ECG classification. By exploiting the semantic correlations among class labels, SGRL organizes ECG representations into independent semantic spaces while reducing inter-class correlations. Specifically, LSPA is designed to generate patient-adaptive label text prompts, while SGRD decorrelates class semantic anchors. Extensive experiments conducted on multiple public long-tailed ECG datasets demonstrate that SGRL consistently outperforms existing baselines, validating its effectiveness in improving both overall classification performance and tailclass recognition. Despite its effectiveness, SGRL depends on predefined diagnostic label categories, whose granularity and annotation quality may affect semantic modeling. In future work, we plan to incorporate self-supervised learning to reduce label dependence and improve the robustness.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No.U25A20449, 62372476). This work was also carried out in part using computing resources at the High Performance Computing Center of Central South University.