Contrastive Flow Matching for Sequential Recommendation

1 Introduction

Sequential recommendation (SR) [Liu et al., 2021; Wang et al., 2022] stands as a cornerstone of modern e-commerce [Wang et al., 2020] and content platforms [Schedl et al., 2018; Hansen et al., 2020], which aims to predict the next item by modeling users' personalized preferences based on their historical interaction sequences. Recently, generative sequential recommendation [Ni et al., 2023; Yuan et al., 2019] has emerged as a promising paradigm that goes beyond NDCG@10 (%)

∗ Corresponding author.

8

6

4

2

0

ically lacks explicit inter-user separation signals. This deficiency raises a critical concern regarding the model's ability to accurately capture unique user interests and generate personalized items that satisfy individual preferences. Our empirical study on FM4Rec across four datasets validates this concern. As shown in Figure 1, the model performs consistently well on popular targets but deteriorates substantially on unpopular ones. This indicates that, limited by its personalization capacity, the model tends to generate common patterns for all users.

This tendency can be attributed to a critical phenomenon in flow matching known as averaged flow, where the learned velocity field (and thus its implied velocity directions) becomes overly similar across different users. Specifically, FM-based models construct an interpolated state by fusing the target item with noise, and utilize a shared predictor to estimate the item representation at this state. Crucially, this estimated representation is then used to estimate the corresponding velocity field for transport. However, due to the lack of inter-user separation guidance, the shared predictor tends to yield overly similar predicted representations for different users. Subsequently, this similarity propagates to the velocity space, causing the derived velocity directions to become averaged across users. Consequently, during inference, these averaged velocities drive the generation toward generic outcomes for all users, resulting in a severe loss of personalization.

To address the above limitations, we propose Contrastive Flow Matching for sequential recommendation (ConFM). Motivated by the causes of averaged flow, ConFM introduces inter-user separation signals from two perspectives, enabling the model to better preserve user personalization. Specifically, to tackle the issue that predicted item representations at interpolation states become insufficiently distinguishable, we design a Representation Contrastive Regularizer (RCR) that explicitly pushes different users' predicted item representations apart, thereby better retaining user-specific preferences. In addition, we propose a Velocity Contrastive Regularizer (VCR), which directly enforces inter-user separation in the induced velocity space, ensuring that each user follows a distinct generation trajectory. By enforcing user separation in both the representation space and the induced velocity space, ConFM produces more user-specific item generation at inference, thereby enhancing personalized recommendation.

The main contributions of this work are summarized as follows:

• We identify the flow averaging issue in FM-based sequential recommendation through diagnostic experiments and analyze that it arises from the lack of explicit inter-user separation constraints.
• We propose a contrastive flow matching (ConFM) for sequential recommendation, which explicitly introduces inter-user separation regularizer in both the representation space and the velocity space to preserve personalization in generated recommendations.
• Extensive experiments on four public datasets show that ConFM consistently outperforms competitive baselines, with particularly significant gains on long-tail items.

2 RelatedWork

2.1 Sequential Recommendation

Sequential recommendation aims to predict the next item by learning user preferences from historical interaction sequences. In early work, recurrent neural networks were introduced to sequence modeling. The representative method GRU4Rec [Hidasi et al., 2016] employs gated recurrent units to encode user interaction sequences, regarding the generated hidden states as representations of current preferences. In recent years, to more accurately capture dependencies within user behavioral sequences, self-attention has been widely explored in sequential recommendation. SASRec [Kang and McAuley, 2018] and STOSA [Fan et al., 2022] respectively introduce causal self-attention and structure-aware attention, which better capture the evolving patterns in user interaction sequences. BERT4Rec [Sun et al., 2019] learns sequence representations through introducing bidirectional transformers and masked word prediction. To mitigate data sparsity, contrastive learning has also been introduced to sequence recommendation. S3 -Rec[Zhou et al., 2020] combines multiple self-supervised pre-training tasks to learn more robust sequence representations. CL4SRec [Xie et al., 2022] constructs positive pairs through data augmentation and uses other sequences as negative samples for contrastive learning, thereby promoting discriminative sequence representations.

2.2 Generative Sequential Recommendation

Generative models have begun to be widely used in sequential recommendations with their advantages in modeling uncertainty and capturing the evolution of user interests. The existing methods can generally be classified into two representative routes: one is based on the diffusion model, and the other is based on flow matching.

Diffusion Model-based Methods. Diffusion models have been successfully adapted to sequential recommendation in several recent studies. Early diffusion-based recommenders, such as DiffuRec [Li et al., 2023] and DreamRec [Yang et al., 2023], typically cast next-item prediction as a historyconditioned denoising process. Recently, PreferDiff [Liu et al., 2025c] further improves training efficiency and preference modeling, while DiQDiff [Mao et al., 2025] alleviates the data bias that diffusion models may suffer from during generation. However, diffusion-based recommenders still rely on stochastic, multi-step sampling at inference time, which is computationally expensive and can introduce generation variance that blurs fine-grained user intent. This drawback motivates more efficient and stable generative alternatives, such as flow matching.

Flow Matching-based Methods. Flow matching learns a continuous-time velocity field to deterministically transport samples from a noise prior toward the target distribution [Lipman et al., 2023]. Building on this idea, it has been progressively introduced into social graph recommendation [Huang et al., 2025] and collaborative filtering [Liu et al., 2025a]. In sequential recommendation, FM4Rec [Liu et al., 2025b] performs history-conditioned deterministic transport to generate next-item representations. However, FM4Rec shares a single estimator across users. As a result, it provides limited objective-level encouragement for user-specific diversity. Therefore, more discriminable models are desirable.

3 Preliminaries

3.1 ProblemFormulation

In sequential recommendation, we consider the user set U = {u1, u2, . . . , u|U|} and the item set I = {i1, i2, . . . , i|I|}, where |U| and |I| denote the numbers of users and items, respectively. For each user u ∈ U, the interaction history is an ordered sequence S = (i1, i2, . . . , im) of length m. Given S, we recommend the next item by

where Θ(i | S) denotes a history-conditioned scoring function (or evaluation metric) for item i.

3.2 FlowMatching

Flow Matching (FM) is a recent continuous-time generative paradigm that learns a transport from a simple base distribution to the target distribution via a velocity field.

Formally, let x0 ∼ p0 be a sample from the source distribution, and let x1 ∼ p1 be a sample from the target distribution. FM constructs an intermediate state at time t ∈ [0, 1] via linear interpolation:

Along this linear path, the ground-truth velocity is

FM trains a time-dependent velocity field vθ(xt, t) by regression:

At inference, we draw an initial sample x0 ∼ p0 and generate ˆx1 by solving the ODE dxt = vθ(xt, t) dt, which is typically integrated using a deterministic Euler method.

4 Methodology

The proposed ConFM framework is motivated by a simple challenge in FM-based sequential recommendation: due to the lack of inter-user separation guidance, a shared predictor tends to produce similar predicted representations, which further leads to averaged velocity directions and ultimately weakens personalized prediction. To address this, ConFM introduces two inter-user contrastive regularizers to the FM-based backbone. The first is the Representation Contrastive Regularizer, which promotes mutual distinguishability among users' predicted next-item embeddings within each mini-batch. The second is the Velocity Contrastive Regularizer, which pushes each user's induced velocity away from the ground-truth velocity targets of randomly sampled other users. These two regularizers explicitly enforce inter-user separation in both the representation and velocity spaces, enhancing the model's ability to capture user-specific preferences. Figure 2 illustrates the overall framework of ConFM.

4.1 Backbone method of ConFM

In this subsection, we first briefly review FM4Rec [Liu et al., 2025b], the backbone model upon which our ConFM is built, which pioneered the introduction of flow matching to sequential recommendation. FM4Rec learns a history-conditioned generation mechanism that transforms a noisy intermediate state into a next-item representation for ranking. It comprises three key steps, detailed as follows.

Interpolation state construction. For each training instance, given the interaction history S = (i1, . . . , im) and the target item im+1, the backbone takes the target item embedding as the target endpoint x1 = Emb(im+1) ∈ Rd , where Emb(·) denotes a trainable embedding layer of item ID. The model then samples a noise endpoint x0 ∼ N(0, I) and draw t ∼ Uniform(0, 1).

Accordingly, the interpolated state on the straight path between these two endpoints is defined as

Finally, the tuple (xt, t, S) is fed into the backbone network to produce the next-item representation for training.

Ground-truth velocity and FM objective. Given the interpolation in Eq. (5), the ground-truth velocity along the straight path is

Following the FM4Rec-style instantiation, the historyconditioned predictor generates an endpoint estimate ˆx1 = fθ(xt, t, S). Instead of employing a separate head for velocity regression, the predicted velocity field is defined as the induced displacement from the noise endpoint:

The model is then optimized via the standard flow matching regression objective, which minimizes the difference between the predicted and ground-truth velocities:

Predictor parameterization. To instantiate the historyconditioned predictor fθ(xt, t, S), a causal Transformer backbone is adopted that conditions each historical token on the interpolation state xt and the time t.

Let S = (i1, . . . , im) denote the interaction history. For the k-th interacted item ik, its embedding is denoted by ek = Emb(ik). The input at position k is formed by fusing ek with the injected state:

where ⊙ denotes element-wise multiplication, t is a learned embedding of the scalar time t, and δ controls the magnitude of the stochastic injection.

A stack of n1 causal self-attention blocks is then applied to encode the noised sequence and obtain contextualized hidden states:

On top of these states, an additional n2-layer stack is applied, and the last-token output is used as the predictor output

The output fθ(xt, t, S) is taken as the predicted next-item representation ˆx1.

4.2 Representation Contrastive Regularizer

As discussed in Section 1, for FM-based methods that derive the velocity field by predicting the next item at interpolated states, the root cause of the average flow issue-which degrades personalization-is that all users share a single predictor without inter-user separation constraints. Even when the input is conditioned on user history, the model tends to output an average pattern across users to minimize the global training loss. Consequently, the predicted embeddings for many users become increasingly similar, leading to a severe degradation in representation separability.

To explicitly enforce inter-user distinguishability, we introduce a Representation Contrastive Regularizer (RCR). Specifically, RCR operates within each mini-batch by penalizing high similarities between the predicted item embeddings of different users, thereby preventing the shared predictor from collapsing into a generic pattern.

Formally, consider a mini-batch of B training instances {(Sb, im+1,b)}Bb=1 . For each instance b, we sample x0,b ∼ N(0, I) and tb ∼ Uniform(0, 1), construct xt,b by Eq. (5), and obtain the predicted next-item representation

We then define an inter-user similarity penalty using a logsum-exp aggregator, and the RCR term is defined as

where sim(·, ·) denote a similarity function, such as cosine similarity. Minimizing LRCR encourages each ˆx1,b to be less similar to the other predictions in the mini-batch. As a result, the model tends to push the predicted embeddings of different users farther apart, making the predictions easier to distinguish.

4.3 Velocity Contrastive Regularizer

RCR promotes separation among prediction embeddings, which can be viewed as an indirect way of separating velocities through representations. However, this remains insufficient because a user's velocity vector is not determined by the prediction alone. Instead, it is computed jointly from the prediction and a randomly sampled state. Given the random sampling of interpolation states, separation only in the embedding space does not guarantee that users' velocity directions are also well separated.

To this end, we propose the Velocity Contrastive Regularizer (VCR), which explicitly constrains the velocity field derived from the user's state. By aligning the predicted velocity with the user's own ground-truth direction while repelling it from the ground-truth velocities of randomly sampled negative users, VCR ensures a clear separation between the user's generated trajectory and the paths of others

Formally, for a mini-batch of size B, we construct an interpolated state xt,b for each instance b and obtain the backbone prediction ˆx1,b = fθ(xt,b, tb, Sb).

We denote the backbone-predicted velocity field by vθ(xt,b, tb, Sb), and the corresponding target velocity by ut,b. The proposed velocity contrastive regularizer is defined as

where ut,b′ denotes the target velocity of a randomly sampled other user (instance) in the mini-batch.

4.4 Overall Objective and Inference

Overall training objective. Our proposed ConFM is built upon an FM-based backbone and further introduces two contrastive regularizers. Therefore, the overall training objective consists of two parts: (1) Following FM4Rec, we adopt its backbone training objectives, which we denote by Lbase, including the standard flow matching loss LFM, an auxiliary mean-squared error loss LMSE, and a cross-entropy loss LCE; (2) Two inter-user contrastive regularizers, namely the representation contrastive regularizer LRCR for encouraging distinguishable next-item representations across users and the velocity contrastive regularizer LVCR for encouraging distinguishable velocity directions across users. Putting them together, the overall objective is

where λR and λv control the strengths of the representationlevel and velocity-level contrastive regularizer, respectively.

Inference. At inference time, given a user history S, we start from a noise endpoint x0 ∼ N(0, I) and iteratively refine it to obtain a next-item representation. Following the FM4Rec-style instantiation, the velocity field at time t is

Given a total number of inference steps q, we set the step size as ∆t = 1q and update xt with an Euler solver:

where t takes values 0,∆t, . . . , 1 -∆t. After q updates, we obtain x1 as the generated next-item representation, which is then scored against candidate item embeddings for retrieval.

4.5 Complexity Analysis

We analyze the time complexity of ConFM for training and inference. Let B, n, d, N, and q denote the batch size, sequence length, representation dimension, number of Transformer blocks, and the number of Euler update steps at inference, respectively.

Training complexity. For each mini-batch, The computational cost of causal self-attention and feed-forward networks is O(N · B · (n2d + nd2 )). In addition, RCR adds an O(B2d) cost for in-batch pairwise similarities, while VCR adds an O(Bd) cost for vector-level operations. Therefore, the overall training complexity can be summarized as O(N · B · (n2d + nd2 ) + B2d).

Inference complexity. Since RCR and VCR are introduced only during training, ConFM incurs no additional inference overhead beyond one computation per Euler update step; therefore, its inference time complexity is O(q · N · B · (n2d + nd2 )).

Overall, compared with the computational cost of causal self-attention and feed-forward networks, RCR and VCR introduce only lightweight additive overhead, indicating that they can improve performance without significantly affecting the training and inference efficiency.

5 Experiments

In this section, we conduct comprehensive experiments to investigate the following research questions:

• RQ1: How does ConFM perform compared with representative baseline methods for sequential recommendation?
• RQ2: Can ConFM alleviate the performance degradation on unpopular targets observed in recommenders?
• RQ3: How do the two proposed components, VCR and RCR, contribute to the recommendation performance of ConFM?
• RQ4: How sensitive is ConFM to different hyperparameter settings?

5.1 Experimental Settings

Datasets. To evaluate the effectiveness of ConFM, we conduct experiments on four widely used datasets: Beauty, Sports, Toys, and Yelp. Beauty, Sports, and Toys are category-specific subsets from the Amazon review corpus [McAuley et al., 2015]. Yelp is a widely adopted benchmark for business recommendation. Following prior practice [Kang and McAuley, 2018], we filter out inactive users and infrequent items with fewer than five interactions. The detailed statistics are summarized in Table 1.

Baselines. We compare ConFM with representative baselines from three categories of sequential recommendation models: (1) Traditional recommenders: GRU4Rec [Hidasi et al., 2016], Caser [Tang and Wang, 2018], SASRec [Kang and McAuley, 2018], and BERT4Rec [Sun et al., 2019] predict the next item with discriminative sequence models (e.g., RNN/CNN/Transformer), which can capture dependency patterns in user behavior sequences. (2) Contrastive-based methods: CL4SRec [Xie et al., 2022] and CaDiRec [Cui et al., 2024] adopt data augmentation and contrastive objectives to alleviate representation degeneration and data sparsity in sequential recommendation. (3) Generative recommenders: DreamRec [Yang et al., 2023], DiQDiff [Mao et al., 2025], DiffuRec [Li et al., 2023], and FM4Rec [Liu et al., 2025b] utilize generative modeling to model item distributions or generate the next-item representation conditioned on interaction sequences.

Evaluation Settings. In our experiments, we evaluate recommendation performance using HR@K and NDCG@K, with results reported for K ∈ {5, 10, 20}. All datasets are split into training, validation, and test sets under the leaveone-out protocol. For a fair comparison, we perform full-sort evaluation over the entire item set without negative sampling.

Implementation Details. All experiments are conducted on a server equipped with eight NVIDIA Tesla V100 GPUs. For all baseline methods, we use their official open-source implementations and adopt the best hyperparameter settings reported or recommended by the original authors for fair comparison. For our ConFM, we train the model with Adam using an initial learning rate of 0.001 and a batch size of 512. The embedding dimension is set to 128, the maximum sequence length is 50, and dropout rates are 0.1 on model representations and 0.3 on the item embedding layer. The weights of VCR and RCR are tuned on the validation set via grid search over λV ∈ {1e-3 , 5e-3 , 1e-2 , 5e-2 , 1e-1 , 5e-1 } and λR ∈ {1e-3 , 5e-3 , 1e-2 , 5e-2 , 1e-1 , 2e-1 , 3e-1 , 4e-1 , 5e-1 , 7e-1 }.

5.2 Overall Performance (RQ1)

To answer RQ1, we compare ConFM with representative baselines on four datasets and report the results in Table 2. From the table, we have the following observations: (1) Contrastive-based methods generally perform better than traditional sequential recommenders. This suggests that contrastive supervision introduces additional representation-level training signals beyond next-item prediction, enabling the model to learn more discriminative and noise-robust user representations under sparse feedback, and consequently enhancing recommendation accuracy. (2) Generative recommenders further perform better than contrastive-based methods. In general, they learn a history-conditioned generative process in the latent space to produce next-item embeddings. This continuous formulation enables smoother preference evolution modeling and can better capture complex and multi-modal user preferences by explicitly modeling uncertainty and producing more expressive retrieval representations. (3) Our ConFM achieves the best performance on four datasets across all evaluation metrics. Compared with the strongest baseline FM4Rec, ConFM achieves an average SASRec CL4SRec CaDiRec DiQDiff DiffuRec FM4Rec ConFM

Figure 3: Popularity-stratified Performance. The arrow annotations indicate the relative improvement of our method over the bestperforming baseline in each setting.

relative improvement of 5.87%, 4.63%, 3.74%, and 2.33% on Yelp, Sports, Toys, and Beauty, respectively.

These improvements are mainly due to two aspects: (i) By pushing the generated next-item embeddings away from each other, ConFM enhances inter-user separability, leading to more distinctive prediction representations; (ii) ConFM suppresses the shared estimator from producing overly similar velocity directions, thereby alleviating the averaged flow effect and improving personalized generation.

5.3 Popularity-stratified Performance (RQ2)

To address RQ2, we conduct a popularity-stratified experiment on the Yelp and Sports datasets. Following previous work [Cheng et al., 2024], we treat the top 20%most frequent items in the training set as popular items and the remaining ones as unpopular items. Test instances are then grouped according to the popularity of their target items. Figure 3 illustrates the performance of our method compared to the most representative baselines from each category. From these results, we make the following observations: First, all methods exhibit significant performance degradation on the unpopular group, indicating that predicting such items demands robust preference modeling capabilities. Second, our ConFM brings clear relative gains on unpopular items: it achieves average relative improvements of 14.81% and 8.70% on Yelp and Sports, respectively. Simultaneously, it demonstrates highly competitive performance on the popular group, indicating that ConFM enhances performance on unpopular items without sacrificing the accuracy for popular ones.

These results suggest that introducing such inter-user separation designals helps produce more distinctive and personalized generated representations, ultimately providing users with more personalized item recommendations.

5.4 Ablation Study (RQ3)

To answer RQ3, we conduct an ablation study to verify the effectiveness of each proposed component in ConFM. Specifically, we construct three variants of ConFM: (i) w/o VCR, which removes the velocity contrastive regularizer from the training objective; (ii) w/o RCR, which removes the representation contrastive regularizer; and (iii) w/ Triplet, which replaces the velocity contrastive regularizer with a triplet-based contrastive objective. The results are reported in Table 3.

As shown in Table 3, all variants perform worse than ConFM. Specifically, the performance drop of w/o VCR suggests that a shared estimator without inter-sample separation signals can make generation trajectories less distinctive. This weakens fine-grained personalization, thereby degrading the final performance. Moreover, the larger degradation of w/o RCR suggests that strengthening inter-user discriminability at the representation level is crucial for improving the overall metrics. Finally, the performance drop of w/ Triplet may be attributed to the relatively weaker supervision provided by a generic triplet objective: compared with the more direct MSE regression supervision, the triplet loss only imposes relative distance constraints and offers less explicit optimization guidance, which can lead to less stable model performance.

5.5 Sensitivity Analysis (RQ4)

To answer RQ4, we further evaluate the sensitivity of ConFM to the two regularizer weights, λV and λR, on Yelp and Sports. The results are reported in Figure 4.

For λV , when varying it from small to large values, the performance on both datasets shows a clear increase-thendecrease trend. A moderate λV consistently yields the best results, suggesting that an appropriate degree of inter-flow separation helps the model generate more personalized trajectories. However, overly large λV leads to noticeable degradation, indicating that excessively strong inter-flow constraints may overly separate the generation trajectories and interfere with fitting the main recommendation objective, thereby hurting overall performance. For λR, varying it over a wide range leads to relatively mild performance fluctuations on both Yelp and Sports. This indicates that encouraging separation among the predicted embeddings yields relatively stable gains, and the model is comparatively less sensitive to the exact choice of λR. One possible reason is that λR mainly acts on the output embedding space to enlarge the differences among generated embeddings, without altering the trajectory-level dynamic generation process; therefore, changes in its weight tend to have a smoother impact on the overall performance.

6 Conclusion

In this paper, we identify a key challenge in FM-based sequential recommendation: during the generation process, the averaged flow phenomenon can make the generated item representations less distinguishable. To mitigate this issue, we propose ConFM, which introduces explicit inter-user separation into flow matching training by separating predicted representations and induced velocity directions, thereby promoting more distinguishable generation. Extensive experiments on four public datasets demonstrate that ConFM effectively alleviates the indistinguishable generation issue and consistently outperforms strong baseline.

Acknowledgements

This research was partially supported by the NSFC (62376180, 62506254), the National Key Research and Development Program of China (2023YFF0725002), the NSF of Jiangsu Province (BK20250822), Suzhou Science and Technology Development Program (SYG202328), the NSF of Jiangsu Higher Education Institutions (25KJB520042) and the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Contribution Statement

Wangyu Jin and Yang Xue contributed equally to this work.