LLM-Summarized Interest Distillation for Multimedia Recommendation

1 Introduction

Due to e-commerce and various online services, the rapid growth of digital multimedia data increases the need for personalized services. Identifying user-preferred content from vast data effectively is a crucial topic in recent years. Collaborative filtering, at the core of recommendation systems, predicts user preferences by mining historical user-item interactions. However, data sparsity, particularly with long-tail items [Yin et al., 2012] and users with limited activity records [Idrissi and Zellou, 2020], poses a significant challenge in modeling interests based solely on interactions. Aiming at enhancing recommendations, recent studies have incorporated auxiliary textual data to enrich user and item representations. Despite their enhancements, inadequate exploration of semantic signals also limits the understanding of users' personalized interests.

∗Corresponding author.

Pioneering studies [Wang et al., 2024] leverage contextual signals from interactions by encoding textual data of items and aggregating them to represent user interests. However, empirical findings indicate that lengthy and unstructured text sequences struggle to depict user interests as they primarily describe items' attributes rather than user interests. On this point, we conduct an empirical study in Figure 1 illustrating that interest-irrelevant descriptions of items lead interest modeling to deviate from downstream recommendation objectives, thereby hindering recommendation performance. This phenomenon inspires a systematic manner to remove redundant data and extract interest-relevant signals when depicting users' interests through item description texts, thereby enhancing the description of interactions. Additionally, the structural neighbors of the sparse interaction graph characterize incomplete interest patterns, leading to biased recommendations. Most recent works have turned to data augmentation [Yu et al., 2024; Wu et al., 2024] for alleviating the data sparsity issue by incorporating task-relevant semantic neighbors. As the empirical study in Figure 1, data augmentation broadens the data landscape and better infers the interests of sparse users, enhancing the recommendation performance. This result suggests exploring an effective data augmentation method to provide more collaborative signals and enhance interest modeling for recommendations.

For interest modeling, this work investigates data enhancement with contextual texts and semantic neighbors of interactions. We propose a LLM-summarized interest distillation (SID) model to enrich interactions in interest description quality and collaborative neighbors quantity for multimedia recommendation. It employs LLMs' understanding and reasoning ability to summarize users' interests from their interactions for depicting users' interests in compacted texts. With this dedicated interest representation, semantically similar neighbors are retrieved, which serve as teachers to transfer multimodal collaborative knowledge to behavioral sparse users. Both interest summarization and retrieval augmentation work to enrich interest modeling for promoting recommendations. The primary contributions of this work are summarized as follows.

• We propose an LLM-summarized interest distillation (SID) model by interest summarization and retrieval augmentation to enhance interactions in both interest descriptions and collaborative neighbors.
• LLMs' understanding and reasoning capabilities are utilized to summarize users' interests from interaction contextual texts, removing interest irrelevant descriptions, thereby improving interest modeling in texts.
• Instead of directly augmenting graph structure, the retrieval-augmented neighbors work as multiple teachers to transfer their multimodal knowledge to the student node, adaptively augmenting collaborative signals for recommendation.

2 RelatedWork

2.1 Multimedia Recommendation

The multimedia recommendation system integrates contextual information of behavioral interactions for recommendation. Representative VBPR [He and McAuley, 2016] integrates visual information into embeddings of matrix factorization [Rendle et al., 2009] to jointly represent user interests. LGMRec [Guo et al., 2024] learns multimodal embeddings in a local graph and combines the global embeddings from a hypergraph to comprehensively represent interests. SMORE [Ong and Khong, 2025] integrates multimodal features in the frequency domain and performs multimodal graph learning to represent interests. Besides modality integration, recent works optimize modality alignment to fuse multiple modalities in interest modeling. BM3 [Zhou et al., 2023] optimizes interaction graph reconstruction with inter- and intra-modality alignment to aggregate contextual contents into recommender learning. DiffMM [Jiang et al., 2024] injects modality-aware knowledge into graph learning of interactions and highlights modality consistency for multimodal interest representation. Existing multimedia models usually focus on embedding and aligning modalities to represent interests. However, they ignore to explore the quality of modality data for learning interests and recommenders. It motivates the extraction of interest-relevant signals from behavior contextual modalities to improve their effectiveness in aiding behavioral data for interest learning.

2.2 LLM-Based Recommendation

Large language models have gained widespread recognition due to their textual understanding capability with extensive world knowledge. A growing body of studies [Zhang et al., 2024; Sheng et al., 2025] has leveraged LLM to encode users and items, extracting semantic features to depict the interaction graph in recommender learning. For example, NoteLLM [Zhang et al., 2024] utilizes LLM to encode items into a special token, which is fed into a graph contrastive learning for representing and recommending items. LRD [Yang et al., 2024] follows discrete variational autoencoder to encode items with LLM and extracts latent relations for reconstructing items and modeling users. AlphaRec [Sheng et al., 2025] encodes items via LLM to initialize graph nodes and performs graph propagation to capture user interests with both semantic and structural signals. Beyond the LLMs' understanding capability, recent research [Tsai et al., 2024; Ren et al., 2024] has also witnessed their reasoning advantage to mine interests implicitly hidden in users' interactions. Representative RLMRec [Ren et al., 2024] harnesses LLM to generate detailed user and item profiles for aiding behavioral signals. It effectively addresses the potential noise and task-irrelevant information from items' descriptions. The advanced reasoning and summarization abilities of large language models, surpassing their encoding proficiency, position them as a powerful tool for investigating the contextual texts of interest learning.

2.3 Contrastive Learning-Based Recommendation

Contrastive learning encodes the data structure by discerning samples, where the inherent self-supervised signals effectively address the data sparsity in personalized recommendation tasks. Representative SGL [Wu et al., 2021] augments graph views through random perturbation and adopts contrastive learning to ensure view consistency for embedding learning. CLCRec [Wei et al., 2021] uses contrastive learning to maximize mutual information between user-item and itemitem pairs to reformulate representation learning. SimGCL [Yu et al., 2022] adds random noises on node embeddings to form contrastive views for contrastive learning, demonstrating InfoNCE loss is primary to align learning views. LightGCL [Cai et al., 2023] contrasts global and local collaborative views, achieving unconstrained structural refinement in embedding learning. LightCCF [Zhang et al., 2025] builds contrastive learning to bring users closer to their positive items and separate them from other users' positive pairs to improve neighborhood aggregation. Since InfoNCE loss possesses excellent neighborhood aggregation capability in contrastive learning, it is desired to transfer knowledge frommultimodal neighbors for interest learning.

3 ProblemDefinition

Let U and I denote the sets of users and items with |U| = M and |I| = N. For user u ∈ U, Iu is the set of items that user u has interacted with, Iu ⊆ I. The historical behaviors are recorded in a binary interaction matrix A ∈ {0, 1}M×N , where Aui = 1 if i ∈ Iu, otherwise Aui = 0. Besides, item i is semantically depicted with textual attributes of title, brand, category, and description, aggregated in Pi. The target of this work is to learn a recommender model f(·) for predicting user-item interaction score ˆ yui as

where Θ is the set of trainable parameters. The all-ranking protocol sorts the items by descending ˆ yui to recommend high-score ones for users.

4 Methodology

This section details the proposed LLM-summarized interest distillation (SID) for multimedia recommendation. As illustrated in Figure 2, SID involves three core modules of interest summarization, semantic retrieval, and multimodal selfdistillation.

4.1 Interest Summarization

In multimedia scenarios, items' textual attributes that users have interacted with carry far richer semantic collaborative signals than implicit behavioral interaction data. They can directly describe user interests through textual semantics, helping alleviate data sparsity issues. However, items' attribute texts inevitably introduce data noise and interest-irrelevant information, which can interfere with interest modeling and increase the risk of language model hallucination. For describing user interests with texts, this work investigates prompt engineering, which emphasizes the goal of interest modeling, assigns the model a specific role-playing identity, and imposes clear formatting constraints. It leverages LLM's understanding and reasoning capabilities to filter out interestirrelevant content, which guarantees the quality of semantic texts in describing user interests.

In the proposed SID, LLM (Llama3-8B) works with prompt engineering, including a system prompt Qs and a user prompt Qu, as illustrated in Figure 3. Qs defines the LLM's role as a precision assistant for JSON summaries, specifies the focus of personal interests only, describes item attributes by title, brand, category, and description, and outlines the input format as a list of JSON strings. Besides, Qu provides a user-specific interaction history Pi, i = 1, 2, ..., n in JSON format, specifies output requirements with 100-200 words, and includes a reference user interest profile for guidance. The prompt Qs and Qu guide LLM to summarize the user's interests and generate a comprehensive user profile that adheres to the specified JSON format represented in Equation (2).

where Pu is the generated user profile to describe user interests in texts. This profile comprehensively describes user interests by capturing both explicit preferences from interaction history and implicit preferences inferred from contextual and attribute-based contents. As Figure 2, the textual user profile Pu and item attributes Pi are encoded with a standard language model to embed users and items as Equation (3).

where e(0) u,t and e(0) i,t are the initialized semantic embeddings of user u and item i, the semantic projector proj(·) performs two linear projections with a LeakyReLU(·) nonlinear activation, and Enc(·) represents the textual encoder Qwen2.53B in which the last hidden state of the last token are taken as the encoder output.

Figure 3: An example of system prompt, user prompt, and the generated user profile in interest summarization.

4.2 Semantic Retrieval

To extract semantic collaborative signals, the proposed SID propagates the textual embeddings of users and items on the interaction graph by graph convolution as:

where e(l) u,t and e(l) i,t represent the learned semantic embeddings of user u and item i at the lth propagation layer, l = 1, 2, ..., L, Nu and Ni denote the first-hop neighboring node set of user u and item i in the interaction graph. The multiorder semantic collaborative signals are then aggregated by integrating the layerwise semantic embeddings as:

where eu,t and ei,t are the aggregated semantic embeddings of user u and item i. Here, semantic pretraining is performed with the recommendation task in Section 4.4 to optimize the semantic projector proj(·) in Equation (3), which guarantees the quality of semantic embeddings in the subsequent retrieval augmentation. As Figure 2, homogeneous similar users and items are retrieved by evaluating interest consistency along the semantic embeddings eu,t and ei,t as:

where gu and gi are the set of users/items retrieved for user u and item i, cos(·) measures the semantic cosine similarity, and topk(·) returns top-k users u′ for u and top-k items i′ for i. This semantic retrieval measures pairwise semantic similarity between the target user/item and all the other users/items and selects the top-k neighbors by descending the candidates. The retrieved users/items act to augment neighboring collaborative signals for the sparse target node from the semantic view. It augments the sparse interaction data in collaborative neighbors quantitatively.

4.3 Multimodal Self-Distillation

In addition to the semantic collaboration, the behavioral collaborative signals are propagated on the interaction graph to learn behavior embeddings for users and items. As Figure 2, the behavior propagation performs the same graph convolution in Equations (4) and (5), where the behavior embeddings e(0) u,b and e(0) i,b are randomly initialized as learnable parameters in the proposed SID. The derived behavior embeddings eu,b and ei,b aggregate multi-order behavioral collaborative signals. SID simply fuses the semantic and behavior embeddings as:

where eu and ei are the fused multimodal embeddings of user u and item i. In the multimodal view, the retrieved neighbors gu and gi are treated as multiple teachers to augment the sparse interactions. Beyond directly augmenting the graph structure [Fan et al., 2023], we deploy an adaptive knowledge transfer manner by multimodal self-distillation to augment the interactions. The self-distillation is implemented with InfoNCE contrastive learning [Yuan et al., 2025] to transfer knowledge from teachers to the target user/item in Equation (8).

where s(eu, egu ) = ∑j∈gu cos(eu, ej) and s(ei, egi ) = ∑j∈gi cos(ei, ej) aggregate the cosine similarity between eu/i and ej, τsd is the temperature coefficient, Bu and Bi are respectively the sets of users and items in a batch. In optimizing LuSD(u) and LiSD(i), the gradient flow through the teacher embeddings is halted, keeping them frozen. It ensures the stability of the teachers in guiding interest learning. By minimizing the embedding disparity, the multimodal collaborative knowledge is aggregated from multiple teachers and injected into the target interest embedding for augmenting the sparse learning.

4.4 Recommendation and Optimization

As Figure 2, the learned multimodal embeddings eu and ei are utilized to match user interests. The pairwise matching is implemented with the inner product to predict the interaction score as Equation (9).

To optimize the recommendation task, a adaptive weighted Bayesian personalized ranking loss Lrec is applied in SID to encourage positive item ranked above negative ones as Equation (10).

where O = {(u, i, j) | (u, i) ∈ O+, (u, j) ∈ O-}, O+ denotes the positive instance set, and O- is the negative instance set, σ(·) is a sigmoid function, ωu,i is the adaptive weight to measure the discrepancy between the target positive u - i and negative instances for user u in a batch, and τp is the temperature coefficient. Particularly, in the semantic retrieval, the recommendation loss Lrec is performed to pretrain the semantic embeddings eu,t and ei,t to ensure the quality of retrieval augmentation. Besides, the recommendation and the auxiliary self-distillation tasks are jointly optimized with a curriculum learning strategy [Wang et al., 2021], which gradually introduces distillation from a single side to dual sides. Following the sequential modeling of items and users in the text, we first perform self-distillation on the item side. Subsequently, we introduce the more complex user-side distillation, thereby reducing the optimization difficulty during the model training phase. The curriculum learning on multimodal embeddings is represented in Equation (11).

where β is the weight to control the contribution of selfdistillation, t represents the number of training epochs, and T denotes the threshold of training epochs to control the training complexity, converting from a single side to dual sides.

5 Experiments

5.1 Experiment Settings

Datasets

To verify the effectiveness of SID in multimedia recommendation, experiments are performed on three public datasets [McAuley et al., 2015], including Beauty, Toys, and Cellphones. These datasets provide user-item interaction records and textual descriptions for each item. The key statistical

indicators of the datasets are summarized in Table 1. Each dataset is split into training and test sets at an 8 : 2 ratio. User-item interaction pairs recorded in the interaction history are defined as positive pairs, while user-item pairs without observed interactions in the history are treated as negative pairs. For negative sampling [Robinson et al., 2021], 256 negative pairs are randomly selected for each positive pair to support self-distillation and the downstream recommendation optimization.

Evaluation Metrics

To evaluate the performance, the all-ranking protocol is employed on the test set. For each user, all the items are ranked based on their predicted interaction scores, and the top-N unobserved items are selected to form the user's recommendation list, where N is the length of the recommendation list, which is set to 20 by default. For quantitative evaluation, three widely used metrics are adopted, including Recall@N, NDCG@N, and Precision@N, where higher values of these metrics indicate better performance. To ensure the reliability and statistical significance of the experimental results, each experiment is repeated 10 times, and the average value of them is reported in the analysis.

Parameters

The semantic projector proj(·) and behavior embeddings {e(0) u,b , e(0) i,b } are initialized using the Xavier normal distribution. The textual encoder derives a 2048-dimensional textual representation, and the semantic and behavior embeddings are set to 64 dimensions. The graph convolution is performed with 3 layers for semantic and behavior propagation. The learning rate is set to 3e-4, and the Adam optimizer [Zhang, 2018] is applied to train the model. The hyperparameters are tuned with empirical experiments. By default, on the datasets, the temperature coefficient for recommendation loss τp is set as 0.2, τsd in self-distillation loss is set by {0.2, 0.2, 0.3}, the strength β of self-distillation is set 0.2, and the size k of the retrieved neighbors is set as 10. To ensure experimental fairness, the original parameter settings of all baseline models are retained, and the encoder is unified to be consistent with Qwen2.5-3B as in this work. All experiments are conducted in an environment with PyTorch 2.2.0 and Python 3.8.19.

5.2 Performance Comparison

We compare the proposed SID with representative baselines, including graph-based models (MFBPR [Koren et al., 2009], NGCF [Wang et al., 2019], and LightGCN [He et al., 2020]), contrastive learning-based models (SGL [Wu et al., 2021], SimGCL [Yu et al., 2022]), multimedia models (BPR T [He and McAuley, 2016], BM3 T [Zhou et al., 2023], SMORE T [Ong and Khong, 2025]), and LLM-based models (RLMRecCon [Ren et al., 2024], RLM-Gen [Ren et al., 2024], AlphaRec [Sheng et al., 2025]). For fair comparison, we remove or institute the visual modality with the textual one in multimedia models, retaining the behavioral and the textual modalities in comparison. The performance of SID and the baselines is compared by Recall@N in Figure 4. The performance by NDCG and Precision exhibits a similar trend; therefore, their figures are omitted. Table 2 presents the top-20 recommendation performance comparing SID and the baselines. In the table, the suboptimal performance is underlined, and the optimal performance is shown in bold.

It can be observed that SID outperforms the baselines consistently on the datasets. By Recall@20, SID yields relative increases of 10.33%, 13.75%, and 4.82%, respectively. The improvements verify the effectiveness of interest summarization and retrieval augmentation in SID to enhance multimodal interest learning for recommendation. SGL and SimGCL outperformMFBPR, NGCF, and LightGCN. It confirms that contrastive learning can effectively enhance graph learning for recommendation. Meanwhile, the superiority of SID over SGL and SimGCL indicates that retrieval augmented knowledge works more significant than self-discrimination in contrastive learning for the downstream recommendation task. The representative multimedia SMORE T achieves better performance than LightGCN on the Beauty and Toys datasets, but slightly lower performance on the Cellphones dataset, which implies the external semantic information works actively but requires an elaborate design for interest modeling. Additionally, SMORE T yields lower performance than SimGCL, which underscores the significance of contrastive learning for recommendation. Representative LLMmodel, AlphaRec, exhibits comparable performance to SimGCL. The performance validates the potential of LLM in depicting interests with semantic information to support the downstream recommendation task.

5.3 Ablation Study

We conduct experiments to verify the role of interest summarization and retrieval augmentation in SID in Table 3. The key operations are removed from SID successively to derive variants compared with SID. SID w/o reasoning removes LLM reasoning and directly encodes items' textual attributes to embed user interest in semantics. SID w/o text removes the textual modality and retains only the behavioral modality for interest modeling. SID w/o RA removes retrieval augmentation and perturbs the target user/item to augment data for contrastive learning. SID w/o SD removes selfdistillation and uses the retrieved users/items to augment the graph structure for recommendation. SID w/o RA+SD removes retrieval augmentation and self-distillation, preserving single task Lrec to train the model. SID w/o uSD and w/o iSD removes the self-distillation on the user/item side and merely performs knowledge transfer on the item/user side. SID w/o curriculum removes the curriculum learning strategy and implements knowledge transfer immediately on both dual sides. The performance of SID w/o reasoning decreases compared to SID, which indicates that LLM can filter out interestirrelevant signals and improve the quality of texts in describing interests. SID w/o text drops the performance sharply, proving that the semantic embedding encoded by LLM acts crucially to capture interest signals in the downstream retrieval and distillation. The performance of SID w/o RA, w/o SD, and w/o RA+SD degrades apparently, confirming that retrieval augmentation contributes to enhancing the interactions with semantic neighbors. SID w/o SD exhibits a more notable decline than that w/o RA. These trends highlight that both semantic retrieval and self-distillation play important roles in multimodal interest learning, and the multimodal distillation is relatively more critical. The performance degradation by SID w/o uSD and w/o iSD verifies the effectiveness of dualside knowledge transfer. SID w/o curriculum decreases the performance, illustrating that knowledge transfer on the user side is more complex than the item side, and their joint learning would disturb each other, hindering the performance. It can be concluded that interest summarization and retrieval augmentation are active in enriching the sparse interactions for recommendation.

5.4 Long-Tail Analysis

We conduct experiments on items of varying popularity to verify the ability of SID in recommending long-tail items. The performance of SID is compared with representative baselines, SimGCL and AlphaRec, which are provided in Figure 5. In each dataset, items are sorted by their interaction size in descending order and divided into 5 groups, where each group contains approximately the same number of interactions. For instance, on the Beauty dataset, the minimum number of interactions for items in each group is 80, 38, 18, 9, and 5, respectively. As the popularity decreases, SID degrades less than SimGCL and AlphaRec. In groups with either moderate or extremely sparse interactions, SID consistently achieves superior performance, which indicates that SID recommends more long-tail items. This characteristic makes item recommendations more balanced and effectively mitigates overemphasizing popular items over long-tail ones. Overall, SID is effective and competitive in recommending long-tail items.

5.5 Sparsity Analysis

We conduct experiments on users of varying interaction sparsity to validate the ability of SID to deal with the sparsity issue. The performance comparison on varying sparsity is given in Figure 6. In each dataset, users are partitioned into 5 groups based on interaction density, ranking from dense to sparse. Within each group, the number of user interactions is approximately uniform. For instance, in the Beauty dataset, users were categorized into groups with the lower bound of interactions 21, 11, 7, 6, and 5, respectively. SID outperforms AlphaRec and SimGCL on most user groups. As interactions become increasingly sparse, the degradation of SID is relatively small, illustrating a more pronounced advantage. In summary, SID is competitive in addressing the sparse interaction issue.

5.6 Visualization

To show the discriminative ability, we further visualize the embedding distribution by t-SNE to compare SID with AlphaRec in Figure 7. The results reveal notable differences between SID and AlphaRec. By SID, users and their interacted items exhibit more compactness. There is minimal intermixing between clusters of users and their interactions. In comparison, AlphaRec displays a far more disordered distribution where the clusters are in a mixed, chaotic state. It validates the effectiveness of SID in capturing users' discriminative personality in interest modeling.

6 Conclusion

We have proposed an LLM-summarized interest distillation (SID) model to enhance users' sparse interactions with highquality interest descriptions and more collaborative neighbors for multimedia recommendation. SID leverages LLMs' understanding and reasoning capabilities to depict user interest with semantic texts, which effectively reduces interestirrelevant descriptions from their interactions. With these summarized texts, we retrieve semantic neighbors to transfer their multimodal knowledge to augment the target user for promoting recommendations. Extensive experiments on real-world datasets demonstrate that SID significantly outperforms representative baselines, verifying the ability of interest summarization and retrieval augmentation in enriching the sparse interactions. Ablation studies further validate the necessity of the compacted interest description and augmented knowledge distillation. The long-tail and sparsity analysis confirm that SID can effectively alleviate the sparse interaction issue. In future work, we will explore interest summarization ability to further promote interest learning with semantic augmentation.

Acknowledgements

This work was supported by the Beijing Natural Science Foundation under Grant NO. L241053, S&T Program of Hebei under Grant No.202621502030001, and the National Natural Science Foundation of China under Grant NO. 62176011.