---
title: "Learning Fair Representations for Recommendation via Information Bottleneck Principle"
authors:
  - "Junsong Xie"
  - "Yonghui Yang"
  - "Zihan Wang"
  - "Le Wu"
doi: "10.24963/ijcai.2024/273"
url: "https://ijcai.org/proceedings/2024/273"
pdf: "https://ijcai.org/proceedings/2024/0273.pdf"
venue: "Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence"
year: 2024
pages: "2469-2477"
track: "Main Track"
keywords:
  - "Data Mining: DM: Recommender systems"
  - "AI Ethics, Trust, Fairness: ETF: Fairness and diversity"
  - "Data Mining: DM: Collaborative filtering"
  - "Machine Learning: ML: Representation learning"
---

# Learning Fair Representations for Recommendation via Information Bottleneck Principle

Junsong Xie, Yonghui Yang, Zihan Wang, Le Wu

*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence*, pages 2469-2477. DOI: [10.24963/ijcai.2024/273](https://doi.org/10.24963/ijcai.2024/273)

## Abstract

User-oriented recommender systems (RS) characterize users' preferences based on observed behaviors and are widely deployed in personalized services. However, RS may unintentionally capture biases related to sensitive attributes (e.g., gender) from behavioral data, leading to unfair issues and discrimination against particular groups (e.g., females). Adversarial training is a popular technique for fairness-aware RS, when filtering sensitive information in user modeling. Despite advancements in fairness, achieving a good accuracy-fairness trade-off remains a challenge in adversarial training. In this paper, we investigate fair representation learning from a novel information theory perspective. Specifically, we propose a model-agnostic Fair recommendation method via the Information Bottleneck principle FairIB. The learning objective of FairIB is to maximize the mutual information between user representations and observed interactions, while simultaneously minimizing it between user representations and sensitive attributes. This approach facilitates the capturing of essential collaborative signals in user representations while mitigating the inclusion of unnecessary sensitive information. Empirical studies on two real-world datasets demonstrate the effectiveness of the proposed FairIB, which significantly improves fairness while maintaining competitive recommendation accuracy, either in single or multiple sensitive scenarios. The code is available at https://github.com/jsxie9/IJCAI_FairIB.

## Full text

## 1 Introduction

In the modern era, recommender systems have become essential allies, seamlessly delivering personalized content across diverse domains [Covington et al., 2016; Chen et al., 2020]. Fueled by sophisticated algorithms and user-centric models [Tan et al., 2021; Yang et al., 2023; Wu et al., 2023], these systems shine in anticipating user preferences, elevating overall user experience and engagement. While recommender systems achieve remarkable success in tailoring recommendations to individual users, they frequently grapple with fairness challenges stemming from biased historical interactions, particularly concerning sensitive attributes such as gender and race [Ekstrand et al., 2018b; Shao et al., 2022]. For example, news recommender systems may disproportionately recommend certain political ideologies over others, which may manipulate user's opinions [Li et al., 2023]. Job recommender systems may display racial or gender discrimination by disproportionately suggesting low-paying jobs to protected user groups [Dong et al., 2023].

* Corresponding author.

Due to the progressive advancement of trustworthy AI, fairness-aware recommendations are capturing the increasing attention of researchers [Li et al., 2023]. Learning fair recommendation representations from user-item interactions is one of the promising methods [Bose and Hamilton, 2019; Xie et al., 2017]. To achieve the goal of fair representations without sensitive information, researchers have proposed various methods [Yao and Huang, 2017; Zhu et al., 2018; Wu et al., 2021a; Wu et al., 2021b]. Among them, adversarial training is the mainstream technique for recommendation fairness, and a series of adversarial-based works have been proposed [Bose and Hamilton, 2019; Dai and Wang, 2021; Wu et al., 2021a]. While learning user representations from observed interactions, adversarial-based methods encourage that sensitive attributes cannot be inferred from the representations. These methods operate within a two-player minimax game framework, effectively safeguarding against the disclosure of such attributes within the learned representations. Despite advancements in fairness, achieving an efficient accuracy-fairness trade-off remains a challenge in adversarial training. Due to the instability of adversarial training [Salimans et al., 2016; Arjovsky et al., 2017], a commonly employed solution is to pre-train recommendation representations based on observed interactions and subsequently optimize sensitive attribute filters through adversarial training [Wu et al., 2021a]. However, this solution is hard to balance accuracy-fairness trade-off, leading to unsatisfied recommendations.

In this paper, we revisit the fairness-aware recommendation from an information theory perspective, and propose a novel FairIB via the information bottleneck principle. As shown in Figure 1, we illustrate the optimization objectives of the traditional recommendation and our propose FairIB.

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000000_c677529c9423fe211218ef2c84a1c66c72c9aa921433f9666465adb3b89d3637.png)

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000001_b60fe3ea47cad5f14e1cf25d927c942aa1dae4140f41ec699afb3e45bc4c39c6.png)

ሻܵ -𝛽𝐼ሺ𝑋; 𝑋 ;ܴ 𝑀𝑎𝑥: 𝐼

- (b) Our proposed FairIB
- interactions User-item :ܴ
- attributes sensitive User :ܵ
- 𝑋: User representations

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000002_75b55d2a4b6442c895fabed25d09e84df42f8567780561cea7517261b9ae9aae.png)

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000003_9da26e4bc83ee963998388b1d09b78bd7fbebbdf38a30105514666c4a5d7ec7b.png)

Collaborative information

Sensitive information

Figure 1: Information diagram of user representations X, user sensitive attributes S, and user-item interactions R. (a) Traditional recommendation methods focus on maximizing the mutual information between representations X and user-item interactions R. As users' behavior exposes their sensitive attributes, maximizing I(R;X) will lead to representations involving sensitive information. (b) Our proposed FairIB via Information Bottleneck principle. It not only maximizes I(R;X), meanwhile minimizing the mutual information between representations X and sensitive attributes S. Therefore, FairIB acquires fair representations X through IB learning, successfully achieving an efficient trade-off of recommendation accuracy and fairness.

Traditional recommender systems focus on the accuracy task, the optimal user representations X are obtained by maximizing the mutual information between X and user-item observed interactions R (I(R;X)). However, empirical studies indicate that users' behaviors can expose their sensitive attributes [Kosinski et al., 2013; Wu et al., 2021a], simply maximizing I(R;X) will introduce sensitive information to representations, leading to unfair recommendation results. To tackle this issue, our proposed FairIB acquires fair representations X based on information bottleneck learning. Besides maximizing the mutual information I(R;X), FairIB simultaneously minimizes the mutual information I(X; S). Thus, FairIB acquires fair yet precise representations in an IB manner, successfully achieving an efficient trade-off of recommendation accuracy and fairness.

However, directly optimizing the learning objective of Max : I(R;X) - βI(X; S) poses three challenges. First, the calculation of I(R;X) is difficult. Each user representation should match all interacted items, as they interact with different items. Second, it's hard to minimize I(X; S) because estimating the upper bound of mutual information is an intractable problem. While current methods use variational techniques for estimation, their heavy reliance on prior assumptions (e.g., Gaussian marginal distribution) can result in the upper bound failing to accurately estimate mutual information with low bias in practical applications [Alemi et al., 2017; Cheng et al., 2020]. Third, users' sensitive attributes are not only exposed in their representations, but also in subgraph structure [Wu et al., 2021a], so it's necessary to filter sensitive information in user-centric sub-graphs.

To address the above challenges, we elaborately implement FairIB as follows. For maximization of I(R; X), we establish that the goal is equivalent to maximizing the inner product between a user and her interacted items representations. This aligns with the typical optimization objective of general recommendation systems. For minimization of I(X; S), we utilize the Hilbert-Schmidt Independence Criterion (HSIC) regularizer [Ma et al., 2020; Wang et al., 2023] to approximate the optimization, without prior assumptions. Besides, we extend the HSIC regularizer to user-centric sub-graphs, further eliminating the sensitive information. In summary, our main contributions are listed as follows:

- In this paper, we revisit fairness-aware recommendations from an information theory lens, and propose a novel model-agnostic fair representation learning method FairIB.
- Technically, FairIB introduces the HSIC-based bottleneck to improve recommendation fairness, effectively eliminating the sensitive information from user and subgraph perspectives.
- Extensive experiments demonstrate the effectiveness of our proposed FairIB, which can effectively achieve recommendation accuracy-fairness trade-off.

## 2 RelatedWork

## 2.1 Fairness in Recommendation

Recommendation is a widely deployed user-centric application [Gao et al., 2023], and an increasing number of researchers are focusing on the issue of fairness [Xiao et al., 2017; Li et al., 2021; Shao et al., 2024]. In recommendation systems, there is the potential for unequal treatment of sensitive user groups [Ekstrand et al., 2018a]. For instance, an unfair job recommendation system may exhibit preferences towards user groups with specific sensitive attributes, inevitably resulting in biased recommendations [Dong et al., 2023]. Many fairness definitions are proposed to measure potential unfairness [Mehrabi et al., 2021; Biega et al., 2018; Wu et al., 2021b]. For example, individual fairness requires that a model produces similar decisions for similar individuals [Biega et al., 2018]. Group fairness advocates against discrimination of a specific user group based on sensitive attributes [Wu et al., 2021b]. In this work, our emphasis is on group fairness because it can quantitatively measure the differences in how sensitive groups are treated. [Hardt et al., 2016]. In response to fairness concerns, researchers have designed numerous models [Yao and Huang, 2017;

Wu et al., 2021a; Dai and Wang, 2021; Wu et al., 2021b]. [Yao and Huang, 2017] proposes five fairness metrics as regularization terms to measure the discrepancy between the prediction behavior for the female and male groups. Based on adversarial learning, FairRec [Wu et al., 2021a] decomposes adversarial learning and orthogonality regularization. FairGo [Wu et al., 2021b] further develops a more sophisticated graph-based adversarial learning for fairness modeling. FairGNN [Dai and Wang, 2021] proposes a sensitive attribute estimator and incorporates adversarial debiasing and covariance constraints to regularize the GNN for fair node representations and predictions. Recently, FairMI [Zhao et al., 2023] attempts to mitigate the impact of sensitive attributes on the final recommendation representation by applying two aspects of mutual information. Different from these approaches, our approach utilizes the information bottleneck principle for fair representation learning, effectively achieving the best tradeoff between recommendation accuracy and fairness.

## 2.2 Information Bottleneck Principle

The Information Bottleneck (IB) principle seeks to encapsulate the balance in the to-be-learned representation between the essential information required for decision and the information retained from the input [Tishby et al., 2000]. IB has been employed to enhance the interpretability and disentangle representation in deep learning-based tasks [Bao, 2021; Jeon et al., 2021]. However, precise calculation of the mutual information between two high-dimensional random variables poses a significant challenge. To address this challenge, researchers use neural networks to approximate and estimate mutual information [Alemi et al., 2017]. For instance, InfoNCE [Oord et al., 2018] and variational based method [Alemi et al., 2017] have been introduced to estimate the lower bound of mutual information. Recently, CLUB is proposed to estimate the upper bound of mutual information based on the log-ratio contrastive learning [Cheng et al., 2020], which heavily relies on prior assumptions. In addition to directly optimizing mutual information objectives, researchers employ the Hilbert-Schmidt Independence Criterion (HSIC) as an alternative to mutual information estimation, which can assess the independence of two variables [Ma et al., 2020]. Given the challenge of estimating the upper bound of mutual information, we opt for HSIC as an approximation to minimize the mutual information between the learned representations and sensitive attributes.

## 3 Preliminary

## 3.1 Problem Statement

In a recommender system, there are two entity sets: a user set U(|U| = M) and an item set V (|V | = N). The interactions between users and items are represented by the matrix R ∈ RM×N . Specifically, in scenarios involving common implicit feedback, the notation rai = 1 signifies that user a has interacted with item i, otherwise rai = 0. The interactions form a user-item bipartite graph, which can be formulated as G =< U ∪ V, A >. The adjacency matrix A can be formulated as:

<!-- formula-not-decoded -->

X ∈ RM×D and Y ∈ RN×D denote the learned user and item representations, where D refers to the dimension of representation. The goal of recommendation is to predict the potential preference ˆ rai of user a to item i, and can be calculated with ˆ rai = xTa yi, where xa is user a's representation, yi is item i's representation. Given a binary sensitive attribute k ∈ {0, 1}, sa refers to user a's attribute value. Then, we can partition the user set U into two subsets:U0 and U1, where U0 represents the set of users with a sensitive attribute value of 0, and U1 represents the set of users with a sensitive attribute value of 1. Our goal is to learn fair user representations X while maintaining competitive recommendation accuracy.

## 3.2 Hilbert-Schmidt Independence Criterion (HSIC)

HSIC is a statistical method employed to quantify the independence between two variables. Followed by [Ma et al., 2020], HSIC(A, B) can be formulated as:

<!-- formula-not-decoded -->

where CAB is the cross-covariance operator between the Reproducing Kernel Hilbert Spaces (RKHSs) of A and B, ||·||2hs refers to Hilbert-Schmidt norm. Given the sampled instances (ai, bi)ni=1 from the batch training data, we estimate HISC as:

<!-- formula-not-decoded -->

where KA and KB are kernel Gram matrices [Ham et al., 2004] of A and B, H = I - 1n 11T is a centering matrix, and Tr(·) refers to the trace of the matrix. In our work, we use the commonly employed radial basis function (RBF) [Vert et al., 2004] as the kernel function KA and KB, as follows:

<!-- formula-not-decoded -->

where σ is a hyperparameter that controls the sharpness of RBF.

## 4 The Proposed FairIB

## 4.1 Overall Objective of FairIB

Here, we first present the overall optimization objective of our proposed FairIB. The goal of FairIB is to learn the optimal user representations X by Max : I(R;X) - βI(X; S). Considering that each user interacts with different items, the optimal user representations X should match all interacted items. We relax the maximization I(R;X) to maximization I(R;X,Y). Besides, users' sensitive attributes are not only exposed in their representations, but also in user-centric subgraphs [Wu et al., 2021b]. Aiming to eliminate user-sensitive attributes further, we additionally minimize the mutual information of sensitive attributes S and user-centric sub-graph representations G of all users. Therefore, the overall objective of FairIB is formulated as follows:

<!-- formula-not-decoded -->

where β and γ are two parameters that control the bottleneck scales, G denotes user-centric sub-graph representations. Eq.(5) describes that the optimal user representations are acquired in the trade-off of both recommendation accuracy and fairness demand.

## 4.2 Maximization of I(R;X,Y)

We first present how to maximize the mutual information I(R;X,Y), ensuring that the representations of users and items contain sufficient information for recommendation tasks. Based on the properties of mutual information and entropy [Kraskov et al., 2004], we have the following derivation:

<!-- formula-not-decoded -->

As H(R) is a constant, we only need to maximize the second term, as follows:

<!-- formula-not-decoded -->

Therefore, maximizing I(R; X, Y) is equivalent to maximizing the log likelihood p(R|X,Y), which is the typical optimization objective for general recommender systems. In other words, we can employ any recommendation model to realize the goal of maximization I(R; X, Y).

## 4.3 Minimization of I(X, S) and I(G, S)

Sensitive Attribute Encoder. Following the previous works [Wu et al., 2021b; Zhao et al., 2023], we train a sensitive encoder to represent sensitive attributes S. Considering that users' behaviors reflect their attributes [Kosinski et al., 2013; Wu et al., 2021a], we build a graph-based encoder for extraction from the interaction data, followed by a sensitive attribute classifier. After training the encoder, we obtain sensitive representations for all users, i.e., {ha|a ∈ U}. To further get the sensitive representations for each user group (i.e., U0 and U1), we calculate the mean value within each group. This process is calculated as follows:

<!-- formula-not-decoded -->

where k ∈ {0, 1} refers to different sensitive attribute values, Uk denotes different user groups, and ha denotes the sensitive representation for user a learned from the encoder. Finally, we obtain the sensitive representations E = [e0, e1].

HSIC-based MI Minimization. Given the encoded sensitive representations E, we instead I(X; S) of I(X;E). In the same manner, we instead I(G; S) of I(G;E). However, it's hard to minimize the mutual information because estimating the upper bound of MI is still an intractable problem. Although previous studies use variational based methods to estimate the upper bound of MI, they heavily rely on the prior distribution and the quality of sampling influences the accuracy of the estimation [Alemi et al., 2017; Cheng et al., 2020]. Here we introduce HSIC to replace mutual information for optimization. Therefore, the optimization objective for user side minimization is:

<!-- formula-not-decoded -->

where Ub denote users in the batch training data, xa and esa denote user a's representation and corresponding sensitive attribute representation. For sub-graph side minimization, we first compute user-centric sub-graph representations G as follows:

<!-- formula-not-decoded -->

where Xl and Yl denote user and item representations in lth layer, Xl+1 and Yl+1 denote user and item representations in l +1th layer. DA is the degree of the adjacent matrix A and L is the depth of sub-graph, g(·) is the readout function. Similar to the user side, we implement the minimization of the sub-graph side as follows:

<!-- formula-not-decoded -->

where Ub denote users in the training batch data, ga is user a's sub-graph representation. Thus, we obtain the minimization objective for a specific sensitive attribute (such as gender):

<!-- formula-not-decoded -->

Here we only present the loss function for a single sensitive attribute. In fact, FairIB is flexible to extend multiple sensitive attributes, which only performs multiple bottleneck learning for each sensitive attribute. Then, we can get multisensitive loss based on HSIC, as follows:

<!-- formula-not-decoded -->

where T denotes the set of multi-sensitive attributes, LtHSIC denotes the loss function of tth sensitive attribute.

## 4.4 Model Optimization

As we mentioned in section 4.2, we employ common pair-wise ranking loss to optimize the maximization of I (R; X, Y):

<!-- formula-not-decoded -->

Table 1: The statistics of two datasets.

| Datasets     |   Users |   Items |   Interactions | Attributes   |
|--------------|---------|---------|----------------|--------------|
| Movielens-1M |   6,040 |   3,952 |      1,000,209 | gender, age  |
| LastFM       |  48,386 |  21,711 |      2,045,305 | gender       |

where σ(·) is the sigmoid function, Θ = [X,Y] is user and item free embedding matrices and α controls the L2 regularization coefficient. Da = {(i, j)|i ∈ Ra ∩ j ∈ V - Ra}, Ra denotes user a interacted items. By combining the independence constraints of HSIC-based bottleneck on the user side and the sub-graph side, we obtain the final optimization objective of FairIB:Lall = Lrec + LHSIC. (15)

## 5 Experiments

In this section, we first introduce our experimental settings. Then, we conduct extensive comparisons with SOTA methods to verify the effectiveness of our proposed FairIB. Finally, we give a detailed analysis of our method, including ablation studies and parameter sensitivities.

## 5.1 Experimental Settings

Datasets. To evaluate the effectiveness of our proposed method, we select two real-world recommendation datasets: Movielens-1M [Harper and Konstan, 2015; Wu et al., 2020] and LastFM [Celma Herrada and others, 2009]. Following the previous works [Wu et al., 2021a; Zhao et al., 2023], we split all interactions into training, validation, and test data. For Movielens-1M, we treat gender as the single attribute, and select gender and age as the compositional setting for further experimental study. For LastFM, we only use gender as the sensitive attribute. Detailed statistics of the two datasets are summarized in Table 1.

Evaluation Metrics. As we concentrate on the trade-off between recommendation accuracy and fairness, it is essential to evaluate and report results for both aspects. For measuring the recommendation accuracy, we employ two widely used ranking metrics: NDCG [J¨arvelin and Kek¨al¨ainen, 2017] and Recall [Gunawardana and Shani, 2009]. Larger values of NDCG and Recall indicate superior recommendation accuracy performance. For fairness evaluation, we also employ two popular group fairness metrics: Demographic Parity (DP) [Caton and Haas, 2020] and Equal Opportunity (EO) [Hardt et al., 2016]. Specifically, we calculate DP as follows:

<!-- formula-not-decoded -->

where Qa = TopKa, and TopKa is Top-K ranked items for user a; i ∈ V , U0 and U1 denote user group with different sensitive attributes; I is an indicator function, if item i is in the set Qa, then I = 1, otherwise I = 0. Then we compute Jensen-Shannon divergence to compare two groups:

<!-- formula-not-decoded -->

The definition of DP requires that item i be recommended to the two groups with equal probability, regardless of the actual preferences of the population [Caton and Haas, 2020]. EO is proposed by considering user actual preferences, which is calculated as:

<!-- formula-not-decoded -->

where Ra denotes the items genuinely liked by user a in the test data. The smaller values of DP and EO, the better recommendation fairness.

Baselines. As our proposed FairIB is a model-agnostic method, we implement FairIB on two representative recommendation backbones: BPR-MF [Rendle et al., 2012] and LightGCN [He et al., 2020]. These two have emerged as representative recommendation backbones in recent years. We use the full-ranking strategy to evaluate all methods for a fair comparison. We compare FairIB with SOTA fairness-aware recommendation methods:

- Reg [Yao and Huang, 2017]: is a regularization-based model that incorporates various statistical fairness regularization terms.
- Adv [Bose and Hamilton, 2019]: is an adversarial learning method designed to minimize the correlation between sensitive attributes and the learned representation.
- FairRec [Wu et al., 2021a]: is a fairness-aware approach with decomposed adversarial learning and orthogonality regularization.
- FairGo [Wu et al., 2021b]: develops a more sophisticated adversarial learning approach that takes into account the hidden unfairness within a user-centric graph.
- FairGNN [Dai and Wang, 2021]: incorporates adversarial debiasing and covariance constraints to regularize the GNN for fair node representations and predictions. It also includes a sensitive attribute estimator to address the challenge of missing sensitive attribute information.
- FairMI [Zhao et al., 2023]: proposes a two-fold mutual information optimization framework, which employs a self-supervised learning approach to maximize the mutual information between pre-trained SOTA model representation and to-be-learned representation. Please note that, we removed the self-supervised learning part for a fair comparison (which is only designed for enhancing accuracy), denoted as FairMI*.

Implementation Details. We conduct experiments on an NVIDIA A40 GPU with Pytorch-2.1.2. For the sensitive attribute encoder, we utilize LightGCN as the backbone and a one-layer fully connected network as the attribute classifier. For model training, we set the latent embedding size as D = 64, the batch size is set to 2048 for Movielens-1M and 4096 for LastFM. The regularization parameter α is set to 0.001. We adopt the Adam optimizer with a learning rate of 0.001. We repeat experiments 10 times and report the average results.

## 5.2 Overall Performance

Table 2 and 3 report the overall experimental results on Movielens-1M and LastFM. From these Tables, we have the following observations:

Table 2: Recommendation accuracy and fairness performances on the Movielens-1M dataset. We report comparisons across different Top-K values. We compare all fairness-aware methods, the best results are highlighted in bold and the second best results are displayed in underline.

| Models   | Methods         |   NDCG@K↑ - 10 |   NDCG@K↑ - 30 |   Recall@K↑ - 10 |   Recall@K↑ - 30 |   DP@K↓ - 10 |   DP@K↓ - 30 |   EO@K↓ - 10 |   EO@K↓ - 30 |
|----------|-----------------|----------------|----------------|------------------|------------------|--------------|--------------|--------------|--------------|
|          | Base            |         0.1943 |         0.2926 |           0.1437 |           0.2916 |       0.2854 |       0.2412 |       0.3580 |       0.3122 |
|          | Reg             |         0.1899 |         0.2859 |           0.1402 |           0.2845 |       0.1954 |       0.1575 |       0.2904 |       0.2576 |
|          | Adv             |         0.1900 |         0.2866 |           0.1404 |           0.2858 |       0.1684 |       0.1214 |       0.2736 |       0.2363 |
|          | BPR-MF FairRec  |         0.1896 |         0.2860 |           0.1407 |           0.2847 |       0.1656 |       0.1191 |       0.2714 |       0.2334 |
|          | FairMI*         |         0.1901 |         0.2867 |           0.1410 |           0.2855 |       0.1521 |       0.1145 |       0.2608 |       0.2199 |
|          | FairIB          |         0.1922 |         0.2903 |           0.1428 |           0.2907 |       0.1453 |       0.1140 |       0.2117 |       0.1740 |
|          | Base            |         0.2018 |         0.3060 |           0.1511 |           0.3085 |       0.2919 |       0.2449 |       0.3609 |       0.3085 |
|          | Reg             |         0.1961 |         0.2938 |           0.1422 |           0.2828 |       0.2097 |       0.1545 |       0.3047 |       0.2611 |
|          | Adv             |         0.1963 |         0.2975 |           0.1469 |           0.2998 |       0.1532 |       0.1068 |       0.2694 |       0.2203 |
|          | FairRec         |         0.1950 |         0.2955 |           0.1472 |           0.2986 |       0.1536 |       0.1042 |       0.2590 |       0.2243 |
|          | LightGCN FairGo |         0.1822 |         0.2741 |           0.1336 |           0.2710 |       0.2728 |       0.2275 |       0.3382 |       0.2921 |
|          | FairGNN         |         0.1964 |         0.2963 |           0.1466 |           0.2969 |       0.1472 |       0.1045 |       0.2608 |       0.2221 |
|          | FairMI*         |         0.1978 |         0.2967 |           0.1480 |           0.2980 |       0.1436 |       0.1046 |       0.2560 |       0.2148 |
|          | FairIB          |         0.2003 |         0.3013 |           0.1502 |           0.3061 |       0.1408 |       0.1033 |       0.2045 |       0.1686 |

Table 3: Recommendation accuracy and fairness performances on the LastFM dataset. We report comparisons across different Top-K values. We compare all fairness-aware methods, the best results are highlighted in bold and the second best results are displayed in underline.

| Models   | Methods         |   NDCG@K↑ - 10 |   NDCG@K↑ - 30 |   Recall@K↑ - 10 |   Recall@K↑ - 30 |   DP@K↓ - 10 |   DP@K↓ - 30 |   EO@K↓ - 10 |   EO@K↓ - 30 |
|----------|-----------------|----------------|----------------|------------------|------------------|--------------|--------------|--------------|--------------|
|          | Base            |         0.1959 |         0.2743 |           0.1564 |           0.2943 |       0.2664 |       0.2376 |       0.3345 |       0.3122 |
|          | Reg             |         0.1830 |         0.2477 |           0.1445 |           0.2819 |       0.2072 |       0.1728 |       0.2953 |       0.2710 |
|          | Adv             |         0.1813 |         0.2518 |           0.1443 |           0.2791 |       0.1335 |       0.1101 |       0.2618 |       0.2501 |
|          | BPR-MF FairRec  |         0.1876 |         0.2644 |           0.1502 |           0.2853 |       0.1484 |       0.1169 |       0.2712 |       0.2592 |
|          | FairMI*         |         0.1884 |         0.2595 |           0.1505 |           0.2827 |       0.1273 |       0.0958 |       0.2580 |       0.2476 |
|          | FairIB          |         0.1897 |         0.2665 |           0.1517 |           0.2861 |       0.1243 |       0.0930 |       0.2474 |       0.2403 |
|          | Base            |         0.1971 |         0.2762 |           0.1572 |           0.2964 |       0.2860 |       0.2569 |       0.3508 |       0.3247 |
|          | Reg             |         0.1863 |         0.2619 |           0.1484 |           0.2865 |       0.2201 |       0.1965 |       0.3012 |       0.2867 |
|          | Adv             |         0.1887 |         0.2652 |           0.1499 |           0.2853 |       0.1382 |       0.1163 |       0.2682 |       0.2576 |
|          | FairRec         |         0.1892 |         0.2627 |           0.1505 |           0.2837 |       0.1397 |       0.1157 |       0.2700 |       0.2596 |
|          | LightGCN FairGo |         0.1693 |         0.2389 |           0.1371 |           0.2627 |       0.2626 |       0.2338 |       0.3282 |       0.3059 |
|          | FairGNN         |         0.1879 |         0.2651 |           0.1501 |           0.2859 |       0.1372 |       0.1171 |       0.2690 |       0.2592 |
|          | FairMI*         |         0.1888 |         0.2662 |           0.1506 |           0.2866 |       0.1381 |       0.1058 |       0.2653 |       0.2548 |
|          | FairIB          |         0.1900 |         0.2678 |           0.1520 |           0.2876 |       0.1313 |       0.0998 |       0.2325 |       0.2314 |

- Compared with base models (BPR-MF, LightGCN), all fairness-aware methods present better recommendation fairness but a slight accuracy drop. This is caused by the natural data distribution, accuracy-fairness is a trade-off process. Furthermore, we find that graph-based backbone (LightGCN) has higher recommendation accuracy but worse recommendation fairness. The reason is that graph convolutions strengthen the collaborative signals, also amplify the sensitive attributes.
- Compared with fairness regularization, adversarialbased methods achieve better fairness performances, which demonstrates that learning fair representations is a more effective technique rather than a simple statisticbased constraint.
- Our proposed FairIB consistently outperforms all baselines on fairness evaluation. Specifically, FairIB improves LightGCN w.r.t DP@30 by 57.82%, 61.15% on Movielens-1M and LastFM, respectively. Besides,
- compared with the strongest fairness baseline, i.e., FairMI, FairIB also presents better fairness performances, demonstrating the effectiveness of our designed HSIC-based bottleneck learning.
- Besides fairness metrics, we observe that FairIB also obtains the best recommendation accuracy results except for the base model. Experiments show the superiority of FairIB in producing the fairest recommendation results with the fewest accuracy sacrifice. All indicate that FairIB acquires the efficient accuracy-fairness trade-off in the IB manner.

Multi-sensitive Setting. Here, we present comparisons of all methods under multiple sensitive scenarios. As illustrated in Table 4, we conduct experiments on Movielens-1M, and analyze model performances on both gender and age fairness. For the age attribute, we divide users into two groups based on binarization. From the Table 4, we have the following observations. First, FairIB achieves the best accuracy and multiple sensitive attributes fairness performance among all fairnessaware baselines. This demonstrates that our proposed FairIB also acquires an efficient accuracy-fairness trade-off under multi-sensitive attributes. Besides, under the multi-sensitive scenario, FairIB is more convenient compared with adversarial methods as we don't need multiple discriminator learning. Overall, either single-sensitive or multi-sensitive attributes, FairIB is effective and efficient in achieving recommendation accuracy-fairness trade-off.

Table 4: Comparisons on multiple sensitive attributes (Top-K=10).

| Models   |   NDCG↑ |   Recall↑ |   DP↓ - Gender |   DP↓ - Age |   EO↓ - Gender |   EO↓ - Age |
|----------|---------|-----------|----------------|-------------|----------------|-------------|
| Base     |  0.2018 |    0.1511 |         0.2919 |      0.1548 |         0.3609 |      0.2237 |
| Reg      |  0.1953 |    0.1425 |         0.2120 |      0.1483 |         0.3120 |      0.2210 |
| Adv      |  0.1940 |    0.1448 |         0.1477 |      0.1231 |         0.2588 |      0.2013 |
| FairRec  |  0.1920 |    0.1432 |         0.1511 |      0.1327 |         0.2535 |      0.1996 |
| FairGo   |  0.1817 |    0.1304 |         0.2730 |      0.1410 |         0.3256 |      0.2178 |
| FairGNN  |  0.1943 |    0.1454 |         0.1503 |      0.1313 |         0.2440 |      0.1905 |
| FairMI*  |  0.1969 |    0.1469 |         0.1428 |      0.1523 |         0.2585 |      0.2180 |
| FairIB   |  0.2002 |    0.1487 |         0.1397 |      0.1137 |         0.2046 |      0.1763 |

Figure 2: Ablation study with Top-K=10 on Movielens-1M.

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000004_8e44eaf4081f444177c32246c6f154e66a73d3351be459438bb729d703e3b97f.png)

## 5.3 Model Analysis

Ablation Studies. In this part, we conduct ablation studies on Movielens-1M to verify the effectiveness of each component in FairIB. As illustrated in Figure 2, we conduct experiments on BPR-MF and LightGCN backbones. Among them, "w/o U" and "w/o G" denote the variants of FairIB without user and sub-graph sides bottleneck learning (Eq.(5)), respectively. From Figure 2, we have the following observations. First, "w/o U" shows better performance both in accuracy and fairness compared to "w/o G". This indicates that bottleneck learning on the user side is effective in eliminating more unnecessary sensitive information while capturing essential collaborative signals more effectively. Second, by equipping bottleneck learning on both the user side and sub-graph side, our method can further improve recommendation accuracy and fairness.

Parameter Sensitivity Analyses. We investigate the impact of user and sub-graph bottleneck learning scales β and γ in Eq. (12) on Movielens-1M. Since the optimal parameters for FairIB on the Movielens-1M dataset are β=40 and γ=10, we fix β=40 and analyze the sensitivity of γ. Similarly, we fix γ=10 and analyze the sensitivity of β. We have several observations from Figure 3. First, with the increase of β, this implies the enhancement of on user-side bottleneck learning, user representations will contain less sensitive information, which achieves a better fairness performance accompanied by a decrease in accuracy. However, a larger β will impact the model's training, resulting in a deterioration of both accuracy and fairness. The optimal balance is achieved when β equals 40. Second, after reaching the optimal value for β, a moderate γ can simultaneously improve both fairness and accuracy. We speculate that this might be a proper bottleneck learning on the sub-graph side helping to capture noise-sensitive information in the graph structure, thereby enhancing both model accuracy and fairness simultaneously. Finally, FairIB achieves an efficient trade-off of recommendation accuracy and fairness when β is set to 40 and γ to 10.

Figure 3: FairIB based on LightGCN with different β and γ.

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000005_6256baa50c40ad6cb83ce569cfe91d46d3295fdb2fafa4412f3cd4712b32f99a.png)

Figure 4: Impact of different IB loss parameters (β, σ2 ) and (γ, σ2 ).

![Image](https://ijcai.org/proceedings/2024/fulltext/273/image_000006_5520f62f01b3c4135bfcf7602949dd948ee926d379cb3dbadd8489e9afd1ecb2.png)

Furthermore, we also delve into the impact of kernel function parameter σ2 in Eq. (4) on Movielens-1M. We investigated various combinations involving σ2 in [0.1, 0.2, 0.3, 0.4], β in [20, 30, 40, 50] and γ in [0, 5, 10, 15]. As shown in Figure 4, FairIB achieve the best fairness performance (EO@10) when (β = 40, σ2 = 0.3) and (γ = 10, σ2 = 0.3). Comprehensive experiments have verified that a moderate value for σ2 (i.e., 0.3) significantly contributes to improving the model's fairness performance.

## 6 Conclusion

In this paper, we researched fairness-aware recommender systems from the information theory perspective. Motivated by the information bottleneck principle, we proposed a novel model-agnostic fair representation method FairIB to eliminate the sensitive information from the learned representations. Specifically, FairIB maximizes the mutual information between learned representations and observed interactions, meanwhile minimizing it between representations and user sensitive attributes. To achieve this goal, we introduced HSIC-based bottleneck to recommender systems, and applied to both the user and sub-graph sides. Extensive experiments on two real-world datasets demonstrated FairIB is effective in balancing recommendation accuracy-fairness trade-off, either in single or multiple sensitive scenarios. In the future, we would like to extend the proposed method by mining attribute correlations to address the practical scenario of missing values for sensitive attributes.

## Acknowledgments

This work was supported in part by grants from the National Key Research and Development Program of China (Grant No.2021ZD0111802), the National Natural Science Foundation of China (Grant No.U23B2031, 72188101).

## Contribution Statement

Junsong Xie and Yonghui Yang contribute equally to this work.

## References

1. [Alemi et al., 2017] Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. In ICLR, 2017.
2. [Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In ICML, pages 214-223. PMLR, 2017.
3. [Bao, 2021] Feng Bao. Disentangled variational information bottleneck for multiview representation learning. In Arti- ficial Intelligence: First CAAI International Conference, CICAI 2021, Hangzhou, China, June 5-6, 2021, Proceed- ings, Part II 1, pages 91-102. Springer, 2021.
4. [Biega et al., 2018] Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. Equity of attention: Amortizing individ- ual fairness in rankings. In SIGIR, pages 405-414, 2018.
5. [Bose and Hamilton, 2019] Avishek Bose and William Hamilton. Compositional fairness constraints for graph embeddings. In ICML, pages 715-724. PMLR, 2019.
6. [Caton and Haas, 2020] Simon Caton and Christian Haas. Fairness in machine learning: A survey. CSUR, 2020.
7. [Celma Herrada and others, 2009] Òscar Celma Herrada et al. Music recommendation and discovery in the long tail. Universitat Pompeu Fabra, 2009.
8. [Chen et al., 2020] Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. Revisiting graph based collabo- rative filtering: A linear residual graph convolutional net- work approach. In AAAI, volume 34, pages 27-34, 2020.
9. [Cheng et al., 2020] Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio upper bound of mutual information. In ICML, pages 1779-1788. PMLR, 2020.
10. [Covington et al., 2016] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recom- mendations. In RecSys, pages 191-198, 2016.
11. [Dai and Wang, 2021] Enyan Dai and Suhang Wang. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In WSDM, pages 680-688, 2021.
12. [Dong et al., 2023] Yushun Dong, Jing Ma, Song Wang, Chen Chen, and Jundong Li. Fairness in graph mining: A survey. TKDE, 2023.
13. [Ekstrand et al., 2018a] Michael D Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Conference on fairness, accountability and transparency, pages 172-186. PMLR, 2018.
14. [Ekstrand et al., 2018b] Michael D Ekstrand, Mucun Tian, Mohammed R Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. Exploring author gender in book rating and rec- ommendation. In RecSys, pages 242-250, 2018.
15. [Gao et al., 2023] Chongming Gao, Shiqi Wang, Shijun Li, Jiawei Chen, Xiangnan He, Wenqiang Lei, Biao Li, Yuan Zhang, and Peng Jiang. Cirs: Bursting filter bubbles by counterfactual interactive recommender system. TOIS, 42(1):1-27, 2023.
16. [Gunawardana and Shani, 2009] Asela Gunawardana and Guy Shani. A survey of accuracy evaluation metrics of recommendation tasks. JMLR, 10(12), 2009.
17. [Ham et al., 2004] Jihun Ham, Daniel D Lee, Sebastian Mika, and Bernhard Schölkopf. A kernel view of the di- mensionality reduction of manifolds. In ICML, page 47, 2004.
18. [Hardt et al., 2016] Moritz Hardt, Eric Price, and Nati Sre- bro. Equality of opportunity in supervised learning. In NeurIPS, volume 29, 2016.
19. [Harper and Konstan, 2015] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. TIIS, 5(4):1-19, 2015.
20. [He et al., 2020] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR, pages 639-648, 2020.
21. [Järvelin and Kekäläinen, 2017] Kalervo Järvelin and Jaana Kekäläinen. Ir evaluation methods for retrieving highly relevant documents. In SIGIR, volume 51, pages 243-250. ACM New York, NY, USA, 2017.
22. [Jeon et al., 2021] Insu Jeon, Wonkwang Lee, Myeongjang Pyeon, and Gunhee Kim. Ib-gan: Disentangled represen- tation learning with information bottleneck generative ad- versarial networks. In AAAI, volume 35, pages 7926-7934, 2021.
23. [Kosinski et al., 2013] Michal Kosinski, David Stillwell, and Thore Graepel. Private traits and attributes are pre- dictable from digital records of human behavior. PNAS, 110(15):5802-5805, 2013.
24. [Kraskov et al., 2004] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mu- tual information. Physical review E, 69(6):066138, 2004.
25. [Li et al., 2021] Yunqi Li, Hanxiong Chen, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. User-oriented fair- ness in recommendation. In WWW, pages 624-632, 2021.
26. [Li et al., 2023] Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, Juntao Tan, Shuchang Liu, and Yongfeng Zhang. Fairness in recommendation: Foundations, meth- ods, and applications. TIST, 14(5):1-48, 2023.
27. [Ma et al., 2020] Wan-Duo Kurt Ma, JP Lewis, and W Bas- tiaan Kleijn. The hsic bottleneck: Deep learning without back-propagation. In AAAI, volume 34, pages 5085-5092, 2020.
28. [Mehrabi et al., 2021] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. CSUR, 54(6):1-35, 2021.
29. [Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv, 2018.
30. [Rendle et al., 2012] Steffen Rendle, Christoph Freuden- thaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. arXiv, 2012.
31. [Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Woj- ciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS, vol- ume 29, 2016.
32. [Shao et al., 2022] Pengyang Shao, Le Wu, Lei Chen, Kun Zhang, and Meng Wang. Faircf: Fairness-aware col- laborative filtering. Science China Information Sciences, 65(12):222102, 2022.
33. [Shao et al., 2024] Pengyang Shao, Le Wu, Kun Zhang, Defu Lian, Richang Hong, Yong Li, and Meng Wang. Av- erage user-side counterfactual fairness for collaborative fil- tering. TOIS, 2024.
34. [Tan et al., 2021] Yanchao Tan, Carl Yang, Xiangyu Wei, Yun Ma, and Xiaolin Zheng. Multi-facet recommender networks with spherical optimization. In ICDE, pages 1524-1535. IEEE, 2021.
35. [Tishby et al., 2000] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv, 2000.
36. [Vert et al., 2004] Jean-Philippe Vert, Koji Tsuda, and Bern- hard Schölkopf. A primer on kernel methods. Kernel meth- ods in computational biology, 47:35-70, 2004.
37. [Wang et al., 2023] Zifeng Wang, Zheng Zhan, Yifan Gong, Yucai Shao, Stratis Ioannidis, Yanzhi Wang, and Jennifer Dy. Dualhsic: Hsic-bottleneck and alignment for continual learning. In ICML, 2023.
38. [Wu et al., 2020] Le Wu, Yonghui Yang, Kun Zhang, Richang Hong, Yanjie Fu, and Meng Wang. Joint item rec- ommendation and attribute inference: An adaptive graph convolutional network approach. In SIGIR, pages 679- 688, 2020.
39. [Wu et al., 2021a] Chuhan Wu, Fangzhao Wu, Xiting Wang, Yongfeng Huang, and Xing Xie. Fairness-aware news rec- ommendation with decomposed adversarial learning. In AAAI, volume 35, pages 4462-4469, 2021.
40. [Wu et al., 2021b] Le Wu, Lei Chen, Pengyang Shao, Richang Hong, Xiting Wang, and Meng Wang. Learning fair representations for recommendation: A graph-based perspective. In WWW, pages 2198-2208, 2021.
41. [Wu et al., 2023] Chenwang Wu, Xiting Wang, Defu Lian, Xing Xie, and Enhong Chen. A causality inspired frame- work for model interpretation. In KDD, pages 2731-2741, 2023.
42. [Xiao et al., 2017] Lin Xiao, Zhang Min, Zhang Yongfeng, Gu Zhaoquan, Liu Yiqun, and Ma Shaoping. Fairness- aware group recommendation with pareto-efficiency. In RecSys, pages 107-115, 2017.
43. [Xie et al., 2017] Qizhe Xie, Zihang Dai, Yulun Du, Ed- uard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. In NeurIPS, vol- ume 30, 2017.
44. [Yang et al., 2023] Yonghui Yang, Zhengwei Wu, Le Wu, Kun Zhang, Richang Hong, Zhiqiang Zhang, Jun Zhou, and Meng Wang. Generative-contrastive graph learning for recommendation. In SIGIR, pages 1117-1126, 2023.
45. [Yao and Huang, 2017] Sirui Yao and Bert Huang. Beyond parity: Fairness objectives for collaborative filtering. In NeurIPS, volume 30, 2017.
46. [Zhao et al., 2023] Chen Zhao, Le Wu, Pengyang Shao, Kun Zhang, Richang Hong, and Meng Wang. Fair representa- tion learning for recommendation: a mutual information perspective. In AAAI, volume 37, pages 4911-4919, 2023.
47. [Zhu et al., 2018] Ziwei Zhu, Xia Hu, and James Caverlee. Fairness-aware tensor-based recommendation. In CIKM, pages 1153-1162, 2018.