Misleading Opinions Provided by Advisors: Dishonesty or Subjectivity

1 Introduction

In large, open online communities, users often encounter entities (e.g. people, products and information) which they have no previous experience with or prior knowledge of. To choose which entities to interact with, they usually rely on the experience or knowledge of other users (advisors). However, advisors might provide misleading opinions for two very different reasons. Firstly, some advisors might be dishonest due to their intrinsic natures such as the low level of benevolence, integrity and competency [McKnight and Chervany, 2001]. They tend to intentionally provide overly positive or negative opinions for some entities which are completely contradict with their real experience. Secondly, advisors might be honest, but are subjectively different [Fang et al., 2012] from users. They provide true opinions based on their experience, which might be unintentionally misleading for users due to their salient subjectivity difference with users.

It is thus important for users to evaluate the quality of advisors' opinions in order to determine how much to be relied on. One generally adopted approach [Teacy et al., 2006] is to model the trustworthiness of advisors. The basic idea is that a more trustworthy advisor to a user will provide higher quality opinions to the user. However, some existing trust models may only consider either advisors' dishonesty [Zhang and Cohen, 2007] or subjectivity difference [Fang et al., 2012], while others cannot accurately distinguish these two different factors [Regan et al., 2006; Noorian et al., 2011]. As indicated, dishonesty is an intrinsic property of advisors, while subjectivity difference exists between users and advisors. Even for a same (dis)honest advisor, different users may have different perceptions on her trustworthiness due to the different level of subjectivity difference between the advisor and each user. It is thus necessary to clearly distinguish these two factors.

In this paper, we propose a novel probabilistic graphical trust model that explicitly distinguishes the factors of dis-

line community involving users, advisors and entities, both users and advisors can provide ratings to entities. Some users may explicitly identify their (dis)trust towards some advisors. Given information about ratings and trust relationships (if any), we model the factors of advisors' intrinsic nature (i.e. benevolence, integrity and competence), users' propensity to trust advisors, and subjectivity difference between users and advisors, as latent variables in the model that may influence users' trust towards advisors. Through detailed experiments on three real datasets, it is confirmed that our model largely outperforms competing approaches for modeling advisor trustworthiness.

2 RelatedWork

Different approaches have been proposed to model advisor trustworthiness for users. Some approaches, such as [Teacy et al., 2006; Zhang and Cohen, 2007; Liu et al., 2011], focus on low quality opinions (unfair ratings) intentionally provided by dishonest advisors. Due to the ignorance of subjectivity difference between advisors and users, they may misuse some important information caused by subjectivity difference, rather than dishonesty. Some other approaches, such as [Fang et al., 2012], only consider subjectivity difference between users and advisors, but ignore the dishonesty characteristic of advisors. They may mistakenly treat dishonest advisors as those having subjectivity difference with users.

To model both dishonesty and subjectivity of advisors, BLADE [Regan et al., 2006] and Prob-Cog [Noorian et al., 2011] are proposed recently. BLADE applies Bayesian learning to model the correlations between entities' properties and ratings of users and advisors. However, it does not explicitly distinguish dishonesty and subjectivity in its modeling process. If the correlations learned for users' ratings are based on entities' properties that are different from those for advisors, it is likely that advisors having subjectivity difference are treated as dishonest. Prob-Cog is a two-layered behavioral modeling approach. First, it filters dishonest advisors according to the rating similarity between users and advisors. Second, trustworthiness of honest advisors is discounted according to their subjective trends. However, Prob-Cog has the assumption that advisors providing very different ratings with a user are dishonest to the user. In consequence, advisors having large subjectivity difference with the user will be misclassified as dishonest. In contrast, our model explicitly distinguishes (dis)honesty and subjectivity difference by modeling them using different sources of rating information, and captures their relationships with trust through the influence of chains in our probabilistic graphical model.

Our model also adopts the trust topology widely studied in Social Science [McKnight and Chervany, 2001] to model advisor (dis)honesty. A similar model is TAF of Chua and Lim [2010], which considers entities' competency, and users' trust propensity and contingency. TAF targets at a different research problem that models users' trust towards entities in online communities. We model advisor trustworthiness and additionally consider benevolence and integrity of advisors.

3 The Probabilistic Graphical Trust Model

We first identify the factors that influence users' trust towards advisors, to construct a conceptual framework of trust shown in Figure 1. More specifically, in Social Science, benevolence, integrity and competence are regarded as antecedents of trust [McKnight and Chervany, 2001] to explore the relationship between a trustor (user) and a trustee (advisor). They are the intrinsic characteristics of advisors' (dis)honesty that may directly influence users' trust towards advisors (drawn as solid lines). An intrinsic characteristic of users-trust propensity, referring to users' initial trust towards unknown advisors before interacting with them, may also have direct effect on users' perception on advisor trustworthiness [Mayer et al., 1995]. In reality, users' different background will conduce to their different initial trust towards others. Different users may have different subjectivity in evaluating same entities. Even an advisor is honest, a user may have different experience with the same entity compared to what the advisor has. In other words, subjectivity difference between a user and an advisor can affect the user's satisfaction level of the opinions provided by the honest advisor, and further indirectly influence the user's perception on the advisor's trustworthiness

(drawn as the dashed line). Furthermore, experience difference between the user and the advisor, as well as that between the advisor and other advisors (users) can have direct impact on the user's perception towards the advisor's benevolence, integrity and competence.

In the next subsections, we first show the probabilistic graphical trust model designed according to the conceptual framework. We then present its parameters and the generative process of sampling observable variables. We also elaborate the inference process of the model and estimation of parameters. Finally, we predict advisor trustworthiness for users.

3.1 Parameters and Generative Process

In some online communities, e.g. Epinions (epinions.com), users may explicitly indicate their trust towards some advisors (trust links), while in others, e.g. eBay (ebay.com), no trust links are available. We thus design two graphical models, shown in Figures 2(a) and 2(b), for the two types of communities, respectively. We use graphical model mainly because it can fully interpret our conceptual framework, and seamlessly merge supervised and unsupervised learning (labeled and unlabeled relationship) [Jordan et al., 1999].

We assume a user u and an advisor a in a community, and denote a's trustworthiness perceived by u as tu,a ∈ [0, 1] where 1 means totally trust and 0 totally distrust 1 . Some other parameters are: 1) u's trust propensity yu; 2) a's competence ca, benevolence ba and integrity ia; and 3) a's subjectivity difference with u, su,a. All these parameters are modeled as distribution parameters, and the expected values are in [0, 1]. Specifically, the expected trust propensity yu of 1 represents complete propensity to trust, while that of 0 no propensity.

Competence (c) indicates whether an advisor has ability to provide reliable ratings (opinions) to users. Hence, it is reasonable to regard it as a latent variable that directly connects with the trustworthiness of the advisor. Its expected value 1 refers to full competence, while 0 means no competence.

Benevolence (b) refers to the degree that an advisor cares about the preferences of users [McKnight and Chervany, 2001]. Thus, we can easily observe its relationship with the rating difference (r) towards the same entity between every user and the advisor. The higher benevolence of the advisor with respect to users will lead to the smaller rating difference between every user and the advisor. Through the chains in the graphical models (see Figure 2, where r is observable), we capture the relationship of benevolence b with advisor trustworthiness t. The benevolence of advisor a consists of two components, one for trust and another for distrust denoted by ba|t=1 and ba|t=0 respectively.

Integrity (i) emphasizes on the degree that a person follows rules in an organization [McKnight and Chervany, 2001]. Then, it can be inferred that integrity affects rating difference (R) between the advisor and average of all other users for the same entity. Higher integrity implies smaller rating difference. Similar to benevolence, we model the relationship of integrity with advisor trustworthiness (see Figure 2, where R is observable), and ia|t=1 and ia|t=0 represent a's integrity for trust and distrust respectively.

1 In model inference, users' trust towards advisors is labeled as either t = 1 (trust) or t = 0 (distrust) with respect to a predefined threshold, and its corresponding probability value refers to the exact trustworthiness of advisors perceived by each specific user.

Subjectivity difference su,a between user u and advisor a may directly influence rating difference (r) between u and a. Through the chains in the models, we seize its influence on trust modeling. Similarly, subjectivity difference also consists of two components, one for trust and another for distrust denoted by su,a|t=1 and su,a|t=0 respectively.

In addition, for communities where trust links are partially observable, we identify a new latent parameter called expressiveness denoted as eu, representing user u's tendency to express her trust eu|t=1 or distrust eu|t=0 links towards advisors. Note that Figure 2(b) is a special case for Figure 2(a), of which the expected expressiveness eu always equals to 0.

Figure 2 shows dependencies among variables and parameters. The directed arrows represent dependency relationships. Variables in shaded circles (e.g. ru,a,k and ou,a) denote observable variables, while the remaining variables are not observable. The variable in grey circle (i.e. tu,a) is partially observable in Figure 2(a). The boxes are "plates" representing replicates, where A represents all advisors, U all users and K all entities, respectively. The generative process of our model follows the steps below:

Step 1: For each user u ∈ U, sample distribution parameter of propensity yu, using Beta distribution with symmetric hyper-parameters, as yu ∼ Beta(ψ).

Step 2: For each advisor a ∈ A, sample distribution parameters: competence ca, benevolence ba|t and integrity ia|t using Beta distribution with symmetric hyper-parameters, as ca ∼ Beta(α), ba|t ∼ Beta(ϕ), and ia|t ∼ Beta(θ).

Step 3: For each user u ∈ U and each advisor a ∈ A: [3.1] u generates trust for a, tu,a (0 or 1), based on u's propensity yu and a's competence ca, as tu,a ∼ Bern(yu) · Bern(ca); [3.2] u samples distribution parameters: expressiveness eu|t and subjectivity difference su,a|t using Beta distribution with symmetric parameters, as eu|t ∼ Beta(ε) and su,a|t ∼ Beta(β); [3.3] u generates observability of link ou,a based on u's expressiveness eu and trust of the link tu,a, as ou,a ∼ Bern(eu|t); [3.4] for each entity k ∈ K, generate a's rating difference with u, ru,a,k, and a's rating difference with all the other users, RU-a,a,k, as ru,a,k ∼ Bin(m, 1 - ba|t) · Bin(m, su,a|t) and RU-a,a,k ∼ Bin(m, 1 - ia|t), where RU-a,a,k and ru,a,k are in [0,m].

We use Binomial distribution [Chua and Lim, 2010] to model rating difference (r and R). The basic idea is that, by dividing an entity into m parts, we obtain rating difference of m if the difference for every part is 1, m - 1 if that for every part is 1 except one part, and 0 if that for every part is 0. We model each part as a Bernoulli event, and then the rating difference can be generated as Binomial distribution. We choose Binomial distribution instead of multinomial distribution mainly because: 1) the assumption of multinomial distribution (each state is independent and identically distributed) does not hold here; and 2) Binomial distribution fits our problem well since r and R are in a finite range.

Also note that there are two kinds of trust links between users and advisors (trust and distrust). We thus choose Beta distribution with symmetric hyper-parameters for parameters, where each expected value of these parameters from prior distribution is assumed to be 0.5.

3.2 Model Inference and Parameter Estimation

We use Gibbs sampling [Casella and George, 1992] to infer our two models (Figures 2(a) and 2(b)), and then update posterior distribution for each parameter, correspondingly.

Model Inference

Gibbs sampling, as a means of approximate inference, is easy to derive for Bayesian inference (our research case), and comparable in speed to other estimators (e.g. EM algorithm). It is well-adapted to sample posterior distribution of a Bayesian network and approximate a global maximum. Because the conjugacy of Beta and Binomial distributions, we apply collapsed Gibbs sampling [Liu, 1994], which integrates the parameters in Figure 2, for model inference.

Specifically, when trust links are partially observable, as shown in Figure 2(a), we conduct sampling whenever we encounter a trust link of tu,a that is needed to be identified. The Gibbs sampling inference process is:

where tu,-a refers to user u's trust links with other advisors except advisor a; ou,-a represents the observability of u's trust links except that with a; RU-a,-a is the rating difference between each of other advisors (except advisor a) and the average rating; and ru,-a is the rating difference between user u and each of other advisors.

We then evaluate the above 4 components independently:

where n(o1u |t) and n(o0u |t) are the numbers of times that user u shows and hides the trust t to advisors, respectively. n(o1u |t) and n(o0u |t) must exclude the counts between u and a because of the condition on ou,-a.

where n(ttu ) is the number of links with trust value t from user u, and n(tta ) is the number of links with value t to advisor a. Similarly, due to tu,-a, n(ttu ) and n(tta ) must exclude the counts between u and a.

where n(Rra |t) refers to the number of rating difference r between a with all the other users if a has been given trust t. n(RU-a,a|t) denotes the number of rating difference r between a and the average of all the other ratings with regard to commonly rated entities with u given that u's trust on a is t.

where n(rru,a |t) and n(rra |t) denote the number of rating difference r between a and u given trust value t, and the number of rating difference r between a and other users given that trust value to a is t, respectively.

When trust links are not observable, as shown in Figure 2(b). The corresponding sampling process is:

The inference process is similar to the Gibbs sampling process where the trust links are partially observable. The only difference is that we need to conduct sampling for each trust link between users and advisors.

Parameter Estimation

After the inference on tu,a, we can update posterior distributions of y, c, e, i, b and s, as follows: 2

2 Due to space limitation, we omit detailed derivation process.

3.3 Trust Prediction

After learning the parameters, we can evaluate the trust between user u and advisor a, tu,a, using the Markov blanket [Pearl, 1988] of node t in Figure 2. The Markov blanket of a node contains all the variables that shield the node from the rest of the network: its parent, its children and its children's parents. This determines that, for our scenario, the Markov blanket of node t becomes the only knowledge needed to predict the behavior of the node t. Thus, we identify the probability of ti,j = t as follows:

We can use the probability value (tu,a = 1) as the trust value if we expect to obtain continues trust values ranged in [0, 1].

4 Experiments

In this section, we carry out experiments to evaluate the performance of our probabilistic graphical trust model (PGTM) on predicting advisors' trustworthiness perceived by users, and conduct comparisons with some competing approaches.

4.1 Benchmark Approaches

We compare our approach with two state-of-the-art models, including BLADE [Regan et al., 2006] and Prob-Cog [Noorian et al., 2011] detailed in Section 2. For BLADE implementation, we treat the average ratings of entities as the attribute modeled on the three datasets. For the Prob-Cog, we tune the parameters so that the model achieves its best performance. Specifically, in the first layer, an advisor is considered as dishonest when its rating difference with a user is larger than threshold μ (μ ∈ [0.5, 0.8]). In the second layer, the trustworthiness of an advisor is further adjusted by her tendency (positive or negative) (β + ϵ ≤ μ). We also show the performance of a naive baseline approach where a user judges an advisor's trustworthiness based on commonly rated entities. If the rating difference on a same entity is smaller than a predefined threshold, the user's experience with the advisor on the entity is positive, otherwise negative. Positive and negative experiences are aggregated to compute the advisor's trustworthiness using the Beta function.

4.2 Data Description

Each of our dataset consists of two files. One file stores the (dis)trust links with 3-tuples (user, user, (dis)trust) which serve as ground-truth, and the other file stores users' ratings of entities with 3-tuples (user, entity, rating). Notice that the links are directed: user a (dis)trusts user b does not imply that b also (dis)trusts a. The goal of all models is to predict the ground-truth (dis)trust links based on the commonly rated entities between users. To obtain our data, we process three available datasets, including FilmTrust (trust.mindswap.org/ FilmTrust), Flixster.com and Epinions.com. Specifically, we use the original FilmTrust dataset, and randomly sample a small portion of original Flixster dataset. For Epinions dataset, the (dis)trust links between users might be due to their direct rating interactions since the entity (i.e. article) is created by users who in turn could rate others' entities. We thus exclude all (dis)trust links in which users have rated some entities created by advisors. The statistical information is summarized in Table 1.

4.3 EvaluationMetrics

To measure the performance of models, we use the commonly used metrics, including precision, recall, f-value, and MAE. Precision= tp tp+fp , recall= tp tp+fn , and f-value= 2×precision×recall precision+recall , where tp, fp and fn is the number of correctly predicted trust links, incorrectly predicted trust links, and incorrectly predicted distrust links respectively. MAE (mean absolute error) refers to the average of the difference between the predicted trust value of each user-advisor pair and the ground-truth trust value (0 or 1). Since there are merely trust links but no distrust links on FilmTrust and Flixster datasets, only precision and MAE are used for these two datasets.

4.4 Results and Discussion

In this section, we first check the effectiveness of latent variables in our model. Then, we present the performance of our model and three benchmark approaches on the three datasets. We further examine these approaches in detail by varying the threshold for trust prediction. If a predicted trust value of advisor a from user u is larger than the threshold, u trusts a.

F-value

Model Effectiveness

In Figure 3, we analyze the effectiveness of each latent variable in our model by showing the performance of the model with only propensity latent variable (PGTM(y)), with propensity and competence latent variables (PGTM(y, c)), with propensity, competence, benevolence and subjectivity difference latent variables (PGTM(y, c, b, s)), with all but expressiveness latent variable (PGTM(y, c, b, s, i)) and the complete model (PGTM(full)) which includes all latent variables. As can be seen, PGTM(y, c) performs better than PGTM(y) when trust threshold is larger than 0.5. This might be due to the fact that PGTM(y, c) rather than PGTM(y) could enable the risk-aversion users (with higher trust threshold) to adjust their propensity to (dis)trust according to advisors' competence through interactions. Next, the superiority of PGTM(y, c, b, s) over PGTM(y, c) becomes more salient when the number of commonly rated entities between users and advisors increases (Epinions>FilmTrust>Flixster as shown in Table 1). This is in accordance with the structure of our probabilistic trust model where subjectivity difference between users and advisors and the benevolence of advisors are directly connected with the rating difference between users and advisors. With more commonly rated items, we could model rating difference more accurately. Moreover, PGTM(y, c, b, s, i) could outperform PGTM(y, c, b, s) saliently when the number of commonly rated items between a user and an advisor is limited. This is because that the additional variable integrity is mainly modeled by the rating difference between the advisor and all other users and thus is less sensitive to the number of commonly rated items. Finally, the complete model PGTM(full) performs best because instead of using the ratings, we use partially observable trust links to model the expressiveness latent variable. Here, 20% of the trust links are observable.

We also investigate the effects of subjectivity difference between users and advisors and dishonesty of advisors in our model. The result on Epinions dataset is illustrated in Figure 4, where subjectivity denotes the model with only propensity and subjectivity difference latent variables, while dishonesty denotes the model with propensity and competence, benevolence, and integrity latent variables. It can be observed that PGTM(y, c, b, s, i) outperforms subjectivity model and dishonesty model, demonstrating that the subjectivity difference factor which is paid less attention before, in addition to the dishonesty factor, improve the performance of our model. We also find that the dishonesty plays a more important role than subjectivity difference. Specifically, the omission of the dishonesty related latent variables (i.e. competence, benevolence and integrity) will lead to a more salient drop than the omission of subjectivity-related latent variable (i.e., subjectivity difference) when the trust threshold is larger than 0.5.

Model Comparison

Table 2 and Figure 5 show the performance comparisons between our approach (PGTM(y, c, b, s, i)) and other approaches on three datasets. In order to make fair comparisons, we assume that all trust links are not observable in our model (i.e. we adopt the model in Figure 2 (b)). First of all, although our approach is a little more time-consuming than other approaches, we find that the running time is acceptable. Specifically, it takes 0.312s (0.061s for Prob-Cog and 0.218s for BLADE) to run each iteration of our model on Epinions dataset. Second, as shown in Table 2, our model (with trust threshold 0.5) achieves much better performance than other approaches in terms of all metrics on all three datasets (except recall on Epinions). The performance of BLADE and ProbCog are better than Baseline on the Epinions and FilmTrust, but worse than that on the Flixster. This is mainly because there are fewer commonly rated entities in Flixster dataset than in other two datasets (see Table 1). Without enough commonly rated items, Prob-Cog may mistakenly treat subjective users as dishonest ones, and thus filter them in the first layer, while BLADE cannot model advisors' evaluation functions on attribute accurately. On the other hand, our model is not so sensitive to the number of commonly rated entities, because only subjectivity difference between a user and an advisor is measured with regard to the commonly rated entities. Benevolence, competence and integrity are measured based on the advisor's past experience with all other users, and propensity is measured according to all the past experiences of the user.

Figure 5 presents the performance of different approaches by varying the trust threshold. It shows that in general our model outperforms the other approaches. It consistently achieves high precision on the three datasets, demonstrating its effectiveness on modeling the trustworthiness of advisors. It also implies the ability of our model on inferring the (unobserved) (dis)trust links. This is especially important in the current online communities, where users are reluctant to explicitly identify their relationship with other users.

5 Conclusions and FutureWork

We proposed a novel probabilistic graphical trust model, separately considering dishonesty of advisors and their subjectivity difference between users, to model the trustworthiness of advisors. Specifically, our model involves three types of latent variables: 1) the dishonesty related variables: benevolence, integrity and competence of advisors; 2) trust propensity of users; and 3) subjectivity difference between users and advisors. We compared our model with a baseline approach, and two state-of-the-art approaches including BLADE and Prob-Cog. Experimental results indicated that the latent variables in our model are both theoretically reasonable and computationally effective, and dishonesty and subjectivity difference are successfully distinguished. Besides, we demonstrated that our model can more accurately model advisor trustworthiness without using the partially observable trust links. Our approach also mitigates the research gap between computational trust in Computer Science and psychological and behavioral trust in Social Science. For future work, we will extend the current model to address other scenarios (e.g., multi-nominal degrees of trust other than binary case).

Acknowledgement

The work is supported by the MoE AcRF Tier 2 Grant M4020110.020 and the Institute for Media Innovation at Nanyang Technological University.