An Adaptive Computational Model for Personalized Persuasion

1 Introduction

Persuasion has been heavily researched in the fields of psychology and social science for many years. It is defined as "a symbolic process in which communicators try to convince other people to change their attitudes or behaviors regarding an issue through the transmission of a message in an atmosphere of free choice" [Perloff, 2010]. It has been illustrated by Reeves and Nass [Reeves and Nass, 2003] that interactive systems have the potential to engage in the same persuasion process as human do. Fogg [Fogg, 2002] then brought persuasion into computing with his valuable work on persuasive technologies. As an extension of Fogg's work, Aarts et al. [Aarts et al., 2007] proposed the combination of ambient intelligent systems and persuasion. In this way, greater persuasive power can be integrated into every aspect of life rather than the traditional box-like machines.

Although these well-established works have been done for years, a number of aspects remain poorly understood, especially how persuasion can really be persuasive at an individual level. On one hand, persuasive technologies have been mostly exploited associated with marketing applications based on the positive averaging effects over groups of people. But it has been proved that the effect for individual attitude and behavior change is less successful [Milgram, 2009]. On the other hand, the design of individual level behavior change is more preferred by the designers for these systems, since more and more technologies are marketed with the promise of changing an individual user's behavior. As far as we know, a comprehensive technology for affecting an individual's behavior or attitudes in a reliable way is still immature. Moreover, Fogg and Eckles have claimed that in order to persuade individuals to change their attitudes or behaviors, three factors must be satisfied: a) the right content, b) at the right timing and c) the right strategy [Fogg and Eckles, 2007]. As the first two aspects are typically fixed, many researchers have suggested that the adaptation of the "way", by focusing on different means to an end, should be the main focus of persuasion at the individual level [Kaptein et al., 2010]. Hence in this paper, we focus on making the persuasion more adaptive to different individuals' personal states through the right strategy.

We propose a unified framework for personalized persuasion based on the Elaboration Likelihood Model (ELM). ELM is a theory of the thinking processes that might occur when we attempt to change a person's attitude through communication; it serves as a major foundation for most of the persuasion-related theories. Based on this framework, we present a computational model called Model for Adaptive Persuasion (MAP) for virtual agents. It is a semi-connected network model which enables an agent to adapt its persuasion strategies through feedback. By incorporating MAP, an agent will be able to identify which route of thinking may be involved and learn the user's personal state from the user's feedback.

We have developed a virtual nurse agent, called Florence, based on MAP in a 3-D virtual home environment. Florence is designed to take care and recommend healthy lifestyle habits to the elderly. A pilot user study has shown that compared with the single best persuasion strategy method, the MAP-based virtual agent achieved a significantly higher rate of persuasion, higher level of social presence, and lower degree of frustration.

The rest of the paper is organized as follows. We describe the theoretical basis of our model in section 2. Section 3

presents the flow of persuasion based on ELM and section 4 discusses the MAP in details. Section 5 presents the implementation of the virtual nurse agent and section 6 reports the user study results. The final section concludes with a highlight of future directions.

2 Theoretical Basis

Most of the persuasion models reviewed above are based on an important theory, namely the Elaboration Likelihood Model (ELM) [Cacioppo et al., 1986], which is essentially a theory of the thinking processes that might occur when we attempt to change a person's attitude through communication. Fig 1 shows that how those variables play different effects within the persuasion processes.

ELM assumes that individuals can differ in the way they think about a message. When they are presented with information, some level of elaboration occurs. Where people fall along this elaboration continuum is determined by their motivation and ability to process the message presented to them. Motivation is used to represent the reasons for our actions, our desires, and our needs for a message; ability is the people's resources and skills to think about the message and it's topic. Motivation can be influenced by the relevance of the issue, personally responsible for processing the message and enjoyment of thinking. Factors that will have impact on ability are intelligence, level of perceived knowledge, time to engage in the message, the amount of distraction in the communication and the number of message repetitions. In summary, when the motivation as well as the ability to think about the message and its topic are high, central route would be used. However, when there is little or no motivation in the subject and lesser ability to process the message, "peripheral route to persuasion" would be used.

2.1 Central Route

Central route processes are those that require a careful and thoughtful consideration, and are likely to predominate under conditions that generate high elaboration.

• Logic is defined as the "interrelation or sequence of facts or events when seen as inevitable or predictable" [Gellner, 1992]. It is normally used to describe facts and figures that support a persuader's claims. Evidence has suggested that using logic can enhance ethos because it makes the persuader look knowledgeable.
• Reasoning is "the use of reason especially to form conclusions, inferences, or judgments". It can be either deductive or inductive, depending on whether it seeks to demonstrate a certainty or a probability.
• An Example can be an element characteristic of its kind or illustrating a general rule. Examples can serve as proof of your arguments. Using examples enhances a persuader's meaning and makes the argument more concrete.
• Evidence is defined as "the data on which a conclusion or judgment may be based; something that furnishes proof". Evidence varies in persuasive power depending on the context in which it is used.
• Facts are defined as a pattern or model, as of something to be imitated or avoided. Using facts is a powerful means of convincing. It can come from readings, observation or personal experience.

2.2 Peripheral Route

The tactics for the peripheral route can be overwhelming to the developers. Various theories have been done to discuss how they influence strategies. We include some of the most acceptable strategies here:

• Reciprocity: People respond to a friendly action with another positive action or more cooperative way to pay back a favor.
• Liking: We are more inclined to act according to the request made by someone we like. This has been supported by overwhelming evidence suggested by researchers who exploit increased liking due to interpersonal similarity.
• Social Proofing: Individuals tend to believe and behave similarly to others who share the same belief and behavior. This effect has been widely used in commercials, e.g. claiming the products are bestsellers so that they can influence people's decision making.
• Consistency: According to studies in cognitive dissonance, people are inclined to maintain consistent beliefs, acts and statements accordingly.
• Authority: In every social community, there exist some levels of responsibility and obedience to authority. This is the reason that authority is considered a form of social influence that is effective. Therefore, customers are frequently exposed to the products' authority endorsements such as "expert reviews".
• Scarcity: humans place a higher value on an object that is scarce, and a lower value on those that are abundant [Mittone, 2009]. According to multiple theories in social psychology, this is because the possession of

scarce products produces feelings of personal distinctiveness or uniqueness.

3 The Flow of Persuasion

Following ELM, we have built a complete model that can be adaptive to users' personal state. As shown in Fig 2, the flow of a persuasion system can be defined as follows. As the user's input message is given, the route judgement model first determines which route to choose. There may be many factors influencing the route judgement model such as motivation and ability. When a route is selected, different strategies may be executed under each route. For the central route, there are five strategies, namely using logic, using reasoning, using examples, providing evidence and giving facts. For the peripheral route, there are six strategies: reciprocation, liking, social proofing, consistency, authority and scarcity. If the selected strategy succeeds, the current persuasion session is considered completed. Otherwise, it will go back to the route judgement model that will determine which route to go through in the following persuasion or whether the persuasion should be paused for a while.

4 The MAP

Given the above guidelines, Fig 3 shows the Model of Adaptive Persuasion (MAP). It is a semi-connected network model which employs three layers, namely an internal layer, a route of thinking layer and a strategy layer. The internal layer comprises two nodes, namely the motivation node and the ability node. The route of thinking layer consists of two nodes: the central route node and the peripheral route node. The strategy layer includes eleven strategy nodes. The first five nodes (S1 -S5) are strategies under the central route, and the rest (S6 -S11) are strategies under the peripheral route. The dynamics of MAP, similar to those found in self-organizing neural network models [Carpenter and Grossberg, 1987; Tan et al., 2007; Tan, 2004; 2007], is described below.

4.1 Definitions

Internal State: Let X = (xm, xa) denote the activation vector representing a user's internal state that will influence that user's Route of Thinking, where xm is the activation value of the user's motivation and xa is the activation value of the user's ability.

Route of Thinking: Let Y = (yc, yp) denote the activation vector representing the user's route of thinking, where yc is the activation value of the central route and yp is the activation value of the peripheral route.

Strategy: Let Z = (z1, z2, ..., zn) denote the activation vector representing the choice of the strategies under consideration, where zj indicates the activation value of strategy j, for j = 1, ..., n.

Eligibility vectors: Let E = (e1, e2, ..., en) denote the eligibility vector of the strategies, where ej indicates the eligibility value of strategy j, for j = 1, ..., n. Initially, ej are all 1's. Once a strategy j is selected for use, the eligibility value ej is set to 0.

Weight vectors for central route: Let Wr = (wcm,wca) denote the weight vector for the central route, where wcm indicates the weight value from the Motivation node to Central route node, and wca indicates the weight value from Ability node toCentral route node.

Weight vectors for strategies: Let Ws indicate the weight vector of strategy. Initially, the weight of strategy j wjk = δ, where j = 1, ..., n, δ ∈ [0, 1], k ∈ {c, p}.

4.2 From Sensory to Action

Given the internal state vector X, the system undergoes two processes, namely from Internal States to Route of Thinking and from Route of Thinking to Strategy Selection, so as to select a strategy based on the output activities of the strategy vector Z. The detailed algorithm is presented below.

From Internal States to Route of Thinking

Given the internal state vector X and the weight vector Wr, the value of the Route of Thinking vector is determined by

The competition process follows the following steps: If yc > 0.5, yc = 1 and yp = 0. Otherwise yp = 1 and yc = 0. In Section 5, we will discuss how these values are obtained in details.

From Route of Thinking to Strategy Selection

1. Code Activation: Given the Route of Thinking vector Y, the weight vectors for strategies Ws, and the eligibility vectors E, the activation values of the Strategy Vector Z (given by zj, for j ∈ [1, ..., n]) is computed as follows

1. Code Competition: All the strategy nodes undergo a code competition process. The winner is indexed at J where

1. Strategy Selection: The chosen node J with the maximum activation value would be the selected strategy. The eligibility value of eJ is then set to be zero.

4.3 FromFeedback to Learning

Upon receiving feedback from the persuadee after applying the strategy J, the system adjusts its internal representation based on the following principles. Given a reward signal r (persuadee agrees), the agent learns that the strategy executed results in a favorable outcome. Therefore, the system learns to associate the node SJ with a higher weight wJk. Conversely, if a penalty is received (persuadee disagrees), there is a decrease of the weight wJk for node J.

Learning: After a node J is selected for firing, wJ is modified by

where

and where α is the learning rate, r is the reward and δ is the decay rate.

In the case where there is no feedback, which means the persuadee did not respond, the weight will not be changed. The same strategy will be used again, but in a different way instead.

4.4 Update the Route of Thinking

If the feedback from the persuadee after performing the strategy J is negative, the activation value of yc and yp would be modified as well. This enables the agent to autonomously switch between the routes of thinking, so that if there is any misjudging of the persuadee's internal states or if the internal states have changed, the agent can still adapt to the persuadee. When r = 0, if j ∈ [1, ..., 5], yc and yp are modified by

where ε is a small positive decimal between zero and one. Otherwise, yp and yc are modified by

The dynamics of the MAP model is summarized in Algorithm 1.

5 ACase Study with Virtual Nurse

We have developed a virtual nurse agent called Florence specializing in healthcare advice and recommendation. Presented as a 3-D high-resolution avatar (implemented using the Unity 3-D engine), the virtual nurse provides a wide range of caregiving functions: (1) monitoring the well-being of the elderly; (2) giving medication reminders; (3) encouraging healthier lifestyle in a holistic manner, such as recommending physical and mental exercises when necessary, introducing healthier dietary choices, promoting lifelong learning, and suggesting social interactions with others; and (4) providing suggestions for stress reduction. Fig 4 shows a screenshot of the virtual nurse.

The system architecture of the virtual nurse is presented in Fig 5. The intention or responses of the elder, which can be either in text or speech, are interpreted by the Speech Recognition and Natural Language Understanding (NLU) module.

Algorithm 1: Dynamics of the MAP model

• 1 Initialize the network;
• 2 Given the internal state vector X and the weight vector Wr, the value of the route of thinking is determined by yc = ∑i=m,a wcixi. If yc > 0.5, the central route is activated, otherwise, the peripheral route is activated;
• 3 while persuasion has not succeeded do
• 4 Given the activation values of the route of thinking vector Y , the weight vectors for strategiesWs, and the eligibility

vectors E, the activation value of zj is computed by zj = ∑k=c,p ykwjkej; 5 All the nodes for strategies undergo a code competition process, zJ =max{zj : for all j = [1, ...n]}; 6 Adopt the winning node J's strategy and update the eligibility value so that eJ = 0; 7 Perform the strategy J, observe the received reward r (if any) from the environment; 8 if r exists then 9 Adjust the weight vector Ws: ∆wJk = α(1 - wJk)r - δwJk; wJk(t + 1) = wJk(t) + ∆wJk; 10 else 11 Reapply the strategy in a different way; 12 Update yc and yp ;

After processing the information, the virtual nurse answers or responds to the elder (both text and speech) through the Dialogue Management and Natural Language Generation (NLG) module. In addition, her facial expression, body posture, and hand gesture are animated through the Non-Verbal Behavior module. The speech-to-text and text-to-speech functions are realized by incorporating the Microsoft Speech Recognition tool.

We have incorporated the MAP into the Dialogue management and NLG module. The values of the internal state vector (motivation and ability) are obtained through two different modules: Natural Language Understanding (NLU) and User Model. During a dialogue, before adopting a strategy to persuade the user, the system first provides a recommendation and observes the user's response. If NLU interprets a positive feedback, it means the motivation is high (a weight of one will be given). On the other hand, ability is obtained through the user model. Specifically, as the user model maintains a record of the user's healthy condition, we may validate the recommendation against the user model to see whether the user has the required ability to accept the recommendation. An example scenario of the agent-based virtual nurse trying to persuade the elderly Lucy to do some exercises is given in Table 1.

During this conversation, when the agent asked why Lucy was not feeling like doing exercises, Lucy replied that she was too old. This was interpreted by the Natural Language Understanding module as low motivation. Hence, wc was given a lower value than wp. Therefore, as described in the previous section, the model first went to the peripheral route, and selected the strategy according to the policy. It first selected "Social proofing" as the strategy. After getting Lucy's attitude, it selected "Authority" which successfully persuaded Lucy to do exercises.

6 Pilot User Study

For empirical comparison, we developed two versions of the virtual nurse. The first nurse (E1) provides the baseline control condition, wherein the virtual nurse Abby persuades using the single best strategy (suggested by the users themselves). The second virtual nurse Florence (E2) is the treatment condition, wherein MAP is embodied. The objective of the user study is twofold. Firstly, we want to observe whether Florence outperforms Abby in persuasion abilities. Secondly, we want to assess the difference in the user experience provided by Florence and Abby, particularly for telepresence, social presence, interactivity and frustration. The first three criteria are considered as the most important features measuring user experience in virtual environments, and frustration has become a popular feature which measures the "dark side" of technology. Specifically, Telepresence refers to a set of technologies which allow a person to feel as if they were present [Steuer, 1992]. Social presence, as a subcomponent of presence, is composed of social richness, social realism and co-presence [Lombard and Ditton, 1997]. Lee et al. [Lee et al., 2006] defined interactivity in media user terms, as the perceived degree that a person in a communication process with at least one more intelligent being can bring a reciprocal effect to other participants of the communication process by turn taking, feedback, and choice behaviors. Frustration refers to the emotional response to opposition. It arises from the perceived resistance to the fulfillment of individual will [Wright et al., 2009].

6.1 Research procedures

The scenario given to the subjects was one wherein the virtual nurse is tasked to persuade the user to do exercises and eat healthy food. 26 subjects with age ranging from 56 to 75 were recruited. Subjects were provided with a set of detailed instructions on the experimental procedures. Before the experiment began, the experimenter conducted a short tutorial session to familiarize the subjects with the basics of the environment and how to interact with the virtual nurse. After the tutorial, the subjects filled out a pre-study questionnaire which included demographics information and preferred persuasion strategy. After a subject finished interacting with the virtual nurses, another questionnaire was administered. We captured four variables, namely telepresence (TP), social presence (SP), interactivity (INT), frustration (FR). The items were captured using a seven point Likert scale where 1 refers to strongly disagree and 7 refers to strongly agree. Given the limited space, we present a sample set of these items in Table 2.

6.2 Data Analysis

Descriptive Statistics: The results show that 17 people out of 26 (65%) have been successfully persuaded by the MAPbased agent Florence, whereas only 9 out of 26 (35%) have been persuaded by single best strategy based Abby. Table 3 shows the means, standard deviations (SD), and confidence intervals (CI, with a confidence level of 95%) of telepresence

(TP), social presence (SP), interactivity (INT) and frustration (FR) for both the E1 and E2 conditions. Table 3 indicates that E2 has better ratings in telepresence, social presence and frustration than E1. Hence, the MAP-based agents outperformed the agents that use the single best persuasion strategy in most of the features.

We used a one-way analysis of variance (ANOVA) to analyze the data. Specifically, the F-test was used to evaluate the hypothesis of whether there are significant differences between the conditions. The p values represent the probability that the test statistics across the conditions are similar (i.e., if p is large) or different (i.e., if p is small). In other words, a small p value indicates a high confidence that the values across the conditions are different. A summary of the F values and p values between E1 and E2 is shown in Table 4. The data analysis revealed significant differences between E1 and E2 in terms of two constructs, namely social presence and frustration: F(1, 51) = 5.23, p < 0.01 for social presence and F(1, 51) = 5.31, p < 0.01 for frustration, where the two parameters (enclosed in parentheses) of F indicate the degrees of freedom of the variances between and within conditions, respectively. Consistent with the statistics in Table 3, the MAP-based virtual nurse generated a higher level of social presence and a lower level of frustration than the agent using the single best strategy, with a mean of 4.76 for social presence and a mean of 3.19 for frustration. There is no significant difference found for both telepresence and interactivity.

7 Conclusion

This paper has examined the gaps between the wellestablished theories of persuasion under the domains of psychology, social science and agent technology, specifically, the lack of computational methods which can adapt to different individuals' preferences based on these theories. To this end, we have developed a computational model for personalized persuasion called MAP, which can be embodied into virtual human-like agents [Kang et al., 2012b; 2012a] to enable adaptive persuasion. Experiments on real users have validated our approach and model.

In the future, we shall extend our pilot user study to a more comprehensive one. We would also like to extend our research on sustained human-agent persuasion. Specifically, we wish to explore the possibility of long-term human-agent persuasion behavior by different means to enhance users' need for cognition (NFC), such as how to move users' attitudes from the peripheral route to the central route.

8 Acknowledgments

This research is supported by the National Research Foundation, Prime Ministers Office, Singapore under its IDM Futures Funding Initiative and administered by the Interactive and Digital Media Programme Office.