A Cognitive Agent Model Displaying and Regulating Different Social Response Patterns

1 Introduction

Human social interaction often goes beyond verbal exchange of information. In recent years neurological mechanisms have been discovered that describe how, for example, direct nonverbal contagion of emotions (e.g., responding to a smile) may take place between agents. It turns out that certain preparation states for actions or for expressing body states (at the neural level called mirror neurons) have multiple functions, not only the function of preparing, but also the function of mirroring a similar state of another person; e.g., [Iacoboni, 2008; Rizzolatti and Sinigaglia, 2008].

Given the substantial differences in social behaviour between different persons, these mechanisms provide a useful point of departure to design agent models that offer a wide human-like social interaction repertoire, from which specific elements can be chosen to be realised. In particular, useful inspiration can be found for nonstandard social responses, shown by persons having an Autism Spectrum Disorder (ASD).

Persons with ASD show a wide range of (gradual) differences in social functioning across populations; e.g., [Frith, 2003; Striano and Reid, 2009]. Especially when talking about men, more or less on a regular basis it is put forward that many of them have certain autistic traits. This does not only happen, for example, when persons are just talking in private, but certain variations of this idea also occur in a scientific context. This is already going back to [Asperger, 1944]: 'The autistic personality is an extreme variant of male intelligence. Even within the normal variation, we find typical sex differences in intelligence'. In statistics it turns out that within populations of persons diagnosed with some form of ASD, men strongly dominate, for example, by a factor 5 for the more extreme side to a factor of more than 10 for the milder side of the spectrum; e.g., [Baron-Cohen, 2002; Keller and Ruta, 2010].

Many persons positioned at the mild side of the ASDspectrum are labeled as 'high functioning'. Especially in societal positions where strong concentration on details and systematic attitude are crucial, they may function quite well, up to high levels of excellence. In particular, this is shown in the academic context, and also has been supported in scientific literature such as [Baron-Cohen et al., 2002, 2007; James, 2010]. Among the famous scientists and philosophers from the past who afterwards have been related to some form of ASD are Newton, Darwin, Einstein, Turing, and Kant.

The cognitive agent model presented here takes inspiration from mechanisms put forward in the recent neurological literature on mirroring, emotion regulation, empathy and ASD. These mechanisms have been incorporated in the agent model in an abstracted form, so that the model can be considered a cognitive model inspired by neurological theories, rather than a neural model. The cognitive agent model can be used as a basis for the development of virtual agents, for example, in the context of simulation-based training, gaming or virtual stories.

In this paper, in Section 2 the design of the cognitive agent model and its background in neurological mechanisms described in the literature are presented. In Section 3 an exploration is presented illustrated by a number of simulation results and (emerging) properties shown by the simulated patterns. Finally, Section 4 is a discussion.

2 The Cognitive Agent Model

First the theories from Social Neuroscience used as a basis for the presented cognitive agent model for social response patterns which will be described briefly. The full response modelled can be considered an empathic response, based on the criteria:

• (a) Showing the same emotion as the other agent
• (b) Telling that the other agent has this emotion

Assuming true, faithful bodily and verbal expression, these two criteria are in line with the four criteria of empathy formulated in [De Vignemont and Singer, 2006], p. 435:

1. Presence of an affective state in a person
2. Isomorphism of the person's own and the other person's affective state
3. Elicitation of the person's affective state upon observation or imagination of the other person's affective state
4. Knowledge of the person that the other person's affective state is the source of the person's own affective state

The discovery of mirror neuron systems and their role in social interaction, has led to a number of hypotheses on the mechanisms behind such empathic social responses, and their possible impairments, for example, in persons with ASD. Phenomena shown in persons with ASD have been related to, for example:

• regulation of enhanced sensory processing sensitivity, in particular for face expressions; e.g., [Neumann et al., 2006; Spezio et al., 2007; Baker et al., 2008; Corden et al., 2008]
• reduced activation of mirror neurons; e.g., [Dapretto et al., 2006; Iacoboni, 2008]
• reduced activation of super mirror neurons for self-other distinctions and control; e.g., [Iacoboni, 2008; Brass and Spengler, 2009]
• reduced emotion integration; e.g., [Grèzes and de Gelder, 2009; Grèzes et al., 2009]

For the first item, emotion regulation mechanisms play a central role; cf. [Gross, 1998; Goldin et al., 2008]. They cover antecedent-focused regulation (e.g., selection and modification of the situation, attentional deployment, and reappraisal) and response-focused regulation (suppression of a response). Adapting to enhanced sensitivity for certain types of stimuli, can take place by forms of emotion regulation by avoiding situations or aspects of situations in which these stimuli occur, or focus attention differently, and/or by suppressing the own bodily response. For example, to get rid of arousal triggered by looking at somebody's eyes, which is experienced as too strong, as a form of antecedent-focused regulation (in particular, attentional deployment) the gaze can be taken away from the observed eyes. According to this perspective, gaze aversion and an expressionless face and voice, as often occur in persons with ASD, can be viewed as forms of regulation of the level of arousal, which otherwise would be experienced as disturbing for the other mental processes.

The elements described above have been exploited in the presented cognitive agent model. Thus a human-like agent model is obtained that, depending on its settings is able to show different types of social response patterns. More specifically, the cognitive agent model designed incorporates mirroring, super mirroring (for self-other distinction and control), emotion integration, and gaze adaptation as a form of emotion regulation to compensate for enhanced sensory processing sensitivity; see Figure 1. Here WS is used to denote world states, SS for sensor states, SR sensory representation states and ES for effector states. Moreover, PB indicates a preparation for a body state and PS a super mirroring state for control. Furthermore, PC indicates a preparation for a communication, and EC the actually performed (expression of) communication.

Note that in the causal graph of the model three loops occur: the body loop to adapt the body, the as-if body loop to adapt the internal body map, and the gaze adaptation loop to regulate the enhanced arousal. The effect of these loops is that for any new external situation encountered, in principle, a (numerical) approximation process may take place until the internal states reach an equilibrium (assuming that the situation does not change too fast). However, as will be discussed in Section 3, it is also possible that a (static) external situation does not lead to an equilibrium, but to periodic oscillations.

The cognitive agent model has been computationally formalised using the hybrid modeling language LEADSTO; cf. [Bosse et al., 2007]. Within LEADSTO a dynamic property or temporal causal relation a →→ b denotes that when a state property a (or conjunction thereof) occurs, then after a certain time delay, state property b will occur; g(s) denotes a gaze avoiding s. Below, this delay will be taken as a uniform time step Δt.

In the model s denotes a stimulus (e.g., a smiling face of another agent B), b a body state and B an agent (another agent or the agent self). A super mirroring state can either refer to an agent B, or to enhanced sensory processing sensitivity, indicated by sens. Note that, following [Damasio, 1999], a body state b is used as a label to indicate an emotion, and SR(b) the feeling of the emotion. Communication of b to B means communication that the agent self knows that B feels b.

Connections between states (the arrows in Figure 1) have weights, as indicated in Table 1. A weight ω may depend on a specific stimulus s, and body state b involved, and on an agent B (self or another agent), when this is indicated by an index B. It usually has a value between 0 and 1, but for suppressing effects it can also be negative. In the column indicated by LP a reference is made to the (temporally) Local Property (LP) that specifies the update dynamics of the activation value of the 'to state' based on the activation levels of the 'from states'; see below.

By varying the connection strengths, different possibilities for the social interaction repertoire offered by the model can be realised. Emotion integration takes place by using a connection from SR(b): in LP4 (mirroring), LP6 (super mirroring), and LP7 (preparing communication).

Reduced emotion integration can be expressed by low weights ω6, ω10, ω11B for these connections. Similarly, low values for ω5 in LP4, resp. ω7B, ω8B in LP5 can be used to achieve reduced mirroring, resp. super mirroring, and higher values for ω9, ω10 in LP6 indicate enhanced sensory processing sensitivity. Below, each of the dynamic properties is described in more detail; each time first a semiformal description is given, and next a formal specification in the hybrid LEADSTO format.

During processing, each state property has a strength represented by a real number between 0 and 1; variables V (possibly with subscripts) run over these values. In dynamic property specifications, this is added as a last argument to the state property expressions (an alternative notation activation(a, V) with a a state property has not been used for the sake of notational simplicity). Parameter γ is a speed factor, which determines how fast a state is changing, based on input received from other states connecting to it. The properties LP1 to LP3 describe how sensory representations are generated for an agent B, stimulus, and body state.

LP1 Sensory representation of an external agent B

If an agent B is sensed with level V1,

and the sensory representation of agent B has level V2.

then after duration Δt the sensory representation of agent B will

have level V2 + γ [ ω1BV1) - V2 ] Δt.

SS(B,V1) & SR(B, V2) →→ SR(B, γ f(ω V1) - V2 ] Δt Here f is a function for which different choices can be made, for example, the identity function f(W) = W, or a continuous logistic threshold function of the form

with σ a steepness and τ a threshold value. Note that for higher values of στ (e.g., σ higher than 20/τ) this threshold function can be approximated by the simpler expression:

In the simulations for properties LP1, LP2, and LP11 to LP14 the function f(W) = W was chosen for f; for properties LP3 to LP10 f is based on the logistic threshold function: f(W1, W2) = th(σ, τ, W1+W2); similarly for more arguments.

Property LP2 is similar to LP1 but applied to stimulus s instead of agent B. The sensory representation of a body state as described by property LP3 is not only affected by a corresponding sensor state (which in turn is affected by the body loop), but also via the as-if body loop by the preparation for this body state. Note that the as-if body loop provides effects on the sensory representation in a shorter time than via the body loop: bodily change usually is a factor slower than neurological change (e.g., one or two seconds vs. 300 to 500 milliseconds).

LP3 Sensory representation of a body state

If the sensor state for body state b has level V1

and the preparation state for body state b has level V2

and the sensory representation of body state b has level V3

then after duration Δt the sensory representation of body state b will have level V3 + γ [f(ω3V1, ω4V2) - V3] Δt.

SS(b, V1) & PB(b, V2) & SR(b, V3) →→ SR(b, V3 + γ [f(ω V1, ω V2) - V3] Δt)

Preparation for bodily change triggered by s (e.g., an observed face) is modelled as follows.

LP4 Preparing for or mirroring a body state

If the sensory representation of s has level V1,

and the sensory representation of b has level V2,

and the preparation for body state b has level V3

then after duration Δt the preparation state for body state b will

SR(s,V1) & SR(b,V2) & PB(b, V3) →→ PB(b, V3 + γ [f(ω V1, ω V2) - V3] Δt)

Super mirroring for an agent B generates a state indicating on which agent (self-other distinction) the focus is, and whether or not to act; this is modelled in LP5.

LP5 Super mirroring for another agent or self

If the sensory representation of agent B (another agent or self) has level V1,

and the sensory representation of s has level V2,

and the super mirroring state for B, s and b has level V3

then after duration Δt the super mirroring for B, s and b will

have level V3 + γ [f(ω7BV1, ω8BV2) - V3] Δt.

SR(B,V1) & SR(s,V2) & PS(B, s, b, V3)

Super mirroring for sensory processing sensitivity, modelled in LP6, generates a state indicating in how far the stimulus induces an inadequately high sensory body representation level. This state is the basis for two possible regulations (modelled in LP8 and LP10 below): of the expressed body state, and of the gaze.

LP6 Super mirroring for enhanced sensitivity

If the sensory representation of s has level V1,

and the sensory representation of b has level V2

and the sensitivity super mirroring state for s and b has level V3 then after duration Δt sensitivity super mirroring for s and b will

have level V3 + γ [f(ω9V1, ω10V2) - V3] Δt.

SR(s,V1) & SR(b,V2) & PS(sens, s, b, V3)

→→ PS(sens, s, b, V3 + γ [f(ω V1, ω V2) - V3] Δt))

The preparation of a verbal empathic reaction to another agent depends on feeling a similar emotion, and on adequate self-other distinction, as modelled in LP7.

LP7 Preparing for communication

If the sensory representation of body state b has level V1,

and the super mirroring for agent B self, s and b has level V2,

and the preparation of communication of b to B has level V3

then after duration Δt the preparation of communication of b to B will have level V3 + γ [f(ω11BV1, ω12BV2) - V3] Δt.

SR(b, V1) & PS(B, s, b, V2) & B self & PC(B, b, V3)

Expressing a (prepared) body state depends on whether a super mirroring state for self is available. However, to cover regulative behaviour to compensate for enhanced sensory processing sensitivity, also the sensitivity super mirroring state is involved, with an inhibiting effect on expressing the prepared body state (ω14 is taken to be negative). Such an effect can achieve that although the agent feels the same as the other agent, the face remains expressionless. In this way LP8 models a mechanism for response-focused regulation (suppression of the own response) to compensate for an undesired level of arousal; cf. [Gross, 1998; Goldin et al., 2008].

LP8 Expressing a body state

If

the super mirroring state for self, s and b has level V1, and the super mirroring state for sensitivity, s and b has level V2, and the preparation for body state b has level V3 and expressing body state b has level V4

then after duration Δt body state b will be expressed with

PS(self, s, b, V1) & PS(sens, s, b, V2) & PB(b, V3) & EB(b, V4)

Note that expression states ES are the agent's effector states (e.g., the muscle states); body and gaze states result from these expression states (via LP11 and LP12 below). A preparation for a verbal empathic reaction leads to expressing this communication in a straightforward manner.

LP9 Expressing communication

If the preparation of communication of b to B has level V1, and the expressed communication for b to B has level V2 then after Δt the agent will express communication of b to B with

PC(B, b, V1) & EC(B, b, V2) →→ EC(B, b, V2 + γ [f(ω V1) - V2] Δt)

Dynamic property LP10 models antecedent-focused regulation (attentional deployment) as described in [Gross, 1998; Goldin et al., 2008]: directing the own gaze away from the stimulus that feels too overwhelming. Note that the gaze direction eg(s) for s is taken to be 1 for total avoidance of stimulus s, and 0 for no avoidance (it indicates the extent of avoidance).

LP10 Expressing gaze for avoidance of s

super mirroring for sensitivity, s and b has level V1, and the expressed gaze for avoidance of s has level V2 then after Δt the expressed gaze avoidance for s will have level V2 + γ [f(ω17V1) - V2] Δt. PS(sens, s, b, V) & ES(g(s), V2) →→ ES(g(s), V2 + γ [f(ω V1) - V2] Δt)

Properties LP11 and LP12 describe how the expression states affect the body and gaze in a straightforward manner.

LP11 From body expression to body state

If the expression state for body state b has level V1, and the body state b has level V2 then after Δt body state b will have level V2 + γ [f(ω18V1) - V2] Δt. ES(b, V1) & WS(b, V2) →→ WS(b, V2 + γ [f(ω18V1) - V2] Δt)

LP 12 is similar to LP11 with gaze instead of body. Sensing a body state and agent B also happen in a straightforward manner, as described by LP13 and LP14.

LP13 Generating a sensor state for a body state

If the body state b has level V1, and the sensor state for body state b has level V2 then after Δt the sensor state for body state b will have level V2 + γ [ f(ω20V1) - V2] Δt WS(b, V1) & SS(b, V ) →→ SS(b, V + γ [ f(ω V1) - V2] Δt)

LP14 is similar to LP13 with agent B instead of body. Within the external world, to generate a sensor state for a stimulus s, the gaze state with respect to s is taken into account. As the gaze state indicates the extent of avoidance of s, it has an inhibiting effect on sensing s (ω23 is taken to be negative); here f has been modelled by f(W1, W2) = W1(1+W2) with -1 ≤ W2 ≤ 0.

LP15 Generating a sensor state for a stimulus

If stimulus s is present with level V1, and gaze state for avoidance of s has V2, and the sensor state for s has V3, then after Δt the sensor state for s will have level V3 + γ [ f(ω22V1, ω23V2) - V3 ] Δt WS(s, V1) & WS(g(s), V2) & SS(s, V3) →→ SS(s, V3 + γ [ f(ω V1, ω V2) - V3 ] Δt

3 Types of Social Response Patterns Shown

To analyse the different types of response patterns shown by the cognitive agent model, some properties were identified and formally specified in a hybrid reified temporal predicate logic (e.g., [Galton, 2006]). Here at(a, T) means that state property a holds at time T, and s(B, b) denotes the stimulus for self consisting of the expression of body state b by agent B. By automated verification they have been checked for generated simulation traces, allowing to evaluate easily the patterns for a variety of parameter values. The simulations discussed first, have been performed with γ = 1, Δt = 0.5, and settings for threshold and steepness values as shown in Table 2. In the graphs in Figures 2 and 3, time is at the horizontal axis and activation levels are at the vertical axis.

The first property expresses that when an agent B is met, showing a certain emotion, within a certain time a response occurs, which can consist of: (1) self feels the same as B, (2) this feeling is bodily expressed by self, and (3) it is communicated by self to B that B feels this.

SBP1(M1, M2, R(b, V)) Response occurrence

When agent B self is present expressing a certain feeling b from some point in time on, then after some time agent self will have a response R (generating the feeling of b, resp. bodily expression, resp. communication).

∀T1 [∀V1,V2,T2≥T1 [ at(WS(B, V1), T2) & at(WS(s(B, b), V2), T2) V1≥M1 & V2≥M1 ] ∃V,T3≥T1 at(R(b, V), T3) & V≥M2 ] with R(b, V) one of SR(b, V), ES(b, V), EC(B, b, V).

By combination 8 different types of response are possible; see Table 3. Some of them are not likely to occur (types 5, 6, and 7): when the agent self does not feel the emotion, it is probably hard to communicate or show it.

The way in which different connections relate to different types of processes, as depicted in Table 1, provides an indication of which deviant connection strengths may lead to which phenomena. For example, when ω5 (connecting SR(s) to PB(b); see Figure 1 and Table 1) is low, mirroring is reduced, and as a consequence low social response (type 8) occurs; cf. [Dapretto et al., 2006]. An example of type 1 is the upper graph (a) shown in Figure 2 displaying the feeling (rep body), mirroring (prep body), expression of body (expr body), and communication (expr comm). Here, ωk = 1 for all k, except for the suppressing connections (from PS(sens, s, b) to ES(b), and from WS(g(s)) to SS(s), respectively): ω14 = ω23 = - 1. The pattern shows an increase of mirroring, followed by bodily expression and feeling, and communication.

Response type 4 only concerns the feeling (not externally observable). For response type 2, the feeling is expressed: it is externally observable, but no verbal communication takes place. Response type 2 with low ω7B

or ω8B (from SR(B), resp. SR(s) to PS(B, s, b)) displays that no adequate self-other distinction is made due to reduced super mirroring (e.g., [Iacoboni, 2008; Brass and Spengler, 2009]). Response type 4 with low ω6 (from SR(b) to PB(b)) can be viewed as a form of emotion contagion without integrating the emotion in responses; cf [Grèzes and de Gelder, 2009; Grèzes et al., 2009]. In contrast, in response type 3 the emotion felt is attributed to the other agent, but no bodily expression is shown.

The middle graph (b) in Figure 2 shows an example of response type 4. The level of emotion felt is becoming high, but due to lack of emotion integration (ω6 = ω11B = 0 and the other ωk the same as for the upper graph), the bodily and verbal expression are reduced. In case of regulation due to enhanced sensory sensitivity (e.g., [Baker et al., 2008; Corden et al., 2008]), patterns occur when a response only lasts for a short time, expressed as:

SBP2(M1, M2, D, R(b, V)) Response withdrawal

When agent B self is present expressing a certain feeling b from some point in time on, and the agent self has response R, then within time duration D this response will disappear.

∀V,T1,T3≥ T1 [

The combination SBP1 & ¬ SBP2 expresses a persistent response, whereas SBP1 & SBP2 specifies only a short occurrence of a response. However, after withdrawal of the response due to regulation, also the arousal level for b will become low, which brings the agent in practically the same state as initially. An oscillatory pattern results, while the environment is fully static. Such oscillatory social response patterns indeed can be observed in persons with some forms of ASD, who let their gaze go back and forth to another person's eyes during a contact, as a way of regulation of enhanced sensitivity. The lower graph (c) in Figure 2 shows an example of such a response pattern, specified as follows.

SBP3(M1, M2, M3, R(b, V)) Response oscillation

When an agent B bodily expressing a certain feeling is present from some point in time on, then:

1. for every time point there is a later time point for which response R occurs
2. for every time point there is a later time point for which response R does not occur

The agent model shows this type of social response when the threshold for sensory sensitivity is set between 1 and 2; for example, for the lower graph (c) Figure 2, it was set to 1.2. Moreover, as for the upper graph ωk = 1 for all k, except for the suppressing connections: ω14 = ω23 = -1. It is shown that body expression and communication last only for short time periods, but recur. If the threshold value is set 1 or lower, no response occurs (type 8); if it is 2 or higher a persistent response is shown (type 1). Note that instead of varying the threshold for sensory sensitivity, similar patterns are generated when the connection strength ω17 (from PS(sens, s, b) to ES(g(s))) is varied. The oscillatory patterns due to regulation for enhanced sensitivity occur for all response types in Table 3.

In the scenarios discussed above and shown in Figure 2 the other agent B and the stimulus were assumed static. However, the cognitive agent model can be applied to agent B as well. In this case it is assumed that the eyes of one agent are the stimulus for the other agent, so that in a mutual manner an avoiding gaze regulation of one agent affects the stimulus for the other agent as well. In this case the interaction starts in an asynchronous and irregular way, as shown in Figure 3. Here the values for γ were taken different expressing individual differences: for agent A it is 1 as for self before, and for B it is (slightly slower) 0.7.

4 Discussion

The presented cognitive agent model for regulated social response patterns uses theories from Social Neuroscience and autism spectrum disorders. The full response modelled can be considered an empathic response, based on the criteria of empathy formulated in [De Vignemont and Singer, 2006], p. 435. Moreover, it was shown how a wider variety of realistic social response patterns can be obtained by varying the agent's makeup of mental structures, inspired by relevant literature on autism spectrum disorders. In contrast to work discussed in [Hendriks and Treur, 2010; Laan and Treur, 2011; Bosse et al., 2011], the presented agent model addresses regulation of enhanced sensory processing sensitivity by super mirroring to control body, face expression and gaze, based on the emotion regulation theory presented in [Gross, 1998; Goldin et al., 2008]. The model provides a basis for human-like behaviour of virtual agents in the context of serious or less serious gaming. For example, it may provide a basis for the implementation of virtual agents for training of psychotherapists, or in applications of human-like virtual characters with realistic body and face expression and gaze.

In a wide literature, the role of emotions in virtual agents in general is addressed; e.g., [Bates et al., 1994; Yang et al., 2008; Gratch et al., 2009]. Usually these approaches are not specifically related to empathic responses, and often use body or face expressions as a way of presentation, and not as a more biologically grounded basis for the emotion as in the neurological perspective of [Damasio, 1999], which was adopted in the current paper. The importance of computational models for 'caring' agents in a virtual context showing empathy has also been recognized in the literature; see, for example [Klein et al., 2002; Bickmore and Picard, 2004; McQuiggan et al., 2008; Bickmore et al., 2010]. The presented cognitive agent model differs from such existing models in that it is grounded in recent insights from neuroscience, emotion regulation and Autism Spectrum Disorders, and reflects the current theories. Moreover, the presented model is able to display social responses in a realistic human-like manner, not only of ideal empathic humans, but also of less perfect humans with some ASD-related characteristics. Whether or not users prefer virtual agents that systematically display ideal empathic social interaction, or also realistic non-ideal social interaction is an open question.

Further work in which the computational model is compared to empirical gaze data obtained using an eyetracker is a next step planned, in line with [Spezio et al., 2007], but then with gaze data from a real conversation. Moreover, in [McQuiggan et al., 2008] the CARE framework for experiments with humans and empathic virtual agents is described. A possibility for future research is to integrate the presented agent model in the CARE environment and conduct experiments with different types of empathic agents. As another example, based on the presented model a social interaction pattern between two agents as shown in Figure 3 can be easily implemented within a displayed agent-based virtual story context. The expressed emotions can be displayed on the faces of the two agents, and gaze regulation can be displayed as eyes or faces turning away from each other. When the agent model described is used as an engine to generate the states and behaviour for each of the two virtual agents, the interactive pattern will automatically be generated.