Modeling Situation Awareness in Human-Like Agents Using Mental Models

1 Introduction

An important aspect of cognition that allows for effective decision making is the assessment of a situation [Wickens and Hollands, 2000]. Endsley [1995] defines it as Situation Awareness (SA) and distinguishes three levels that describe a person's SA: the perception of cues, the comprehension and integration of information and the projection of information into future events. Especially in demanding circumstances (e.g., air traffic control) a reduction in a person's SA can seriously degrade performance. Considering that SA is crucial for human decision making, this process should be taken into account in the development of agents designed to display humanlike behavior. Within the field of serious gaming, such agents are used to create realistic circumstances that allow for training of real-life situations [Silverman et.al., 2006]. An example application is combat flight simulation, a common method used to train fighter pilots, to learn the skills necessary for optimal flight behavior [Jacobs and Dempsey, 1993; Salas and Cannon-Bowers, 2001].

Earlier models have been proposed for the design of such intelligent agents with SA, see e.g. [Jones et.al.,

1999; Wickens et.al., 2008]. However, these models are limited as they do not represent all necessary aspects and stages of SA as have been distinguished above. Also, research on belief updating can be seen as an approach towards modeling situation awareness (see e.g. [Friedman and Halpern, 1997] and [van Benthem, 2007]), however, such belief update approaches again do not include the actual perception of cues, nor the projection of information into future events as distinguished by Endsley.

The purpose of this paper is to create a model that covers the entire cycle of Situation Awareness as identified by Endsley. In this model, the performance of observations is considered, these observations are translated into beliefs, beliefs are then updated and used to form future beliefs, and also to direct new observations. The model allows the distinction between experts and novices as both groups are different in how they obtain awareness [Maes, 1990; Shanteau, 1987; Schriver et al., 2008]. In the proposed model, the degree of awareness within the agent's working memory is represented by an activation value of beliefs on the situation. The model is general as the agent makes use of an available mental model (domain information on concepts and their relations stored in long term memory) to perform mental processing to generate integrated complex beliefs from observations and beliefs on the future from these complex beliefs. These processes cover the process described as cue integration in [Wickens and Hollands, 2000]. The developed model has been applied within the domain of F-16 fighter pilot training.

In this paper, first a theoretical background on modeling situation awareness is given in Section 2, Section 3 explains the cognitive agent model of SA. In Section 4 a case study is presented and the SA of a fighter pilot is simulated. Finally, a discussion of the work is given in Section 5.

2 Theoretical Background

From [e.g. Endsley, 1995; Wickens and Hollands, 2000] it is known that an important role is assigned to memory in achieving the three levels of Situation Awareness (i.e. perception, comprehension, projection). Firstly, representations of domain knowledge are stored in long term memory (often referred to as a mental model or schema of the environment [Endsley, 1995]). The level of SA that is obtained depends on the complexity of the available mental model. As a person becomes more experienced with the environment, the mental model becomes more developed, which also explains why experts are better at integrating multiple cues as opposed to novices [Maes, 1990]. Shanteau [1987] confirms this point by showing that the difference between experts and novices in decision making is also due to a difference in the ability to perceive meaningful patterns. Moreover, acquisition of observations is not static, but concerns a dynamic iterative process taking into account the mental model. Based on the mental model, and the goals aimed for, at each point in time control can take place in order to perform selected additional observations on important but yet insufficiently known aspects of the situation.

Secondly, working memory is important in maintaining awareness of the situation [Kane and Engle, 2002]. More specifically, Kane and Engle address the executive attention component of working memory as responsible for the activeness of memory representations (i.e., stimuli, beliefs, goals); see also, [Barrett et.al., 2004; Kane et.al., 2006]. The higher the attentional focus is on a specific task, the higher the activity value is of beliefs that are important for that task. In contrast, if no attention is contributed to a specific belief, that belief gradually becomes less active. In order to create an agent with a realistic picture of the environment, a model should contain both the characteristics of executive attention as well as the ability to perform further mental processing using domain information stored in long term memory.

Finally, when designing a model of SA an important aspect is the degradation of SA that can arise in demanding circumstances. In [Wickens and Hollands, 2000] it can be found that when time is limited, the integration of cues is impaired and as a result the picture of the environment will be incomplete. In addition, perception will be imperfect when people cannot perform all available observations due to the finite character of working memory capacity [Wickens and Hollands, 2000].

The representation of knowledge in long-term memory (cf. [Endsley, 1995; Wickens and Hollands, 2000] is established by means of the separate mental models. Furthermore, the difference between experts and novices (cf. [Kane and Engle, 2002; Shanteau, 1987] with respect to their knowledge is handled by the level of detail in the mental model as well as their working memory capacity and the time available for reasoning. The direction of observations is performed by considering the goals, future beliefs, and the overall capacity of the working memory. The activeness of memory components [cf. Barrett et.al., 2004; Kane et.al., 2006] is taken into account via the expression of activation levels of beliefs, and the influence of these activation level upon the use of the beliefs in the reasoning. Finally, the degradation of performance under demanding circumstances is established by limiting the available time for the methods, and hence, having a less refined picture of the situation.

3 The Cognitive Model

The general structure of the cognitive model for situation awareness is shown in Figure 1. In this figure, it can be seen that the model consists of four main components. Three components are in line with the model of Endsley (1995) which includes the perception of cues (i.e. component 1), the comprehension and integration of information (the combination of 2 and 3), and the projection of information for future events (component 4). In addition, component 5 represents the mental model.

The model functions as follows. Initially, the agent starts to observe within the world, and obtains the results of these observations. These results are forwarded to the component responsible for the formation of beliefs about the current situation. In this component, two types of beliefs are distinguished, namely simple beliefs, and complex beliefs. The simple beliefs concern simple statements about the current situation that have a one-toone mapping to observations, or have a one-to-one mapping to another simple belief (e.g., I believe that an hostile aircraft has detected me based upon my observation of his radar signal). The complex beliefs are aggregations of multiple beliefs and describe the situation in a composed manner. Using the knowledge stored in the mental model (of which an example is shown in Figure 2 which will be used as a running example throughout the explanation of the components), the component first of all derives simple beliefs about the situation. Thereafter, the complex beliefs are derived from the simple beliefs, again using the knowledge stored in the mental models. In order to project the complex beliefs to the future situation, they are forwarded to the component belief formation on future situation. Herein, again a mental model is used to make the predictions. The judgment of the future situation that then follows is used to direct the observations of the agent. Below, the details of each of the components are described.

3.1 Observations

The observation component receives information from the external world about the observations the agent has decided to perform. Observations are represented by means of degrees of certainty. For example, an agent might observe a familiar person standing nearby, with a very high certainty, whereas observing someone from a distance that looks like a familiar person has a lower degree of certainty. The following predicate is used to represent observations:

observation_result: INFO_ELEMENT x TIME x VALUE

The first argument hereby expresses the element that is observed. Note that also time is included explicitly here. These observations are forwarded to the component belief formation current situation. The decisions to perform specific (additional) observations are discussed in Section 3.4 (observation selection). When looking at the example mental model presented in Figure 2, the observations included involve the detection of another plane in the vicinity. Two observations are included, namely whether a plane is closing in, and whether it comes from a hostile direction.

3.2 Belief Formation for Current Situation

After the observations have been performed, they need to be interpreted by the agent. In this interpretation process, the first phase is to form simple beliefs that are directly related to these observations. Simple beliefs are in this case represented by the following predicate:

simple_belief: INFO_ELEMENT x TIME x VALUE

In the predicate, the value presents the activity of the belief in the mind of the agent, which depends on a number of aspects, among which the certainty of the observation. In order to translate a certainty of an observation into an activation of a belief, the following rule is used:

LP1: Observations to simple beliefs

current_time(t) ∧ observation_result(I, t, V1) ∧ simple_belief(I, t-1, V2) ∧ simple_belief_decay(I, ) ∧ steepness(I, ) ∧ threshold_value(I, ) ∧ recency_influence(I, ) →→ simple_belief(I, t, (1- )⋅ ⋅V2 + ⋅th( , , V1))

This expresses that in the formation process of a simple belief, both the certainty of the observation and the old value of the belief are considered (thereby assuming that at least an initial value is always present). Within the mental model, the parameters of the translation process are expressed (the elements shown in grey bold represent information from the mental model). The parameters influence how much value a new observation has compared to a previous belief value as well as how fast the belief decays (i.e., how fast the belief looses activation). Both heavily depend on the domain and the element it concerns. Furthermore, the mental model expresses the parameters of a so-called threshold function which is used to translate the certainty factor of the observation to a value contributing to the activation value of the simple belief. The parameters of the threshold function include the threshold itself (i.e., the value at which a contribution of 0.5 is established) as well as the steepness of the curve at that point. The idea of the use of such a function is adopted from the neurological domain. Using a threshold function, it can be indicated how important the observation is (i.e., some observations might already become very active in case there is merely the slightest certainty that it is the case). In Figure 2, all these details are not explicitly shown, however the relations between the observations and simple beliefs are. In this case, the two observations (closing_in and from_hostile_direction) are both directly coupled to a similar simple belief.

During times that no observations are done concerning a specific belief, the belief just decays.

LP2: Simple belief decay

current_time(t) ∧ simple_belief(I, t-1, V2) ∧ ¬∃V1:REAL [ observation_result(I, t, V1) ] ∧ simple_belief_decay(I, )

• →→ simple_belief(I, t, ⋅V2)

After the new activation value of simple beliefs has been calculated, the influence of the simple beliefs among each other is determined. Hereby, influence weights reside in the interval [-1,1] (again represented as part of the mental model). Experts will typically have stronger connections, whereas novices have lower connection strengths. Weights can be expressed in two ways: (1) by means of fixed connections (i.e., just predefining the value), or (2) by means of specific rules to derive weights whenever appropriate. In the example mental model, next to the simple beliefs directly coupled to observations, one additional simple belief is present which expressed whether the plane is hostile. Connections in the mental model express that the belief that a plane originates from a hostile direction has a strong influence on the belief whether it is hostile (an influence weight of 0.9) whereas the same simple belief also has a significant influence upon the belief whether the plane is closing in (0.5) due to the fact that planes from hostile directions are likely to close in.

Not all the rules described above are continuously used to calculate the weights as this would not be efficient, nor human-like. Therefore, the assumption is made that only the connections that originate from a simple belief with an activation value above a certain threshold are calculated. In case the values are not above the connection threshold the strengths are assumed to be 0. Given the values of the weights that have been determined, the influence of the simple beliefs among each other can be calculated. For this, an iterative form of updating is used which is based upon calculating all the influences that originate from the simple belief with the highest activation value. The method is expressed in detail below.

Method 1: Updating simple beliefs

1. Search for the simple belief with the highest value that has not been considered yet and is above the threshold.
2. For all connections originating from the selected belief:
3. Select the connection with the highest strength originating from the selected belief that has not been considered yet of which the absolute value is above the minimal connection threshold. In case none are left, go to (d). If none were present in the beginning, go to (e).
4. Perform calculations (LP3)
5. Mark the connection as considered and go to (a).
6. Add 1 to the time used.
7. Mark the selected belief as considered. In case the time has reached the maximum time the algorithm terminates, otherwise go to 1.

As this method has anytime behavior it also enables the mental processing within a specific deadline. The updating of the belief is as follows:

LP3: From simple beliefs to simple beliefs

current_time(t) simple_belief(I1, t, V1) ∧ simple_belief(I2, t, V2) ∧ connection_strength(I1, I2, w1) →→

simple_belief(I2, t, V2 + γ⋅(Neg(V1⋅w1⋅V2) + Pos((1-V2)⋅(V1⋅w1)))

Here, γ is a predefined constant that determines the speed of updating. In this formula Neg(X) evaluates to 0 in case X is positive and X in case X is negative. Pos(X) evaluates to X if X is positive and 0 otherwise. The rule expresses that once the contribution to the current value of a simple belief is positive, then the simple belief's activation value goes up, and otherwise it goes down. The above construct establishes that the activation values remain between 0 and 1.

When all activation values for simple beliefs have been calculated, the complex beliefs can be determined. Note that the complex beliefs are always a combination of multiple simple and/or other complex beliefs. In this case, it is assumed that the complex beliefs are calculated by taking a weighed sum of relevant simple beliefs. Here multiple paths can be present, e.g. a combination of simple belief b1 and b2 that allow the derivation of c1, but also a combination of b3 and b4 might result in the derivation of c1. Note that in this case the weights are assumed to be expressed on the domain [0,1]. The determination of the weights themselves is done in an identical fashion as the calculation of the weights for simple beliefs. In Figure 2, an example of a complex belief is also shown, which expresses the belief that the plane is under attack (by the detected hostile plane). This complex belief is formed by a combination of the simple belief that it concerns a hostile plane with the belief that the plane is closing in. The updating of the complex beliefs once the weights are known is done with a similar method as method 1 (again with anytime behavior), except that all connections pointing to a complex belief are taken into account. The updating of the value itself is expressed as follows (γc is a predefined constant):

LP4: From simple to complex beliefs

complex_belief(CI1, t, VI1) ∧ belief(I1, t, V1) ∧ …. belief(In, t, Vn) ∧ in_same_group(I1, .... In, CI1) ∧ connection_strength(I1, CI1, w1) ∧ ….. connection_strength(In, CI1, wn) steepness(CI1, ) ∧ threshold_value(CI1, ) →→ complex_belief(CI1, t, VI1 + γc⋅(f( w1V1, .... , wnVn) - VI1))

Here, the beliefs that together form a connection to the complex belief (e.g., the example of b1 and b2 before) are taken, and the contributions are calculated using a combination function f. For example, such a combination function can be based on a logistic threshold function or a weighted sum (the latter has been chosen in the example simulations shown). The model described so far results in an overall belief about the current situation. In case no new information is present with respect to a complex belief, a simple decay of the activation value is assumed.

LP5: Complex belief decay

current_time(t) ∧ complex_belief(I, t-1, V2) ∧ complex_belief_decay(I, ) →→ complex_belief(I, t, ⋅V2)

3.3 Formation of Beliefs for the Future

For the formation of beliefs on the future, an identical approach is followed compared to the formation of complex beliefs. The only difference is that and the time delay is also an aspect of the connection strengths and of the model to derive a belief on the future. An agent might for instance know that the belief refers to a state that will happen in 5 time points. This is made explicit as follows:

LP6: From complex to future beliefs

complex_belief(I1, t, V1) ∧ ..... complex_belief(In, t, Vn) ∧ future_belief(FI1, t+D, VI) ∧ in_same_group(I1, .... In, FI1) ∧ delay_parameter(I1, .... In, FI1, D) ∧ connection_strength(I1, FI1, D, w1) ∧ ..... connection_strength(In, FI1, D, wn) ∧ steepness(FI1, ) ∧ threshold_value(FI1, )

• →→ future_belief (FI1, t+D, VI + γf⋅(f(w1V1, .. , wnVn) - VI) )

Note that the future beliefs themselves can be the same as the complex beliefs (except for the explicit aforementioned time parameter). Furthermore (similar to LP3 and LP4), a predefined constant γf (value 0.8) is used to determine the updating speed. In the example mental model one future belief is present (based upon the complex belief that the plane is under attack), expressing that the plane is shot down, which has a delay of 5, and a weight of 0.4.

3.4 Observation Selection

The final step is that the beliefs on the future direct the selection of the observations. In order to select observations first they are rated by relevance values, which depend on various elements: (1) the current goals of the agent (including a particular activity level), (2) the beliefs of the agent on the future, and (3) the maximum amount of observation cost the human can handle (e.g., due to working memory limitations). Initially, all observation relevance values are set to 0:

observation_relevance(O1, t, 0)

Thereafter for each of the observations the relevance value is derived using the aforementioned factors as follows (thereby considering the different goals and future beliefs in a sequential manner one by one):

LP7: Determining observation relevance

observation_relevance(O1, t, V1) ∧ future_belief(FI1, t+D, V2) ∧ goal(G1, t, V3) ∧ relevance_for_belief(O1, FI1, w1) ∧ relevance_for_goal(O1, G1, w2) →→ observation_relevance(O1, t, V1 + γo⋅(w1⋅w2⋅(1-V2)⋅V3 - V1))

This expresses that the current value of the goals and beliefs on the future influence the relevance of observations. To calculate the observation relevance, the importance of an observation for the active goals is taken into account. Also the future beliefs are taken into account; once a future belief is more certain, the need for observations decreases (if it is known that something will certainly happen in the future, there is less need for observing it).

After the relevance for each observation is calculated, the step to come to actual selection of observations is done via observation cost. It is assumed that each observation has a certain cost (e.g., determined by the effort needed to observe) and each human has a maximum amount of observation cost that can be spent per time unit (dependent on the human's working memory capacity). The observations are sorted by relevance and thereafter selected as long as the total of the cost of the selection of the observations does not exceed the overall maximum.

4. Case Study

In this section, an extensive case study is described to show the overall behavior of the presented cognitive model for Situation Awareness. The idea of applying the model in this context is to develop human-like opponents against which human fighter pilots can practice in a simulator. First, the scenario is explained in more detail, followed by a number of simulation results.

4.1 Scenario Description

In this case, the case study concerns a military scenario in which a pilot has to detect whether (enemy) contacts are near and if so, determine what kind of threat these contacts pose. This detection is performed by means of a radar warning receiver, which can provide a number of observations. Example observations that can be provided are the direction of the contact, the direction of the Front Line of Own Troops (FLOT) and a beeping noise with various frequencies, indicating the threat of a contact.

First of all, simple beliefs are present that represent the observations (including their negations). Also, relations are expressed between these beliefs, for example, when a belief is present that no beep is heard, this has a negative impact on the simple belief concerning a continuous beep. Next to this, several other simple beliefs are present that are related via weighed connections with the other simple beliefs. This includes beliefs on whether the ownship is searched (i.e. the pilot's plane has been observed by another plane), tracked (the heading of the pilot's plane is observed by another plane) or locked (a radar lock is placed on the pilot's plane). Based upon these simple beliefs, complex beliefs can be derived. The overall network of observations, simple beliefs, complex beliefs, and future beliefs that results (including a subset of the connections and the associated weights) is shown in appendix A 1 . The weights of this case study have been defined by domain experts, in the future, it is envisioned to learn the weights based upon pilot behavior observed. The appendix also shows knowledge on the relevance of observations for different goals and for different beliefs. In addition, different relevancy values of the observations for each future belief are available (e.g. the observation of a continuous beep is relevant for the future belief probable engaged).

4.2 Results

A number of simulations have been conducted in order to show the influence of various parameter settings upon the overall behavior of the system (the initial parameter values can be found in appendix B 2 ). In Figure 4 an overview is given of the observations that have a value of 1 in the world for all these scenarios. The observations that are not listed have a certainty of 0. All negations of the observations have an opposite certainty value. This means that for example, 'no_beep' is observed with a certainty of 1 from time point 1 to time point 8 and from time point 9 to 26, 'occasional_beep' is observed. However, in this case, 'not_occasional_beep' has a certainty of 0 and 'not no beep' will have a certainty of 1.

1 http://www.cs.vu.nl/~mhoogen/SA/appendix_A.pdf

2 http://www.cs.vu.nl/~mhoogen/SA/appendix_B.pdf

Scenario 1: Enough time available

In the first scenario, the amount of time available to create the awareness of the situation is set to a high value (50 for each phase). This allows for multiple updating steps. Figure 5 shows the results of the activation levels of the resulting simple beliefs on the y-axis and time on the xaxis.

It can be seen that the pilot initially judges that no contact is present; thereafter a contact is present which is searching the plane of the pilot, followed by a track and lock. Of course, more complex beliefs are formed about the situation (figure 6). It can be seen that the activations quickly rise for the fact that the contact is not coming from the front line of own troops (i.e. it is likely that the plane is hostile). Also, the fact that the contact is unknown becomes active, but after a while the pilot notices that the contact is hostile, that he is probably targeted, and that the plane now comes from the front line of own troops (i.e. the contact has changed direction). Eventually, the pilot concludes that he is probably engaged (i.e. a very dangerous situation).

Figure 7 presents the activation of beliefs on the future situation. Note that this figure does not show when these beliefs are derived, but when they are assumed to be the case. The pilot will use this information to select the observations at the next point in time.

According to domain experts, these results are similar to human behavior.

Scenario 2: Limited time available

The second scenario addresses a case whereby there is limited time available for the reasoning (for reasoning to simple and complex beliefs respectively 20 and 5 steps available). However, there is sufficient working memory for performing the observations (and hence, all observations are performed).

Figure 8 shows the simple belief activation for this scenario. As there is less time available for reasoning, the simple beliefs become less active as compared to Scenario 1 with sufficient reasoning time (in this case the maximum activation value is 0.7 as opposed to 0.95 in Scenario 1).

The results of the complex beliefs (figure 9) shows that the belief activation patterns are different to those for Scenario 1 (Figure 4). For example, the probable targeted situation only becomes active for a very short period, and the same holds for probably engaged, whereas these are crucial for the pilot. This is caused by the fact that again, less time is available for belief updating and not all complex beliefs will be updated. In addition, simple beliefs are less active as explained before.

The low activation value of some complex beliefs also has an effect on the activation value of future beliefs (of which a graph is not shown for the sake of brevity). In this case, not all complex beliefs are high enough to activate the future beliefs; only 'from flot' and 'probable targeted' have an activity value higher than zero.

Scenario 3: Enough time but limited working memory

The final scenario involves sufficient time but with a decrease in working memory capacity. This means that particular observations cannot be performed. In Figure 10 the timeline of observations is shown (no box represents the fact the observation has not been selected.)

Figure 11 shows that the strength of the complex beliefs becomes lower due to the inability of the pilot to observe all necessary elements. Whether the plane comes from the front line of own troops is for example never derived since the direction is never observed. The two most important complex beliefs (probably engaged and probably targeted) are however derived since the accompanying observations are performed.

5. Discussion

In this paper, a model has been presented for Situation Awareness. This model has been based upon the three key stages within Situation Awareness as defined by Endsley [1995]: the perception of cues, the comprehension and integration of information, and the projection of information into future events. When comparing the model with the criteria for models for Situation Awareness as expressed in Section 2, it can be seen that all of the prominent concepts as distinguished in the literature have been incorporated in the model.

Other computational models have been proposed for Situation Awareness, also in dynamic environments. For instance, So and Sonenberg [2004] create a computational model for situation awareness for defining pro-activeness of behavior. In their model, they also use the model of Endsley as a basis and they incorporate beliefs with certainty factors. The differences between novices and experts in this reasoning process are however not explicitly taken into account, nor the activations of beliefs taken as seen in human reasoning. A more detailed model of SA can be found in [Juarez-Espinoza and Gonzalez, 2004], but it does not make use of a general method to integrate observations into higher level beliefs and is therefore difficult to apply in new situations. As mentioned in the introduction, BDI models [Rao and Georgeff, 1995] in general can also be seen as models for Situation Awareness, as most of these models incorporate the formation of beliefs based upon observations, and can create a projection for the future to decide which intention to pursue. These models do however not explicitly incorporate all the criteria shown to be important in Situation Awareness (as expressed in Section 2). One particularly interesting field is the domain of situated agents which are "artificial systems capable of effective, rational behavior in dynamic and unpredictable environments" [cf. Kinny and Georgeff, 1991], for which the crucial problem faced is "to ensure that the agent's responses to important changes in its environment are both appropriate and timely" [Kinny and Georgeff, 1991]. Within these approaches however, the emphasis is mainly on selection of appropriate actions, given that beliefs have been formed, and not so much on creating a complete model of the world such that a more accurate description of the situation can be made. When looking at the literature in Psychology, modeling the part concerning the perception and judgment of the situation is crucial to enable good responses [cf. Randel and Pugh, 1996].

For future work, it is planned to combine the Situation Awareness model with a naturalistic decision making approach. Furthermore, it is envisioned to run experiments with this combined model in a fighter pilot training environment to see whether it can indeed result in humanlike behavior.

Acknowledgments

We would like to thank the National Aerospace Laboratory (The Netherlands) for sharing their expert knowledge on the domain of air combat.