On the Logic of Normative Systems

1 Introduction

Normative systems, or social laws, have been widely promoted as an approach to coordinating multi-agent systems [Shoham and Tennenholtz, 1996]. Crudely, a normative system defines a set of constraints on the behaviour of agents, corresponding to obligations, which may or may not be observed by agents. A number of formalisms have been proposed for reasoning about normative behaviour in multiagent systems, typically based on deontic logic [Meyer and Wieringa, 1993]. However the computational properties of such formalisms - in particular, their use in the practical design and synthesis of normative systems and the complexity of reasoning with them - has received little attention. In this paper, we rectify this omission. We present Normative Temporal Logic (NTL), a logic for reasoning about normative systems, which is closely related to the well-known and widelyused branching time logic CTL [Emerson, 1990]. In NTL, the universal and existential path quantifiers of CTL are replaced by indexed deontic operators Oη and Pη, where Oηϕ means that "ϕ is obligatory in the context of the normative system η", and Pηϕ means "ϕ is permissible in the context of the normative system η". Here, ϕ is a temporal logic expression over the usual CTL temporal operators ❣,♦, , and U (every temporal operator must be preceded by a deontic operator, cf. CTL syntax), and η denotes a normative system. In NTL, obligations and permissions are thus, first, contextualised to a normative system η and, second, have a temporal dimension. It has been argued that the latter can help avoid some of the paradoxes of classical deontic logic. NTL generalises CTL because by letting η∅ denote the empty normative system, the universal path quantifier A can be interpreted as Oη∅ ; much of the technical machinery developed for reasoning with CTL can thus be adapted for NTL [Emerson, 1990; Clarke et al., 2000]. NTL is in fact a descendent of the Normative ATL (NATL) logic introduced in [Wooldridge and van der Hoek, 2005]: however, NTL is much simpler (and we believe more intuitive) than NATL, and we are able to present many more technical results for the logic: we first give a sound and complete axiomatisation, and then discuss the logic's relationship to standard deontic logics. We introduce a symbolic representation language for normative systems, and investigate the complexity of model checking for NTL, showing that it varies from P-complete in the simplest case up to EXPTIMEhard in the worst. We present an example to illustrate the approach, and present some brief conclusions.

2 Normative Temporal Logic

Kripke Structures: Let Φ = {p, q, . . .} be a finite set of atomic propositional variables. A Kripke structure (over Φ) is a quad K = 〈S, S0, R, V〉, where: S is a finite, non-empty set of states, with S0 ⊆ S (S0 ̸= ∅) being the initial states; R ⊆ S × S is a total binary relation on S, which we refer to as the transition relation 1 ; and V : S → 2Φ labels each state with the set of propositional variables true in that state. A path over R is an infinite sequence of states π = s0, s1, . . . which must satisfy the property that ∀u ∈ N: (su, su+1) ∈ R. If u ∈ N, then we denote by π[u] the component indexed by u in π (thus π[0] denotes the first element, π[1] the second, and so on). A path π such that π[0] = s is an s-path.

Normative Systems: In this paper, a normative system is a set of constraints on the behaviour of agents in a system.

1 A relation R ⊆ S × S is total iff ∀s ∃s′ : (s, s′ ) ∈ R.

More precisely, a normative system defines, for every possible system transition, whether or not that transition is considered to be legal or not. Different normative systems may differ on whether or not a transition is legal. Formally, a normative system η (w.r.t. a Kripke structure K = 〈S, S0, R, V〉) is simply a subset of R, such that R \ η is a total relation. The requirement that R\η is total is a reasonableness constraint: it prevents normative systems which lead to states with no successor. Let N(R) = {η | (η ⊆ R) & (R \ η is total)} be the set of normative systems over R. The intended interpretation of a normative system η is that (s, s′ ) ∈ η means transition (s, s′ ) is forbidden in the context of η; hence R\η denotes the legal transitions of η. Since it is assumed η is reasonable, we are guaranteed that a legal outward transition exists for every state. If π is a path over R and η is a normative system over R, then π is η-conformant if ∀u ∈ N, (π[u], π[u + 1]) ̸∈ η. Let Cη(s) be the set of η-conformant s-paths (w.r.t. some R).

Since normative systems are just sets (of disallowed transitions), we can compare them, to determine, for example, whether one is more liberal (less restrictive) than another: if η ⊂ η′ , then η places fewer constraints on a system than η′ , hence η is more liberal. Notice that, assuming an explicit representation of normative systems, (i.e., representing a normative system η directly as a subset of R), checking such properties can be done in polynomial time. We can also operate on them with the standard set theoretic operations of union, intersection, etc. Taking the union of two normative systems η1 and η2 may yield (depending on whether R \ (η1 ∪ η2) is total) a normative system that is more restrictive (less liberal) than either of its parent systems, while taking the intersection of two normative systems yields a normative system which is less restrictive (more liberal). Care must be taken when operating on normative systems in this way to ensure the resulting system is reasonable.

Syntax of NTL: The language of NTL is a generalisation of CTL: the only issue that may cause confusion is that, within this language, we refer explicitly to normative systems, which are semantic objects. We will therefore assume a stock of syntactic elements Ση which will denote normative systems. To avoid a proliferation of notation, we will use the symbol η both as a syntactic element for normative systems in the language, and the same symbol to denote the corresponding semantic object. An interpretation for symbols Ση with respect to a transition relation R is a function I : Ση → N(R). When R is a transition relation of Kripke structure K we say that I is an interpretation over K. We will assume that the symbol η∅ always denotes the emptyset normative system, i.e., the normative systemwhich forbids no transitions. Note that this normative system will be reasonable for any Kripke structure. Thus, we require that for all I: I(η∅) = ∅. The syntax of NTL is defined by the following grammar:

where p ∈ Φ and η ∈ Ση. Sometimes we call α occurring in an expression Oηα or Pηα a temporal formula (although such an α is not a well-formed formula).

Semantic Rules: The semantics of NTL are given with respect to the satisfaction relation "|=". K, s |=I ϕ holds when K is a Kripke structure, s is a state in K, I an interpretation over K, and ϕ a formulae of the language, as follows:

The remaining classical logic connectives ("∧", "→", "↔") are assumed to be defined as abbreviations in terms of ¬, ∨, in the conventional manner. We write K |=I ϕ if K, s0 |=I ϕ for all s0 ∈ S0 , K |= ϕ if K |=I ϕ for all I, and |= ϕ if K |= ϕ for all K. The remaining CTL temporal operators are defined:

Recalling that η∅ denotes the empty normative system, we obtain the conventional path quantifiers of CTL as follows: Aα ≡ Oη∅ α, Eα ≡ Pη∅ α.

Properties and Axiomatisation: The following Proposition makes precise the expected property that a less liberal system has more obligations (and less permissions) than a more liberal system.

Proposition 1 Let K be a Kripke structure, I be an interpretation over K, and η1, η2 ∈ Ση: If I(η1) ⊆ I(η2) then K |=I Oη1 ϕ → Oη2 ϕ and K |=I Pη2 ϕ → Pη1 ϕ.

We now present a sound and complete axiomatisation for NTL and some of its variants. First, let NTL- be NTL without the empty normative system η∅. Formally, NTL- is defined exactly as NTL, except for the requirement that Ση contains the η∅ symbol and the corresponding restriction on interpretations. An axiom system for NTL-, denoted ⊢-, is defined by axioms and rules (Ax1)-(R2) in Figure 1. NTL- can be seen as a multi-dimensional variant of CTL, where there are several indexed versions of each path quantifier.

Going on to NTL, we add axioms (Obl) and (Perm) (Figure 1); the corresponding inference system is denoted ⊢. We then, have the following chain of implications in NTL (the second element in the chain is a variant of the deontic axiom discussed below). If something is naturally, or physically inevitable, then it is obligatory in any normative system; if something is an obligation within a given normative system η, then it is permissible in η; and if something is permissible in a given normative system, then it is naturally (physically) possible:

Finally, let NTL+ be the extension of NTL obtained by extending the logical language with propositions on the form

(Ax1) All validities of propositional logic (Ax2) Pη♦ϕ↔Pη(⊤U ϕ) (Ax2b) Oη ϕ↔¬Pη♦¬ϕ (Ax3) Oη♦ϕ↔Oη(⊤U ϕ) (Ax3b) Pη ϕ↔¬Oη♦¬ϕ (Ax4) Pη ❢(ϕ∨ ψ) ↔ (Pη ❢ϕ∨Pη ❢ψ) (Ax5) Oη ❢ϕ↔¬Pη ❢¬ϕ (Ax6) Pη(ϕU ψ) ↔ (ψ ∨ (ϕ∧ Pη ❢Pη(ϕU ψ))) (Ax7) Oη(ϕU ψ) ↔ (ψ ∨ (ϕ∧Oη ❢Oη(ϕU ψ))) (Ax8) Pη ❢⊤∧Oη ❢⊤ (Ax9) Oη (ϕ→ (¬ψ ∧Pη ❢ϕ)) → (ϕ→¬Oη(γ U ψ)) (Ax9b) Oη (ϕ→ (¬ψ ∧Pη ❢ϕ)) → (ϕ→¬Oη♦ψ) (Ax10) Oη (ϕ → (¬ψ ∧ (γ → Oη ❢ϕ))) → (ϕ → ¬Pη(γ U ψ)) (Ax10b) Oη (ϕ→ (¬ψ ∧Oη ❢ϕ)) → (ϕ→¬Pη♦ψ) (Ax11) Oη (ϕ→ψ) → (Pη ❢ϕ→Pη ❢ψ) (R1) If ⊢ ϕ then ⊢ Oη ϕ (generalization) (R2) If ⊢ ϕ and ⊢ ϕ→ψ then ⊢ ψ (modus ponens) (Obl) Oη∅ α→Oηα (Perm) Pηα→Pη∅ α (Obl+) η ⊑ η′ → (Oηα → Oη′ α) (Perm+) η ⊑ η′ → (Pη′ α → Pηα)

η ≡ η′ and η ⊏ η′ (⊑ can then be defined), interpreted in the obvious way (e.g., K, s |=I η ⊏ η′ iff I(η) ⊂ I(η′ )). An axiom system for NTL+, denoted ⊢+, is obtained from ⊢- by adding the schemes (Obl+) and (Perm+) (Figure 1).

Theorem 1 (Soundness and Completeness) The inference mechanism ⊢- is sound and complete with respect to validity of NTL- formulas, i.e., for every formula ϕ in the language of NTL-, we have |= ϕ iff ⊢- ϕ. The same holds for ⊢ with respect to formulas from NTL and ⊢+ with respect to NTL+.

Proof: All three cases are proven by adjusting the technique presented in [Emerson, 1990]. For the NTL- case, the tableau-based construction of [Emerson, 1990] immediately carries through: we will encounter, for every generated state, successors of different dimensions. For the case of NTL, which includes the symbol η∅, we have to add clauses corresponding to (Obl) and (Perm) to the construction of the closure cl(ϕ) of a formula ϕ: if Oη∅ α (respectively, Pηα) is in cl(ϕ) then also Oηα (respectively, Pη∅ α) should be in cl(ϕ). In the case of NTL+, we have to close off cl(ϕ) under the implications of axioms (Obl+) and (Perm+). ✷

Going beyond NTL+, we can impose further structure on Ση and its interpretations. For example, we can add unions and intersections of normative systems by requiring Ση to include symbols η ⊔ η′ , η ⊓ η′ whenever it includes η and η′ , and require interpretations to interpret ⊔ as set union and ⊓ as set intersection. As discussed above, we must then further restrict interpretations such that R \ (I(η1) ∪ I(η2)) always is total. This would give us a kind of calculus of normative systems. Let K be a Kripke structure and I be an interpretation with the mentioned properties:

(all of which follow from Proposition 1). Having such a calculus allows one to reason about the composition of normative systems.

Relationship with Deontic Logic: The two main differences between the language of NTL and the language of conventional deontic logic (henceforth "deontic logic") are, first, contextual deontic operators allowing a formula to refer to several different normative systems, and, second, temporal operators. All deontic expressions in NTL refer to time: Pη ❣ϕ ("it is permissible in the context of η that ϕ is true at the next time point"); Oη ϕ ("it is obligatory in the context of η that ϕ always will be true"); etc. Deontic logic contains no notion of time. In order to compare our temporal deontic statements with those of deontic logic we must take the temporal dimension to be implicit in the latter. Two of the perhaps most natural ways of doing that is to take "obligatory" (Oϕ) to mean "always obligatory" (Oη ϕ), or "obligatory at the next point in time" (Oη ❣ϕ), respectively, and similarly for permission. In either case, all the principles of Standard Deontic Logic (SDL) hold also for NDL, viz., O(ϕ → ψ) → (Oϕ → Oψ) (K); ¬O⊥ (D); and from ϕ infer Oϕ (N). The two mentioned temporal interpretations of the (crucial) deontic axiom D are (both NTL validities):

With these translations, all of the most commonly discussed so-called paradoxes of deontic logic also holds in NTL. However, it has been argued (cf., e.g., [Meyer and Wieringa, 1993]) that one of the causes behind some of the instances of the paradoxes (particularly those involving contrary-to-duty obligations) is that the language of conventional deontic logic is too weak, and that by incorporating temporal operators some instances of the paradoxes can be avoided.

3 Symbolic Representations

In practice, explicit state representations of Kripke structures are rarely if ever used when reasoning about systems, because of the state explosion problem: given a system with n Boolean variables, the system will typically have 2n states. Instead, practical reasoning tools provide succinct, symbolic representation languages for defining Kripke structures. We present such a language for defining models, and also introduce an associated symbolic language for defining normative systems.

A Symbolic Language for Models: We present the SIMPLE REACTIVE MODULES LANGUAGE (SRML), a "stripped down" version of Alur and Henzinger's REACTIVE MODULES

LANGUAGE (RML) [Alur and Henzinger, 1999], which was introduced in [Hoek et al., 2006]. SRML represents the core of RML, with some "syntactic sugar" removed to simplify the presentation and semantics. The basic idea is to present a Kripke structure K by means of a number of symbolically represented agents, where the choices available to every agent are defined by a number of rules, defining which actions are available to the agent in every state; a transition (s, s′ ) in K corresponds to a tuple of actions, one for each agent in the system. Here is an example of an agent definition in SRML (agents are referred to as "modules" in (S)RML):

This module, named toggle, controls a single Boolean variable, x. The choices available to the agent are defined by the init and update rules 2 . The init rules define the choices available to the agent with respect to the initialisation of its variables, while the update rules define the agent's choices subsequently. In this example, there are two init rules and two update rules. The init rules define two choices for the initialisation of variable x: assign it the value ⊤ or the value ⊥. Both of these rules can fire initially, as their conditions (⊤) are always satisfied; in fact, only one of the available rules will ever actually fire, corresponding to the "choice made" by the agent on that decision round. The effect of firing a rule is to execute the assignment statements on the r.h.s. of the rule, which modify the agent's controlled variables. (The "prime" notation for variables, e.g., x′ , means "the value of x afterwards".) Rules are identified by labels (ℓi); these labels do not form part of the original RML language, and in fact play no part in the semantics of SRML - they are used to identify rules in normative systems, as we shall see below. We assume a distinguished label "[]" for rules, which is used to identify rules that should never be made illegal by any normative system. With respect to update rules, the first rule says that if x has the value ⊤, then the corresponding action is to assign it the value ⊥, while the second rule says that if x has the value ⊥, then it can subsequently be assigned the value ⊤. In sum, the module nondeterministically chooses a value for x initially, and then on subsequent rounds toggles this value. In this example, the init rules are non-deterministic, while the update rules are deterministic. An SRML system, ρ, is a set of such modules, where the controlled variables of modules are mutually disjoint.

The Kripke structure Kρ = 〈Sρ, S0ρ , Rρ, Vρ〉 corresponding to SRML system ρ is given as follows: the state set Sρ and valuation function Vρ corresponds to states (valuations of variables) that could be reached by ρ, with initial states S0ρ being states that could be generated by init rules; the transition relation Rρ is defined by (s, s′ ) ∈ Rρ iff there exists a tuple of update rules, one for each module in the system, such that each rule is enabled in s and s′ is obtained from executing this collection of rules on s.

2 To be more precise, the rules are guarded commands.

A Symbolic Language for Normative Systems: We now introduce the SRML Norm Language (SNL) for representing normative systems, which corresponds to the SRML language for models. The general form of an SNL normative system definition is:

Here, id ∈ Ση is the name of the normative system; these names will be used to refer to normative systems in formulae of NTL. The body of the normative system is defined by a set of constraint rules. A constraint rule

consists of a condition part χ, which is a propositional logic formula over the variables of the system, and a set of rule labels {ℓ1, . . . , ℓk} (we require [] ̸∈ {ℓ1, . . . , ℓk}). If χi is satisfied in a particular state, then any SRML rule with a label that appears on the r.h.s. of the constraint rule will be illegal in that state, according to this normative system. An SNL interpretation is then simply a set of SNL normative systems, each with a distinct name.

Given SNL normative systems η1 and η2, for some SRML system ρ, we say: η1 is at least as liberal as η2 in system ρ if for every state s ∈ Sρ, every rule that is legal according to η2 is legal according to η1; and they are equivalent if for every state s ∈ Sρ, the set of rules legal according to η1 and η2 are the same.

Theorem 2 The problem of testing whether SNL normative system η1 is at least as liberal as SNL normative system η2 is PSPACE-complete, as is the problem of testing equivalence of such systems.

Proof: We do the proof for checking equivalence; the liberality case is similar. For membership of PSPACE, consider the complement problem: guess a state s, check that s ∈ Sρ, (reachability of states in RML is in PSPACE [Alur and Henzinger, ]) and check that there is some rule legal in s according to η2 is not legal in s according to η1, or vice versa. Hence the complement problem is in NPSPACE, and so the problem is in PSPACE. For PSPACE-hardness, we reduce the problem of propositional invariant checking over (S)RML modules [Alur and Henzinger, ]. Given an SRML system ρ and propositional formula ϕ, define normative systems η1 and η2 as follows (where ℓ does not occur in ρ):

normative-system η1 normative-system η2 ¬ϕ disables ℓ ⊥ disables ℓ

According to η2, ℓ is always enabled; thus η1 will be equivalent to η2 iff ϕ holds across all reachable states of the system.

✷

4 Model Checking

Model checking is an important computational problem for any modal or temporal logic [Clarke et al., 2000]. We consider two versions of the model checking problem for NTL, depending on whether the model is presented explicitly or symbolically. For each of these cases, there are two further possibilities, depending on whether we are given an interpretation I for normative systems named in formulae or not. If we are given an interpretation for the normative systems named in the formula, then NTL model checking essentially amounts to a conventional verification problem: showing that, under the given interpretation, the model and associated normative systems have certain properties. However, the uninterpreted model checking problem corresponds to the synthesis of normative systems: we ask whether there exist normative systems that would have the desired properties.

Explicit State Model Checking: The interpreted explicit state model checking problem for NTL is as follows.

Given a Kripke structure K = 〈S, S0 , R, V〉, interpretation I : Ση → N(R) and formula ϕ of NTL, is it the case that K |=I ϕ?

The CTL model checking problem is P-complete [Schnoebelen, 2003]. The standard dynamic programming algorithm for CTL model checking may be easily adapted for interpreted explicit state NTL model checking, and has the same worst case time complexity. More interesting is the case where we are not given an interpretation. The uninterpreted explicit state model checking problem for NTL is as follows.

Given a Kripke structure K = 〈S, S0 , R, V〉 and formula ϕ of NTL, does there exist an interpretation I : Ση → N(R) such that K |=I ϕ?

Theorem 3 The uninterpreted explicit state model checking problem for NTL is NP-complete.

Proof: For membership in NP, simply guess an interpretation I and verify that K |=I ϕ. Since interpretations are polynomial in the size of the Kripke structure and formula, guessing can be done in (nondeterministic) polynomial time, and checking is the interpreted explicit state model checking problem. Hence the problem is in NP. For NP-hardness, we reduce SAT. Given SAT instance ϕ over variables x1, . . . , xk, for each variable xi, create two variables t(xi) and f (xi), and define a Kripke structure with 3k + 1 states, as illustrated in Figure 2; state s0 is the initial state, and state s3k is a final state. Let ϕ∗ denote the NTL formula obtained from ϕ by systematically replacing every variable xi with (Pη♦t(xi)). Define the formula to be model checked as:

This formula is satisfied in the structure by some interpretation iff ϕ is satisfiable. ✷

Symbolic Model Checking: As we noted above, explicit state model checking problems are perhaps of limited interest, since such representations are exponentially large in the number of propositional variables. Thus we now consider the SRML model checking problem for NTL. Again, we have two versions, depending on whether we are given an interpretation or not.

Theorem 4 The interpreted SRML model checking problem for NTL is PSPACE-complete.

Proof: PSPACE-hardness is by a reduction from the problem of propositional invariant verification for SRML [Alur and Henzinger, ] 3 . Given a propositional formula ϕ and an (S)RML system ρ, let I = {η∅}, and simply check whether Kρ |=I Oη∅ ϕ. Membership of PSPACE is by adapting the CTL symbolic model checking algorithm of [Cheng, 1995]. ✷

Theorem 5 The uninterpreted SRML model checking problem for NTL is EXPTIME-hard.

Proof: By reduction from the problem of determining whether a given player has a winning strategy in the two-player game PEEK-G4 [Stockmeyer and Chandra, 1979, p.158]. An instance of PEEK-G4 is a quad 〈X1, X2, X3, ϕ〉 where: X1 and X2 are disjoint, finite sets of Boolean variables - variables X1 are under the control of agent 1, and X2 are under the control of agent 2; X3 ⊆ (X1 ∪X2) are the variables true in the initial state of the game; and ϕ is a propositional formula over X1 ∪ X2, representing the winning condition. The agents take try to make ϕ true, by taking it in turns to alter the value of at most one of their variables. The decision problem is to determine whether agent 2 has a winning strategy in a given game. The idea of the proof is to define an SRML system that such that the runs of the system correspond to plays of the given game instance, and then to define an NTL formula to be model checked, which names a normative system η, such that the transitions legal according to η correspond to a winning strategy for player 2. The construction of the SRML system follows that of the EXPTIME-completeness proof of ATL model checking in [Hoek et al., 2006], with the difference that player 2's update rules are given labels (so that they may be disabled). The formula to model check then defines three properties: (i) if it is agent 2's turn, then according to η at most one of its possible moves is legal; (ii) all of agent 1's moves are legal according to η (i.e, agent 2 must win against all of these); and (iii) the legal paths according to η must represent wins for agent 2. ✷

3 In fact, the result of [Alur and Henzinger, 1999] is stated for RML, but the proof only makes use of features from SRML.

5 Example: Traffic Norms

Consider a circular road, with two parallel lanes. Vehicles circulate on the two lanes clockwise. We consider two types of vehicles: cars, and ambulances. The road is discretised in a finite number of positions, each one represented as an instance of a proposition at(lane-number, lane-position, vehicle-id). Thus at(2, 5, car23) means agent car23 is on lane 2 at position 5 (lane 1 is the outer lane, lane 2 is the inner lane). We also refer to lane 1 as the left lane and to lane 2 as the right lane. At each time step, cars can either remain still or change their position by one unit, moving either straight or changing lane. Ambulances can remain still or change their position by one or two units, either straight or changing lanes at will. We are interested in normative systems that prevent crashes, and that permit ambulances take priority over private cars. So consider the following normative systems:

• η1: Ambulances have priority over all other vehicles (i.e., cars stop to allow ambulances overtake them);
• η2: Cars cannot use the rightmost (priority) lane;
• η3: Vehicles have "right" priority (i.e., left lane has to give way to any car running in parallel on the right lane).

We modelled this scenario using an RML-based model checking system for ATL [Alur et al., 2002]. Each vehicle is modelled as a module containing the rules that determine their physically legal movements, and global traffic control is modelled as a set of norms that constrain the application of certain rules. For example, here is the (somewhat simplified) definition of a car (we abuse notation to facilitate comprehension: for example addition and subtraction here are modulo-n operations, where n is the number of road positions, and the at(. . .) predicates are implemented as propositions):

module car-23 controls at(l,p,car-23) init [] // initialise ... update car-23-straight: at(l,p,car-23) & not(at(l,p+1,car-1)) & ... & not(at(l,p+1,vehicle-n)) -> at(l,p+1,car-23)' := T, at(l,p,car-23)' := F; car-23-right: at(1,p,car-23) & not(at(2,p+1,car-1)) & ... & not(at(2,p+1,vehicle-n)) -> at(2,p+1,car-23)' := T, at(1,p,car-23)' := F; car-23-left: at(2,p,car-23) & not(at(1,p+1,car-1)) & ... & not(at(1,p+1,vehicle-n)) -> at(1,p+1,car-23)' := T, at(2,p,car-23)' := F; car-23-still: T -> skip;

We can then define the norms described above using SNL; (again, we abuse notation somewhat in the interests of brevity; variables must be expanded out for each car and position, in the obvious way):

normative-system N1 at(1,p,car-i) and at(1,p-1,amb-j) disables car-i-straight, car-i-left, car-i-right; normative-system N2

at(2,p,car-i) disables car-i-straight, car-i-still; at(1,p,car-i) disables car-i-right; normative-system N3 at(1,p,car-i) and at(2,p,car-j) disables car-i-straight, car-i-right;

Using a model checker, we can then evaluate properties of the system; e.g., if there is only one ambulance then we have Oη1∪η2∪η3 ¬crash.

6 Conclusions &Acknowledgments

Several issues present themselves for future work: tight bounds for complexity of uninterpreted symbolic model checking, the complexity of satisfiability, and a full implementation of a model checker encompassing the variations discussed in section 4 are the most obvious.

We gratefully acknowledge support from the Spanish research project Web(I)-2, TIC2003-08763-c02-00), the EU funded project OpenKnowledge, FP6-027253), and the UK EPSRC project "Virtual Organisations for E-Science".