Using the Probabilistic Logic Programming Language P-log for Causal and Counterfactual Reasoning and Non-Naive Conditioning

1 Introduction andMotivation

In recent years there has been a lot of progress in logical knowledge representation and reasoning as well as probabilistic knowledge representation and reasoning. There have been some attempts at developing formalisms that allow both logical as well as probabilistic knowledge representation.

One such approach is Pearl's probabilistic causal models [Pearl, 2000]. While Pearl's formalism is an improvement over traditional propositional probability representations such as Bayes' nets, there are still the limitations that are associated with propositional representation as well as classical (and hence monotonic) reasoning. In recent years approaches to overcome both have been proposed. Proposed approaches to overcome the limitations of propositional representation include probabilistic relational models [Koller, 1999; Getoor et al., 2001], various probabilistic first-order logics [Nilsson, 1986; Bacchus, 1990; Bacchus et al., 1996; Halpern, 1990; 2003; Pasula and Russell, 2001; Poole, 1993], and approaches based on assigning weights to first-order formulas [Paskin, 2002; Richardson and Domingos, 2006]. There have been also many formalisms that integrate logic programming with probability such as Poole's Probabilistic Horn Abduction (PHA) [Poole, 1993], his Independent Choice Logic (ICL) [Poole, 1997; 2000], the LPAD formalism [Vennekens et al., 2004], Bayesian logic programming [Kersting and Raedt, 2000], ALTERID [Breese, 1990; Wellman et al., 1992], probabilistic knowledge bases [Ngo and Haddawy, 1997], stochastic logic programs [Muggleton, 1995; Cussens, 1999], probabilistic constraint logic programs [Riezler, 1998; Santos Costa et al., 2003], interval based probabilistic logic programming [Ng and Subrahmanian, 1992;

Earlier it was shown that P-log has many advantages over the other languages. In particular, P-log allows a wide variety of updates compared to the other approaches; it allows explicit specification of background knowledge; and it allows logical reasoning to dynamically decide on the range of values that a random variable can take. In this paper we show how P-log can also do causal and counterfactual reasoning of the kind in Pearl's probabilistic causal models. We give an encoding of probabilistic causal models in P-log and state the correspondence. Our correspondence shows that while the computation of causal and counterfactuals in PCMs is algorithmic in nature, in the encoded P-log it is automatically captured by the semantics of the language. We then show that P-log's ability to allow logical reasoning to dynamically decide on the range of values that a random variable can take leads to a elaboration tolerant way to perform non-naive conditioning [Halpern, 2003]. We illustrate this with respect to Halpern's second-ace puzzle example. 1

2 Background

In the next two subsections, we will briefly review the syntax of a simplified version of P-log [Baral et al., 2004] and the syntax and semantics of probabilistic causal models [Pearl, 2000].

2.1 P-log lite: an overview

For simplicity, we will define a subset of the probabilistic logic programming language P-log, which we will call P-log lite. We use logic programming terminology of terms, atoms and literals. A P-log lite program Π consists of (i) a set of boolean attributes {p, ¬p : p is an atom }, (ii) a regular part, (iii) a set of random selection rules, (iv) a probabilistic information part, and (v) a set of observations and actions. In the following by t we denote a vector of parameters consisting of terms, and by ˜ ¬p we refer to p and by ˜ p we refer to ¬p.

1 Earlier work on P-log [Baral et al., 2004] hinted at this with respect to the Monty Hall example. However no general methodologies for "non-naive" conditioning were proposed.

• (ii) Regular part: The regular part of a P-log lite program consists of a collection of logic programming rules (without disjunction and under the stable model semantics).
• (iii) Random Selection: A random selection is a rule of the form

Statement (1) says that one of a(t) or ¬a(t) is selected at random. In P-log with non-boolean attributes the random statement is more general and is of the form: random(Attr : {X : p(X)}).

(iv) Probabilistic Information: Information about probabilities of random attributes taking particular values is given by probability atoms (or simply pr-atoms) which have the form:

where l is a(t) or ¬a(t), and pr(a(t)) = v would cause a(t) with probability v.

• (v) Observations and actions: Observations and actions are statements of the respective forms: obs(l) and do(l).

Observations are used to record the outcomes of random events, i.e., random attributes and attributes dependent on them. The statement do(a(t)) indicates that a(t) is made true as a result of a deliberate (non-random) action.

Semantics of P-log lite:

The semantics of a P-log lite program Π is given by a collection of the possible sets of beliefs of a rational agent associated with Π, together with their probabilities.

We start with translating the logical part of a P-log lite program Π to an Answer Set Prolog program τ(Π) as described below.

• (a) τ(Π) contains the regular part of Π.
• (b) For each attribute atom a(t), τ(Π) has the rules: intervene(a(t)) ← do(a(t)). intervene(a(t)) ← do(¬a(t)).
• (c) For each random(a(t)) in Π, τ(Π) has the Smodels rule: 1{a(t), ¬a(t)}1 ← not intervene(a(t)).

The left hand side of the above rule means that one of a(t) and ¬a(t) must be in the answer set.

• (d) τ(Π) has all the actions and observations of Π.
• (e) For all literals l, τ(Π) has the rules:

An answer set of τ(Π) is called a possible world of Π. The set of all possible worlds of Π is denoted by Ω(Π).

Let W be a possible world of a P-log lite program Π. If Π contains a random selection rule of the form random(a(t)), then we say that both a(t) and ¬a(t) are possible in W with respect to Π. For everyW ∈ Ω(Π) and every literal l possible in W we now define the corresponding probability P (W, l).

For each l possible in W:

If l ∈ W, Π has a pr-atom pr(l) = v and W does not contain intervene(l) then P(W, l) = v, and P(W, ˜l) = 1 - v.

If for any attribute literal l, we do not have pr-atoms for either l or ˜l, l ∈ W and W does not contain intervene(l) then P(W, l) = 0.5, and P(W, ˜l) = 0.5.

Now we are ready to define the measure, μΠ, induced by the P-log lite programΠ.

Definition 1. (Measure) (i) The unnormalized probability, ˆμΠ(W), of a possible world W induced by Π is ˆμΠ(W) = ∏l∈ W P(W, l), where the product is taken over literals for which P(W, l) is defined.

• (ii) The measure (or the normalized probability), μΠ(W), of a possible world W induced by Π is the unnormalized probability of W divided by the sum of the unnormalized probabilities of all possible worlds of Π, i.e., μΠ(W) = ˆμΠ(W) PWi∈Ω ˆμΠ(Wi)

When the program Π is clear from the context we may simply write ˆμ and μ instead of ˆμΠ and μΠ respectively. ✷

The truth and falsity of propositional formulas with respect to possible worlds are defined in the standard way. A formula A is true in W is denoted by W ⊢ A.

Definition 2. (Probability)

The probability with respect to program Π of a formula A, PΠ(A), is the sum of the measures of the possible worlds of Π in which A is true, i.e. PΠ(A) = ∑W⊢A μΠ(W). ✷

When Π is clear from the context we may simply write P instead of PΠ. Conditional probability in P-log lite is defined in the usual way. Moreover, under certain consistency conditions (which hold for the examples discussed in this paper) on P-log lite programs T , formulas A, and a set of literals B such that PT (B) ̸= 0, we have PT (A|B) = PT∪obs(B)(A)

2.2 Probabilistic causal models

In this section we present many definitions from [Pearl, 2000] and give our definition of a PCM-probabilistic query.

Causal model: A causal model is a triple M = 〈U, V, F〉 where

• (i) U is a set of background variables, (also called exogenous), that are determined by factors outside the model;
• (ii) V is a set {V1, V2, . . . Vn} of variables, called endogenous, that are determined by variables in the model - that is, variables in U ∪ V ; and
• (iii) F is a set of functions {f1, f2, . . . fn}, such that each fi is a mapping from U ∪ (V \ Vi) to Vi, and such that the entire set F forms a mapping from U to V . In other words, each fi tells us the value of Vi given the values of all other variables in U ∪V , and the entire set F has a unique solution for V , given a realization of U. Symbolically, the set of equations F can be represented by writing Vi = fi(PAi, Ui), i = 1, . . . , n, where PAi (connoting parents) is a subset of variables in V \ Vi and Ui stands for a subset of variables in U.

We will refer to the realization of a set of variables Y ⊆ V given the causal model M and the background variable realization of U = u as YM(u). Y (u) will be used as shorthand for YM(u), but we will explicitly use the subscript Mx for realizations of submodels defined below.

Example 1. We have a simple causal model consisting of a background variable U, endogenous variables A and B, and the set of the following causal functions:

For each value of U, there is a unique solution for A and B. That is, for U = 1, A = B = 1 and for U = 0, A = B = 0. ✷

Submodel: Let M be a causal model, X be a set of variables in V , and x be a particular realization of X. A submodel Mx of M is the causal model Mx = 〈U, V, Fx〉 where Fx = {fi : Vi ̸∈ X} ∪ {X = x}.

Submodels are useful for representing the effect of local actions and hypothetical changes. Mx represents the model that results from a minimal change to make X = x hold true under any realization of U.

Effect of Action: Let M be a causal model, X be a set of variables in V , and x be a particular realization of X. The effect of action do(X = x) on M is given by the submodel Mx.

Potential Response: Let X and Y be two subsets of variables in V . The potential response of Y to action do(X = x), denoted YMx (u), is the solution for Y of the set of equations Fx, where u is a particular realization of U.

Counterfactual: Let Xi and Y be two subsets of variables in V . The counterfactual sentence "The value that Y would have obtained, had Xi been forced to be xi" is interpreted as denoting the potential response YMxi (u), where u is a particular realization of U.

Probabilistic causal model: A probabilistic causal model (PCM) is a pair 〈M, P(u)〉 where M is a causal model and P(u) is a probability function defined over the domain of U.

Because the set of functional equations forms a mapping from U to V , the probability distribution P also induces a probability distribution over the endogenous variables.

The probability of a counterfactual statement is defined using the potential response, YMx (u), induced by the submodel Mx. It can be expressed in a similar manner: P(YMx = y) = ∑{u | YMx (u)=y} P(u)

PCM probabilistic query: Joint probabilities of counterfactuals that are conditional on observations will be the focus of our counterfactual reasoning. Let YMx1 = y1,. . . , YMxm = ym be counterfactuals and let B and C be subsets of V. A PCM probabilistic query is of the form P(YMx1 = y1,. . . , YMxm = ym, B = b | C = c), and its value is given by:

3 Encoding Pearl's firing squad in P-log lite

In this section we will consider Pearl's firing squad example and show how one can encode this example in P-log lite and ask counterfactual queries.

The firing squad PCM: The PCM of the firing squad has two background variables U and W, and endogenous variables A, B, C, and D; which stand for

U = court orders the execution; C = captain gives a signal; A = rifle A shoots; B = rifle B shoots; D = the prisoner dies; W = rifle A pulls the trigger out of nervousness.

The causal relationships between the variables are described by the functional equations C = U; A = C ∨W; B = C; D = A ∨ B. In addition the probabilities of the background variables are given as P(U) = p and P(W) = q.

3.1 Example of an observation assimilation query

Consider the query that asks the probability that B shot given that the prisoner is dead? In PCM formalism this will be expressed as P (B|D).

To translate this query to an equivalent P-log lite query we will use a program Π1 that captures both the causal relationships and the probabilistic arguments found in the firing squad PCM. The programΠ1 will have explicit rules for the positive and negative boolean variables. The P-log lite predictive query is not conditioned on the fact that the prisoner is dead. Instead, the fact that the prisoner is dead is added to Π1 as an observation rule obs(D). The P-log lite programΠ1 consists of the following:

1. Declarations for each background variable:
2. Declarations for each endogenous variable:
3. Pr atoms and logic programming rules

U : boolean.

W : boolean.

random(U ).

random(W ).

C : boolean.

B : boolean.

A : boolean.

D : boolean.

1. Reminder of the logic programming rules

Result 1. P(B|D) = PΠ1 ∪ obs(D)(B) = p 1-(1-p)(1-q)

3.2 Example of an intervention query

Now let us consider the following intervention query: What is the probability that the prisoner is dead given that A is not allowed to shoot? In PCM formalism this is expressed as P (DM¬A ).

To translate this query to an equivalent P-log lite query we will use a program Π2 which is similar to Π1 with respect to the parts (1)-(3) of Π1, but differs on part (4) by having rules which are blocked when an intervention is done. In this the nonmonotonic operator not of logic programs comes in handy.

Part (4) of Π2

¬C ← ¬U, not do(C), not do(¬C).

¬B ← ¬C, not do(B), not do(¬B).

3.3 Example of a counterfactual query

We now consider the following counterfactual query from [Pearl, 2000]: "What is the probability that the prisoner would be alive if A had not shot given that the prisoner is in fact dead?"

In PCM this counterfactual query is expressed as either P (¬DM¬A | D) or as P (¬DM¬A | DM ), where M is the original causal model.

Thus the PCM counterfactual query has two causal models in the probabilistic query. The conditional variable DM, which is an equivalent representation of variable D, is a member of the original causal model M, while the variable YM¬A is a member of the submodel M¬A.

Our encodings in Π1 refers to the model M and our encoding in Π2 refers to the model M¬A. Now we need both. We achieve this by carefully using the indices 0 and 1 for the endogenous variables. We do not use any index for the exogenous variables as they remain the same in both models.

Note that in Π1 we do not apply do() rules to the original model M, since by definition submodels are the result of an effect of action do() on the original model M. Therefore all variables that are members of the M do not need intervention rules in the translation.

The P-log lite encoding of the above example will consist of the programΠ3 given below. To ask the query all one needs to do is to add obs(D(0)) and do(¬A(1)) to Π3 and compute the probability of ¬D(1). In the syntax of P-log lite this is expressed as PΠ3 ∪ obs(D(0)) ∪ do(¬A(1))(¬D(1)).

The intuition behind adding obs(D(0)) is that D is observed with respect to the original model M which is encoded by the index 0. The intuition behind adding do(¬A(1)) is that ¬A is established (through an action) resulting in a new model M¬A which is encoded by the index 1. The intuition behind asking the probability of ¬D(1) is that the query is asked about the model M¬A which, as we mentioned earlier, is encoded by the index 1. Now we describe the programΠ3. Its constituents are as follows:

• Declarations for each background variable:
• Probability atoms corresponding to the background variables U and W.

U : boolean. random(U ). W : boolean. random(W ).

• Indices to enumerate the modelsM andM¬A: index = {0, 1}.
• Declarations for each endogenous variable:
• Rules that allow us to reason about the model M.

C : index→ boolean. A : index→ boolean. B : index→ boolean. D : index→ boolean.

We use the index 0 for the endogenous variables. The background variables do not have an index as they remain unchanged with respect to the original model and its sub-models.

• Rules that allow us to reason about the model M¬A.

Since doing an action (as an assignment to a variable) necessitates surgery to the model so that the parents of the variable that is assigned no longer affect the variable; we do it logically by adding "not do" in the bodies of the rules. As a result we have the following rules:

To answer PΠ3 ∪ obs(D(0)) ∪ do(¬A(1))(¬D(1)) all one needs to do is to add the intervention and observation facts do(¬A(1)), and obs(D(0)), respectively and compute the probability of ¬D(1) with respect to the resulting theory.

4 A general encoding of PCM to P-log lite

We now generalize the encoding illustrated in the previous section to arbitrary PCM theories and for queries that may refer to multiple hypothetical models. We will refer to our encoding 2 as T . To accommodate multiple submodels, the Plog lite rules will use an index variable S, where S = 0 will correspond to the original model of the PCM theory and, S = 1 . . . m, will correspond to the m sub-models necessitated by the probabilistic query. The encoding T will consist of the following.

Step 1: Enumerate the models and submodels necessitated by the probabilistic query 0, . . . ,m. The variable S used in the endogenous variables rules is in the domain of index:

2 Our encoding of a PCM theory and a query will be independent of each other.

Step 2: For each background variable u ∈ U, for which the PCM has the probability p(u) = q, T will have the following:

Step 3: For each endogenous variable v, with x1, . . . , xn ∈ U ∪ (V \ v) and with the associated function fv(x1, . . . , xn), we do the following:

1. Add the following declaration: v : index→ boolean.
2. Let the disjunctive normal form of fv(x1, . . . , xn) be c1 ∨ . . . ∨ cn, where each ci is a conjunction of literals made up of from x1, . . . , xn. In the translation the parameter S is added to each endogenous variable xi of x1, . . . , xn, such that xi becomes xi(S). The P-log lite program T will have the following rules:

1. Let the disjunctive normal form of ¬fv(x1, . . . , xn) be d1 ∨ . . . ∨ dn, where each di is a conjunction of literals made up of from x1, . . . , xn. In the translation the parameter S is added to each endogenous variable xi of x1, . . . , xn, such that xi becomes xi(S). The P-log lite program T will have the following rules:

Step 4: Encoding the query P(b, YMx1 = y1, . . . , YMxm = ym|c)

• 4.1: For each counterfactual statement YMxi = yi:

For each variable v ∈ V that is a member of the realization of Xi = xi: if v is positive, we add do(v(S = i)); else (i.e., v is negative) we add do(¬v(S = i)).

• 4.2: For each variable w ∈ c: if w is positive, we add obs(w(0)); else (i.e., w is negative) we add obs(¬w(0)). ✷

Before presenting a theorem that formally relates PCM theories and queries with our encoding, we need to consider a notion of "being able to compute unique solutions of a causal model using three valued iteration". Consider the causal model from Example 1 which had the functions: A = U ∧B. B = U ∨ A. In Example 1 we showed that for each value of U, there is a unique solution for A and B. In this case, these unique solutions can be computed in an iterative fashion. Suppose we initially assign the value false to U. We then assign unknown to A and B. We now use 3-valued logic on the causal functions and compute the values of A and B. After the first iteration A has value false and B has value unknown. After the second iteration we reach a fixpoint where A and B are both assigned to false. This is what we refer to as "being able to compute unique solutions of a causal model using three valued iteration."

Having unique solution does not necessarily mean that it can be obtained using the above mentioned three valued iteration. Following is a counterexample 3 :

Let A = ¬B ∨¬C ∨U, B = ¬A∧¬C, and C = A∨¬B. When U is false we can not derive the value of A, B, or C by a three valued iteration. But, there is the unique solution A = true, B = false, and C = true that satisfies the causal functions.

Theorem 1. Given a PCMM = 〈M, P(U)〉, let P denote the probabilistic query with respect to it and let the P-log lite program T be obtained by the encoding of M as described in the previous section. Let b, c, y1, . . . , ym and x1, . . . , xm be realizations of subsets of the endogenous variables V . Also let YMxi = xi be counterfactual statements, where Mxi is a submodel of M. If all realizations of U = u give unique solutions for endogenous variables in the submodelsMxi and the unique solutions can be computed in an iterative fashion starting from the background variables we have the following:

Discussion on the theorem and its proof:

Here the encoding has three main aspects: (i) the encoding of the PCM equations, (ii) taking into account the severing of connections between variables when an action is performed, and (iii) considering models and submodels in parallel during counterfactual reasoning. For part (i) the notion of "unique solutions being computed in an iterative fashion" is important as that leads to a correspondence between the unique solutions and the answer sets. To do (ii) we use the negation as failure operator "not" and to do (iii) we introduce multiple time lines. Overall, the proof is based on splitting the logic programming translation of T to many layers, where the bottom layer consists of the enumeration of the background variables.

4.1 PCM, PAL and P-log lite

A related work is [Tran and Baral, 2004], where an encoding of PCM in an action language PAL is given. The encoding here to a probabilistic logic programming language P-log lite is different from the encoding in [Tran and Baral, 2004]. There the key issue was to encode the severing of connection between variables when an action is performed. It was achieved by introducing an ab predicate and having effect axiom make(vi) causes {ab(vi), vi} and a static causal law ¬ab(vi) causes vi ⇔ fi(PAi, ui) to capture the PCM equation vi = fi(PAi, ui).

5 Non-naive conditioning in P-log

We now show how P-log provides an elaboration tolerant way to perform non-naive conditioning in that it shows a way to incorporate new observations through simple addition of knowledge rather than a surgery of existing knowledge. Halpern in [Halpern, 2003], uses the term naive conditioning to refer to conditioning with respect to the 'naive' state space one usually constructs when trying to reason. He discusses several examples, where naive conditioning gives the wrong answer. Following is his second-ace puzzle:

3 This has significance beyond AI, as the recent thesis [Riedel, 2003] (awarded the best Ph.D thesis of Electrical Engineering at the California Institute of Technology for the year 2003) mentions three valued iteration but does not point out that the iteration does not necessarily lead to the unique solution.

A deck has four cards: the ace and the deuce of hearts and spades. After a fair shuffle, two cards are dealt to Alice. It is easy to see that, at this point, there is a probability of 1/6 that Alice has both aces, a probability of 5/6 that Alice has at least one ace, a probability of 1/2 that Alice has ace of spades and a probability of 1/2 that Alice has ace of hearts.

Alice then says, "I have an ace." Conditioning on this information (by discarding the possibility that Alice was dealt no aces), Bob computes the probability that Alice holds both aces to be 1/5. This seems reasonable. Next, Alice says, "I have the ace of spades." Conditioning on this new information, Bob now computes the probability that Alice holds both aces to be 1/3. But suppose that Alice had instead said, "I have the ace of hearts." It seems that a similar argument again shows that the conditional probability that Alice holds both aces is 1/3.

Is this reasonable? When Bob learns that Alice has an ace, he knows that she must have either the ace of hearts or the ace of spades. Why should finding out which particular ace it is raise the conditional probability of Alice having two aces? Put another way, if this probability goes up from 1/5 to 1/3 whichever ace Alice says she has, and Bob knows that she has an ace, then why isn't it 1/3 all along?

Halpern [Halpern, 2003] analyzes the above example and points out that it is important to know to what question Alice answered: "I have ace of spades." He goes on to say that if she is asked (Q1) to name an ace if she has any and it is given that if she has both aces she will pick one of their colors with equal probability, naive conditioning does not lead to the right answer. We agree with Halpern. In this case instead of the naive possible worlds, one needs to take into account the new question and create a new set of possible worlds. Plog allows us to do that within the language in a nice way. On the other hand if Alice's answer is with respect to the question (Q2) which asks if Alice has the ace of spades, then the assimilation of Alice's answer is different, and in this case naive conditioning works.

The second-aces puzzle in P-log:

Consider the following encoding Π1 of the second-ace puzzle in P-log. 4

card1 and card2 have the domain card = {1s, 2s, 1h, 2h}. random(card1). ¬p2(X) ← card1 = X. p2(X) ←not ¬p2(X). random(card2 : {X : p2(X)}).

4 Here we use full P-log rather than P-log lite that we presented earlier. Among the differences, in full P-log the literals are more general and attributes can take values from a declared domain rather than being just true or false.

has alice(X) ← card1 = X. has alice(X) ← card2 = X. Result 4. Probabilities w.r.t. Π1 (i) PΠ1 (has alice(1s) ∧ has alice(1h)) = 1/6 (ii) PΠ1 (has alice(1s) ∨ has alice(1h)) = 5/6 (iii) PΠ1 (has alice(1s)) = 1/2 (iv) PΠ1 (has alice(1s) ∧ has alice(1h)|has alice(1s)) = 1/3

Now we will consider that Alice's response of "I have ace of spades" is with respect to the question Q1. In that case we propose that the new information is incorporated to Π1 by adding the following to Π1:

random(o : {X : p(X)}). p(1s) ← has alice(1s). p(1h) ← has alice(1h).

We refer to the resulting P-log program as Π2.

Result 5. PΠ2 (has alice(1s) ∧ has alice(1h)|o = 1s) = 1/5

A general methodology:

From the encodings in the previous section one can draw the lesson that when an observation x about a particular concept is made, one can incorporate that observation correctly and avoid possible error associated with naive conditioning by:

1. Adding rules for a new predicate p(X) that define the value that o can take.
2. Adding the random declaration for a new attribute o: random(o : {X : p(X)}).
3. Adding the observation obs(o = x).

Moreover the steps (1) and (2) are useful for defining the values that an attribute can take. It is used in Π1 when defining the values the attribute card2 can take, after card1 has been assigned a value.

6 Conclusion

In this paper we have shown how to do non-naive conditioning and encode Pearl's probabilistic causal models (PCMs) in P-log lite. We show how one can reason about simple observations, observations as a result of answers given to specific queries, interventions and counterfactuals, in P-log lite. There is a distinct difference between how reasoning about interventions and counterfactuals is done in PCMs and in Plog lite. In the former, surgery is done on a given PCM theory to obtain sub-models due to interventions, while in the later one just needs to add do facts, and let the semantics take care of the reasoning. Another difference between the two approaches is that P-log, because of its superior logical knowledge representation aspect, allows one to specify more nuanced theories. For example, with respect to the firing squad example, one can express knowledge such as "Rifleman B normally follows the captain's order". An elaboration tolerant representation of "normally" will allow easy elaboration of new knowledge such as: "B, as a conscientious objector to death penalty, defies his captain with respect to shooting." Such representation and elaboration is straightforward in Plog, while the corresponding PCM theory will need surgery.

7 Acknowledgement

This work was supported by a contract from ARDA and NSF grant 0412000. We thank Michael Gelfond and Nelson Rushton for discussions on this topic.