A Society of Mind-Multiple Perspectives, Reasoned Assumptions, and Virtual Copies

ABSTRACT

Little i' clearly unrlerstood about the similarities, dilferences, ·and com­ parative computational arHl representational advantages of the many proposals extant for organizing minds into collections of "mental sub­ agents." Using a new mathematical framework for exactly specify­ ing the structure of mental organizations, we formulate separately t-he ideas of multiple pe.·spect.ivcs, reasoned assumptions, and virtual copies. WhP-n combined, thf'se notions form a conimon backbone for systems as diverse a.· CONLAN, NETL, and FOL, and show many particular characteristics of those systems results of the "language of thought" adopted for representing the contents of mental subagents. The framework also sug~csts connections between the "strengths" of r;,ental attitudes, the ambiguity of "self," and the possibilit.ies for self­ on,niscicnce.

• jl. Several recent proposals in artificial intelligence reformulate an­ oicnt doubts about the reality of the "sell" by explaining or constructing ~geuts in terms of a collection of int.cracting, simpler subagents. Some of these proposals discuss the agent's actions without reference to any "self" at all, and others presume "selfhood" to Hit epiphcnomcnally from subagent to subagent as dictated by needs to communicate with the external world or to assign credit or blame for actions. While thinkers throughout history have occasionally doubted on philosophical grounds the r.ommon "single-agent" view of the human mind, the new proposals suggest that there may be computational difficulties inherent in singk~agent psyc!.ologics that arc only overcome by the multi-agent viewpoint. MINSKY has called this approach the "society of mind." In some proposals the subagents comprising the society are numerous, very simple, neurologi~ally conceived mechanisms. Other proposals suggest more ccmplex subageuts, ranging from the coroutine collections of "hetcrarchy," to the "knowledge sources" of production systems, to frame systems, to collections of mutually referring logical theories, to F'REUD'S com;nit.tee of id, ego, and superego, and modern split-brain thL'Ories, in which the complexities of the subagents rival that of the agent as a whole. We need not view these proposals as mutually ex­ clusive, if we can subdivide subagents into sub-subagents, but questions like t,Ws arc difficult to pursue without reasonably precise characteriza­ tions of the dilfcrent sorts of subagents to be related. More specifically,

This research wa." supported by the Defense Advanced Research Pro­ jects Agency {DOD), ARPA Order No. 3597, monitored by the Air Force Avionics Laboratory under Contract F33615-81-K-1539. The views and conclusions contained in this document are those of the au thor, and should not be interpreted as representing the official p:>licies, either expressed or implied, of the Defense Advanced Research Projects Agency or the Government of the United States of America.

A SOCIETY OF MIND

Multiple peospcctives, reasoned assumptions, and virtual copies

Jon Doyle Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 U.S.A.

the knowledge representation literature is filled with proposals for com­ plex organizations based on widely dilfering sorts of "languages of thought," such as logir. {l<'OL [WEYHRAUCH 1980]), list structures and rational :dgebraic functions {CONLAN [S.USSMAN AND STEELE 1980]), nodes a.nd links {NETL [[>'AHLMAN 1979]), etc. Although their abstract structures seem related, there is little hope for understand­ ing the relations among these proposals. and for making rapid further progress without clearly formulating the underlying ideas separately and then analyzing their range of combinations. Toward this end, we present a mathematical framework for exactly specifying the structure of mental societies. Since the framework is fairly general, we illustrate it by characterizing a particular society of mind which incorporates three often-proposed capabilities of subagents and relations between subagents, namely multiple perspectives, reasoned a.~"umptions,. and virtual copies. While these characteristics of societie~ are sometunes thought to require the usc of logical or quasi-logical languages as sys­ tems of representation, our formulation makes few structural or rep­ resentational demands, and so permits usc of any desired system of representation {including logical languages) in which the few required structures may be encoded. ~·or example, we can reconstruct FOL and CONLAN at the end of our formulation largely by choosing logical or LISP-like languages for the contents of mental subagents.

The mathematical framework is developed and otherwise ap­ plied in [DOYLE 1982] ""d [DOYI,E 1983n] . While I formulate the particular society he re t.o gencrali>.e the organization suggested in mY t!H'"is ([DOYLE 1980]), the ideas involved have an older, wider history, and I have worked to incorporate the insights of JOllA N DE KLEER, MERRICK FURST, KURT KONOLIGE, MARVIN MINSKY, BRIAN SMITH, RICHARD STALLMAN, GUY STEELE, GERALD SUSSMAN, DAVID TOURET:6KY, and RICHARD WEYIIRAUCII into this exposi­ tion.

• §2. Researchers frequently motivate proposed decompositions ol mind with concerns about self-knowledge, that is, information and mechanisms the agent employs to understand, predict, control, and modify its structures and actions. Although specific tasks appear amen­ able to specific solu Lions, students of the broad problems of repre­ sentation, decision-making, and learning come to appreciate the utility, if not importance, of self-knowledge in adaptive agents. Artificial in tel· ligence studies many sorts of self-knowledge, but for brevity we consider only three.

One corn111only studied sort of self.-knowlcdge involves mul­ tiple corefcrential representations. Since art.ilirial intelligence proposals often suppose reprcscnt.at.ional agents, individual representations and their relations form natural objects of self-knowledge. Since the feasible mechanization of thinking demands concern for the difTiculty of reach­ ing conclusions and solving problems, one of the most studied relations between representations is the ease of thin!<ing about something in i<!fms of one representation relative to the case of thinking ahout it in terms of an alternative representation. MINSKY emphasizes refor­ mulation or representation switching as the h<>art of problem solv­ ing; BOBROW and WINOGRAD make similar opportunism -the basis of KilL; and SUSSMJ\ N and STEELE illustrate t.hc infcrcnti"l impor­ tance of interactions between multiple corcfercnt.ial rcprcsent.at.ions. Thus useful sorts of self-knowledge include the possible alt.ernativcs to a particular representation, their relative cfTicicnci<,s, and how to as­ sign credit or blame to choices among lhcsc alt.ern:ttivcs upon unusual succtosses or failures. The motivations for employing multiple repre­ sentations of Pxtra-mcntal objects ;dso apply to the representations of t.he menial objects figuring in self-knowledge. It is natural to iden­ tif)' these different. views of objects as individual rnental subagents, each with it" own distinguished view of parts of the world, parts of the agent, or parts of both. This is roughly the position taken by MINSKY. However, if these different perspectives or subagents are t.o influence each other, they must be connected somehow, and the basic sorts of connections arc those of reference and corcfcrcnce. Both no­ tions are necessary, for while one subagent might refer to another, in­ tended extra-menial referents cannot be "grasped" in the same way, so at most. the agent can intend thai. its representations of the>!c objects share referent..·.

Unfortunately, the introduction of mutually knowledgeable and influential mental subagents int.o psychological theories poses many puzzling diificulties of formulation and interpretation. These difficulties appear most viciously in agents employing logical languages as sys­ tems of representation. Where classical logic and metamathematics usually seek ways of avoiding paradoxes of self-reference, th· designers of artificial agents instead seem t.o seck them out.. Fort.unat.ely, analysis of a narrow sort of self-knowledge (discussed presently) suggests a for­ mal interprct.ation for these more widely self-referential systems, one which docs not force us to accept any particular psychology for our agents, but instead allows similar formulation and exact comparison of the many variations we might think t.o explore.

Another sort of self-knowledge concerns t.hc inl'ercntial rela­ t.im'" between arbitrary representations instead of the economic rela­ tions between alternative corcfcrential representations. Many research­ ers have studied the uses of explicitly represented infereJotial relation­ ships in const.ructiug explanations, assigning blame for mistakes, and revising the agent's state of mind when it.s assumptions change. These inferential relationships need not. be strictly deductive. While the most general use simply indicates what representations were computed from what, inferential records play a crucial role in so-called default reason­ ing. Default reasoning involves drawing conclusions in the absence of definite supporting or contrary evidence. Representations of the partial evidence for and the missing evidence against a conclusion permit the agent. to make reasoned assumptions, "reasoned" in the sense that the age01t can identify both the sources of the assumed conclusion and t.he sper.ific informatioll which indicates its retraction or reconsideration. The representations of inferential relationships describing reasoned as­ sumptions also pose problems of interpretation, since the agent's draw­ ing one conclusion may prevent it from drawing another. Fortunately, this problem has been solved, and below we extend the solution to handle the problem of multiple perspectives mentioned above.

A third important sort of self-knowledge concerns structural relationships between representations. The most. studied structural relationship is that of structure sharing. Like the technique of mul­ tiple perspectives, structure sharing has economic motivat ions, namely minimizing the number of times one has to encode similar information and t.he amount of storage t.hc agent must consume for the cncodings. Like general inferential rclal.ions)lips, however, st.ruct.mc sharing need not entail corefcrence of the related representations. Fer example, the species of the cat family (lions, tigers, cheetahs, persinns, etc.) may have no properties in common beyond those of mam nw.ls, since each cat species may lack some property shared by all other cat species. But to write down descriptions of each species is very tedious unless we write down a single description of a "prototypical" cat. species (which we may choose t.o be one of the actual species) and describe every other species by its (presumably few) differences from the prototype. Since such family resemblances occur among the members of every 11atural kind, great economies can be realized in representing our common knowledge of the world. The most common sorts of structure sharing relations usually go by the names of "inheritance relations" and "virtual copies." As we demonstrate below, it is easy to interpret. some of these structural relationships between subagents along with the previously mentioned ones.

§3. \Vc first. rlcsco-ibe the mathematical framework in which we work, and then introduce t.hc particular constit.utivc as"'"n pt.ions which characterize t.hc three represcnt.at.ional notions outlined above.

Our first fundamental >~;;surnpt.ion is that st.at.es of t.hc agent. can be decomposed into sets of mental clements or components. We write D to mean the domain of all pos";blc mental components, so that if Sis a st.at.e of t.he agent, t.hen S ~D. Similarly, we "-'sume that Jl is the set of all possible subagents, and that. states of subagents may also be decomposed into elements of D. For each a E A, we write Da ~ D to mean the subdomain of possible state-components of a, and require that. every state component. belong to at least one subagent., that. is, D = UaEA D, . Thus if sis a state of t.he agent and a E A, then s n D. is t.he state of subagent"· We do not require that every subagent cxict. in every slate of the wliole agent, t.hat is, we allow S n D. = 0.

Our second fundamental assumption is that. every slate com­ ponent has an interpretation as a restriction on the possible s.tatcs which contain it. Formally, we assume (and concretely Plcfinc below) a func­ tion I: D--+ P P D (P means power set), so that if dE D, t.hcn I(d) ~ P D is the set of potential states eanctioned by t.hc clement d. We encode in I the intended meanings of subag,nt st.at.e components for the relations standing between the state of the subagent and the states of other subagents. Note that these two assumptions permit several levels of decomposition of subagents into ot.hcr subagents, as long as the interpretations chosen capture the intended synonymy of subagents with their subsocieties, for example by ruling out states in which one occurs without the other.

We define the component-admissible set., Q ~ P D to contain just the "sclf-8atisfying" sets of state components. Formally,

That is, if S E Q, t.hcn all subagent states are in agreement as far as individual components of t.he stale can tell. The third fundamental assumption of the framework is that every admwsible state of the agent must be component-admissible. We write ,$ for the set of admissible stales of the agent, so our a.9sumption is that,$ ~ Q. If,$= Q, then all restrictions on admissible slates are expressed in the "local" restric­ tions given by I, and if ,$ i' Q, t.hen l,herc arc "global" restrictions not expressed by I. For example, 0 E Q no matter how we choose I, so non emptiness of admissible states cannot be expressed as a local restriction.

These three fundamental assumptions exhaust the basic frame­ work used in this paper. We now fill in the details of D, I, and,$ to characterize our particular mental society.

§4. While we do not require that subat;cnis be completely repre­ scut.ational, or that they employ any particular system of representation if t.hey arc completely representational, we do J"cquirc a few minimal capabilities with which subagents can discuss each other. Our first particular constitutive assumpt.i.on is that the stale components of sub­ agents can he further dec:omposed into "contents" indexeu by the sub­ agent. l"orrnally, for each subagent a E A, we assume a set C. such that Da = {a} X C.. We further facilitate mutual reference by ad­ mitting subagents as possible contents, that is, A c:;:; C. for each a E A. To simplify matters, we assume that all contcut sets are the same set C, and pretend that every content is a (possibly t.rivial) ·subagent by assuming A = C. These simplifications are innocuous since we can always rule out senseless clements by giving them the empty inter­ pretation I(d) = 0 which prevents their inClusion in any admissible stale. With these simplifications, we have D = C X C, and read (a, b) ED as subagent a making the (possibly trivial) statement b. (We say "statement" here for want of a bct,ter term. Contents of subagents are statements only when C is a language, which we do not require.) For each stale 8 E j, we fino out what. statements subagent a makes by means of the projection or perspective operator

With this minimnl notion of statements by subagents, we can describe the vocabularies of multiple perspectives, reasoned assump­ tions, and v.irtual copies. We introduce these vocabularies in turn by means of abstract syntax functions, We also introdur.e separate inter­ pretations It, I2, and I3 for elements expressed in these ~ocabularies and define I = It n I2 n I3, so that when more than one of these inter­ pretations applies to a singie clement, their intersection is the complete interpretation of the element,

35. The vocabulary of multiple perspectives is captured with three ;yntactic constructors on the sci of contents. We assume the existcnc;, of functions (,) [enlargeu parentheses], (, ), and ~ from C X C into C, so that for every a,b E C, we have (a,b) E C, (a,b) E C, and a~bE C.

The (,) constructor permits subagents to discuss the contents of other subagents, where we read (a, (b, c)) E D as the statement made by a that subagent b makes the statement c. Since the const.ructor m<>y be iterated, we can construct even more complex statements, such as

whose reading is left as an exercise.

The(,) constructor is the dual of(,), and produces names for the multitude of relative perspectives. That is, we read ((a, b), c) as the 3tatement c made by the subagent corresponding to a's view of b. The corresponding reading exercise for this constructor is

We make no assumptions of correctness or completeness ~bout the "views" held by subagents about other subagents. That is, we allow an admissible state S to contain (a, (b, c)) even if (b, c) f/:. S, and to contain (b,c) even if (a,(b,c)) f/:. S. We leave pursuit of constitutive assumptions like correctness and completeness to future work. The only requirement we make is the lnt.ended connection between the dual constructor functi~ns. This we exprrns with the interpretation function I1 by requiring, for every a, b, c E C, and

These interpretations have the consequence that

which makes formulation of reasoned assumptions and virtual copies much easier.

Subagents usc the ~ constructor to specify coreferences. We read (a, b ~c) as a's thought that to it, band c mean the same, Thus in a's view, every statement made by b will also be rnadc by c, and vice versa. We capture this formally by defining, for eve.·y a, b, c E C,

I'Ve complete the definition of 11 by defining lt(e) = P D for every ' E D not covered above.

While the corcferencc constructor allows subagents to relate lome of their own subpcrspectives, it cannot be used to relate "top­ level" subagents. Since we require that every domain clement belong to at least one subagent, every coreference statement must occur within ·orne subagent, and hence only relate its subperspectives. That is, (a, b~c) relates the perspcct.ives of (a, b) and (a, c), not those of band c. If our society is not to be a crowd of sleepwalkers, each unrelated to the others no matter bow it dreams it is related, there must, be connections between the subagents expressed either in I (which we uo not do here) or in the(,) constructor. That is, we c<>n read (a, b) = cas a's reference to c by means of b. Since the (,) constructor is defined, along with D, independent of the element interpretations, all such references arc "hard-wired" into the agent's realization, and cannot be changed by any action of the agent. We do not ~equire that (,) be 1-1, and this allows us to "wire together" subagents by defining common references, for example by defining the constructor so that (b, a) =a= (c, a), in which b and c can communicate and otherwise influence each other through a. If we imagine the human mind described in this way, (,) reflects the actual neural connections in the br_ain, while stated coreference relations using ~ simply reflect the decisions of mental subagents.

We could of course introduce modifiable references by incor­ porating the (,) table into states. To do this, we need only redefine D to be C 2 U C 3 , where elements (a, b) E C 2 are as hdorc, and elements (a, b, c) E C 3 indicate entries in the constructor table, specifically, (a, b, c) E S means that (a, b) = c in S. We require that (,) be single­ valued (but not necessarily complete) with the mouified interpretation function

We give subagents the capability to specify references by means of a constructor ~ fror.; C 2 to c, where (a, b~ c) is a's (ostensive) decision to use b to refer to c. This is formalized with the interpretation

Of course, we can get ostensive coreference from reference by using (a, b~d) and (a, c~d) instead of (a, b~c). However, to keep the rest of the discussion as simple as possible, we forgo modifiable references for our original definitions, and leave recasting the subsequent definition~ in terms of modifiable references as an easy exercise for the reader.

§6. The vor.:cbulary of reasoned :.~.>sumptious is captured with a single syntadic constructor of so-called finite simple reasons (sec [DOYLE 1982)). This constructor encodes each three finite subsets A, B , C <::;;; C as a single clement of C written A II B If- C. We read (d,A II B If­ C) E D as d's cou11nitmcnt to make every statement in C whenr:ver it also makes every statement in A and none of those in 13. Formally, we delinc for every finite A, fJ, C <::;;; C and d E C

and l2(e) = P D for every other e E D. (W means the set-complement of U.) Note tha t clements of the form (d, 0 II 0 If- A) correspond roughly to MINSKY'S "K-nodm." Combining simple reasons with men­ t ioned perspectives a llows phrasing versions of STALLMAN'S [1981] "inter-theory inference rules," for instance

§7. The vocabulary of .virtual copies is captured in seven syntactic constructors, each of whi ch represents an "i11dircct refcrcnccn version of the simple reason constructor. Spcr.ilically, we may substitute an indirect reference to the contents of a single subagent for one or more of the concrete sets mentioned by simple reasons. We indicate indirect reference to the content. of sub".gent a by the notation @a, so our constructors range from @a II B If- C to @a II @b If- @c. The usual notion of simple structure sharing is then captured in elements like (b, (/) 11011- @") ,which we interpret as b's commitment to make every statement it thinks a makes. Formally, we define h so that

• §8. These defini tions exhaust the vocabulary and interprc t,.tions of our simple society. We define l(d) = J1(d)n l2(d) n l3(d) for every dE D and take ) = Q. Although very brief, these con·truclions capture a su rprisingly large part of the structure of important representa tional systems. We have left C unspecified, and some well-known repre­ sentational systems can be captured largely as specific choices for C . For example, if we choose C to be the set of all LISP S-expressions, the sor.icty bear. close resernhlance to SUSSMAN and STilEL~;'s constraint language. In this case, subpcrspcctives correspond to the "parts" of constraints , or at t.hc very deepest levels of embedding, the "values" of cells. The constraint language system also involves further restrictions on admissible staLe3, such as closure under solutions to sP.ts of equa­ tions between rational functions, but we avoid formalizing those here. A13 another example, if we choose C to be the set. of sentences in a first­ order logical language and adopt the modifiable reference definitions discussed earlier, the society bears close resemblance to the variant of WEYIIRAUCII'S FOL system used in tDoYLE 1980] . In this case, sub­ agents and perspectives correspond to "theories" and "subtheorics," and =l corresponds to "semantic attachment." WgYHHAUCII'S sys­ tem also incorporates a simplifier, evaluator, and automatic reflection mechanism, but we avoid pur~uing these here as well.

~ 0. Our task is not yet douc, however , for we have not yet formul ated the "virtual" sense of virtual copies. lf S E ) , the above interpreta­ tions ensure that S contai ns a ll condu,;ions sanctioned by reasons and by structure sharing relations. That is, if (b, 0 II 0 If- A) E S, then A <::;;; Jib(S), and if (b , 0 II 0 If- @a) E S, then P(b,.j(S) <::;;; Pb(S). If the agent must realize a ll these elements in permanent storage, we have not achieved any economy of storage, even though we may have achieved economy in writing the information in the first place.

Similarly , admissible states contain all rewritings of all their clements in all equivalent perspectives. Since these agree in informa­ tion, explicit realization in storage is uneconomical. One might also worry that admissible states rnust he infinite, but that is·not so. Even if the constructor functions forceD to be infinite, admi ssible s tates need not be infinite since we do not require tha t subagents be complete in their knowledge of each other . This means t.h a t most perspectives may be void, indeed, that finite states contain only finitely many nonempty perspectives. This situation is altered if we employ the sentences of a logical·language for C and further requ ire perspectives to he deduc­ tively closed, but we avoid those assumptions here.

We captme the motivations of virtual copies and virtual per­ spectives with the notious of extensions and admissible ex tensions. Suppose the agent only records some set S <::;;; D in storage. S need not be admissible itself if the agent in terprets it as tl·e "kernel" information from which to reconstruct a "full" admissible state. That is, if the agent . needs to cheek the presence of some clement in its state, it rc<:onstructs the full state from S, checks for the clement, remembers the answer, nnd then reclaims all storage except that used by S itself. We define Bxts(S) , the extensions of S, by Exts(S) '= { E E ) I S <::;;; B}. We require that the full state reconstructed from S must be some E E Exts(S).

Unfortunately, extensions of Scan contai n, in addition to the ~is,; i ng clcm<'nt" virtually specified by 8, clemcilts complct.cly unrc­ lat.ed t.o Lite kernel specifications. To sec this, con~id c r an analogous situation from logic. We may choose to economize storage in a logically structured agent hy choosing and explicitly storing some axiomatiza­ tion A of a deductively closed set S, that is, by picking A so that 8 = Th(A). In such an agent, deductively closed supersets correspond to ex tensions. But to reconstruct the initial set S from A, we cannot simply pick any deductively closed superset of A, since S is the small­ est of these, and larger ones will contain extra unintended axioms and their consequences. To avoid the correspond ing problem in our society, we ir1t.rouuce the- notion of admissible exten.·ion. We s"y that E is an admissible extension of S, writ ten E E AExts(S) , if and only if E E F:xt.,(S) and E is jinit~ly grounded inS. By finitely grounded we mean that, for every e E E there is a finite sequence u of elements of E (" "proof" of e in H) such that e E u and every element of u is either in S or is a required consequence of some elements appearing earlier in a . Thus if a; = (a, (b,c)), ei ther (1) a; E S; or (2) for some j < i, a; = ((a,b),c); or (3) for some j < i, a; = (a,A II B If- C) where (b, c) E C and for every d E A , (a, rL) precedes a; in a and for every d E B, (a, d) ~ E; etc. We avoid presenting the full definition here, "" it is not uifficult to construct, merely tedious. A detailed develop­ men t for the sp<!cial case of finite simple reasons alone can be found in [DOYLF. 1982] . Since states contain rell."3oned assumptions, there may be several sets of assumptions possible in the kernel set S, and these can lead to distinct admissible extensions. For example, if just (a,0 II {b} If- {c}) and (a,0 II {c} If- {b}) are inS, then there will be two finitely grounded extensio·ns, one containing (a, c) but not (a, b) , and the other containing (a, b) but not (a, c).

While we here accept finitely grounded <:xtcn~ions as admis­ sible <;xteusions, they are inadequate to fully r.apturc the usual notion of virtual copy. In current practice, it is cr ucial that successive quedes agree, that is, thai virtual iurormation is conserved acro:-~s reconstruc­ tions. But this cannot be guaranteed with multiple admissible exten­ sions, since the agen t might for one query construct E and next time construct E' ~ E, differing in some answers even though no kernel information has changed. Thus the virtual state is conserved only if the agent computes a unique admissible extension. TOURWfZKY [1983] is currently developing restrictions on the sorts of information states can contain, restrictions designed to guarantee the existence and uniqueness of admissible extensions. He also motivates the aim of uniquene,;.' by sceki ug pa rallel algorithms for reconstructing the virtual clements, and requiring that concurrent processes computing subsets of the virtual clements agree on their overlap. TOURETZKY'S dis­ coveries notwithstanding, I doubt that completely unoffensivc restric­ tions on the vocabulary of the agent cau alone guarantee uniqueness of finitely grounded extensions. 1 suspect that some appl ications demand a vocabulary sufficient to phrase ambiguities, and for these one appeal­ ing approach is t.o make the reco-nstruction algorithm, whether con­ currently or serially realized,. be a probabilistic algorithm. That is, when an ambiguity arises during reconstruction, the algorithm makes a random choice (random, not arbitrary) . The intent of such deliberate randomization is to make every possible reconstruction equally likely or to occur with ~orne specilied frequency. If the agent wishes to judge its certainty on some question, it asb that question repeated ly. Questions with answers common to or absent from a ll adinissible extensions never vary in their answer, while other queries exhibit uncertainty, wallling in response over time. If the alternative admissible extensions do not differ greatly, then most answers will be the same anyway no matter which admissible extension is chosen. [DOYLE 1982] develops a theory of subjective probability by measuring the rel..'ltive frequencies of different answers, but we cannot go into that here.

§10. Even if the ambiguities of admissible extensions are resolved, am· biguities due to multiple perspectives remain. MINSKY and others have suggested that some abrupt changes in human behaviors and attitudes stem from changes in which subagent has control as "spokesman" over the communication or motor channels to the external world. In that view, there is no fixed notion of "self," but a different sense of self depending on which subagent gains control. One advantage of that view is that abrupt changes of attitudes arc computationally trivial, for they stern from switching vantage points rather than from laborious revision of the state itself. The framework proposed here facilitates consideration of such proposals. For example, a natural problem is that of formulating precise notions of "abrupt" changes. If we ded<le when perspectives of different subagents are "similar," we can allow wide variations in which subagent is currently "self" as long as most of the self-image is conserved across self-changes, and single out as abrupt those changes of self which bring large or significant changes in the self-perspective. That is, if p4 (S) and P&(S) are. very similar, say if p 4 (8) and P&(S) differ by no more than 7 (±2) elements, we might say ~t no major self-changes, only changes of attention, arc involved in switches between a-self and b-self. Indeed, if the probabilistic approach to ambiguities of interpretation is adopted, then one need make no special provision for ambiguities due to self-changes. Can we develop measures of similarity on ·both states and perspectives so that if S and S' are similar, so arc p4 (S) and p4 (S'), or vice versa? Unfortunately, we cannot pursue such questions here.

§ ll. There arc many other possibilities to be explored in introducing notions of self into soeieties such as this . In [DOYLE 1980]1 advocated distinguishing a particular subagent, called ME, as the self. (I arn less committ<'d to tha t approach now.) When compared to the free­ floating approach just discussed, the usc of a fLXcd self-subagent appears to require significant computational costs for subst,antial perspective changes. (Uut sec [MCDERMOTT 1982] and [MARTINS 1983].) In any event, distinguished perspectives sti ll merit consideration, for it may be easier to endow them with limited completeness and correct­ ness properties than amorphous agents. Specifically, one of the intents of my earlier proposal was to have the subagent ME he the authority on just what the state contained. The idea here is to construct the agent so that (modifiable references or not) (ME, a) = a for every a E C (including ME itself) , if that is possible. I suspect it is not too difficult to achieve, and such organir.ations have obvious attractions for constructing agents possessing reflective powers. To pursue this idea, if one perspective admits the limited self-omniscience described above, docs it follow that all do? That is, does global self-omniscience follow from local self-omniscience? I suspect not, but have no coun­ terexample. H also seems certain that cliiTercnt perspectives can differ arbitrarily much even if both are mutually omniscient. If the subagents a ll use a logical language as a system of representation, well-known results indicate general limits to self-omniscience, but which sorts of limited sel f-knowledge can be introduced without difficulties arising! KRIPKE'S analysis of truth indicates that even seemingly innocuous statements of mutual knowledge can in concert produce unrcwncilable paradoxes. Since his theory involves a notion of groundedncss resem­ bling our notion of grounded extension, similar result.s seem likely here. Unfortunately, we must leave these questions for future study.

ACKNOWLEDGEMENTS

I thank JOSEPH SCHATZ for advice, and IJEN COHEN for reading a distant ancestor of this paper.

RrmmENCI~S

1. Bobrow, D. C., and Winograd, T., 1977. An overview of KllL, a knowledge rPprescnLaLion language, Cognitive Science l, 3-16.
2. Ooyle, J., 1980. A rnodd for deliberation, aclion, and introspec­ tion, Carnbrirlge: MIT Artincial Intelligence Laboratory, TR-581 (Ph.D. thesis).
3. Doyle, J. , 1982. Some theories of reasoned ;LSsllmpLions: an essay in rational psychology, l'i ·tsbu rgh: Depe rtment of C"rnpu ter Science, Carnegie-Mellon University.
4. Doyle, J., I !J83a. Methodological simplicity in expert system const.ruclion: the case of judgments and rea:;oned "ssumptions, AI Magazine, to appear.
5. Doyle, .J., 1083b. Admissible staLe semantics for representational systems, iEEE Computer, to appear.
6. F'ahlman, S. L~., 1979. NETL: A System for Represwting and Using Real World Knowledge, Cambridge: MIT Press.
7. '/. Kripke, S. A., 1975. Outline of a theory of truth, Journal oJ Philosophy 72, 690-716.
8. McDermott, D., 1982. ConLcxts and data-dependencies: a syn­ thesis, Department of Computer Scienre, Yale University.
9. Martins, J.P., 1983. Reasoning in multiple belief spaces, Ph.D. thesis, Department of Computer Sci~nce, State University of New York at Buffalo.
10. Minsky, M., 1965. Matter, mind, and models, Proc. IFIP Congress, 45-49.
11. Minsky, M., 1975. A framework for representing knowledge, The Psychology of Computer Vision (P. Winston, ed.), New York: McCraw-llill . Appendix in MIT AI Laboratory Memo 306.
12. Minsky, M., 1980. K-ljnes: a theory of memory, Cognitive Science 4, 117-133.
13. Stallman, R. M., 1981. A new way of representing defaults, Cambridge: Massar.huscLts Institute of Technology (unpublished manu~cript).
14. Sussman, G. J ., and G. L. Steele Jr., 1980. CONSTRAINTS- A language for expressing almost-hierarchical descriptions, Artifi· cia/ Intelligence 14, 1-39.
15. Touretzky, D., 1983. Multiple inheritance and exceptions, Pitts· burgh: Department of Computer Science, Carnegie-Mellon Uni· versity.
16. Weyhrauch, R. W., 1980. Prolegomena to a theory of rnecha­ niwd formal reasoning, A rtific:ial lntc/ligence 13, 133-170.