An Approach to Medical Diagnosis Based on Conceptual Structures

AN Al'PROACH TO MEDICAL DIAG!WSIS BASED ON CONCEPTUAL STRUCTURES

B. Chandrasekaran, F. Gomez, S. Mi ttal, and J. Smith, M.D. Artificial Intellip,ence Group Department of Computer and Information Science The Ohio State University

Columbus, OH 43210

The power of an experienced medical diaanostician seens to us to arise from a knowlada· oraantzat ion whosE: very structure helps keep the combinatorial growth of processing under control. Thi· ·tructure mirrora the deep conceptual structure of the field. We propose a knowledge representation achPme called concaptual ~tru~tures, which ~rganj,~ea_ knowled&e J n the form of product ion rules or nore complex procedures l.n Ruch a way that they are acceased and uaed as needed. This structure is doninantly hierarchical, the successors of a conceptual node stand in& for aubconcepta which refine that concept. Associated with each concept is a aet of procedures (experts) which attempt to apply the relevant knowledge to the case at hand; this might include turninc over control tr aelected aubconcepts for more detn1lec' anAlysis. Thus, ahstractly, the crnce!ltuAl structure organizes th· invocation of procedures available to the diar,nosticinn. ThP calling· C'f " specinliAt by 8 physician is akin, we think, tC' control transfer to a subconcept in the Cc>f.nitive structure of a diRr,nosticisn durin· problem aolvinr.; the difference is in thP. level of det111l of the corritive m:~p. In thiR pAper · so!"e principles auvernint the deRi~;n of conceptual structures are presPntPc. l?e :~pply the i ~en11 to the dct<ign of t:IJX, a system to perform di~tgnoses in a syndrone called CholeBtasis 1 anc~ rliscu,;s the perfornAnce of the system.

1,0 lN'fRODUCTIUN

In this paper we shall describe an approach to the design of systems to solve problems in knowledge-rich domains. We shall consider in particular a system intended to perform medical diaanosis related to a liver .yndrome na~oted Cholestaais. The underlying metaphor is one of a society of experts amona whom the knowledge of the don~in und the procedure· for actin& out that knowleuke are distributed. The organization of knowled11e aa experts raisea n1any important questions, such aa what knowled&e should be organized into an expert and what other knowledae dtatributed het··een them; the nature of control tranafar and communication between experts; the data-base organization and access; and the relation between redundancy of knowledge and data on one hand and the speed of processi n& on the other. While the idea of experts is not new, we believe that the epistemological principles needed as criteria for decomposing a body of knowledge into experts, in order that their interaction is controlled rather than chaotic, have been missing. Our discusaion in the next few sections will attempt to highlight some of these principles.

·n,e paper is orljanized as follows. In Sect ion 2, ""' Jiacuss how concevtual ltructures effectively organize a body oi knowla<.i&" into experts. In Section J "" diseuse, for purpose~ of illustration, the knowledre representation of a t:ragment of Cholestflsis. This leads, in Section 4, to a discussion of different types o1 experts and th.,ir structural repres.,ntation. In Section 5, a r·rototype l'ledical diagnosis systcl'l basetl on these principles ia described. The description is accompanied by an illustration of the syat"m diSf_nosinr a real medical ca11e, Other contcntporsry approaches to nadical diagnosis using Artificial Intelliccncc techniques are discuaseu briefly in Section 6. In the concluuing section, we outline sone exten,;ions that are conteutplatcti tor the near 1uture.

2.0 fQNCEP'l'UAL STRUCTURES AND ORGANIZATION OF ~DICAL K!I_':!_W~_!::lJ�E

we adopt the view that a g iven field of knowledge has an

unaerlyina conceptual _s tructure which enables effective uae of that body of knowledge. This is the deer structure to which the knowledge representation in the fieltl ahould Htrive, and this is what effectlv" teachers communicate and effective learners build. This conceptual atructure can be viewed as a way of oq,anizin& knowledge in "uch a way that purposeful, focuased acceas to the relevant parte of the knowledg~ structure is achieved. Consider medical diagnoals as an example. It i s our view that the power of an experienced medical diagnostician arises from a knowled11e oraanization whose very st ructure helps keep the con~inatorial growth of processing under control. Knowled11e in the form of facts, heuristics, production rules or r.tore complex procedures cannot be effectively used unleas it is empedde~ in this underlying structure in s uch a way as to facilitate access as needed.

What fire the principle· by which domains of knowledge are to be organized so that they can be used effectively? Our theais is that the major criterion has to do with the efficiency of transfers of control or access to di f ferent parts of the knowledge ntructure. There are correct and incorrect analyses of the conceptual structure of a field. Incorrect analyses l ead to a proliferation of control transfers; correct ones ahould generally lead to f ocusHed and controlled con®unication between di fferent parts of the structure.

The organization of the medical community provides a concrete caae study in the principles of knowledge structurin&. A phenomenological account of medical decisi on making s eema to us to be very revealing in thi s re!i,ard, One of our guiding notions is that there is a correapondence between the broad conceptual organization in a diaanoatician and the organization of specialties in a medical community. The calling of a specialist by a physician is not dissimilar, in our view, to control baing handed over to a subconcept in the cognitive structu=e o~ & diaanostician durinK the course of his problem sJ:ving. Thus this is a concrete realization of the !.QS!_.g_/ £.f minds paradigm that has been put forward by Minsky and Papert[l], In order for this paradigm to 10 beyond being a metaphor, we need to know how a IIOCiety of minds, ~ in .an. individual or as a collection .QJ. experts :..QW..tl£..!!. ~ p..I:Ql!.lru!!, ~h~id coordinate the activities of and communication among membera to produce intelligent behavior, We feel that the organization of specialties in the medical community reflect& the underlying conceptual struclure, and, to a certain extent, the requirements of efficiency in control and communication structures as well*, The same principle& that lead to effective communication among real experts in a medical col!l1lunity are to be used in organizing and structuring knowledge within a specialty. A aimplified rendering of these principles are:

• Pl) There U some sort of a hierarchical organization with respect to control tra~fer. That is, the GPwho had original reaponsibility might, after deciding that it ia a liver problem, tranfer control to the liver specialist, The liver specialist might in some cases hand over control to a super specialist in a subarea, but it would be highly unusual for him to transfer the patient to a heart specialist. If such a thing is aver done, it'd be usually through return of control to the GP who would then accomplish such a transfer.
• P2) The expert who is contemplating control transfer knowe enough about the domain of knowledge of the aubexperta who are likely to be called by him to decide, generally correctly, which subexpert is relevant, but not enough to solve the problem himself.
• P.3) each expert knows enough about the domain of others in his level of the hierarchy to decide when he himself haa been mistakenly called.
• P4) There are acme experts who do not belong in the main hierarchy, but who are outside of it, so to speak. They are not generally given control for the purpose of diagnosing the original complaint. They are used aa reaource experts to accomplish specific subtasks requiring their expertise. They communicate back to the expert who called them with the answer. An example of thia type of expert is the radiologist.
• P5) The communication between an expert and a subexpert ia of aeveral forma: answer specific questions

* It might be argued that medical specialties historically grew up around organ systems, i.e., along physiological and anatomical lines, and that control and communication efficiency is not a paramount criterion. In fact all this means is that the criterion of control transfer and communication efficiency is often met by organizing many apecialties along these linea- after all one would expect that there would be a certain coherence within each auch specialty. Further, not all medical apacialties are organ-based; consider rheumatology, radiology, emergency room medicine, clinical pathology, toxicology and infectious diseases as counter examples, ()!r thesis is that the intuitive notion of internal coherence of a specialty is closely related to the control tranafer· and communication efficiency, and thus the latter becomes the concrete criterion of "natural ·lsction" in the evolution of specialties.

dealing with the subspecialty, confirm or reject possibilities, or, in difficult cases, hand over total control, Further, in some instances a mere number atanding for weight or probability is returned (see P7 below); in most cases, however, the communication is symbolic in nature, in a vocabulary which, in parts, is particular to the specialty subject matter.

• P6) An expert often has enough knowledge of the subject utter of the subexperts to be able to solve easier caaes. For example, most GP's know enough about liver diseaaes to solve "easy" liver cases, enou~h about heart diseases to solve easy heart cases,etc. Amon~ the criteria that determine which knowledge should be abatracted at this level are: how common is this kind of malfunction, how conceptually complex is this etc, Thia aort of organization keeps the cornbinatorics of control transfer to a min1mun, since only in a ralati valy few cases will there be a large nwnber of accesses to subexperts.
• P7) While many significant decisions involve manipulating symbol structures associated with concepts, there are very definite situations which are best viewed as weighting several pieces of evidence to produce a numerical weight for a hypothesis. This piece of information is eventually incorporated in aymbolic structures which are used for problem aolving,

~plying these principles to organization of knowled~ \~!thin a specialty such as liver disease or Cholestasis, we have found that the conceptual structure can be uaefully viewed as a tree, where a node stands for a particular concept and the successors of the node stand fOr subconcepts that help refine that concept; e.g., hepatitis, whose successors might be acute, fulr.inant and chronic types of hepatitis. Associated with each node is a set of procedures which decide on the applicability of the concept to the case at hand. Part of auch decision-making in a node is often the decision to turn the control over to subconcepts and their aaaociated procedures to check on their aprlicability, In this aense, the conceptual structure organizes the invocation of procedures available to the diagnostician.

Notice that the form of the structure as a tree already exercises a stron~ constraint over cocmunication. An sxpert cannot call other experts arbitrarily, but can only conrnunicate via their super anll subconcepts. 'l.'hile it il l~rd at this stage to advance this principle as an absolute requirement, we have found that violations indicate either that the analysis was incorrect or that the concept actually belongs outs ide the structure in some sense, ~ith specific retricted modes of communication with the experts in the main structure. Thia 1a consistent with p~inciple 4 above.

We note in passinc that the principles governinc the organization of conceptual structures are meant not only to make effective ~ of knowledr,e, but also to aid in the acquisition of it. The concepts a;;et refined as a function of further learninr,, and the node at which the refinement takes place gets appropriate abstractions about the subconcepts, Another point to be noted is that often heuristics are formed which enable the system to go directly to a concept several levels below, sidesteppina;; the methodical, level by level communication. But concentration on this second order optimization device obscures the clarity of the. underlying structure. In fact expertise often consists of an accumulation of such heuristics, and, for this teaaon, experta tend to be poor elucidators of the undarlyin& structure, These, however, are issues which are not the lltain concern of this paper, and will be d1ecuesed in detail elsewhere, In the next section, we illuetrate the usefulness of the lbove ideaa by the concrete example of a fragment of the chole1t11is aystem, While we have performed the analysis for choleatasis in general, we present here anly a port~on thereof, in order to concentrate on the principlel.

J,O CONCEPT'.:AL SnllcrURE OF CHOLESTASIS

Cllole1tasis is a condition caused by disruption of the 18cretion of bile or of its excretion into the duodenum. It happena that ··hile the identificatj.on of the ceuae of choleataeis mi&ht be co~plicated, the determinatioc that it is present or not present can be accompiished with relative ea1e, This fact makea the cholestasis knovledae b81e a u1eful object of study: it ·:r,s large cough to be an adequate arena in which to ·~;est the ideaa, and at tha sa=e time, its interaction with knovled&e basel concerned with other diseases can be kept under 'control becauee of the relative certainty of 1t1 applicability . (We call node· corresponding to oe~ncepta with the latter property "anchor nodes,") It hal been found useful to group the causesof cholestaaia a1 occurrin' within the liver itaelf(intra-hepatic) or in tha bili·ry tree cutside the liver (extra-hepatic), there are often broad indications of whether a case is extra- or intra-hepatic, which, while not 100~ reliable, U6 aenerally helpful in decidin& >~hich is mora likely* and thua ahould ba investi&ated first.

W8 ahall shortly aive the details of Procedure Clloleataais, which >~ill be repreaentative of the procedurea in molt of these conceptual nodes. But first we need to refine the concept of Extra-hepatic Clloleataais in some detail. The movement of bile throuah the biliary tree can be obstructed by a physical object such aa stone, stricture or cancer of varioua kind· or by swelling cauaed by inflammation due to a ll.lmber of cauaea, Aasociated with eaeh subconcept are proceourea indicating how to establiah it by sy~~toms, ld&nl and lab data; information about dectdina which of their subconcepts are applicable, etc. In accor.dance with the principle& aet forth earlier, our analysis of Choleata$is recommend· the conce?tual structure in Fig . 1, Becauae of the procedure· attached to each of these nodea, the tree can be viewed as a structured oraanization of experts involved in diagnosing Cholestaais. The creation of such a hierarchy is the teault of the analysis of the content of a field of knowledae. The dif!i~ultles involved in the uncoverin3 of such a atructure can be illustrated by considerin~ tha "atone" and "inflam'Uat ion" (cholangitis) nodes. Taxtbooka on liver diseases inform us that infl8l:ll!lation may be cauaed by the preaence of stones and, conversely, ltonea may be formed if there is persistent inflau:maticn, A direct render in& of this information would result in tha mutual callin& of these two experts, ~olatin& the hierarchical ~xpert call structure. The difficulty h cle'ired by notin& that thene two nodes do not correspond to the general concepts of "biliary atone" and "bile duct inflammation." Inatead, they stand

* 1 'Likely" here does not mean that probabilities are computed; rather, certain aspects of the problem daacription are looked at for information ~out which ahould be ~iven priority in investigation, for "stone aa cause of Chole1taais" and "inflanmation as cau1a of Cholestaais." The "stone" node then has the information ebout "inflam:nation" needed for its work localized there (see III in Proc<·dure Stone-Consistency 1n 1ec ,4 .4) and similarly for the "inflammation" node . It 1aema to us that such a decouplina in knowledae oraanization ~s easential for cperational efficiency .

4,0 ~ SOLVING BY EXPERTS

4.1 Trpes of Experts

Thera are two types of experts - characterized by the control they exert over the problem solving process . Experts like Cholestasis, EY.tra-hep, Intra-hep, etc . , play &reater role in decision making. They cecide, for ax811tple 0 which expert to call next; what sub-problemo to 1olve; decide it the .problem is solved; mediate between competin& advice from subconcepts (in differential diagnosis situations), etc , The other claaa of experts, such as Stone, Stricture, etc . , primarily evaluate the problem-data to dete~ine how vall the concepts they represent fit the data and to say what additional data will enable a batter decision.

Coacepts auch aa Stone are tip nodes in the concert tree (primitive concepts). Deciding if t he presence of atonea characterizes the case is done mainly by weighin& different pieces of evidence, both pro and con, into a whole.* This does not entail any search t hrou~rh a srace of hypotheses, On the other hand, nodes higher in the

* Thi1 ia not to say that numbers in this situation, human diagnosticians use The important point is that tree need to aearch below in order to fulfill their teak, It often happens, however, that a node which was incorrectly analyzed as a tip node ia indeed a concept which ia aufficiently co~plex that further refining is oece .. ary,

4,2 Problem Solving Strategy

'.lhe basic atrategy in problem aolving is one of 81tabliahing and refining the concepts, Experts .. sociated with the concepts actually carry out these proceases; i.e., there ia no unifor~ mechanis~ ~eratin& on different concepts for this purpose, This 1a very important, as each concept may have different koowledse and thus associated atrategies for establishment and refinement,

Some nonprimitive concepts are anchoring concepts, because they can be established with a high degree of cartainty. In pathognomonic aituations, they can be eatablished by one datum or one set of data, but geoerally they are established by accumulatin~ evidence from a small n=ber of highly reliable syn11>torns/lab data, Once established, the concept is ~ by the process of dete~ining which of the subconcepts, alone or in combination, fit the data which this particular CDncept ·was originally asked to explain, These ~achorin& concepts provide a strong global focus to the problem-solving process, Control is syster.atically distributed to other experts so that each may solve a part of the refinement prob l.em. The expert which provides the focus also retains the control over the problem-solving process, thus largely eliminating the "focus of attention" problem."'

How to handle additional data while the diagnosis is in pro&ress is an issue in any diagnosis syste~. Our system is designed to begin with some initial data and ek for more, when needed. The new data do not prompt IPY costly computations or drastic change of focus; instead, the controlline expert uses them to re-compute only what was affected within its do~in. As control is puaed back and forth, each expert deternines the relevance of the data and aces accordingly but only when it seta the control.

How the above problem solving strategy is ·~bodied in the aystem can be best understood by considerin~ sorne ecamples of experts, ~:e shall consider Procedure Ololestasis and rrocedure Stone -consistency as examples of non-tip and tip node experts respectively, Cholestasis is also an anchor node and thus provides a i!POd illustration of how control is carefully distributed with a clear focus,

4.3 Procedure Cholestasis

1. CllOLESTASIS has been handed control and asked to firs t ESTABLISH itself and then IU:FlNE the concept it

eycbolic te~plate matching of some kind is needed. In Proceduru Stonu-Consistency that follows, the nur:bers really stand for diacretized 0 symbolized degrees of evidence,

• Reference [ 2 J discusses the focus of at tent ion problems that arise in IIL::ARSAY-II, which, even though it is not a medical diabnosis project, is of interest due to its experts-based orr.anization, Our additional principles tor const raining control and communication seem to make the difference,

embodies,

• 1, Eatabliah choleatasia on hiatorical data, aigns, aymptoms, an11 clinical data, Thia portion is a combination of heuristic rules and numerical weights. (Thia ia atrai&htforward and can ba done quite rdiably,)
• 2, Conaider first the disease-hypotheses invoked by patient data in INVOKED-co~c list and made available to CHOLESTASIS by the calling expert. (As a doctor scans through the data, some possibilities co~e to mind, Thia is the typical heuristic knowledge of ~ost doctora about cholestatic diseases, However these,!!:! .. rely ·uggestions to be evaluated,)
• a, Generate calla to immediate sub-concepts of CHOU:STASIS, which are the auperconcepts of those in the liat. For exa~ple, if IN\'OKED-cONC contains SCL-cUOLA!lGITIS, BD-CANC£R 0 PERICliOLAI.~�ITIS and PRlK-BIL-ciRRHOSIS, these calls would be ---

Call txTRA-liEl' (consider SCL-chOL 0 BD-c&-.CER)

Call INTRA-liEP (consider PERICHOL, PRIM-BIL-ciR)

(The eXperts corresponding to these eventually get control and they evaluate data more carefully and return four liata: CONSISTE~IT-DATA, COiiTRADICTillG-DATA, 1:\ECONSIDERATION- DATA, and kJ:;COMHE.NDATIOU, which together consist of weir.hted recommendation anu reasons, Note that CHOLESTASIS kDowa which aubconcepts to call, even though it doaan't know enough to make a decision about them. Tbia ia consistent with P2,)

• b, Place members of INVOKED-coNe into three lists: ACTIVL, INACTIVE and ~IKELY. INACTIVE holds those for which data are insufficient to decide one way or the other,
• Check if concepts in ACTIVE list can explain all ABNORM-DATA (use CONSISTENT-, CONTRAI>ICTING-DATA lists for thia),
• io Yea, also no ai.nificant overlap, return ACTIVE liat as answer (strai&htforward cases will be solved at this stage).
• No, some data in ABNORM-DATA cannot be explained. Co to DEFAULT-PROC.
• Yea, but overlap. Go to OVERLAP-HANDLE*, return modified ACTIVE list,
• II, DEFAULT-I'ROC INVOKED-cO!IC fails choleatasis,) (Control transfers to explain all here facets when of
• Use data to decide which of EXTRA or INTRA is more likely. (Imagery information is often very effective in this discrimination, Errors here will delay correct diagnosis, but not prevent it.)
• 2, Execute X:XTRA-lN-cllOLI:sTASIS INTRA-IU-cHOLE.SlASIS .acccording to result from 1. solved, return ACTIVE list, Else call the other, that fails go to LOOK-FURTIIER. or If If

III, EXTRA- and INTR.A-IN-cHOLESTASI S

(EXTRA- and IllTRA-IN-OlOLESTAS IS contain some knowleuge about subconcepts that is of a more usual nature and thus helps deal with a large number of cases, Relatively easy cases will be generally handled by this, Again, putting this knowledge here reflect& P2 and ~) Theae consist of a collection of rule-baaed calla of the form:

* 0\il:.RLAr-liA:UJLE. which deals with the notion of two diseases explainin(! substantially the same data, is not currently implemented and will not be further discussed here. Details will be &iven in [3],

If history of acute/chronic cholecystitis, Then Call EXTRA (Consider Stone)

If air in x-ray of biliary tree, Than Call EXTRA (Conaider Stricture)

JY, LOOK-FURTHER

(Control transfers here when everything CHOLESTASIS ~ directly baa been tried, but there is still "dgnificant" data left to be explained.) ~rder EXTRA and INTRA in terma of likelihood (aa in II-1 above) and tranafer control to one, and then the other if necessary, with request to diasnose unexplained abnormal data.

Principle 1 ia implicit in the organization of the canceptual structure, We saw several explicit examples of P2 in I,2a. and II,2 above. This particular example does not involve Pl, Interpretation of ima~in~data will involve calls to the radiology expert, which is not explicitly indicated in the above procedure, but which is an example of P4. The different forma of messages, · indicated by PS, are exemplified throughout the Procedure. DEFAULT-PROC employs P6 partially. This procedure baa enough knowledge about subconcepts to call them for anawers to &pacific questions, but keep the cantrol for eaeier caaea. The aatablishment of cholaateais in I.l ia dona in our implementation partly by waighta aasigned to different aymptom.. This is an aample of P7.

4,4 Procedure Stone-Consistency*

( We show only thatpart of the expert which determines the fit; message and explanation details are not

lhown,) I. CALCULATE X-RAY EVIDENCE If Cholangiogram& available --~ If Stones seen in Biiary-tree than XEV .,._ 3; else XEV..,. -1 .!m.!!. plain-film X-ray available then if stones seen in Biliary-tree ---then XEV .. l; else XEv..-o .J1u XEv..-o -- .It XEV > 2 ~ poaitive consistency evaluation II, CALCULATE HISTORY EVIDENCE This decision is made by asking about atone-related diseases: hemolytic- disease, cholecystitis etc. III, CALCULATE CLINICAL EVIDENO:: This deciaion is made on the basis of the evidence for Chlangitis, colicky-pain, vomitting and nausea in a weighted-sum logic. IV, _CALCULATE SUMMARY EVIDENCE Evidence from 1, 2 and 3 is weighted-sum logic to obtain evaluation. combined in a the consistency 5,0 PERFORMANCE OF MDX - A PROTOTYPE ~ 5.1 ..!!!!.!!!! ..!!! Interaction In thia aection we diacuas the performance of the * Each of these four major deciaions are made by a aaparate group of rules, which are evaluated only when aaeded, ror details aee[J].

prototype diagnosis syatem(~IDX} on a real case. Currently the syatem can only dialJloaa extra-hepatic Cboleatatic syndrome, which consists of about nine dia ... ea represented as terminal nodes. The system is interactive in two aanaea. The user can enter as much data as he wants, and later on the system will ask for additional data, if needed, Interaction of the aacond kind ariaea when an expert which is not yet implemented is nevertheleas invoked by some other expert. In this aituation, if a message ia aant to an expert which is kDown to the system but not implemented, the message is zoutad to the user who can provide answers on behalf of tba unimplemented expert& (see [3) for details). Part of the attraction of expert systems ia that they permit IUCh a modular development in a conceptually natural way. This approach has the additional advantage that tha logic of the system'& decision-making becomes ~latively clear to the physician and he can view the ayatam as a consultant and not aa a competitor. We have apace here to discuss only one case in some detail.

5.2 ~.£!!!

this case ia discussed in [4].0ur diacuasion will be in the form of computer printout fragments (in Gothic) interspersed with commentary. User response to queries during diagnosis is preceded by $ aiBP· In most plac~s, the explanation generated by the system should be IUfficient, Patient data ara first entered in episodal fOra, For this caae the following data are entered for peat medical history and the ~ admitsion episode.

PATIENT RECORD SYST El~

PATIENTS NUMBER IS PAT01 AGE? 39 SEX? H EI~TER HISTORICAL EVIDENCE ? HALOTHANE ? (ULC-COLITIS F) 1 (BIL-SURGERY F) 1 (CHOLECYSTITIS F) 1 E TIME? (AT AO~IISSION) 1 JAUNUICE 1 PRURITUS 1 VOMITTING 1 (WTLOSS (~~UNT (LB 16.0))) 1 (PAIN F ABDOMEN) 1 (LIVER (SIZE iiORIW..)) ? (PAIN LIVER F) 1 (PAW T EPIGASTRIUM MILD) 1 (SPLEEN (FELT F)) 1 (URINE (BILIRUBIN)) 1 (STOOL (BLOOD F)) 1 (BILIRUBIN (DIRECT 10,799998} 12.200000) 1 (PROTEIN 8.1 (ALBUMIN 3.9) (GLOBULIN 4.2)) 1 (SGOT 32) 1 (ALK-PHOS 19.200000) 1 (PT 11.399999 (CONTROL 10.5)) 1 (ESR 16.0) 1 (WBC 9200 (PMN(PER 69.0))(LYIIPHOCYTE (PER 19 .0)) (MONOCYTl (PER 10.0)l (BASOPHIL (PER 2.0)) ? (ULTRASOtlAGRAM (BIL-TREE (IHD DILATE))> ? (PLAIN-FILI-I (LIVER (NORHAL))(DUODENUM (NORMAL)) ) ? (ERCP (BIL-TREt:: (CHEP OBSTRUCTION CONICAL) (CBILE (NORIW..))) (GALL-BLADDER (I~ORMAL))

(PANCREAS (PANC-DUCT (NORMAL)))) ? ~�I-XRAY (DUODEllU~1 (CONPRESSION))) ? LIVER-SCAI~ (LIVER LARGE) (SPLEEN LARGE)) ? PERC-TRAilS-CHOL (BIL-TREE (IHU DILATE)(CHEP NARROW) (LHEP MULTIFOCAL)(RHEP r1ULTIFOCAL))) ? (SPLAN-AGRM1 (LIVER (SURFACE (TU1~0R-BLUSH F) (NEOVASCULATURE F)))) END OF lJATA ENTRY FOR PAT IEiH PATOl

The entry of negative information, such aa (PAIN F ABDOMEN) meanini "no abdoCiinal pz.in", needs ~ome explanation, Much of this !nformation is obtatned by routine physical examination, historical questionnaire md lab teat a. However, information auch aa (ULC-COLITIS F) is not routine; it ~ be avoilable from P"tient hi ,~tory charts, but more often it is extra~ted by questions cr tests when a particular hypothesis is hein& considered. Tte syste~ is capable of aakin& for more data as needed, though that aspect is not our main concern here, We ahall consider the veraion of our implementation in which any data not pven are assumed to be unknown, unless they can be inferred from known data (with the exception of queries anawered by the User for uni~plemented experts). In any caae, the data that are entered here- both positive and aeaative- are those directly taken from the case deacription in the journal. The important point is that there exists a genuine intellectual problem of putting the piece· toaether into a coherent whole even when only zelevant data are entered, The problem of extraneous data is a bit more subtle. Clearly, totally irrelevant data should not cause any problems, and in fact they don't. No data-directed component haa any rules that would fire in this case, and no hypothesis-driven component ~uld look for them. Potentially relevant ~traneoua data are more problematic, but that applies to our system as well as to human diagnosticians. In this case blind alleys are possible, but erroneous diaanoaes usually are not. Space limitations prevent a fuller discuasion of this iasue in thia paper.

5.2.1 The Diagnosis

The topmost expert in the system is r.p. As the data are entered, some rulea are fired, suggesting hypotheses, Iha hypotheses become the starting point for GP to diaanose the case. In this case, rules suggesting Ololeataais and ~~.r_!lt_ive-cho!_e!!_tasis were fired,

BEGIJl DIAGNOSIS FOR PATIENT PATOl TRYI1tG RULES TO INVOKE HYPOTHESIS TRYING RULE: ((( PRESENT?(URINE BILIRUBIN))(NORM? ALK-PHOS AN)) ((l1jVOKE CHOLESTASIS))) SATIS~H:.u EXECUT!flG ACTIONS: ((IIWOKE CHOLESTASIS)) TRYIUG RULE: ( ((HISTORY? ANESTHETICS)) \(INVOKE (POST-OPER ANESTHETICS)))) SATISFIEu EXECUTJiHi ACTIONS: ((INVOKE (PUST-OPER ANESTHETICS))) TRYING RULE: (((DEFORMITY? BIL-TREE ? STONE)) ((INVOKE STONE))) UNSATISFIEIJ:

Other rules involving Pruritua, Bilirubin and Jaundice and invokin& choleataais were also triggered.

IIHTIAL UATA IUDICATES --­ Ct()LESTASIS (PUST-OPtR ANESTHETICS) CHOLESTASIS IS INuiCATEO BY PRURITUS PRESENT .JAUNUICE PRESENT URINE BILl RUB IN NORM OF ALK-PHOS IS (AN) NORM OF BILIRUB Ill IS (AN) RATIO OF DIRECT VALUE VS BILIRUBIN VALUE IS 0.885 (POST-OPER ANESTHETICS) IS INDICATED BY HISTORY OF ANESTHETICS

CHOLESTASIS being the highest anchor node suggested, CJintrol 1a transferred to it with the message to Establish and Kefine itself. The !'less ages and some of the rules* uae~stablishing cholea tseia are shown.

HANLliNG OVER COIHROL TO CHOLESTASIS EXPERT EXPEkT-CALLEO: CHOLESTASIS CALLEIJ·BY: GP MESSAGE (OUERY (ESTABLISH CHOLESTASIS) (REFINE CHOltSTASiS))

Procedure Cholestasis described in an earlier section is naw operational,

TRYING TO ESTABLISH CHOLESTASIS EoHE.IUI~li TABLE LOGIC TRYING RULE: (((DUCT? IHD BIOPSY BILE-STASIS) (PRESENT? JAUNDICE)) ((T 3))) UI~SATISFII::D: E.X ITI!ili TABLE LOGIC PATHOGllOHONIC tVIDENCE IS 0

Similarly rules such as that involving large liver size result in accumulating physical evidence, and rules triuered by bilirubin, SCOT and other lab dat:~ are used to construct a more structured clinical evidence.

PHYSICAL tVIDENCE 4 CLII1ICAL EVIDEIM (8 4 0 2)

A symbol-matching table such as in Procedure Stone-Consistency is finally used to decide cholestasis GD a scale of -3 to 3. liotice the difference between the use of rules for suggestinG cholestasis in CP and for establishing cholestasis in this node, In the latter, evidence is method! cally accumulated and waighted to reach a decision.

CHOLESTASIS FIT: (ESTABLISiiED CHOLESTASIS 3)

Q' had psased two relate<! suggestions to Cholestasis, namely Post-oper and Anesthetics as invoked ·uggestions (i.e. suaaeations made in a data-directed fashion )and these experts are now called(see I.2 in Procedure Cholestasia). The mesaap;e actually gets route~* to the

* In the rest of the discussion only a few of the rules from each conceptual &roup will be shown. rach decision within an expert is macie by a rulc-&roup or a rrocedure. The representational details will be described in[3). Rule conditions are actually logical queries to the data-base. The details of the date-base organization

and queriea can be found in[5],

*·The routin& waa done automatically by the communication system and CHOLESTASIS did not know that POST-OPER and ita aneeator INTRA-HEP arc not implemented, The answer does indicate, however, that the decision was made by the USER. Because POST-OPER is the immediate auper-concept of ANESTIIETICS, only one mea sage waa aent, though individual answers were teturned,

USER who make a the deciaion that Poa t Ope r 1a u nlilcdy

EXPERT-CALLEu: INTRA-HEP CALLED-BY: CHOLESTASIS 14ESSAGE (OUERY (REFINE POST-OPER) (CONSIDER ANESTHETICS)) EXPERT -CAL LEU: POST -OPER CALLED -BY: ItiTRA-HEP MESSAGE (OUERY (REFINE POST-OPER) (COiiSIDER ANESTHETICS)) FOR lACH DISEASE LISTED IUOICATE WHETHER IT IS ESTABLISilEO U1lLIKEL Y OR UNKNOWN · ANSWER Y-11-U POST -OPER?$N ANESTHETICS?$N EXPERT-CALLEU: INTRA-HEP CALLED-BY: USER MESSAGE (ANSWER (UIILIKEL Y ANESTHETICS BY-USER) (UNLIKELY POST -OPER BY -IJSER)) Control now passes to 11.1, i.e., CHOLESTASIS now tries co decide which of EXTRA or INTRA is more likely. TRYINb TO UECiuE BETWEEN EXTRA AND INTRA-HEPATIC CONUITION: (DEFORMITY? IHu (CT-SCAN ULTRASONAGRAM) &JILATE) VALUE: T COUuiTIOII: (~RPH? BIL-TREE PERC-TRAIIS-CHOL (NORMAL)) VALLE: F COIIIHTIOU: (MORPH? BIL-TREE ERCP (NORHAI..)) VALUE: F VALUE RETURNED: ( EXTRA-HEP T F F) SELECTED: EXTRA-HEP Followina ll.2, control transfers to EXTRA-IN-CHOLESTASIS. Here heuristic rules are tried to IU&&est possibilities from amona the subconcepts of axtra-hepatic cholaataaie. TRYING RULES TO INVOKE HYPOTHESIS TRYING RULE : (((PAW? ABOCIHEN COLICKY)) ((SUGGEST STONE))) UNSJ.TISFIEu: TRYING RULE: (((11lRPH? GALL-BLADDER 7 GB-SHADOW)) ((SUGGEST GB-CAIICER))) UI~KNOWN TRYING RULE: (((DEFORMITY? BIL-TREE 7 (NARROW))) ((SUGGEST SCL-CHOLANGITIS SO-CANCER))) SATISFIEu EXECUTING ACTIONS : ( (SUG(;EST SCL-CHOLAIIGITIS 60-CANCER)) SUGGESTluliS Ill EX fRA-CHCL (BO-CANCE~ SCL-CHOLANGIT IS) CHOLESTASIS firat callR BD-CANCER(via EXTRA-REP). TRYIIilo TO lSTABLISH BO-CANCER EVIUEIM-SITE OF BIL-TLI~OR (T ( CHEP)) TRYING TO I!lFER BILIARY OBSTRUCT I CN EVIIJEIICE-SJTE OF BIL-JBSTRUCTION (T (CHEP)) TRYirlG RULE: (((ANO-TRI (CAR TUIIOR) (CAR OBST) (AROUNU (CAOR TUMOR) (CAOR OBST))) l (T 3l ll SATISFIEu: TRYING RULE: ({(DEFORMITY? BIL-TREE BIOPSY TUMOR)) ((T -3))) UNKNOWN: EVIDENCE-OF TUMOR Ill BILE-DUCT 3 ESTABLISHED SO-CANCER 3

NESSAGE (AI~SWER( ESTABLISHED SO-CANCER 3)) raachea CHOLESTASIS via EXTRA-HEP. BD-CANCER made the decision after asking questions about wnere the obstruction and tumor in the biliary-tr ee may be(both questions are complicated queries poaed to the Radiology expert , not shown here, which examines the relevant radiological inforoation to make t he mference). SCL-cHOLANGITIS is called next, but returns an unlikely evaluation after detailed processing of relevant information. TRYING TO ESTABLISH SCL-CHOLANGITIS TRYING RULE: (((ANU-TRI (DUCT? BIL-TREE BIOPSY FIBROSIS) (DUCT? BIL-TREE BIOPSY (CARCINOMA ATY-EPITH-CELL) ) (HISTORY? ULC-COLITIS))) ((T 3))) UNSATISFIEu : ... PATHOGNOMONIC EVIDENCE . IS 0 x-ray evidence is evaluated to be -2 by use of a loaic 8imilar to that ahovn in the dec iaion between EXTRA and JNTilA earlier. XRAY EVIDEI~CE IS -2 SUI+IARY EV IDE liCE IS -2 UllliKEL Y SCL -CHULANGIT IS -2 EXPEkT-CALLED: EXTRA-HEP CALLED-BY: SCL-CHOLANGITIS I<IESSAGE (ANSWER (UilLIKEL Y SCL-CHOLANGITIS -2)) At tbia point, there ia one refinement of EXTRA-HEP in ACTIVE list and CHOLESTASIS haa to decide if all taportant data have been covered by the active b7Patheaaa. This deciaion ia currently made by the uaer. ACTIVE: 00-CANCER EXTRA-HEP UNLIKELY: SCL-CHOLANGITIS POST-OPER ANESTHETICS UllKNOWii: the problem of when to atop is atill an open problem i n diarnoatic knovled&e-baaed systems, though some ifproachea baaed on threaholdin& confidence measures have been tried(6,7]. We sre workin& on a aymboli c .. tching criterion for thia purpose and preliminary deaian haa been completed for explaining the major findinaa in extra-hepatic cholestaais(see (3] for dataile). tn this cue, as the CPC discussion ahowa(4] , all the data have been explained, However, in or der t o demonatrate the reat of the ays tem, let ua aay NO to the quaation: ARE ALL UATA EXPLAINEU BY ACTIVE DIAGNOSIS?$N WI nov are in IV in Procedure Choleatasis. LOOK-FURTHER banda over control to EXTRA-HEP, askin& it to aee i f i t can refine the diarnoais any further (but limiting its aearch by asking it to consider only the t ypi cal diaeaaea). EXTRA-HeP tries its~ heuristic rules, but 110 auaaeationa <Are made. It then falls back on an ~hauative aearch and callA* each typical expert under it, one by one. Aa it turna out, all return an unlikely evaluation, ~XPERT-CALLEO: lXTRA-HEP CALLEU-BY: CHOLESTASIS MESSAGE (OUERY (REFINE EXTRA-HEP) (CONSIDER TYPICAL)) TRYINb RULES TO INVOKl HYPOTHESIS

6 Details of calls and procesain& by these experts have bean alr.ipped.

TRYING RULE: ((~EOUAL? JAUNUICE INTENSITY SEVERE) EQUAL? JAUNDICE TRENu UNRELENTING)) ( SUGtiEST AV-CAIICER))) UNKNOWN

SUGGESTIONS IN EXTRA-HEP iHL

EXTRA-REP returns control back to CHOLESTASIS with the result of ita processing namely failure to establish any more concepts, but able to pretty much rule out all t he other possibilities, The point of continuing the diagnosis, even though the correct diagnosis had already bean made, waa to demonstrate the role of suggestion rulas at all level11 in the conceptual hierarchy 811 a means of achieving efficiency. If bd-cancer had not been suggested at the level of CHOLESTASIS, it would hava been eatablished when control waa handed over to !XTRA-HEP. The same effect occurs at all lavela(including within tip-node experts, which usually have pathognomonic rulea) in the hierarchy.

EXPERT-CALLED: CHOLESTASIS CALLED-BY: EXTRA-HEP MESSAGE {ANSWER {REFINED EXTRA-HEP))

ACT IVE : Bu-CANCER EXTRA-HEP UIILIKEL Y: AV-CAUCER AMP-CANCER GB-CANCER STRICTURE STONE SCL-CHOLANGITIS POST-OPER ANESTHETICS U·IKNOWN:

ARE ALL iJATA EXPLAINED BY ACTIVE DIAGNOSIS ?$YES tha diagnosis is summarized on a scale of -)(Reject) to J(Definite),

txPERT-CALLED: GP CALLED-BY : CHOLESTASIS MESSAGE (~~SWER (ESTABLISHED CHOLESTASIS 3))

DISEASES CONSIDERED AND FOUNU UNLIKELY

AMP-CANCER -2

UB..CANCER ·2

STRICTURE -3 STONE -2

SCL-CHOLANGITIS -2 POST-OPER BY-USER ANESTHETICS ~Y-uSER

AV-CANCER -3

CONSIDERED ANU UNABLE TO uECIDE

THE FINAL DIAGNOSIS IS --­

CHOLESTASIS 3

EXTRA-HEPATIC IrlDICATED WITH OBSTRUCTION AT (CHEP) BD-CANCER 3

There are three important points to note about this example. Firat, the diagnosis is correct (though aystema based on other approaches may have done as well in t his case). Second, the diaanostic knowledge was uaed at the correct place, once the proper context had been established, For example, it would be premature and often disastroua to try to establish tip-level concepts like bile-duct cancer, unless cholestasis had been established and extra-hepatic indicated as more likel y than intra-hepatic. Finally, the problem of coping with a large volume of data at a single level was .voided by using the patient data at the appropriate level of spec'ificity and importance. In other words,

the 1ystem tries to explain different types of data at thoae levels in the conceptual hierarchy where they are 111011 t relevant,

6.0 COHPARISON WITH RELATED SYSTEI'IS

6,1 :;NTERNIST

INTERNIST is perhaps the most comprehensive AI medical program: it deals with 807. of internal medicine. The diseases ar e organized in an orgnn-hased hi erarchy . Disease noJea arc linked by causal and associative links , lr· addition, each datum (type) has "manifestation of" and "invoked bv" links with disease nodal (with ·orne wei&!tts attached to these linka), INTERNIST performs diagnoaes by calculating a numerical ~eora for diseases baaed on how many manifestations are pre1ent, fraction of prototypical finding· present, IUpport from c:au1al diseases and a measure of how ~uch UDaxplained data is explained by a hypothesis. The diaaaae hierarchy and a partitionina of current bypotheaea into complenentary and supplementary diseases bal~a tb create some focus and cut down the complexity of processina. The basic problem-solving strategy is ana of purauina the higheat scoring hypothesis in a alobal way, A new datum introduced at any time leads to a recomputation of all estimates, This could also lead tiD sudden shifts of focus. Huch of the intelligence of the program is embodied in the complex scorin& function. 1hua explanation of diagnosis is conceptually opaque,

6.2 Present Illness Proaram (PIP)

this program is designed to suggest oossible diseases in the area of renal diseases, Knowledge about clinical aDd physiological states (not all being diseases) is oraanized into frames. Each frame contains information lbout relationships with other states; triggers to iPvoke states and differential diagnosis; and a table of probabilistic information to estimate the likelihood of the state· baaed on patient data, There are also tulea which can determine the likelihood of states or tule them out in some cases,

the basic strategy is similar to that of INTERNIST, i.e ·· the invoked hypotheses are ordered by their likelihoods and the top one at any given time is puraued, A hypothesis is accepted or rejected if it crosses a threshold , The likelihood estimate includes tha support from complementary states, match with the prototypical finding·, and the degree to which patient data are explained by the hypothesis ,

Alain, aa in INTERNIST. a new datum leads to potentially expenaive recomputation of all estimates, and possible fOcus ahifts. Further the use of Bayesian estimation in a global way would add to the computational and focus problems, Discussion of some problems associated with Bayesian estimates can be found in [11) and [12). As a laat comment, the frames are interpreted in a uniform way, It would be difficult to use specialized knowledge to bear on the problem at the appropriate place, unless it can be caat in the general mold,

6.3~

~CIN advises clinicians in the area of infectious diseases. The knowledge is o;tored in the form of deductive rules, with some associated certainty factor. tha program also has available to it a context hierarchy, which allows it to select rules based on the context and ask for appropriate data.

The rules are used in a backward-chaining deductive fashion, to first find the cause(~) of infection and then sug&est therapy for it, r:uch of the real power of the system seems to us to come from the ability of the context hierarchy to use the rules in a controlled way, More generally, while rules are a convenient way to express certain kinds of knowledge, many other kinds of knowledge are not easily encoded in rule form (see [13) for a case study in this regard).

6,4 CASNI;;T

CASNET performs diagnosis and recommends therapy in aJ.aucoma dise&Ha. The knowledae is represented in a cauaal network of dyafunctional states with well defined startin& (etioloaic) and endin& etatea. Evidence about the likelihood of these atates can be Obtained from another network relatin& states to teats. A teat may be a patient datum or combinations of those. Dlaeaaea are defined aa patha in the caueal net.

the ayatem calculate& the statue of each state, baaed on the teat valuea. This computation is repeated as new data are entered. It also computes the weight of each uode. which ie independent of its statue, but depends on tboae nodea which occur in the aame path. Diseaaes are bypotheaized and verified on the baais of which paths are moat likely or potentially able to account for all confirmed atatea. There is a computational burden in calculatina and recalculating the status and weight of .. ch node in the network. Given the exponential nature of auch computation. it seem& unlikely that such an approach would be eaeily extended to larger domains. PUrther. the uae of patho-physioloaical states is, perhaps, not poasible for most of medicine. as such .. chaniama are known only spottily, A relatively minor problem ia created by the fact that evidence from different teata can be combined by creatin& a "higher" taet-entity. data combine in different ways to auport or disprove diseaae hypotheses, many new teat-entities would need to

In many medical .situations, where patient be created for such combinations.

7,0 CONCLUDIN�~

WAile the precedina paae· can be viewed aa a partial attempt at uncoverina the structure of ht:man coanition ill the diaanoaia teak · the real teat of the ideas propoaed is in how well the aystem baaed on them works. In addition to the caae preaented in an earlier &action. aaveral other extra-hepatic caaes (both from medical journal& aa well aa from a practicin& hepatoloaist) have been tried. and in all caaee the diagnoatic behavior waa aatural and cloaely correaponded to the physician·' reaaoning, The aystem is conatantly being modified, but DOt in the sense of "patchina" but to increase the quality of the knowledge. For inatance, it has been fOund uaeful to place certain rulea higher in the b1erarchv. In soma caaea, the table loaic of aome tin uodea haa been found inaccurate. and, after consultation with medical texta and experts, modifications have been made. Theae seem to ua to be natural ways of improving the power. Of course. the modifications to local procedures should converge eventually. We cannot say that the system haa yet passed thia test.

wa envisaae the next stage in our work to be enlarging the aystem to handle the 'whole of the Cholestasis -rndrome. We hope to study the effect& of upward acalin& in thia enlargement. For one thin&, a more powerful data base organization for temporal phenomena will be needed. However. we shall not opt for a aaparate temporal expert, aa this would violate our conetraint on control tranafers. Instead temporal knowledae will be dietributed as needed.

OUr current approach conaista of giving the aystem all the data that are available in the journal description of the caee or from the collaborating hepatologist. In either case · the case is in a sense "complete." In reality, however. aa the diagnostic process unfolds, testa are ordered to confirm or reject hypotheses. Thus the ability to order testa would be a useful extension to our ayatem. The conceptual information needed is partially implicit in the knowledge structure (queries to the data baee can be converted to ordere for testa). Siailarly. since the inter-expert messaaes contain purpoeee and reeponsea, the basis for an explanation capability is already present. However, more work is oeeded to realize these facilities in an elegant manner, i···· without too much detail and with appropriate lbetractions. Explanation and test-ordering facilities would enhance the uee of the system as a consultant.

In aummary, we have attempted, in this paper. to lay the fOundation& for a knowledae representation scheme, which derivee ita power becauae of its ability to capture the atructure of a body of knowledge at an appropriate level of depth. There are many interestina issue· related to conceptual structure· we have not had occasion to diecues in this paper. These and our experiences with problem-solvina eystems for larger domains of knowledge will be the subject of future reporta.

ACKNOWLEDGMENT: We are indebted to Douglas Levin,M.D., tor his interest and consultations.

IIEI'ERENCES

1. Hineky.H. "Plain Talk about Neurodevelopmental Epietemology." In Proc. IJCAI-77. NIT. Cambridge, Haea ·· Ausust. 1977. pp.lOSJ-1092
2. (2] Hayas-F..oth.F ·· Lesser,V.k, "Focus of Attention in the Hearsay- II Speech Unuers tanding System." In ~ IJCAI-77, ~IIT, Cambridge. ~lase ·· August. 1977. pp.27-35
3. Hittal,S. rh.D. lliasertation(forthcomin~;) · Dept. of Comp. and Info. Sc ·· The Ohio State University, Dec. · 1979
4. "Case 45-1977· l·lassachueeetts General Hospital." New En&land Journal .£!.. Hedicina. Nov.lO, 197"7; pp.l054-1059
5. Hittal.s.. Chandrasekaran .B. "Conceptual ltepreeentation of P01tient lJata Bases." OSU-TR-79, Dept. of Comr. and Info. Sc ·· The Ohio State University, April,l979
6. (6] Pople,H.£.,tiyers.J.D ·· Hiller,k.A. "DIALOG: A Hodel of Diasnostic Locic for Internal Hedicine." In Proc. lJCAI-75. Tbilsit ~SSR, Sept ·· l975, pp.848-855 -----
7. (7] Shortliffe.E.H. Conputer Cased l:edical Consultations: !1\"C!;; f.!sevie r :lorth-Holland Inc., 1976
8. [8) Pople.H.E. ''The ~·ormation of Composite Hypothesis in lliaGnostic Problem Solving: An Excercise in Synthetic Reasoning." In .!:r££.... IJCAI-77. !.IT · Cambridae. ~l<lss., /.ugust. 1977, pp.lOJ0-1037
9. [9) Pauker.s ·· corry.G.A ·· 1~01ss irer ,J. ,Schwartz, w. "Towards a Simulation of Clinical Co:;nition." ~ American Journal of :ledicine 60(1976)981-996
10. Weiss .s .i·l. ,Kulikowski ,c .A. ,Aluarel,S. · Sa fir ,A. "A Hodel-based i·.ethod for Computer-Aided Nedical Deciaion-Naking." Artificial Intelli&ence. 11:2(1978) 145-172
11. [ll] Szolovits,P ·· rauur.s.G. "Categorical and Probabilistic i\ensoning in :tedical Diacnosis." Artificial Intellicencc. 11:2(1978) 115-144
12. l'einstein,A.K. "l.linical lliostatistics XY.XIX. The haze of Bayes. the Aerial Palaces of Decision Analyab and the Computerized Ouija Board." C:lir.ic.,l Pharmacolor.y and Thcrap~utics. 21:4(1977) 482-490
13. Ke!Ogia,J.A. "A Production 1.ule Systen for Neurological Localization." In Proc. $econu Annual Symposium 2!!_ Conputer Applications in ·>edical Care. November. 197&· pp.254-260