NEOMYCIN: Reconfiguring a Rule-Based Expert System for Application to Teaching

NEOMYCIN: RECONFICURINC A RULE-BASED EXPERT SYSTEM FOR APPLICATION TO TEACHING

William J. Clancey and Reed Letsinger

Computer Science Department Stanford University; Stanford, CA tfS06

hiRh level nl expertise (31]. 1nd 2) modul!r design 1nd

ABSTRACT

NFOUVCIN" Is e medicel conaultelion system In which t.AYCIN's kfiOWiedge base is reorg1nized end extended for use in GUIDON, e l11chlng proge~m. The new system constitutes e psycholoaicel model for doing di1gnosis, designed to provide e b!Sis for Interpreting student beh1vior 1nd te~ehing di1gnostic strategy. The model aep1r11es out kinds of knowledge thet ,,. procedurally embedded In hAVCIN's rules 1nd so inaccessible to the te~ching progrem. The key ide· is to represent explicitly end sepertlely: 1 dOm!ln-lndependenl di1gnoslic strategy in the form of mel1-rules, knowledge ·bout the structure of the problem spiCe, c·us·l end dele/hypothesis rules, end world feels.

• As 1 psychologlcel model, NEOMYCIN ceplures the forw~rd­ direcled, 'compiled !Ssoci!tion" mode ot re~sonlng th·l ch~reclerlzes expert beh·vior. Collection and lnlerprel·lion of dele ere focused by the "differenliel" or working memory of hypolhests. Moreover, the knowledge b11e Ia broldened so the! GUOON cen leech 1 student when to conaider 1 specific infectious di11111 end what compel ina hypotheses to consider, essenllelly the knowledae 1 humen would need In order to use the t.AYCIN consultation eyslem properly.

INTROOUCTION

In order to use 1 knowledge bese IS subject materiel for leeching, it is imporlenl that the kinds of things 1 student needs to be told be represented flexibly, so the! they c1n be singled out and erliculeled. Development of intelligent tutoring systems such es SOPHIE [5]. WHY (27], WUM'US [16], end GUIDON [II] (12] cen be viewed, In pert, as 1 problem of knowledge represenlelion. This reuerch hes shown the advanteaes of:

• -- multiple represenl1lions of knowledge (e.g., the slmuletion model end semantic network in SQPHE),
• -- representetions that cen be both interpreted end used to generate leaching text (e.g., Brown's meteoroloaicel eutomele (4), end production rules used in WUt.4PUS end GUIDON),
• -- network representations of knowledge lh·t uplure "imporlence" (SCHOLAR (9]), "c~mplexit~· or ;pre-re~ul~ile: essocielions (WUMPUS, BIP [3]), 1n1logy end aeneraitzahon relelions (WUMPUS),
• -- end represenl·lions the! ·llow for verlents on expert l)lrtormence (for modelling the student) (WEST [1]. BUGGY (7]).

In the GUIDON program we h1ve been exploring the problem of using hAVCIN's rule set Is le~ching meteri1l. hAVCIN [26] is I rule-besed . expert system that provides therapy 1dvice for cerl1in kinds of infectious diseases. II has spewned · cless of systems, eelled "E:hAVCIN systems,· which all use the seme production rule lenguege end interpreter [29). GUIDON can operate using the rule

set of eny EhAVCIN system as subject m·leri1l.

MYCIN's rules were thought to be potentially useful for

teachinR because: 1) form1l evalu1lions indic1le that it C!plures 1

· This research hes been supported in p!rl by ARPA ·nd ~ "onlracl N00014-79C-0302. Compulelion·l resources were provided by the SUMEX-Ait.A f~eilily (NIH aranl RR 00785~7).

represenl1lion·l mel·-knowled&l! enable the program to expl1in its re!SOni"ni (13). lronic·lly, we h1ve found that it is in precisely these two erees--experllse end .explenelory cepebilily--so lmporlenl for 1 successful le!Ching progrem, the! t.AYCIN fell· ahorl. To solve these problems, we heve implemented 1 new ayalem we cell NEOMYCIN.

A. The limitations of hAVCIN for !pplic·lion lo teachina

First, hAVCIN is designed to be uaed . IS a consull·nh its knowledge is too n1rrow to be used for le!chtng 1 student to be a prim1ry diagnostician. The knowledge base is designed to interpret culture results from the blood 1nd the cerebrel-spinel fluid. But wh·l expertise su&&ests lh!l such · culture should be l1ken? Wh·l knowledge does 1 human dr~w upon for focusing on b~eleremi1 or meningitis, end wh·l competing hypotheses (!nd medic!l tests) need to be considered before hAVCIN should even be used? This knowledge is cerl·inly 1 crilic·l p!rl of le!Ching infectious dise~se diagnosis, but hAVCIN knows nothing about it.

Second, protocols of experts solving the 11me e~ses presented to hAVCIN indic1le lh!l the progrtm does n_ot org!nl.ze or use its knowledge the w1y 1 hum1n experl does. Th11 result 11 ~I surprising, for it is consistent with 1 h!lf-dec·de of psychologtcll research into medic1l problem-solving [II]. [24). [20]. [21]. [21] [14]. [17). If GUIDON, our lutorl·l progrem, is to ~rticulale 1nd recognize the hiererchicel org1nizalions of knowledge end seerch slr·legies that humans find useful, we need I~ reorgenize ~CIN's rule set end incorporate en explicit model of dtegnosllc thinktng. In particular, the model must exhibit: focused, forward-directed use of d1l11 tri11er essocietions lhet suggest new hypothe.aes; follow-up questions that esleblish the disease process ("ptclure of the patient"); and management of 1 ch·naing "working '!"mory" (hereafter, "differenti!l") of hypotheses under considerthon. To this extent, the development of NEOMYCIN Is en eltempl to eynthesize previous rese1rch, 1nd to 1n1lyze its !ppliulion to our

infectious dise11e Droblem domlrin.

B. Developtng 1 Rsychologic·l model by mo<lifying Et.AYCIN

• A psychologicel model of dieanoslic thinking cennol be represented usina the Et.AYCIN represenl11ion elone, lhet. Is, by simply rewriting hAVCIN's rules. lnsleed, the represenlehon and interpreter must be eugmenled end the rules orginized by multiple, "'rlhogon·l structures.

For ex1mple 1 simple interpreter chenge is to ellow Incoming dale to c1use new' subgoels to be setup end pursued. Conaider the triBfltlr enlecedenl rule "if the petienl h!S 1 stiff neck end e heldectl!, then consider meningilis." .. When e physiclen heers that the pelienl h1s 1 stiff neck, the essocietion to m_eningilis might come to mind, prompting him to determine if the pehent has 1 held!Che IS well. To bring eboul this effect In t..EOMYCIN. 1 new type l:lf enlecedenl rule hid to be 1llowed, 1nd 1 loc·l change mede to the EhAVCIN control structure.

Besides interpreter changes, different kinds of knowledge hid to be sep!rlled out of the rules 1nd represented explicitly. Fig. l ahows e typical (p!r !phrased) t.AYCIN rule in which different kinds of knowledge ere procedur~lly embedded.

-,.he medical exemples in this peper are simplified; we m1ka no clelms eboul completeness or accuriCy. They are for purpose· of illustration only.

• If: 1) The infection Is meningitit
• The subtype of meningitis Is btcltritl
• Only circumstentltl evidence ia evaileble
• The pelient is et least 17 y11r1 oltl
• The petient Is en elcohollc

Then: there is suuestive ev1oence the! c:IIOiococcus­ pneumoniae is en orgenism uusing the meningitla.

Fiaure I. Typictl t.AYCIN Rule

This rule is en example of "compiled expertise.· We can list bOme of the individual steps of rusoning end knowledge sources out of which it is composed, unknown to MYCIN, but explicitly represented in NEO~CIN:

• -- Analysis of other rules shows thet this rule (to determine the organism) is only invoked efter it hes been established the! the petient hes en infection. Thus, four mejor subgoals are established in this order: Is there en infection? Is It meningitis? Is it becterial? Is it diplococcus-pneumoniH? Each of these subgoals hypothesizes · more specific uuse of disuse. Thus, ths ordsrin1 of clwses constitutes · top-down refinsmttnf stretssy. However, MYCIN does not know about this speeieliutlon hlererchy. It does not even know thet diplococeus-pneumoniM is 1 bacter ium. Perhaps most serious of ell for meeting our leeching goals, MYCIN omits intermediete utegorles such IS ecule/ehronie meningit is &nd "gram negelive meningitis" thet physicians find helpful. In NEOMYCIN these eetegories ere represented explicitly In an stio/o,icel texonomy by ellowing peremeters to be apeeieliutions of one enother.
• -- The eleuse ebout the pelient's ege prevents t.AYCIN from esking if a child is an elcoholie. MYCIN does not know thet the ordering of these clauses is importent, or what the reletionship is. In NEOMYCIN these world reletions ere uptured by seperete "screening"· rules.
• -- Whsn there Is leboretory evidence (I culture with visible organisms), this rule does not epply (cleuse 3). However, 1 eompenlon rule still ellows the eireumstentiel evidence of elcoholism to be considered, but gives it less weight. This principle of considering elrcumstentlel evidence even when there ere hard, phyeleel observetlons of the eeuse, is not explicitly known to MYCIN. The principle Is compiled idenlicelly Into 40 peira of rulea, relher then being steted es 1 reasoning rule for combinina herd end 10ft evidence. NEOMYCIN hes rules for reasoning eboul the evidence II has collected, so connections between dele end hypolhesta ere seperete from the contexts in which they will be used.

These forms of knowledge--· (top-down) atretegy, en et iologicel taxonomy, world feels, evidence weighing rules--form 1 basis for · psychologicel model eboul knowledge orgeniulion end access, but they ere · not sufficient. Consider the ebove rule egeln. How does a physician remember to esk eboul elcoholism? How does he remember the connection with diplococcus? Experts use 1 rich set of orgeniutional eids end mnemonics for eceessing their knowledge.

For example, one een think of "I eking the petlent's history· IS e process of "determining the differenliel of possible eeuus." Under this strete1y, the expert follows the principle (rule model) lhel "compromised host conditions broaden the differenlitl by auaaestlng speclel ceuses." Alcoholism Is one of Illest conditions. The essociet lon to diplococcus might be remembered 11 1 aimple uusel story: elcohollcs bruthe In their own secretions, 10 organisma found in the mouth lind their way to the lungs, eeusing pneumonle. NEOMYCIN incorporates these psyehoto&ictl elds: 1) 1 reprssent·tion of di·snostic str·te1y thel providea 1 meeningful, useful orientetion for collecting dele ("ettempt to brotden the di fferential"); 2) structur~l associelions for indexing evidence to consider (abstrections such IS "compromised host conditions" and rule models that use them); end 3) rule justific·tions that relete dele/hypothesis essocietions to underlying ctustl proceaua.

As we mentioned ebove, we cen't use t.AYCIN for leeching ebout diegnosis because the range of problems it knows ebout Is not realistically wide enough. But if we simply ldded knowledae ebo~o~t more dis.,ses end when to order ltboretory tests we would be in trouble: · top-down diegnostle strelegy Is intdequete for a broader range of problems. The eombinetories of the medieel diegnosis search problem meke it impossible for an expert to consider every infection, to work top-down. lnititl Information most commonly brings the physician into the middle of his texonomlc hiererchy (via the "compiled associ.,tions" such IS the triaaer rule given ebove). Working from the middle, the physieien must first look upwards to focus the possibilities (is it a treumatic process? cencer?) and lhsn refine downwards. The epproach used by MYCIN's rules only works beeeuse ·the user of the progrem Is the one who focuses on meningitis. t.AYCIN can verily lhet the hisloricel end leboralory evidence is consiatent with meningitis, but II dolan, heve the knowledae for considering It in the first piece. The progrem hes only two infections to consider and does not know ebout other causes of lhs findings reported by the uur.

For the proarem itself to shoulder thia focusina burden (lo lhet GUIOON een t .. ch It to a student), we ahould more properly think of its ar., of expertise as being related to the oburvelions 1 user will bring to It, rather than the problems it knows how to confirm end refine. Thus, MYCIN's er11 of expertise is "menlngitls"1 in conlrest, NEOMYCIN deals with "ebnormel neurologicel signs" or "headechs end fever.· In order to alve NEOMYCIN the captbllity to deel with a broeder renge of problems, to ectually hive It think of other causes of headeehe end fever, we: 1) tllpend«/ the etloloflicel llnowltld6s to include broed eelegorlea of other, non-infectious problems, such es ·toxic problem, · "neoplestlc problem"! 2) incorpor·ttld ths focusin1 strate1y of "1roup and diffsrentiete· 10 the program could manaae this broeder rena· of possibllllitsl end 3) to enhence the progrem'a ability to epply this strategy we 8dded llnowltld6e ebout diss..se procefSes, knowledge thlt cuts orthogonelly across the etioloaieal taxonomy.

II AN OVERYIEW Of NEOMYCIN

A few words about .the charecter of MYCIN's problem domeln are in order. We essume thet a diegnosis or problem solution consists of en ordered list of problem causes lhel heve been selected from 1 fixed, hierarehietl space of hypotheses (e.g., "cencer process·, "chronic meningitis") or stele cetegories (e.g., "mesa lesion in the brain"). We assume that en Informant presents a problem to the program, which acts es 1 consultant, the role pleyed by e student using GUIDON. There are two types of dele: soft dele (circumstantial or historical) end hard dele (laboretory or direct meesurements). Some of the evidence may be missing, end conclusions will usuelly be uncerteln.

A schematic of the NEOMYCIN system (Fig. 2) illuslralea the various knowledge sources end their reletlon to ,.,. stretegic knowledge end differential. These components are shown 11 Icons expended in subsequent figur11. The interpretation of Fig. 2 follows.

• There are four kinds llf domain rules: 1) cauul rules form a net of physioloaical sletas ,;,d disease eeleaories, ultimately linklnc raw observations (incominc data) to the etiolocieel taxonomy, 2) lriiJIJer rules associate d1ta with etioloaies, which are pi1ced ·· hypOtheses in the differential (maintained so that caneral causes are replaced by their mora specific descandants)J 3) data/hypothesis rules essociata clrcumstantiel and laboratory data with disuses, as do trluer rules, but only those rules focused by the differential ere tried when the data Is clrcumalantial (the! Ia, the asaociations lh·t "come . to mind" are those hypotheses already In the differenli·l, 11 well as the nodes of the etiolocical taxonomy which hanc below the hypotheses of the differential)! 4) acreei'WniJ rule· (not shown) form a hierarchy of abstractions and restrictioM on data (e.a., "if the p·lienl is not immunosuppressed, then he ia not an alcoholic") which are applied by backward chaininc, in an attempt to determine a d·lum without ul\inc the user.
• -- Other dom·in knowled&e (not shown), orlhoconal to the hierarchies of cause, considers diseases 11 processes hevinc a location, extent, procression of symptoms, etc. One form of disease process knowledge is represented as a frama-lika description associated with diseases in the eliolocical taxonomy, and is used to differentiate amonc them. A second form consists of e list of process-oriented, follow-up question· that should be immediately asked when some disease catecory or physiolocical slate Is Implicated (for example, to est1blish when symptoms occurred and their orderinc and chance in severity).
• The met1-strateay for doin& diacnosis consists of 1 hierarchy of domain-independent meta-rules. In caneral, these meta-rules eKamine the differential, and make use of the atioloalcal taxonomy, causal associations, and disease process knowledaa to decide what datum to request next. A typical atrata&y rule It ahown in Fia. 3.

~ETA-RULE397 (for the task Group-and-Differenliete)

If: there are two items on the differential thet differ

in some disease process feature

then: ask a question th1t differentiates between these two kinds of processes

Ficura 3. A typical alntacy rule.

The annotated typescript in the neKI aecllon ahowa how these forms of knowledge interact In practice. Subsequent section· provide 1 lAw more details about the representation.

Ill A SAUPLE CASE

To illustrate the ideas presented above, a aimpla case waa presented to 1 stripped-down version of NEOMYCIN (there are generally only one or two rules to establish each hypothesis). Only the collection of circumstantial evidence is shown. Observe the many different reasons why a question is asked1 this is a reflection of the compleKity of the diagnostic strategy. NEOUVCIN Is hypothesis- and data-directed. In contrast, MVCIN only asks 1 question to evaluate a clause of 1 rule for the aoal it is pursulna. its rules are not sorted by conclusion, so its question· appear to skip back and forth rtndomly among hypotheses. It is not "b1ckward chaining· per se that dislinauishes the two syalems, for NEOMYCIN essenli·lly backward chains throuch ita stratealcal rulea. It is NEOMYCIN's forward, non-eKhausllve r11aoninc and m1n1cement of 1 space of hypotheses that mkaa It reaaon more like 1 hum1n.

(For brevity, the trace of rule conclusion· Ia of the form ."Conclude: PARA~TER is VALUE (CF)." The context i1 elweya PATIENT-2. Slratecic1l tiSks are sometime· abbreviated. Fla. 4 relates meta-rules to. tasks.)

7·Dec·IO 16:11:25

········PATIENT·2········

task ~AKE-DIA�NOSIS ~TARULE384 succeeded. fiSk IDENTIFY-PROBLEU ~TARULE385 succeeded.

lnlll~llu/ca ue uncondlllonal. TIN llr·l ·l·p I· lo col/eel ldenllfyln· dala, followed by liN reaaona lor ... 111,. Mlvlce.

Plene enter inform1tion about the patient.

Name Ace Sax R.ca

• I ) ** J.S~ITH 4 UALE CAUCASIAN
• Ple11e deacribe the chief complalnh

(Enter kayword or phr11e.)

• **DIPLOPIA

Diplopia I· reco.nlaed lo be a neu,_,lo.lcal ·l·n whlcb lrluera menln./1/a, and ieeda 10 a follow-up queal/on (auoclaled wllh N.UfiOSIGN) ....

1nlecedent RULE376 succeeded. Conclude: NEUROSIGN is YES (1.0) antecedent RULE426 succeeded. Conclude: INFECTIOUS-PROCESS is UENINGITIS (.7)

Differential: Addin& ~NIN�ITIS

1. How many daya hu J.amlth had abnormal neuroloclc ai&Mf ** 13

Trying evidence rules: LENSIGN ·> CHRONIC-MENINGITIS Trying RULE52il/PATIENT -2; RULE524 succeeded. Conclude: MENINGITIS is ACUTE-MENINGITIS (·.3) Conclude: MENINGITIS is CHRONIC-MENINGITIS (.3)

Different ial :

replace MENINGITIS with CHRONIC-MENINGITIS

Another antecedent rulea Ia auoclaced wllb DII'I.OPIA ····

antecedent RULE373 succeeded. Conclude: BRAIN-PRESSURE is YES (.8)

BIIAIN·PIIESSU118 auueala a MAIS·UIION··&Iteae antecedent rulu ·II model lmmedl·te, forw·rd /Dierellc .. Ch·C we obnrved In experla ....

antecedent RULE375 succeeded. Conclude: MASS-LESION is YES (.72)

Differential: Adding MASS-LESION

Any other information? **HEADACHE

Here · lriUer antecedent rule require· more laform.Cion before II can be applied ..··

4) Does J.smith have a a tiff neck?

**YES

antecedent RULEil24 succeeded. Conclude: INFECTIOUS-PROCESS Ia MENINGITIS (.91)

Any other inform1tionf **NO

The pro&r·m now I a/I.e~ lnlllallve, acattiiJI Ita m·ln hl~cory·calclnl ca~ll of ·e~Cabl/ah Cite /lypotheala apace· (herufcer, EHS). The differential hu MW, unexamined elemenca, ao Cite "&roup and dlfferenCiace· (GicD) t··ll Ia Invoiced flrat. METAIIUU400 reco&niNII Chat Cite ellolo&lcal anceacora of chronic meniiJIIC/a bave 1101 Men explored, ao an accempc Ia mede co confirm (caalii'UIISUI· HYPOTHESIS · PH) Chat an lnfec&loua proceaa Ia o..,.lq &he problem ....

Enter EHS loop: focus · NIL task METARULEil27 succeeded.

Review differential: MASS-LESION ~-MEMN�ITIS

Enter G&D loop: focus · NIL task METARULEilOO succeeded.

Enter PH loop: focus · INFECTIOUS-PROCESS task METARULEil10 succeeded.

!5) Ia J.amith febrile?

·· y

antecedent RULE423 succeeded.

Conclude:

DISORDER-ETIOLOGY is 'INFECTIOUS-PROCESS (.7)

There are no rurcher que1Cion· co adr an lncerrupc recurna control lo lhe GltD lull. No rulea aucceed, ao control return· &o the EHS &ad. Tile ·explore aM refine· ( 1411) caall al/encly refine· m····lealon, expaadlq Cite differential by dlfferen& eCiolo&lcal caCe&orlea, aM ao &riUerln& return co the !HI 1·~11.

observed STOP-PURSUING METARULE408

Repeat ing G&D loop: focus · INFECTIOUS-PROCESS

Repeating EHS loop: focus · INFECTIOUS-PROCESS tuk METARUL~428 succeeded.

Ent,r E&R loop: focus · INFECTIOUS-PROCESS task METARULE429 succeeded.

observed STOP-EXPLORING METARULE407

Repeating EHS loop: focus · INFECTIOUS-PROCESS tuk METARULEil27 succeeded.

The flrac alep Ia ·&·In &o review &lie dlfferenCial, a proceu obaerved In expetta. Proceaa fe·curea of brain abacea. and chronic menln&l&ll are comp·red· &hey both occur In &he centr·l nervoua ayllem, ·re chronic problema, and ·re lnfectloua, bu& brain abaceaa Ia a locaUzed problem. NEOMYCIN aalla a queaCioa 10 dl~crlmlnale on I hi~ bula ....

Review differential: BRAIN-ABSCESS .HEMATOMA PUS-IN-BRAIN CHRONIC-MENINGITIS

Enter G&D loop: focus · INFECTIOUS-PROCESS task METARULE397 succeeded.

6) Do.a J.amith h1ve foul neurolo&icllai&naf· **NO Trying evidel"'ce rules: FOCALSIGNS -> BRAIN-ABSCESS RIA.E179 failed due to clause 1

The pro&r·m ha~ not been aupplled wllh llnowled&e for conflrmln& oCher cau~ea of mau lulon ( ·-'·· lraumallc hemorrh·&e, tumor), 10 II Ia unable Co conllnue 1111 &roupln& operallon, ·nd be&ln· an explorallon c¥cle ····

Repeating G&D loop: focus · INFECTIOUS-PROCESS

Reputing EHS loop: focus · INFECTIOUS-PROCESS tesk METARULEil28 succeeded.

Enter E&R loop: focus · INFECTIOUS-PROCESS task METARULEil02 succeeded.

Enter PH loop: focus · BRAIN-ABSCESS task METARULE409 succeeded.

Now directly focu~lnl on braln·ab·ceu, lhe pro&ram · re·Jize~· Ch·c dac· aupp/led e·rller Ia relev·n& ( rule433 ) . Chronic menln&llll Ia Chen conaldered by reflnln& II and pur·uln& ·peclflc c·u·... Pur·uiiJI TB, NEOMYCIN follow· the ·traCe&Y of conflrmllll lite flral ("·n·bllnl") alep In the dl1ea·e proce ... coot·c& wiCb &be orl·nl~m ...·

Trying evidence rules: MASS-LESION ·> BRAIN-ABSCESS Trying RULE433/PATIENT ·2; RULE433 succeeded. Conclude: INFECTIOUS-PROCESS is BRAIN-ABSCESS (.216)

Obaerved STOP-PURSUNG MET ARlA.E408

Repeating E&R loop: focus · BRAIN-ABSCESS task METARULE429 succeeded.

Enter PH loop: focus · TB-MENINGITIS task METARULEil11 succeeded.

7) Do.a J.amith have · TB risk fKtorf **YES

Trying evidence rules: TBRISK -> TB-MENINGITIS Trying RlA.E525/PATIENT -2; RULE525 succeeded. observed STOP-PURSUING METARULE408

'ocualnl acr·te&le~ dlct·te thai · ·lb/IIJI be conaldered nexc. Tun&·l menln&lll· I· refined, aad 1 child, cryptococcua, pur·ued ·...

Repeatina E&R loop: focus · TB·MENINGITIS task METARlA.E401 succeeded. Enter PH loop: focus · FUNGAL-MENINGITIS

Repeating E&R loop: focus · FUNGAL-MENINGITIS task MET ARULE399 succeeded. Enter PH loop: focus · CRYPTOCOCCUS

A cancer patlenc Ill .c ·ome Tlllll of &ecctn& crypcococc·l menln&lll·. ll.acher Chlln llllllln& dlreccly If the patient hllll cancer, che pro&r·m model· lln expen'· efflclenc c·alln& of · wider net by ullin& · more lenllrlll qw·llon. lpeclflclllly, there are ··c,..nln& t'lllell," Chill l.,d lc to decermlne flrac If the pllllent Ill lmmuno·upprelllled (rule3115} llnd then comproml·ed (1111e343} . Thill I· che only form of bllcllw·nl chlllnlnl chat occur· In NEOMYCIN.

teak METARULE431 succeeded.

• --[0] Findout: LEUKEMIA

--(1] Flndout: IWlMSIPPRESSED

Tryina RULE343/PATIENT -2;

.1) II J.amlth 1 compromiaed holt (e.&: 1lcoholic,

alckle-cell·dlaene, Immunosuppressed)?

**YES

Rll.E343 f1iled due to el1use 1

If Che pallenc were not comproml·ed, che pro&r·m could havil concluded Chill he I· not lmmuno·upprelllled ( rule343 ). Now Ill· un·ure llnd mulll ··II dlreclly. If che pallenc Ill not lmmunoauppr····d, the pro1r·m will .know Chill he doe· no4 have leul<emlll '( l'llle3115 ). The ·n·wer of ·teull.emla· below lmplle· ·lmmunolluppre··ed," H ru/elJI15 fall·, and che orl&ln·l &oall· determined.

• --[1] Finished: I~SIPPRESSED

t) II J.amlth 'lmmunoauppreaaed (e.a. corticosteroid lher~py, cytotoxic druc lhlnpy, ndiallon lhlr~py, leukemia)? **LEUKEMIA

I will naume th1l leuklmil It one of the dia&nOIIla of J.lmith

Rll.E395 f1iled due to cl1use 1

• --[0] Finished: LEUKEMIA

Tryin&. evidence rules: LEUKEMIA -> CRYPTOCOCCUS Trying RULE056/PATIENT -2; RULE056 succeeded. Conclude: FUNGAL-MENINGITIS is CRYPTOCOCCUS (.3)

Repeeling E&R loop: focus · CRYPTOCOCCUS Ink METARlA.E401 sucee~d.

Allenllon turn· to · lllb/1111. Al·ln, che ·enllblln& ·cep· Ill llllll.ed abouc flrllC ....

Enter PH loop: focus · COCCIDIOIDES lesk METARULE411 succeeded.

1 0) Hn the pelienl ever been to 1 cocci-endemic ···' **NO

Trying evidence rules: COCCI-ENDEMIC -> COCCIDIOIDES RUL£570 f1iled due to clause 1 RULE287 f1iled due to clause 1

observed STOP-PURSUNG METARULE408

Repelling E&R loop: focus · COCCIDIOIDES

Repealing EHS loop: focus · COCCIDIOIDES Ink ~TARULE430 succeeded.

Hllvlnl eXhllUilled llll llmlled .know/ed,., Che Pt'OIIIllll flndll no o4her telllVIllll, /typo4belllll·OIIemld qllllllllO .. CO aall. leveral 1eneral qwatton· .,. ··.ked ...

1 I ) 11 J.amlth reeelvlnc any medic1lionaf **NO

Repe11ina EHS loop: focus · COCCIDIOIDES lllk RULE430 succeeded.

12) Hn J.amlth been recently holplllllzedf **NO

Repelling EHS loop: focus · COCCIDIOIDES

If llddlllon·l d·C· hlld been llupplled, new /typo4hellell mllhC h·ve been pl·ced on che dlfferenll·l llnd lllrlllelle· for 1roupln1 or reflnln& ml&ht h·v· been Clllled /nco plq once a&aln. Thla end· che hi·Cory·tlll<lnl procellll. Nuc che pro&r·m would order l·bor·tory ceau, procellll chem, and perhapa return to &·Cherln& clrcum·c·nll·l·vlllence.

IV THE DIAGNQSTIC META-STRATEGY

Formalizing the diagnostic strategy from protocol an1lysis was the most difficult part of designing NEOMYCIN. Fia. 4 shows the general outline of the meta-strategy. Each non-terminal node in the tree stands for a task thai is achieved by a set of rules. An important aspect of our model of diagnosis Is that the process Cln be l1ughl as 1 task-posin& activity: the problem-solver thinks in terms of what he is trying to do (e.g., to consider unusu1l Cluses and so broaden the differenliel) in order to brin& knowledce sources to mind. Thus, the meta-strategy Is structured so the 111111 m1ke sense as things lhel experts try to do.

Fia. 4 shows lh1l the main object of the 111111-str~teay Ia to decide whet dele to collect next (invoke MYCIN's FINDOUT routine), gener~lly by focusing on some hypothesis in the differentiel. Aside from collecting lnitiel informelion, the basic Idee is lh1l collectina circumsl1nli1l evidence is 1 process of ·sfablishillf IM hypoiMsis sp.ce. This process l1kes the form of eonsiderina wh1l could uuse the reported d1l1, grouping and reflnina the differential, and llkina aener1l questions ". A creel de1l of what we micht ull heuristic confidence is plated In the aenerel questions, which constitute the outline of the "history laking process" es It Ia aenerelly tauaht to medical atudents. However, strategies for ualna causal and diaeaae proceu knowledae enable the expert to be an efficient problem solver In 1 comblnatorlally lara· space, and these strateaies are aenerally not tauaht. (A Ieier publication will discuss NEOMYCIN's meta-strategy in more detail.) ·

" "Group and differentiate· is used here in the loose sensa of esl1blishing general focus on 1 process that is consistent with hypotheses suggested independently by the d1la. Clusferillf (in multiple ways) and discrimin.1lin1, the usual 11111ning of the term, Ia one operation for ~thieving this focus.

The implementation is In terms of hierarchical meta-rules, which as 1 whole constitute the meta-strale&Y. Fia. !I illustrates how the rules for · given task ere treated ·II 1 pure production system-- they ere repeatedly tried in order, returnina to the head of the list when one succeeds, stoppina when no rule succeeds or en end condition Is true.

/\\

Ficure 5. Rule-based Invocation and Interruption of str~tecicel tasks

The end condition is itself determined by rules, and Is inherited as we descend into the hierarchy of tesks. The rnein uae for this feature Is to allow refocusing when new data chenaes the 1tete of the differential, as well 11 non-exhaustive COMideratlon of hypotheses.

V ETIOLOGICAL TAXONQMY. CAUSAL AND DISEASE PROCESS KNQWLEOGE

Some details of the implementation ere aiven in this aection. The etiological taxonomy (Fla. 6) Is implemented es EMYCIN parameters in which the values "" one parameter <·;&.;_ ~hronlc menlnaitis) ere themselves parameters <·.a~ Tb·menln&lhl and funaai-meningili·). We cell theae ta/tOitOIIIIc par-t·a.

Causal knowledge (fig. 7) is represented 11 rules marked 11 being causal, and modified by · certainly lector, es all MYCIN rules. A causal rule of the form "if A then e· implies that A Is cauaed by B, the direction of the association which is most genertlly uaeful for Interpreting data and refining hypotheses. Theae rules mention data para,..terl, taxonomic .parameters or llate-caterory paratnefers. State-cateaory parametera ·land for pathophysiological slates or eateaories of diaeiM (e.a., 1 rn11s lesion in the brain). .We are invesliaalina the possibility of usi"l .Pople's "planning links" [22] to more preclaely distinauish between causal and subtype links ··. Causal rules are used by the "explore end refine" task to work backwardl from state-ceteaory hypotheaes In the differential to prior causea, end ultirnetely to diqnostic hypotheses in the etioloalcal taxonomy (es 1hown In Fit- 7).

INCR BRAIN PRESSURE

MASS. LESION

TU.Io:. HE~T~US· IN·BRAIN

....-.:;::::· '- ,

" '--- v"

--..._-...... ' ) //1

/

~ \ \ \,.., HEMORRHAGE

I "'j'"''"""·l.

~- .1\ I\\/!

I t ANEU YSM

SUBDURAL

BRAIN

EMPYEMA

ABSCESS

\

l_

\

I

I

I ~ /

1 MYcof.c ANEURYSM 1

.... "'?'

~ ,/

"-

I

-- --

I

I _.

....... --

Fir;ure 7. Portion of causal rule network, showin& cOMeetto,llo eliolo&ical taxonomy

· So called because they Indirectly control the invocation of the domain-dependent object rule1. Davis' conception of meta-rules was thai they would directly order objecl-level rules. However, in our theory of diagnostic strategy, meta.:rutes reason ebout the stat,e of lhe differential and knowledge sources (kinds of evidence) that could change il In desirable ways. Thus, our meta-rule· chooae kinds of object rules (hypothesis-confirmina, proea11-orlenled, ceusal).

""While we might uy that an unknown mess le1ion (a lpece­ oc:cupylna substance) Is caused by · tumor, It is more proper to repre·ent 1 tumor as 1 kind of m111 lesion.

I

I

1

/ .......

/

I

Diseese process knowledse Is represented 11 e freme nsocleted with c:erlein lexonomic paremeters. Slots are process descriptors such as "extent," "loc1tion," and "course", associated with 1 literal value and .a pointer to the parameter to establish it. For example, 1ssociated with breln abscess is the triple (EXTENT FOCAL FOCALSIGNS), meanins that the extent of the disease is focel end this can be determined by 11kina about "focal sisns.· Disease process knowledge is ·orthosonal to the etiotosical taxonomy, makins It useful for groupins and discrimlnalins hypotheses (sn typeac:rlpt before question 6).

VI RELATED RESEARCH

Besides the ICAI projects cited In the introduction, our work he1 been motivated by previous research In teac:hina problem­ solvlna slretegies (e.g., (19) (6) (30)). We believe NEOMYCIN is the first eltempl to · formelize e ruMable psycholosical model of dieanostlc: strategy which can be presented to 1 student. AI should be obvious from our represenlations, a considerable debt Is owed to the medical problem-solvin' literature, cited above.

Both· Reggie (23) and Aikins (I] modified the MYCIN system to meke it more acceptable to physicians, particularly to improve knowledge ecquisition. Aikins use of an eliologicel taxonomy end trlaaer rules, derived from Rubin's work, is pertic:ularly close to our epproac:h. However, we go e step further by representina slretegicel knowledge separately in domain-independent form. Our leeching epplic:alion has elso made clear the importance of disease process knowledge for broedening the diagnostic: renae of a consultation program.

Other research in cognitive psychology has been helpful to us, perticularly studies at the Learning Research Development Center (2), (I 0], (15] in modelling the differences between experts end novices in geometry and physics problem solvinc. To some extent, our eltempt to "decompile" MYCIN's knowledge is the inverse of Anderson's tesk of modellina how e novice composes end aenerellzes knowledae from experience.

VII SOME LIMITATIONS

Pople's experience has been useful to point out limitations in our desian. He shows that a simplistic causal network is not ,edequate when en ellempt Is made to represent ell of seneral lnlernel medicine (22~ For ex1mple, when the ceusal connections between data end the taxonomy ere long and complex, it may not be feeslble to follow each path (possible cause). His "bridge concepts· (similar to Feltovich's "logical competitor sets" (15)) are attempts to model how an expert jumps over to distal, tentative hypotheses. They essentially provide a quick way to lind the intersection ol ceuses for 1 set ol disease symptoms.

Similarly, Rubin's thesis illuslretes a number ol strategies lor combining .hypotheses (for example, relating complications and causes) that we have not yet found to be important In MYCIN's domein. To this extent, our model Is not the complete story of humen dleanoslic: reesonlna, but II c:en be buill on 11 we expand our experience Into other domeins. We do not yet understand how en expert oraanizes his differential! how context is saved and restored from interruplat how ursency, cost, end human velues factor into the dleanostlc: processt end so on.

VIII SUMMARY OF WHAT WE LEARNED

In order to teach diagnosis we need 1 psyc:hotosical model ol problem-solving. In particular, (lesson 1) we need to incorporate into our model the medical knowledse end strategies an expert uses for lnltlel problem formulation. An expert thinks in terms ol e hlererc:hy ol causes end the process characteristics ol a disease so thet he can order the data and his search. Moreover, en expert has learned "compiled associations" that allow him to efficiently 1) essociete hypotheses with data (trigger rules, Pople's bridge concepts), end 2) cast e wide net of questions (general, sc:reenlna, end follow-up (pinning-down) questions).

Also, (lesson 2) we need to represent the various kinds of knowledge explicitly so that they can be accessible lor teachins. Our method is to represent strategical knowledge In dom1in independent form, wholly separate from the medlcel knowledce described above. This requ.ires that the medic1i knowledce be oraenized so that it c:1n be indexed by the stretecies (e.a., as the disease process frame links abstract features to data points).

In 1 sense, we have re-discovered the procedural/declarative problem, as have other cognitive psycholosists (e.c., Anderson, Rumelhert (25]). We allow lor the l1cl th1! the expert hu composed associations, so he does not think about the justifications ol data/hypothesis links, and he makes wide tentative jumps to "bridge concepts." However, we represent these compiled ~ssociali~ns dec:lar~tively lor flexible use, and redundenlly store ~ntermed1ale steps (as text) to ellow lor explanation of reasonina.

IX FUTURE RESEARCH

Development ol NEOMYCIN and GUIDON will proceed in p~rallel. We intend to compare NEOMYCIN's performance to MYCIN to determine if our more principled representation has changed the performance of the system. This is a possibility beceuse we have simplified some rules so they represent more closely the associations a human expert normally remembers. Preliminery runs give comparable results, thouch NEOMYCIN asks fewer questions because ol its focused approach. We also intend to use our new representation for a computer failure di·cnosis consultant, to leal the domain-independence ol our model.

GUIDON2, 1 new version ol GUIOON, will use the NEOMYCIN representation, making it possible to articulate diaanostic strategy. A new phase of development will begin as we try to use the diagnostic strategies (and variants ol them) lor interpreting student behavior, leading to capabilities to evaluate partial solutions and provide essislanc:e. The first version of GUIDON ellempted these things, but was not eble to recocnize or ·ucaesl psyc:holosically velid approaches.

ACKNOWLEDGEMENTS

We are especially grateful to Or. Timothy Beckett, MD, lor his patient explanations, and willingness to be directed in our discussions. Bruce Buchanan and Bob London have al10 contributed to the GLNDON project.