A Model-Based Consultation System for the Long-Term Management of Glaucoma

~ MODEL-BASED CONSULTATION SYSTEM FOR THE LONG-TERM MANAGEMENT OF GLAUCOMA*

Sholan M. Weiss

Institute of Computer Science, Mount Sinai School of Medicine, New York Cas~ir A. Kulikowski

Depart:Jr.ent of Computer Science, Rutgers University, New Brunswick, New Jersey Aran Safir

Institute of Cotr1puter Science, Mount Sinai School of Medicine, New York

ABSTRACT

This paper describes a computer-based systerr. for consultation in the diagnosis and therapy of glaucoma. The reasoning procedures interpret the findings of a particular patient in terrr·s of a causal-associational network (CASNET) rr~el that characterizes the pathophysiological mechanisms and clinical course of treated and untreated diseases.

The major new features of this CASNET/glaucoma program are: a) generation of detailed and medically acceptable interpretations from a qualitative model of glaucoma; b) reasoning about detailed follow-up management of a patient; c) incorporation of alternative expert opinions about subjects under debate; and d) its testing and updating by a collaborative computer-based network of glaucoma

researchers.

INTRODUCTION

Consultation with a specialist is often sought by a practitioner who encounters a particularly complex case, or one that fails to respond to conventional courses of therapy. The consultant brings to bear detailed and up-to-date information from research and clinical practice in several principal tasks: ellcitation of patient history, performance of the physical examination, selection of tests to be performed, and the subsequent interpretation of the case and recommendations for therapy. A corr.puter consultation system that incorporates the knowledge and experience of expert clinicians can be a flexible and useful tool for providing advice in at least three of the above tasks.

In the past few years, artificial intelligence (AI) ideas and methods have been proposed and applied to problems of medical decision-making [1,2,3,4,5]. We have developed an approach that uses a causal-associational network ( CASNET) rr~el of disease to represent the medical knowledge to be used in reasoning about a given patient [1,2,6].

I Thls research was supported in part by grants RR-643 of the NIH Biotechnology Resources Program, and 1-R01-MB-00161 of the Health Resources Administration, HEW.

In this paper we report on a high performance consultation prograrr. for the d1agnosis, prognosis, and therapy of glaucoma based on a CASNET model. This CASNET/glaucorr.a program provides consultation for complex clinical cases, including those with involved histories and multiple follow-up visits. Our approach has . been to separate the descriptive model of disease from the decision-making strategies. Although the data base of medical knowledge constantly changes to incorporate current research, the strategies of decision-making should be able to "roarr. over" that data base without needing change. We also wanted a model of disease that could generate strategies for explanation, question-answering, teaching, and testing as well as those for diagnoses, prognoses, and therapies. Our representation is best suited for qualitative descriptions of diseases as dynamically evolving processes. We use a causal network of events to express the mechanisms of the disease. This formal framework for the prediction and assessment of the time course of the disease before and after treatment has proven to be both an efficient and realistic representation for several types of diseases. Reasoning about . the selection of long-term sequences of therapies based on an explicit descriptive model of d1sease is a characteristic feature of the CASNET method, in contrast to MYCIN's purely norrr.ative or rule-based approach[3], and the concentration on diagnostic reasoning that characterizes the INTERNIST [4] and Present Illness [5,7] programs. Pople has canpared the first three approaches in terms of their

inferential methods [8].

An ~portant characteristic of our system is that it can present alternative opinions derived fran different consultants. To obtain a variety of opinions we have established a national network of collaborating investigator-consultants who share in the development and testing of the program [9]. This .QPhthalmological Network, or ONET, includes glaucana researchers at the Mt. Sinai School of Medicine, Washington University, Johns Hopkins University, the University of Illinois at Chicago, and the University of Miami. The ONET members present the corr.puter consultation program with a variety of complex cases, and weigh its performance against their own judgements. Their subsequent suggestions are used to refine the diagnostic and therapeutic recommendations, to tmprove the system's assessment of signs and symptans, and to perfect specific techniques for acquiring and displaying clinical data. A model-building and editing progran for designing CASNET-type models has also been developed [10], This facilitates the incorporation of new information into the glaucana model or other models of disease. To establish a data base canposed of the glaucana cases presented to the system, we have designed data analysis prograns [ 11] compatible with our mod.els. These permit a review of individual cases over time, as well as the selection of groups of cases according to specified conditions. One of our goals is to develop a mechanism for the routine inclusion of significant new results from current research. Because these are included in the progran as they are produced by clinical researchers, they enhance the progran's capabilities as a teaching tool as well as a consultant.

DESCRIPTION OF TilE PROBLEM OOMAIN

OUr fundamental approach has been to develop methods for describing diseases in terms of models that incorporate current medical knowledge from a variety of sources. Because we wished to include in our model a detailed description of the disease process, we restricted our initial design to a medical problem that is well-defined and circl.lllscribed, yet i~mportant enough to elicit interest from the clinical research cgmmunity. Glaucana was chosen with these constraints in mind. The eye can be adequately described in terms of relatively few anatomical structures, and glaucar1a is a disease whose mechanisns are largely l:united to the eye. Yet, glaucc:rna is a leading cause of blindness in the United States, and a disease whose subtleties are often overlooked until irreparable loss of vision has

occurred.

Several prototype models of diseases other than glaucar1a have been developed using the CASNET rejresentation (e.g. diabetes, anemias, thyroid diseases) · In cc:rnparing these experiences, we have obtained sc:rne insight into the advantages and ltmitations of the CASNET modeling approach. When an understanding of the mechanisns of disease serves as a basis for decision-making, the CASNET approach is most valuable. When reasoning is mostly judgmental, and based more on empirical information than knowledge of the disease mechanisns, other decision models may prove more appropriate [3,12].

The mechaniSJ1S of glaucoma [13,14] are sufficiently well known that they can be used to explain most of the observed patterns of patient f1ndings in terms of causal models, at least to a first approxtmation. Although some aspects of the disease are as yet not well understood, those mechanisns that are known provide a rational franework for diagnostic interpretation and therapeutic planning. In glaucoma, understanding of the different alternative mechanisms and causative factors directly affects choices of therapies. A clinically useful in-depth model can thus be developed. In contrast, less well-understood diseases, such as inflarmatory disorders within the eye, would be poor choices for developing a prototype model, because they follow a mor~ varied and unpredictable course for which detailed mechanisms are as yet poorly understood.

CAUSAL ASSOCIATIONAL NETWORK (CASNET) MODELS

A causal-associational network is a particular type of semantic network [15] designed to describe:

1. pathophysiological mechanisms by causal pathways,
2. external observations by an associational structure related to the pathophysiological states,
3. diseases by classifications irr1posed on the above.

CASNET models can be used to describe many different car1plex processes, although we have developed therr1 to describe pathophysiological medical events [6]. Knowledge, in our scheme, is represented by three types of data elements, corresponding to the three kinds of description outlined above. These are: observations of the patient; states of the causal net; and the diagnostic, prognostic, and therapeutic categories. Observations are the direct evidence obtained about a patient. Pathophysiological states are intermediate constructs that describe conditions or mechanisms that summarize results from many different observations. Categories of disease are conceptually at the highest level of abstraction, summarizing both states and observations. An illustrative example is shown

in Figure 1.

Typical relationships between the elements are as follows:

a) Observations-to-state: ((Decreased Visual Acuity) AND (Perimetry: Paracentral Scotoma)) --irr1plies--> (Field Loss) with strong confidence (Level 4).

b) State-to-state: (Elevated Intraocular Pressure) --causes--> (Field Loss)) with frequency (often - level 0.3).

c)State-to-disease category: (Elevated Intraocular Pressure) AND (no( Field Loss)) --irr.plles--> (High Risk Glaucoma SUspect) under the condition that (Open Angle Glaucoma Mechanisn) has been confirmed.

THE CONSULTATION PROGRAM: CASNET/glaucoma

CASNET/glaucoma is an interactive progran running in 35K words of memory on the PDP-10 computer under either the TOPs-10 or TENEX operating systems. Because of speed and efficiency considerations, it is written in FORTRAN. Modifications and updating of the glaucoma model are carried out by interaction with a separate editing progran [10], written in SNOBOL. This progran checks the model for consistency and compiles it so that it will run efficiently under CASNET/glaucoma.

Diagnostic Interpretation:

Diagnosis is implemented by first interpret1ng the patient's observations in terms of their underlying pathophysiological states. A three-valued logic (confirmed, denied, undetenmined) is used to summarize the truth value of each state taken as a hypothesis for the patient. The truth value is derived by setting a threshold on the confidence measures of all applicable observation-to-state mappings. Diagnoses are then triggered by various configurations of confirrr·ed and denied states

within the causal network.

An important aspect of our approach is that we consider diagnostic interpretation to be much more than the simple assignment of a patient to some pre-specified category. Evaluation of the patient's status is an on-going dynamic process. The patient's clinical status is re-evaluated on successive visits as changes in the presenting signs occur. The causal model summarizes the findings, and guides in the construction of diagnostic and prognostic hypotheses. These hypotheses may be simple hypotheses, such as "very elevated intraocular pressure" or more complex hypotheses (composed of a set of related simple hypotheses) such as "chronic angle closure glaucoma." All hypotheses may inclooe modifiers that fUrther specify a condition by its intensity, duration, progression, topographical distribution, or other characterizing features.

In addition, the statement of the hypothesis will often inclooe a qualitative, verbal estimate of its degree of confirmation (e.g. "very strong likelihood of developing field loss"). Thus, although several measures of confirmation are used in the corr.putations that lead to the selection of the elements that form a hypothesis, no explicit weight has been attached to a hypothesis itself. This has been adequate for glaucorr.a. Extensions of the CASNET model approach may involve the assignment of weights for ranking hypotheses.

Generation of Treatment Recomrr.endations

Among the major factors affecting therapy selection, we consider current diagnostic status, past history, and the desired expected outcome for the patient. Once the patient is undergoing treatment, the effectiveness of the current medication must be assessed, and new factors considered, such as side ·~ffects, corr.plications of the disease for which the current therapy is not effective, conditions not detected at the initial visit, etc. An lll'lportant aspect of our system is that it does not "freeze" past diagnoses as a permanent interpretat1on·, though past diagnoses can be recalled. At each follow-up visit, the entire past history and set of updated findings are re-evaluated, and the possibility of a modified diagnosis considered.

The main representational elements that enter into the generation of a therapy recommendation are illustrated in Figure 2. Each diagnostic conclusion points to a general class of therapies, represented as a ranked preference list of specific therapy states. The glaucoma experts have been able to specify various typical sequences of treatments that they follow as the disease fails to be adequately controlled by a therapy at the previous stage. Thus, the ranking within each general class reflects an increasing degree of severity in the disease and the corresponding treatment. To choose a specific therapy within a given general class, the detailed flndings of the individual patient must be taken into consideration. Factors that do not directly bear on the diagnostic interpretation (allergies, occupational constraints, etc.) often affect the choice of therapy. Each specific therapy state is assigned a weight, derived from the observations, indicating a measure of confidence in the success of a treatment. The treatment with the highest weight is selected from the list of treatinents within a single general class. This scheme permits encoding of conditions that trigger selection of a therapy at a higher level of severity, bypassing the lower-ranked ones (indicated by transition paths in Figure 2). For exanple, if progression of visual f1eld loss occurs more rapidly than expected, a higher dosage of the controlling medication is likely

to be suggested.

If several different recommendations classes are derived treatment frorr, the

diagnostic conclusions a master list is consulted to see if any of the tentatively recommended treatrrtents is covered by the others and is therefore unnecessary, or if there is the possibility of drug interactions or if there is sorr.e binocular constraint that must be taken into account. Such restriction rules are indicated by the arrows labelled R 1 ,R 2 ,R 3 in Figure 2. If complications arise ffom a treatment, or if the original diagnosis is subsequently found to be incorrect, this will lead to a new diagnostic categorization of the pat1ent, which in turn will lead to the selection of a new general class of therapies.

Further Information and Research Studies

There are currently over 200 possible diagnosis and therapy staterrtents, many of which are not mutually exclusive. For most cases, more than a single staterr.ent will be appropriate. These conclusions and recorr~tendations are often explanatory in nature, and sum.arize particularly significant features of a case. · These statements reflect the judgement of our panel of experts and may contain alternative recommendations for diagnoses and therapies. Even so, they must be concise and brief. Another section of the progrcrn has been designed to crnplify the conclusions in some cases. It emphasizes citations from research studies, and quotations frorr. experts to support the progrcrn·s conclusions. Partial or alternative results from studies that may not yet be ready for incorporation into the logic of the model are also included here. The diagnostic and therapy recamrr.endations, for a patient with pigmentary glaucorr.a, followed over an extended period of time, are illustrated in Figure 3. The additional commentary on this form of glaucoma is shown for the first visit of this patient.

RESULTS AND DISCUSSION

The consultation progrcrn was developed in the course of an investigation into the use of artificial intelligence approaches to the representation of disease processes for corrtputer-ba.sed consultation. From this evolved the causal associational network (CASNET) representation, with glaucoma chosen as the disease which would serve for the developnent of a prototype progrcrn. As work progressed, the goals beccrne more clearly those of developing an expert consultation system, in collaboration with a network of glaucoma researchers.

Because the progran' s logic is contained in general strategies that analyze the CASNET model, it is relatively easy to incorporate new medical knowledge or improve existing parts of the model by increasing its depth and breadth of description. The prototype model, which covered the description of the disease in a single eye for a single visit of the patient, was succeeded by a binocular model, which in turn evolved into a model that incorporated follow-up visit information.

Early in our work, we collected a sample of 40 difficult cases. Initially, the progran did not classify (diagnose) all cases correctly. However, as our model improved, it was soon able to diagnose the 40 cases correctly. This result demonstrated, at a relatively early stage, that our approach did provide an incremental means of 1mproving the progrcrn' s performance. We beccrne confident that poor or inaccurate conclusions could be corrected, that cases diagnosed correctly would remain correct, and that diagnostic and therapeutic recorr.mendations could be improved.

This first cycle of . developnent was followed by a second stage, which began when an improved prototype of the glaucoma consultation progrcrn was presented at the annual meeting of the Association for Research in Vision and Ophthalmology in 1973. Interest in the progrcrn, arising from this demonstration of its capabilities, led to the collaboration with several expert glaucoma researchers. It was hoped that their advice would result in a progrcrn that could provide consultation at a sophisticated level. Another motivation for this approach was the authors' desire to test the applicability of the consultation system as a tool for clinical research. The consul tat ion system would serve as a point of entry for difficult clinical cases. A data base of such cases would then be accunulated to test the consultation progrcrn. Statistical analysis of the data might provide new insights into the disease process itself.

Since 1975, researchers from the five different medical centers participating in the Ophthalmological Network (ONET) have been accessing and testing the glaucoma consulation progrcrn. Each has chosen a sub-specialty of glaucoma for testing and developing the progrcrn in depth. Many subtleties of description and reasoning in several types of glaucoma (primary open angle, angle closure, and the secondary glaucomas) have been added in this manner. In addition each ONET member is free to enter any case of glaucoma, and is also able to review the cases entered by his colleagues. There are periodic meetings of the ONET group at which opinions on the different glaucoma topics are discussed and compared. Thus, the sources for alternative opinions that are incorporated into the progrcrn include: a)an ONET member's comments about another's cases; b)suggestions by the originator of a case on alternative conclusions; c)systematic review by the canputer science group, which discovers differences in the handling of similar cases; d)review of the research literature that suggests new alternatives.

Although we currently have several hundred cases on file, it is difficult to evaluate program performance in a simple manner. Classifying conclusions as being merely correct or incorrect is an oversimplification. The program's conclusions are presented not as single unique diagnoses but rather as combinations of judgments about a patient's status. These may inclt.de such factors as: the type and severity of disease, evaluation of current therapy, and recorr~endations for future testing or therapy. In an objective evaluation of program performance, each of these elements must be considered. 1-\:)st of the cases selected by our clinical collaborators are complex and difficult ones requiring expert judgment. Our sanple of cases has thus been deliberately biased to enable us to develop an expert consultant program rather than one that merely does well in a large percentage of typical glaucoma cases. The ONET members have estimated that the prograr. arrives at reasonable and often sophisticated judgrr.ents in about 75~ of the difficult cases of glaucana [16].

The CASNET/glaucoma systerr. was subject to an intensive evaluation by a large and varied group of ophthalmologists during the 1976 meeting of the American Acader.y of Ophthalmology and Otolaryngology. The consul tat ion program was used to summarize results of cases and present its recanmendations, contrasting therr, to the opinions of a panel of experts, at the glaucoma symposiur· just preceding the formal opening of the Acader.y convention. The cases had been entered into the canputer in advance of the symposiun, but the program's conclusions were left unaltered. The panel gave a variety of opinions about the cases, and in almost all of therr. the prograr. included in its alternatives the main interpretation given by the panel.

We also tested the program in a more detailed manner. It was one of the scienti fie exhibits at the Academy meeting. It was displayed and made available for testing by all conference attendees. Evaluation questionnaires were filled out by those ophthalmologists who tested the program. Forty nine responses were obtained. The results are surliT.arized in Table I.

A 95~ acceptance rate for clinical proficiency in the sample questioned is high given the amount of unknown material presented to the co:1sul tat ion systerr. by the ophthalmologists, who were encouraged to test it with their difficult cases. The t~ cases (5~)

in which the prograr. was judged to perform inadequately corresponded to situations in which diseases other than glaucoma formed a significant part of the diagnosis, and the appropriate infom.ation had not yet been included in the model. The 77% rate of high corr.petence (the "expert" and "very competent" responses) ascribed to the systerr. by this independent sample of ophthalmologists accords well with the previously cited judger.ent of our glaucoma collaborators. Efforts currently being devoted to inclt.ding alternative expert opinions in complex cases are expected to improve this performance index in the coming year. The answers to the second q~estion listed in Table I indicate the strong potential that the ophthalmologists saw in using the consul tat ion program as a support tool for organizing clinical trials, and for Sltr.rr.arizing and

analyzing their results.

It is interesting to observe the differences between the responses to the second and third questions. Clearly, the ophthalmologists see an important ultimate contribution to health care (87~ for very or moderately important), but this is secondary to the applicability to glaucarr.a research (95~ for the t1o0 top responses) ·