Integration of Neural Networks and Expert Systems for Process Fault Diagnosis

1 Introduction

The diagnosis of process faults is a difficult, yet vital job for plant operators in modem chemical and nuclear process plants. Many automatic control systems currently used in industry are inefficient in communicating the primary cause of a process upset to plant operators. As exemplified by the incident at Three Mile Island, plant operators can quickly become inundated with alarms and process information immediately following an abnormal event in the process operation. Even for experienced plant operators, timely and accurate fault diagnosis remains difficult [Himmelblau, 1978].

Fault diagnosis can be viewed as the process of linking symptoms to causes, paralleling the field of medical diagnosis. Thus, the goal of process fault diagnosis is to match patterns of sensor measurements and process alarms (the symptoms) to specific equipment malfunctions and operational faults (the causes). Numerous methods and technologies have been developed to aid the plant operator in the diagnosis of faults in chemical and nuclear process plants. Each of these competing technologies have their own distinct advantages and disadvantages as reviewed by Becraft et al. [1991]. Within the past few years, expert systems and neural networks have emerged as the leading methods with which to aid operators in this arduous task.

1.1 Expert System Fault Diagnosis

Many expert systems have been developed for fault diagnosis in several different domains, with varying degrees of success. Expert systems are intuitively attractive for process fault diagnosis, as rules can be explicitly listed linking symptoms (process measurements and alarms) to causes (specific equipment malfunctions and operational faults). These heuristic rules are a form of compiled, or shallow knowledge of an expert. That is, a distinct symptom-cause linkage is expressed without any deep knowledge of the system's structure, function, or principles of operation.

The limitations of rule-based expert systems are revealed when they are confronted with novel fault situations for which no specific rules exist. If the knowledge base does not contain the necessary information about a particular fault situation, the expert system will be unable to diagnose the fault. However, this diagnostic brittleness can be overcome through the inclusion of deep knowledge of the process in the knowledge base, usually in the form of models of the process structure and function [Davis, 1984]. These models can be either qualitative or quantitative simulations of the process. The expert system can then reason from first principles [Davis, 1983] to diagnose the novel faults.

1.2 Neural Network Fault Diagnosis

The use of neural networks is another diagnostic methodology which has shown promise as an aid to plant operators. As opposed to expert systems where the knowledge contained by the system is stored explicitly in the knowledge base as symbols such as words and phrases, the knowledge learned by a neural network is stored implicitly in a distributed manner throughout the network, as the numerical values computed for the different synaptic weights and neuron biases.

When using neural networks for fault diagnosis in chemical and nuclear process plants, the inputs to the network include the sensor values or alarm states of the process, and the outputs of the network represent the presence or absence of particular faults. In many fault diagnostic neural network systems [Dietz et al., 1988; Hoskins and Himmelblau, 1988; Venkatasubramanian and Chen, 1989; Venkatasubramanian et al., 1990; Watanabe et al., 1989], each output node corresponds to one particular fault possibility. An output value of 1 indicates the presence of the fault and an output value of 0 indicates the absence of the fault. The use of output node values to determine fault magnitudes has been examined [Venkatasubramanian et al., 1990], but fault type must still be assigned to a particular output node. Novel faults, for which the neural network has not been trained, or for which no output neuron has been assigned, are generalized and matched to the closest fault scenario for which the network has been trained. Unfortunately, this generalization may or may not be valid, depending on the closeness of match between the novel fault and the faults contained within the network training set.

1.3 Diagnostic Focus Using Neural Networks

Becraft and Lee [1991] investigate a neural diagnostic strategy that solves the problem of diagnosis of novel fault types by using increasing resolution of diagnostic focus. The diagnostic strategy developed uses a hierarchical neural network structure (Figure 1) to successively narrow the diagnostic focus of the system to isolate the fault cause.

The initial neural network examines the overall process and determines in which plant battery, or group of unit operations, the fault is occurring. Each of the second layer neural networks is trained to localize faults within a particular plant battery and assign the cause to a unit operation within that battery. Additional layers may be trained to diagnose the specific fault within the unit operation, and the magnitude of the fault as proposed in previous investigations [Deitz et al., 1988; Venkatasubramanian and Chen, 1989; Watanabe et al., 1989], if it is deemed necessary.

One of the advantages of this diagnostic strategy is the system's ability to degrade gracefully in novel fault situations. Rather than giving a diagnosis of "unknown", or else giving a false positive classification, the system narrows the diagnostic resolution to the particular plant battery, and then to the unit operation where the fault is occurring. Thus, even though the system might be unable to diagnose the actual cause of the fault, useful information is still obtained concerning the fault's location. This information may then be passed on to an expert system for further analysis, or used by an operator to complete the diagnosis and take the appropriate corrective actions.

2 Neural Network I Expert System Integration

The dichotomy of AI approaches to fault diagnosis can be expressed as being either numeric I symbolic, algorithmic I heuristic, or neural network I expert system methodologies. Each approach contains its own strengths and weaknesses: neural networks are extremely fast pattern associators, can handle sensor noise, learn from experience, and generalize in novel fault situations if sufficiently structured and trained, however they are black-box operators unable to explain their own reasoning methodology, incorrectly generalize novel faults if improperly trained, and forget past training if retrained on new data; expert systems have explicit representations of knowledge which eases the modification and validation of the systems, are able to generate explanations for their reasoning methodology, and can use deep knowledge to reason about novel events, but they are unable to learn from experience, are hard to maintain if the knowledge base becomes extremely large, and require extensive computational time if a deep model of the process must be consulted. To take advantage of the strengths of each technique, as well as avoiding the weaknesses of either individual technique, an integrated neural network I expert system diagnostic strategy is used.

The idea of an integrated approach is well supported by authorities in both fields of AI. On the neural network side, Norman [1986] mentions the need for an extra evaluative structure to interpret the results of a parallel distributed processing model. In the afterword of their compendium of neural network research, Anderson and Rosenfeld [1988]

state that hybrid architectures could exploit the potentially powerful synergy obtainable through the use of "high level" traditional artificial intelligence methods and "low level" neural network methods. Elaine Rich [Myers, 1990] describes the various structures that such hybrid architectures could take. Although Edward Feigenbaum dismisses neural networks as "not of much current interest to those who are working on expert systems", he concedes that neural networks could perform a useful pre-symbolic processing function for sensory inputs [Owen, 1989], thus indirect! y supporting the integrated approach.

2.1 Interpretation of Neural Network Diagnoses

In most cases of fault diagnostic neural networks based on continuous activation functions, the values of the neurons in the output layer will not be either one or zero as desired, but fall somewhere in between these two values. When this occurs, the diagnosis presented by the neural network is not automatically generated, but must be interpreted by some external means. The usual method by which non-binary diagnostic neural network outputs are distinguished is through the use of a diagnostic threshold, or referent value [Weiss et al., 1990]. This threshold is a cutoff point for neural outputs, values above which indicate the presence of the fault and values below which indicate the absence of the fault. Depending upon the value of the diagnostic threshold, the number of incorrect diagnoses of the neural network can vary significantly. With a high diagnostic threshold, fewer cases will be diagnosed correctly as indicating the presence of a fault. Thus, some cases in which a fault is actually present will be incorrectly diagnosed as being fault-free. With a low diagnostic threshold, more cases will be incorrectly diagnosed as indicating the presence of a fault when none is actually present. Depending upon the diagnostic philosophy of the particular plant operation, a suitable diagnostic threshold must be selected in order to evaluate the neural network outputs.

An alternative to selecting a set diagnostic threshold is to evaluate the classification order of the output values. Thus, a range of possible fault causes could be evaluated according to the magnitudes of their neural outputs. A large output value may not be high enough to warrant an absolute diagnosis according to a set diagnostic threshold, but may indicate the high possibility of the fault's presence when compared to the other alternatives. It is such an analysis of neural outputs which is extremely suitable for higher intelligence processes, such as an expert systems.

2.2 Embedding Neural Networks in Expert Systems

The key to successful fault diagnosis using the combined methodology is the integration of the neural networks and expert systems. Embedding a neural network within an expert system appears to be an effective architecture for a process fault diagnostic system (Figure 2). Much of the information required by the diagnostic system is numeric in nature (process sensor readings), which is easily handled by the neural networks. The hierarchical neural diagnostic structure developed by Becraft et al. [1991], provides useful fault detection and location information, even if it cannot determine the actual fault cause directly. The expert system activates the neural diagnosis to generate the potential fault candidate list. It then analyses the neural network outputs, and either confirms the diagnosis, or offers an alternative solution. The expert system takes into account symbolic information issues which are poorly handled by neural networks.

3 Fault Diagnosis Case Study

In order to illustrate the integration of the neural networks and expert systems for process fault diagnosis, a case study is presented. The case study investigates the use of the hierarchical neural network diagnostic strategy as well as the interpretation of the results by the expert system.

3.1 Case Study Process Description

The process chosen to demonstrate the operator advisory system's capabilities is the multi-column distillation plant, shown in Figure 3. The objective of the distillation plant is to separate the feedstock mixture containing benzene, chlorobenzene, and meta-dichlorobenzene, obtained from a hydrocarbon chlorination plant, into the desired distillation products consisting of a 99% pure meta-dichlorobenzene stream, an 88% pure benzene stream, and a dichlorobenzene stream of at least 95% purity. This objective is accomplished by using two distillation columns in series, with their associated condensers, reflux drums, reboilers, and pumps. A feed stream preparation battery, consisting of a pump, a flow controller, and a heat exchanger, precedes the first column in the flowsheet. The heat exchanger in the feed preparation battery uses the energy from the first column's bottoms product to preheat the feed stream to the desired level. The control system for the plant consists of the necessary sensors, control valves, and controllers required to maintain the desired flowrates, pressures, and levels of specific streams and units throughout the plant. The flowsheet contains 34 process streams connecting 14 major unit operations and 4 minor unit operations, with 11 control valves and associated controller loops. The case study process is simulated using HYSIM, a process flowsheet simulator developed by Hyprotech, Ltd [1989].

3.2 Neural Network Development and Training

The neural networks used in the case study were built, trained, and tested using the Intelligent Neural Network Application Testing Environment (INNATE), a simulation package written in FORTRAN by Becraft [1991]. The neural network development effort was completed on an Apollo DN10000 workstation, with a modified version of INNATE embedded into the PC-based expert system, enabling easy portability of the trained neural networks used in the diagnostic strategy.

The basic neural network utilized in the operator advisory system's fault diagnostic strategy is the backwards­ propagating multilayer perceptron network incorporating the generalized delta rule, as developed by Rumelhart, Hinton and Williams [1986]. A learning rate, fl, of 0.65 and a momentum term, a, of 0.80 were chosen as the training parameters for the networks. A convergence criteria, e, of 0.1, the allowable difference between the actual output of a neuron in the output layer and the desired or target output, is used to determine the extent of further network training required. The training methodology used is reported in [Becraft, 1991]. The standard sigmoidal activation function

[Rumelhart et al., 1986] is used to calculate each neuron's activation.

Two levels of neural networks are required to implement the hierarchical neural network diagnostic strategy. The first level, or Main Plant neural network is trained to identify the particular plant battery where the fault originated. The main network contains 42 input nodes, corresponding to the process measurements from 12 streams in the plant. These streams include all of the streams entering and leaving the plant, as well as several internal streams which would normally be monitored if the distillation plant were an actual industrial installation. One hidden layer containing 42 nodes is used in the main network. The output layer consists of three output nodes, assigned to represent the three different plant batteries: the Preheat Battery, the Column 1 Battery, and the Column 2 Battery.

Each of the second level networks, one for each plant battery in the distillation plant, are trained to further isolate the cause to a particular unit within the plant battery. The inputs used in each network are the process measurements of the streams crossing its battery limits, as well as measurements from internal streams which would normally be monitored. The second level network inputs include additional process measurements not presented to the Main Plant neural network, thus providing a greater degree of resolution of the plant battery state. One hidden layer is used in each network, with the number of hidden nodes being equal to the number of input nodes for that particular network. Each node in the output layer corresponds to a unit contained within the plant battery limits. An additional output node is assigned to signify a process disturbance occurring upstream from the plant battery, thereby negating incorrect diagnoses due to fault propagation effects. Consequently, the Preheat neural network consists of 21 input nodes, 21 hidden nodes, and 5 output nodes, and each of the column battery networks consists of 33 input nodes, 33 hidden nodes, and 15 output nodes.

3.3 QUALMS Expert System Development

The Queensland University ALarm Management System (QUALMS) is a deep knowledge, model-based expert system for the diagnosis of faults in chemical process plants. QUALMS was developed using a PC-based expert system shell, Intelligence/Compiler, developed by Intelligence Ware, Inc [1987]. Intelligence/Compiler supports backward chaining, forward chaining, exact reasoning, semi-exact reasoning, inexact reasoning, and the use relational database facilities within its inference engine structure. The object­ oriented knowledge base of QUALMS consists of frames, rules, facts, and lists encoding the knowledge required by the expert system. The expert system utilizes simulation­ derived data about the operation of the process plant being diagnosed, which is generated by HYSIM. The QUALMS knowledge base contains generic process knowledge about chemical plants including frames for standard chemical plant unit operations, as well as process specific knowledge such as the plant topology and the functionality of specific units within the distiilation plant case study process. Thus, the expert system is designed for easy portability between industrial installations, with only the plant specific knowledge contained within the modular knowledge base requiring to be changed, as well as the development and training of neural networks specific to each plant.

The diagnostic metaknowledge contained within QUALMS includes the necessary information to collect the current sensor data from the process, run the modified version of INNATE to classify the present fault scenario using previously trained neural networks, and to analyze the results of the hierarchical neural diagnostic strategy. Table 1 shows some of the rules contained in QUALMS which embed the the neural network program, INNATE, within the expert system. The results of the neural diagnosis are presented to the user in the form of both the numerical values of the output neurons and a relative classification ordering, and confirmation of the diagnosis by the deep knowledge expert system begins.

Table 1. INNATE/QUALMS Integration Rules

ACTIVATE Main Net, 'tablename'

PRINT NEURAL ACTIVATIONS Main Net,'tablename'

ACTIVATE 'network', 'tablename' If

WRITE "Running INNATE classification" and

'netbatch' : = (batchfile of 'network') and

C DOS, 'rcode' ,INNATE,"<", 'netbatch' and

CONCATENATE 'network', "-output" ,'tablename' and GET DATA 'network','tablename';

PRINT NEURAL ACTIVATIONS 'network','tablename' If

WRITE "Neural activations for: ",'network' and PRINT-TABLE 'tablename';

4 INNATE/QUALMS Diagnostic Performance Analysis

In addition to the correctly functioning, fault-free scenario simulation of the case study process, 35 single fault scenario simulations were developed for the process. These fault scenarios cover an extensive range of possible faults which could occur in the case study process. The data from these examples were used to train the neural networks used in the hierarchical neural diagnostic strategy.

4.1 Noise-free Fault Scenario Diagnosis

Each individual neural network in the hierarchical diagnostic strategy was tested on the noise-free sensor data from all 36 scenarios. With the noise-free sensor measurements, each network performed flawlessly, with zero incorrect diagnoses throughout the entire test set. This result was to be expected, as these were the fault scenarios for which either the neural networks were trained, or which had no effect on the particular network's plant battery. The expert system analyzed the results and confirmed the diagnoses obtained from the neural diagnostic strategy.

4.2 Novel Fault Diagnosis

To test the system's diagnostic abilities in novel fault situations, an additional fault scenario was conceived for which the neural networks were not trained. A simulation of the distillation plant was developed, containing a pipe burst or leak in the stream connecting the second distillation column and the pump immediately prior to it. Using a pipe burst or leak as the novel fault scenario was deemed appropriate as it is a commonly encountered process fault in industrial situations. Additionally, the location of the fault was not considered to be a unit operation, therefore the individual neural networks did not have a specific output node assigned to diagnose the fault.

When presented with the sensor data from the novel fault scenario, the first level Main Plant neural network correctly diagnosed the fault to be located in the Column 1 plant battery, even though the fault was occurring downstream from the last unit operation in the plant battery. Each of the second level neural networks also correctly diagnosed the fault. The Preheat neural network correctly indicated that no faults were occurring in the Preheat Battery. The Column 2 neural network correctly identified the fault as a process disturbance occurring upstream of the Column 2 Battery. The Column 1 Battery neural network indicated that none of the unit operations in its plant battery were the cause of the fault, which is the correct diagnosis for that battery. Thus, while no individual second level neural network was able to give the correct diagnosis of the situation, the expert system was able to integrate the results from each of the individual neural networks to give the proper diagnosis for the novel fault situation: the first level network for the Main Plant narrows the diagnosis to the Column 1 plant battery, and the individual second level networks indicate that the fault is occurring in the stream connecting the Column 1 and Column 2 plant batteries, thus providing the correct diagnosis.

4.3 Robustness to Sensor Noise

The diagnostic system was also tested in the presence of sensor noise. Random fluctuations of up to 5% and 10% noise levels were introduced into the process measurements used by the system. The diagnostic performance of the hierarchical neural networks decreased slightly at the 5% noise level, with a greater degradation of performance occurring at the 10% levels, as seen in Table 2. For the majority of cases, the expert system was able to confirm the faults diagnosed.

5 CONCLUSIONS

An artificially intelligent operators' advisory system, INNATE/QUALMS, was designed to aid in the diagnosis of faults in large-scale chemical process plants. The integration of two fundamentally different diagnostic techniques, neural networks and expert systems, was used successfully to diagnose faults in a example case study process. The integration of the two technologies was investigated. The diagnostic system which was developed exhibited good diagnostic performance under a variety of conditions including novel faults, and the presence of sensor nmse.