Model-based Diagnosis in the Real World: Lessons Learned and Challenges Remaining
Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Invited Speakers. Pages 1393-1399.
© 1999 International Joint Conferences on Artificial Intelligence
Abstract
This paper discusses some trends in modelbased diagnosis. We consider some recent applications and discuss why they were possible, the lessons we learned from them, the new impulse that they gave to research in the field and the new challenges that emerged from them.
- [Allred et al., 1991] D. Allred, Y. Lichtenstein, C. Preist, M. Bennet, and A. Gupta. AGATHA: an integrated expert system to test and di- agnose personal computers boards. In R. Smith and A. Carlisle Scott, editors, Innovative Applications of AI 3, pages 87-104. AAAI Press, 1991.
- [Baroglio et al., 1994] C. Baroglio, M. Botta, and L. Saitta. WHY: a system that learns using causal models and examples. In R. Michalski and G. Tecuci, editors, Machine Learning, an Artificial Intelligence Approach (vol IV).
- Morgan Kaufman, 1994.
- [Brusoni et al., 1998] V. Brusoni, L. Console, P. Teren- ziani, and D. Theseider Dupre. A spectrum of defi- nitions for temporal model-based diagnosis. Artificial Intelligence, 102(1):39-79, 1998.
- [Bylander et al., 1991] T . By- lander, D. Allemang, M. Tanner, and J. Josephson. The computational complexity of abduction. Artifi- cial Intelligence, 49(1-3):25-60, 1991.
- [Cascio and Sanseverino, 1997] F. Cascio and M. San- severino. IDEA (Integrated Diagnostic Expert Assis- tant) model-based diagnosis in the the car repair cen- ters. IEEE Expert, 12(6), 1997.
- [Cascio et al., 1999] F. Cascio, L. Console, M. Guagli- umi, M. Osella, A. Panati, S. Sottano, and D. The- seider Dupre. Generating.on-board diagnostics of dy- namic automotive systems based on qualitative devi- ations. AI Communications, 12(1), 1999.
- [Chandrasekaran and Milne, 1985] B. Chandrasekaran and R. Milne. Special section on reasoning about structure, behavior and function. In Sigart Newsletter, volume 93, pages 4-55. 1985.
- [Chantler et al., 1996] M.J. Chantler, S. Daus, T . Vikatos, and G.M. Coghill. The use of quantitative dynamic models and dependency record- ing for diagnosis. In '7>roc. 7th Int. Work. on Principles of Diagnosis, pages 59-68, 1996.
- [Chantler et al., 1998] M.J. Chantler, G.M. Coghill, Q Shen, and R.R. Leitch. Selecting tools and tech- niques for model-based diagnosis. Artificial Intelli- gence in Engineering, 12:81-98, 1998.
- [Console and Torasso, 1991] L. Console and P. Torasso. A spectrum of logical definitions of model-based diag- nosis. Computational Intelligence, 7(3):133-141, 1991.
- [Console et al., 1996] L. Console, L. Portinale, and D. Theseider Dupre. Using compiled knowledge to guide and focus abductive diagnosis. IEEE TI-ansac- tions on Knowledge and Data Engineering, 8(5):690- 706, 1996.
- [Cordier, 1998] M.-0. Cordier. When abductive diag- nosis fails to explain too precise observations: an ex- tended spectrum of definitions of MBD based on ab- stracting observations. In Proc. 9th Int. Work. on Principles of Diagnosis, pages 24-31, 1998.
- [Davis and Hamscher, 1988] R. Davis and W. Ham- scher. Model-based reasoning: Troubleshooting. In H.E. Shrobe, editor, Exploring Artificial Intelligence, pages 297-346.
- Morgan Kaufman, 1988.
- [Davis, 1982] R. Davis. Expert systems: Where are we? And where do we go from here? AI Magazine, 3(2):3- 22, 1982.
- [Davis, 1983] R. Davis. Diagnosis via causal reasoning: paths of interaction and the locality principle. In Proc AAAI 83, pages 88-94, 1983.
- [Davis, 1984] R. Davis. Diagnostic reasoning based on structure and behavior. Artificial Intelligence, 24(1- 3):347-410, 1984.
- [de Kleer and Williams, 1987] J. de Kleer and B.C. Williams. Diagnosing multiple faults. Artificial In- telligence, 32(1):97-130, 1987.
- [de Kleer and Williams, 1989] J . de Kleer and B.C. Williams. Diagnosis with behavioral modes. In Proc. 11th IJCAI, pages 1324-1330, Detroit, 1989. IJCAI
- [de Kleer et al., 1992] J . de Kleer, A. Mackworth, and R. Reiter. Characterizing diagnoses and systems. Ar- tificial Intelligence, 56(2-3):197-222, 1992.
- [de Kleer , 1986] J. de Kleer. An . assumption-based TMS. Artificial Intelligence, 28(2) :127-162, 1986.
- [de Kleer, 1991] J. de Kleer. Focusing on probable diag- noses. In Proc. AAAI 91, pages 842-848, 1991.
- [Doyle, 1997] R. Doyle. The emergence of spacecraft au- tonomy. In Proc. AAAI 97, pages 756-761 , 1997.
- [Dressler and Freitag, 1994] Dressler and H. Freitag. Prediction sharing among time and contexts. In Proc. AAAI 94, pages 1136-1141, 1994.
- [Dressler and Struss, 1994] Dressler and P. Struss. Model-based diagnosis with the default-based diag- nosis engine: Effective control strategies the work in practice. In Proc. 11th EGA!, pages 677-681, 1994.
- [Dvorak and Kuipers, 1989] D. Dvorak and B. Kuipers. Model-based monitoring of dynamic systems. In Proc. 11th IJCAI, pages 1238-1243, Detroit, 1989. IJCAI
- [Eiter and Gottlob, 1995] T. Eiter and G. Gottlob. The complexity of logic-based abduction. Journal of the ACM, 42(1):3-42, 1995.
- [Friedrich and Nejdl, 1992] G. Friedrich and W. Nejdl. MOMO: Model-based diagnosis for everybody. In W. Hamscher, L. Console, and J . de Kleer, editors, Readings in Model-based Diagnosis, pages 143-149.
- Morgan Kaufman, 1992.
- [Friedrich et al., 1994] G. Friedrich, G. Gottlob, and W. Nejdl. Formalizing the repair process -extended report. Annals of Mathematics and Artificial Intelli- gence, 11(1-4):187-202, 1994.
- [Genesereth, 1984] M.R. Genesereth. The use of design descriptions in automated diagnosis. Artificial Intelli- gence, 24(1-3):411-436, 1984.
- [Hamscher and Davis, 1984] W. Hamscher and R. Davis. Diagnosing circuit with state: an in- herently underconstrained problem. In Proc. AAAI 84, pages 142-147, 1984.
- [Hamscher et al., 1992] W. Hamscher, L. Console, and J. de Kleer, editors. Readings in Model-Based Diag- nosis. Morgan Kaufmann, 1992.
- [Hamscher, 1991] W. Hamscher. Principles of diagnosis: Current trends and report on the first international workshop. AI Magazine, 12(4):15-23, 1991.
- [Hart, 1982] P.E. Hart. Directions for AI in the eighties. Sigart Newsletter, 79:11-16, 1982.
- [Lackinger and Nejdl, 1991] F. Lackinger and W. Nejdl. Integrating model-based monitoring and diagnosis of complex dynamic systems. In Proc. 12th IJCAI, pages 1123-1128,Sydne~ 1991. IJCAI
- [Lamperti and Pogliano, 1997] G. Lamperti and P. Pogliano. Event-based reason- ing for short circuit diagnosis in power transmission networks. In Proc. IJGAI 97, pages 446-451, 1997. IJCAI
- [Leitch et al., 1992] R. Leitch, H. Freitag, Q. Shen, P. Struss, and G. Tornielli. Artist: A methodolog- ical approach to specifying model-based diagnostic systems. In G. Guida and A. Stefanini, editors, In- dustrial Applications of Knowledge-Based Diagnosis, pages 183-208. Elsevier, 1992.
- [Leitch et al., 1994] R. Leitch, M. Chantler, Q. Shen, and G. Coghill. A preliminary specification methodol- ogy for model-based diagnosis. Annals of Mathematics and Artificial Intelligence, 11(1-4):11-32, 1994.
- [Levesque, 1989] H.J. Levesque. A knowledge-level ac- count of abduction. In Proc. 11th IJGAI, pages 1061- 1067, Detroit, 1989. IJCAI
- [Loiez and Taillibert, 1997] E. Loiez and P. Taillibert. Polynomial temporal band sequences for analog di- agnosis. In Proc. IJGAI 97, pages 474-479, 1997. IJCAI
- [Lucas, 1998] P. Lucas. Analysis of notions of diagnosis. Artificial Intelligence, 105(1-2):295-343, 1998.
- [Malik and Struss, 1996] A. Malik and P. Struss. Diag- nosis of dynamic systems does not necessarily require simulation. In Proc. 7th Int. Work. on Principles of Diagnosis, pages 147-156, 1996.
- Monet,] Monet. The European network of excel- lence in model-based systems and qualitative reason- ing. http:/ /monet.aber.ac.uk. Link
- [Masterman and Biswas, 1997] P.J. Masterman and G. Biswas. Monitoring, prediction and fault isolation in dynamic physical systems. In Proc. AAAI 97, pages 100-105, 1997.
- [Masterman et al., 1998] P.J. Masterman, G. Biswas, and E. Manders. A comprehensive framework for model based diagnosis. In Proc. 9th Int. Work. on Principles of Diagnosis, pages 86-93, 1998.
- [Mozetic, 1990] I. Mozetic. Diagnostic efficiency of deep and surface knowledge in KARDIO. Artificial Intelli- gence in Medicine, 2(2):67-83, 1990.
- [Mozetic, 1991] I. Mozetic. Hierarchical model-based di- agnosis. Int. J. of Man-Machine Studies, 35(3):329- 362, 1991.
- [Patil, 1981] R. Patil. Causal representation of patient illness for electrolyte and acid-base diagnosis. Techni- cal Report LCS-267, MIT, Cambridge, MA, 1981.
- [Pearl, 1989] J. Pearl. Probabilistic Reasoning in Intel- ligent Systems. Morgan Kaufmann, 1989.
- [Peng and Reggia, 1991] Y. Peng and J. Reggia. Abduc- tive inference models for diagnostic problem solving. Springer-Verlag, 1991.
- [Poole, 1989] D. Poole. Normality and faults in logic- based diagnosis. In Proc. 11th IJGAI, pages 1304- 1310, Detroit, 1989.
- [Porcheron et al., 1994] M. Porcheron, B. Ricard, J .-L. Busquet, and P. Parent. DIAPO: a case study in ap- plying advanced AI techniques to the diagnosis of a complex system. In Proc. 11th EGA!, pages 43-47, 1994.
- [Portinale and Torasso, 1996] L. Portinale and P. Torasso. On the usefulness of reusing diag- nostic solutions. In W. Wahlster, editor, Proc. EGA! 96, pages 137-141.
- J. Wiley, 1996.
- [Price, 1998] C. Price. FUnction-directed electrical de- sign analysis. Artificial Intelligence in Engineering, 12:445-456, 1998.
- [Provan, 1998] G. Provan. Modeling and diagnosing rocket engines: a multimodeling approach. In Proc. 9th Int. Work. on Principles of Diagnosis, pages 102- 109, 1998.
- [Raiman and de Kleer, 1995] Raiman and J. de Kleer. Trading off the cost of inference vs. probing in diagnosis. In Proc. 14th IJGAI, pages 1736-1741 , Sydney, 1995.
- [Reiter, 1987] R. Reiter. A theory of diagnosis from first principles. Artificial Intelligence, 32(1):57-96, 1987.
- [Searl, 1994] E. Searl. Sensor placement for diagnosabil- ity. Annals of Mathematics and Artificial Intelligence, 11(1-4):493-510, 1994.
- [Struss and Dressler, 1989] P. Struss and 0. Dressler. Physical negation -integrating fault models into the general diagnostic engine. In Proc. 11th IJGAI, pages 1318-1323, Detroit, 1989. IJCAI
- 1992a] P. Struss. Diagnosis as a process. In W. Hamscher, L. Console, and J. de Kleer, editors, Readings in Model-based Diagnosis, pages 408-418.
- Morgan Kaufman, 1992.
- 1992b] P. Struss. Knowledge-based diagnosis: an important challenge and touchstone for AI. In Proc. 10th EGA!, pages 863-874, Vienna, 1992.
- [ten Teije and van Harmelen, 1994] A. ten Teije and F. van Harmelen. An extended spectrum of logical definitions of diagnosis. In Proc. 5th Int. Work. on Principles of Diagnosis, pages 334-342, 1994.
- [Trave Massuyes and Milne, 1997] L. Trave Massuyes and R. Milne. Gas turbines condition monitoring us- ing qualitative model-based diagnosis. IEEE Expert, 12(3):21-31, 1997.
- [Weiss et al. , 1978] S. Weiss, C. Kulikowski, S. Amarel, and A. Safir. A model based method for computer- aided medical decision making. Artificial Intelligence, 11(1-2) :145-172, 1978.
- [Williams and Nayak, 1996] B. Williams and P. Nayak. A model-based approach to reactive self-configuring systems. In Proc. AAAI 96, pages 971-978, 1996.
- [Williams and Parker, 1983] T.W. Williams and K.P. Parker. Design for testability: A survey. Proceedings of the IEEE, 71(1):98-112, 1983.
References were extracted automatically from the PDF and may contain errors.
Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.
1 Introduction
The aim of this paper is to discuss some recent stories concerning the application of model-based diagnosis to industrial domains and the lessons we (as individuals and as a community) learned from such stories.
Model-based Diagnosis (MBD in the following) is an approach to diagnosis that was proposed in the early 80's to overcome limitations of the traditional expert systems approach (see [Hamscher et al., 1992] for a collection of papers and [Davis and Hamscher, 1988] for an introduc tion). The philosophy of the model-based approach can be sketched as follows. Diagnosis should be based on an objective model of the device (system) to be diagnosed. More specifically, different types of models can be con sidered: structural (concerning the physical or logical structure of a device), functional (describing the func tions of a device), behav:ioral (describing how a device works, i.e., how its functions are achieved), teleological (describing the purposes of the use of a device), or a combination of them. Models should be reusable, in two ways. On the one hand, the same model of a device should be used for different problem solving tasks (such as diagnosis, simulation, reconfiguration, ... ). On the other hand, models should be compositional: the model of a device should be usable in all the cases where the device is used as a component of a larger system.
Since the 80's a lot of work has been carried on and
- several approaches to modeling have been proposed and investigated;
- definitions of diagnosis and formal accounts of the task have been presented;
- algorithms for solving the problem and overcoming its complexity have been designed;
- applications have been developed.
Oskar Dressler
Occ'm Software GmbH,
D-82041 Deisenhofen (Germany)
email: dressler@occm.de
http:/ /www.occm.de
MBD attracted researchers from many different fields of AI (and engineering) and played both the role of testbed of several approaches to KR and reasoning and the role of generator of new problems and approaches.
From an industrial point of view, diagnosis (or, more generally, guaranteeing the availability of systems) is a primary need. Indeed, several diagnostic systems have been implemented since the mid 70's and some of them were success stories for AI. More recently, however, some links became stronger and this led, at least in some areas, to important projects and applications. In particular, the two authors were involved in some of these projects.
In the paper we analyze the reasons for such important evolution pointing out why, in our view, MBD is ready for applications (Sect. 2), the applications that have been built and the lessons we learned from them (Sect. 3), the new research issues that arose from applications (Sect. 4), concluding with some considerations on the implications for future activity in the field (Sect. 5).
2 Why MBD is ready for application
In this section we discuss the reasons that in our view make MBD ready for real application. In order to do that, it is interesting to start by looking at some past seminal talks and papers that strongly influenced our ap proach to the field [Davis, 1982; 1983; 1984; Hart, 1982; Chandrasekaran and Milne, 1985] (and recent ones, such as [Hamscher, 1991; Struss, 1992b]). Those paper pointed out the potential advantages of the model-based approach, as well as opportunities and goals.
Did we achieve all of those goals and see all the op portunities? Certainly not. Some of them have been widely studied and we now have a clear understand ing and available solutions (e.g., many of the problems concerning modeling, the definition of diagnosis and al gorithms for computing diagnoses efficiently had major advances). On the other hand, there are many aspects (such as dealing with dynamics or integrating models) that are still open research problems.
However, what was achieved in these years is in some sense more than solving many specific problems. In our view we succeeded in establishing ourselves as a com munity, with its agreed language and basic definitions. The foundations of the field are well established and we share a clear specification of what the problem is and what its basic ingredients are. A good deal of theoretical work provided semantic foundations for our definitions and almost all of us have this semantics in mind when discussing about research topics or applications.
Let us analyze these foundations in more detail.
First of all, the diagnostic process relies on models which are in most cases component-oriented. A device is described in terms of its minimal replaceable or re pairable components. For each type of component the model includes: a list of its variables (interface, inter nal or state variables, parameters) , as well as a defi nition of its modes of behavior (including correct and fault modes). Then the behavior of the component is described via a set of relations (constraints). Both the nominal (correct) behavior and a set of known faulty be haviors can be described [de Kleer and Williams, 1989; Struss and Dressler, 1989], exploiting different modeling assumptions. For example, a model may be quantita tive or qualitative (using different types of qualitative abstractions); dynamic or static (in many cases the dy namics of a ·system is abstracted for the sake of sim plicity); crisp or probabilistic (where incomplete and/or imprecise information is represented using numeric in formation, as e.g., in probabilistic networks).
The models of the components constitute a library of basic units that can be used for building models of com plex devices (which only need to specify the structure of the device in terms of its components). Such com positionality assumption is a fundamental one, allowing the re-use of models (and thus the possibility of building the diagnostic system for a new device without major ef forts) . Obviously, the assumption has some major conse quences on modeling (e.g., the well-known "no structure in function" principle which requires that the model of a component does not depend on the contexts in which the component may be used in a complex device).
Although the component oriented approach is the most widely adopted, it is not the only one. There are cases, in fact, where it is not easy or interesting to iso late components or the behavior of a system is not easily obtained as the composition of the behavior of its com ponents. This is the case, for example, when processes have to be modeled or when the structure of the sys tem is complex, as, e.g., in physiological systems. Thus, there are approaches that are based on process-oriented models that describe the global behavior of a system, usually in terms of causal networks (see e.g., [Patil, 1981; Porcheron et al., 1994; Provan, 1998]).
The second foundation is the definition of diagnosis. There is a diagnostic problem whenever the observed behavior of a device is not in accordance with the ex pected behavior, i.e. , the behavior that can be predicted from the model, assuming that all the components are in a normal mode (fault detection). Diagnosis corre sponds to removing such an inconsistency and to ex plaining the faulty behavior. This amounts to finding which components of the system may be faulty (fault isolation) or, more precisely, finding an assignment of a mode to each component in such a way that the ob served behavior is explained (fault identification). Dif ferent notions of explanation can be adopted, ranging from weaker ones based on consistency [Reiter, 1987; de Kleer et al. , 1992] to stronger ones based on ab duction [Poole, 1989]; see also the analysis in [Console and Torasso, 1991]. Besides the logical characteriza tion, there are also set-theoretic [Peng and Reggia, 1991; Lucas, 1998] and probabilistic ones [Pearl, 1989].
Several other aspects of the diagnostic process have been analysed and formalized: e.g., diagnosis as an it erative process [Struss, 1992a] test/probe selection [de Kleer and Williams, 1987], repair [Friedrich et al., 1994], integration with monitoring [Dvorak and Kuipers, 1989; Lackinger and Nejdl, 1991].
Characterizing diagnoses is a necessary and interest ing problem but it is not sufficient . Several algorithms for computing diagnoses that are correct wrt the char acterization have been designed. The efficiency issue has been tackled from different points of view. From the theoretical one there are analyses of the complex ity of the problem and of the special cases in which it is tractable (e.g., see [Levesque, 1989; Bylander et al., 1991; Eiter and Gottlob, 1995; Friedrich and Nejdl, 1992]). From the practical point of vie"', several strategies for computing diagnoses efficiently have been devised. On the one hand, some approaches defined notions of "pre ferred diagnoses" (e.g., based on cardinality - sin gle faults diagnoses are the preferred ones - or on other orderings or on fault probabilities) and algorithms for generating only those diagnoses [de Kleer, 1991; Raiman and de Kleer, 1995; Dressler and Struss, 1994]; other introduced focusing techniques for avoiding useless computations (e.g. , [Console et al. , 1996]) or techniques for re-using inferences [Dressler and Freitag, 1994]; oth ers discussed how to exploit hierarchical models for fo cusing diagnosis [Genesereth, 1984; Mozetic, 1991]; yet others coupled MBD with other techniques such as case based reasoning (for caching results, e.g. [Portinale and Torasso, 1996]), or machine learning. As regards the latter, there are approaches that use simple tech niques to induce diagnostic rules from simulated cases (e.g., [Mozetic, 1990; Cascio et al., 1999]), or approaches proposing a closer integration between MBD and learn ing (e.g., [Baroglio et al., 1994]).
The research on MBD benefited from interactions with many other disciplines of AI and of computer sci ence; to mention some: knowledge representation, non monotonic reasoning, qualitative reasoning, constraint problem solving, probabilistic reasoning, machine learn ing, temporal reasoning, control theory, . . . In other words, model-based diagnosis has been in the late 80's and 90's what heuristic diagnosis was ten years before: an area that originated new ideas that spread to other fields of AI (the use of the ATMS [de Kleer, 1986] and of abduction are two examples) and an area where new ideas have been tested in practice (e.g., qualitative or non-monotonic reasoning).
The fact that several aspects in the field are rnature is demonstrated by the existence of papers that review the diagnostic task systematically and at the knowledge level, mapping the techniques that can be used for solving diagnostic problems to the features of the application domains. Papers such as [Poole, 1989; Console and Torasso, 1991; ten Teije and van Harmelen, 1994; de Kleer et al., 1992; Cordier, 1998; Lucas, 1998; Brusoni et al., 1998]) pointed out the different alterna tives that can be considered for characterizing diagnoses, discussing their properties and comparing them. More over, some recent projects defined frameworks for select ing the most appropriate approach to diagnosis, given a specific application (e.g., the ARTIST [Leitch et al., 1992; 1994] and PRIDE [Chantler et al., 1998] projects).
Finally, but not less important, MBD has been a meet ing point for theoretical and application work. This is not to say that this is not true of other areas of AI, but what happened in MBD is that many people have been active on both sides. This led to interesting research projects and applications. This is a very important topic and we shall devote the next section to it.
In conclusion, let us return to the initial question "why is MBD ready for applications?". A tentative answer is: "Because we have reasonable foundations for the field and understanding of how a diagnostic problem can be tackled" . This means that, given a problem, we can en visage how it could be tackled, which techniques could be adopted and which aspects of the problem could be criti cal or beyond available techniques (and thus would need either to be abstracted or solved in some approximate way) . And we have the instruments to actually build systems. Thus, although there are many open problems for research, we can tackle applications without cheating and claiming that everything is solved, but also without having to consider each new application starting from scratch and as a sort of bet.
3 What we did learn from applications
After the discussion in section 2, it is interesting to anal yse some of the applications that have been built recently and the lessons we learned from them, focusing on a specific field, automotive systems, on which we gained experience over the last years.
Model-based diagnosis has been applied to many dif ferent fields. Historically, one can recognize two main streams. On the one hand, there is the approach that focused on the diagnosis of technical systems (mainly cir cuits in the early years), based on a structural decompo sition of the device to be diagnosed and on a description of the (correct or faulty) behavior of each type of com ponent. On the other hand, it is important to mention the important works on medical diagnosis. However, the latter had no strong impact on applications (for several reasons whose analysis is outside the scope of this paper) , but had important impacts on the genesis of MBD (con sider experiences such as ABEL [Patil, 1981] and CASNET [Weiss et al., 1978]), although the community recognizes itself mostly in the first mainstream.
As mentioned above, electronic devices were the main application used in the early work to experiment ideas and techniques (see most of the papers in [Hamscher et al. , 1992]). The diagnosis of copiers led the Xerox Pare group to interesting research on efficient algorithms and focusing strategies. As an example of industrial appli cation, it is worth mentioning the system developed by Hewlett Packard [Allred et al., 1991]; CDIAG by Dassault Electronique [Loiez and Taillibert, 1997] is a commer cial system for the diagnosis of different types of circuits (including analog one; this shows that it is interesting to have working applications also for domains in which there are still interesting open problems for research).
Although electronic devices were one of the main field for experimenting ideas, most of the applications that are currently on the field come from other areas. For ex ample, the experience in TIGER (an EU founded project) originated a commercial system for the dia&nosis of gas turbines that is used in several locations [Trave Mas suyes and Milne, 1997]. Interesting work was performed on power generation and distribution systems: two ap plications that are being moved to the field are the DIAPO system by Electricite de France (for nuclear power plants [Porcheron et al., 1994]) and a system by Cise for power distribution lines [Lamperti and Pogliano, 1997].
Aerospace is another important area of application (on which many details are not disclosed). Among dif ferent applications, it is interesting to mention the role that model-based reasoning and diagnosis are playing in the NASA REMOTE AGENTS project for autonomy in space [Doyle, 1997]; particularly interesting is the Liv INGSTONE system [Williams and Nayak, 1996] for the di agnosis and reconfiguration of space autonomous agents. Other interesting aerospace applications have been de veloped by Rockwell (see [Provan, 1998] for a recent ex ample) ; Aerospatiale is experimenting the CDIAG system mentioned above in an automatic test equipment for ana log and hybrid boards; similarly Boeing developed inter esting applications.
Let us now turn to discussing the automotive domain. There are several reasons why diagnosis (and MBD in particular) became more and more important in this field. First of all, the increasing complexity of the cars (especially from the electronic point of view) called for more sophisticated diagnostic techniques (both on-board the car and in the workshop) . Second, legislation re quired the presence of diagnostic systems in the Elec tronic Control Unit (ECU) of the car. Third, competi tion between manufactures led them to investigate new features for attracting customers and for augmenting their satisfaction. Thus, the interest of car manufactur ers is growing and they are looking at model-based rea soning as one way of managing the increasing complexity of cars and the high maintenance costs and unnecessary downtime deriving from such a complexity.
The application to the automotive field is very promis ing from a commercial point of view, but also challeng ing from the technical point of view, under several view points. First of all, the systems to be diagnosed are complex, but many of them can be managed with state-of the-art technologies. Second, different physical domains are involved, ranging from mechanical, to hydraulic, elec tric and electronic systems. Thus the field is an ideal one for applying and experimenting modeling and diagnos tic techniques. Third, most of the critical devices to be diagnosed (especially on-board) are dynamic feedback system with an active control (performed in most of the cases by the software running on the ECUs); diagnosing these system is a very challenging problem and stimu lated a lot of work from the research perspective. The application on-board the car imposed several new prob lems and constraints, mainly due to the resources that are available on-board and to the real-time constraints.
In summary, the application to such a domain led MBD researcher to confront themselves with many in teresting problems and, as we shall discuss later, this had a very positive feedback on research. This was due to the possibility of working on real systems, together with the end users. Several results came out of this co operation. First of all, there is a number of systems, of various nature, that are deployed and used in practice. Among them, it is worth mentioning IDEA, developed by Centro Ricerche Fiat and currently used in more than 1500 workshops around Italy to diagnose some 80 differ ent types of subsystems within a car [Cascio and San severino, 1997). Another interesting system is MDS by Daimler-Chrysler, which is going to become a commer cial product. AUTOSTEVE is a system for the automatic generation of the FMEA (Failure Modes Effect Analy sis) in case of electrical circuits [Price, 1998) used by Ford and Jaguar. An approach to diagnose Chevrolet engines is discussed in [Masterman et al., 1998). RAZ'R by Oc c'm is a modeling and diagnosis development and run time environment; it is used by Bosch and Volvo within the VMBD project (see below).
Last but not least it is fundamental to mention the VMBD project (funded by the European Union), which involved a number of partners including car manufac tures (Daimler-Benz, Fiat, Volvo) , suppliers (Bosch, Dassault Electronique, GenRad, Magneti Marelli and Occ'm) and Universities (Aberystwyth, Paris XIII and Turin), and whose goal was to make a step forward in the use of model-based diagnosis both on-board the car and in the workshop. The project was successful from several points of view. As an immediate effect, it led to the development of three prototype applications, based on state of the art techniques, and demonstrated on real cars: the Common Rail fuel delivery system (on aLan cia car), the DTI fuel delivery system and the automatic transmission (on a Volvo car). The models we adopted are mostly qualitative ones (some simple quantitative models had to be used for measurement interpretation and for fault detection). For the two fuel delivery sys tems we adopted models based on qualitative deviations [Malik and Struss, 1996]. The components are modeled in terms of qualitative differential equations that include appropriate parameters whose values allow for the rep resentation of fault modes of the components. The equations for deviations (in which the variables represent de viations of quantities from expected values) are obtained via algebraic transformations. This form of modeling proved to be very useful since in most cases it is rele vant to reason in terms of variables deviating from their expected values, rather than in term of absolute values.
For the off-board system we adopted standard MBD; for the on-board one we experimented the direct use of MBD (for the DTI using Raz'r) and a precompilation approach: in the Common Rail diagnostic trees are syn thesized starting from the solutions to a set of simulated cases computed using the model-based approach. This allowed us to produce diagnostic systems that can be easily implemented on current ECUs ( see [Cascio et al., 1999) for details). ·
Probably the most important (even if not immediate) effect of the project is that it contributed to pushing MBD; manufactures and suppliers had the opportunity of having a close look at the technology, using it on their systems and viewing the advantages (and problems) that it can lead to; as a result, some of them decided to in vest on MBD. The project was a unique opportunity for us as researchers, not only to experience our ideas and techniques, but also to face new problems and generate a huge number of new research issues. Finally, the project stimulated the interest of other manufactures and suppli ers and will probably originate a number of new projects, in several different directions.
In summary, we have that although automotive ap plications are complex and there are still open research problems, the systems that have been implemented in the last years witness the fact that applications solving real problems can be built. The same story could repeat in other application areas, provided that similar condi tions are met. This would require the involvement of end-users in application projects, in co-operation with research centers and suppliers of solutions. In order for this to happen, as naive as it may sound, it is important that researchers work actively in trying to involve com panies in their local environment and start projects on real applications. In this sense, in the European context, the role of the funding from the European Union has been fundamental in the automotive domain and could play the same role also in other domains.
':'.urn a technical point of view, it is important that the experiences gained in the past projects are re-used. In fact , they can tell a lot on which problems can be solved and which are curr~ntly beyond the state of the art and about which techniques are suitable for a given problem. In particular, it would be very important to have frame works for analysing problems and tools for supporting the construction of model-based systems, without having tore-implement most of the software every time. Some advances in this direction are currently being made but many efforts are still needed. In particular, it would be interesting and important to design open architectures for diagnostic systems and to make available software modules (e.g., as COM or CORBA components) that can be combined to create MBD applications.
4 Applications stimulated new research
In the previous sections we stressed the idea that it is important to work on applications even if not all the research problems are solved. In this section we want to stress another issue: building applications is not the end of research. Actually, working on applications is a big advantage for research, rising new relevant research issues. This indeed has been the case for model-based diagnosis, in our experience. Thus, it is fundamental that researchers work on both theory and applications; keeping the two aspects too distant is a big mistake, especially for researchers who risk to spend a lot of time and efforts on artificial problems, each one attracting the interest only of a closed community.
Thus, let us analyse in more detail some of the re search issues that arose from the recent applications of model-based diagnosis. As in previous sections, we shall base the discussion on our experience in the automotive domain (and obviously the discussion will be partial) .
Different modeling techniques have been used in the applications discussed in the previous section. Each model relies on some assumptions, being an abstraction of the actual system. Although the problem of choosing the appropriate models, given the requirements of the diagnostic task, has been pointed out already in [Davis, 1983], there is still a lot to be done for clarifying the issue. In particular, it would be interesting to have a general characterization and criteria that could help in the choice of the model that is most suitable for a given problem. This does not mean, however, that new mod eling techniques are not needed. Several applications, for example, pointed out that although qualitative mod els are very useful, some form of quantitative reason ing would be important, especially in the early phases of the diagnostic process (fault detection in particular) and for discrimination purposes. Thus, research on the integration of qualitative and quantitative reasoning (or on semi-quantitative reasoning) is very important.
An ideal output of research on modeling should lead to a sort of "intelligent model library" , i.e., a repository of models of components that could be used in different applications. Reusability is in fact an important issue which still needs a lot of research. In fact, although in principle models of components are reusable, in practice this is a difficult problem. In order to improve reusability one should have different models of the same component, at different levels of granularity and based on different modeling assumptions. The repository should be intel ligent and should provide guidelines to select the most appropriate model for a given application.
One specific research issue on which a lot of attention concentrated since the early age is the problem of deal ing with dynamics [Hamscher and Davis, 1984]. This is in fact a critical aspect to be taken into account in most applications. Several approaches to deal with dynamic (and time-varying) behavior have been proposed (see the discussion in [Brusoni et al., 1998]) and there is a very active debate (in which the experiments on applications play a fundamental role). It is worth noting that some of the proposals introduced in the MBD community tech niques that are used in other fields ; two examples are the use of Bond graphs in [Mosterman and Biswas, 1997] and of techniques coming from system engineering, as e.g., in [Chantler et al. , 1996]. Closer connections with these (and other) fields is a topic for future research.
The work on automotive applications pointed out that the diagnostic process, especially when performed on board the car, is quite complex. The need of integrat ing diagnosis (in its restricted sense of interpretation or explanation of observations) and planning for schedul ing (and performing) tests or actions was recognized already several years ago (e.g., [Friedrich et al., 1994; Struss, 1992a]). However, in many cases the problem is more articulated and difficult than anticipated and there are aspects and opportunities that have not been considered yet . For example, in several systems it is interesting and important to perform active tests when the operating conditions allow them (e.g., when a car is running at a constant engine speed and the pedals are in specific positions). This requires the ability of scheduling such tests (to avoid performing them in criti cal situations) , performing some action and interpreting its effects. There are many opportunities for research on this problem, linking work on planning and theories of action and change and on perception (e.g. vision or sensing) more closely to the diagnostic task.
The future of diagnosis in automotive domains is the application on-board the car. In VMBD we experimented the use of MBD for on board diagnosis but in order to sit on current technology ECUs (which have a few kilobytes of memory), we also considered the use of MBD to gen erate decision trees to be used on-board. MBD proved to be very flexible for such a task. However, the possibil ity of actually using model-based diagnosis on-board has to be investigated further. This, in fact, would be more flexible, opening new opportunities (and a closer inte gration with the off-board system to which the on-board one could pass useful information). Would it be con ceivable to define a run-time model-based system, with reduced functionality but that could be implemented in next generation ECUs? Will the idea of having an on board car-PC be pursued by car manufactures and sup pliers? This would be an fundamental opportunity for the growth of our field.
Still considering on-board applications, it would be in teresting to investigate tete-diagnostics, having a satel lite connection between a car and an assistance center (for performing diagnosis or actions on the car or sug gesting the driver what to do or how to reach the closest workshop, which, at the same time could be alerted) . Such a centralized model would be important also for simplifying the maintenance of models which could be done only on the server. An important role could be played also by approaches to preventive diagnosis, which needs further studies. For example, one could recognize a non-optimal behavior of some component (possibly ex ploiting wear models) , and tune the system to avoid such behaviors or warn the user or just keep track of the prob lem until the next maintenance.
However, the most important research (and practical) issue risen by research on automotive domains does not concern a specific step within the diagnostic process, but rather the role of diagnosis along the design manufac turing and service chain. What emerged clearly is that the current situation in which diagnosis is not integrated along this chain is not working. Building models for diagnosis a-posteriori, when devices are manufactured is not a good choice. It makes diagnosis more difficult (especially as regards modeling) and does not allow to exploit all the potential advantages of the model-based approach. Indeed, model-based diagnosis should be in tegrated along the process; in such a way, the diagnos ability of the system being designed could be tested and design choices (e.g., selecting which sensors to use and where) could be influenced by these tests. This requires the availability of a library of models of basic compo nents, (library that could be extended when new compo nents are in turn designed) and the integration of model based diagnosis into design tools. In this way, the models for diagnosis could be an automatic result of the design process, as well as other aspects such as, e.g., the FMEA of the device to be used in documentation manuals. This integration would provide significant benefits also for de sign since it would avoid the redesign of systems which do not meet diagnosability requirements. Some ideas in the direction of integrating diagnosis and design were pre sented in the past (e.g., the work on design for testabil ity [Williams and Parker, 1983] and on sensor placement [Searl, 1994]). However, this is a topic requiring many efforts, possibly involving people working on design and companies that produce design tools.
The list of new topics could be longer, but providing it is not the goal of this paper. Our goal is to stress that many of these problems would have not been recognized without working on real with end users. This work pro vided a new important impulse to research, impulse that would have been impossible without the applications.
5 Some implications for the future
In the previous sections we discussed the important role that applications had in the recent advances in MBD. They allowed researchers to experiment and validate their techniques; this work pointed out a number of new important research issues. Thus, applications al lowed the field to grow with new impulses, opening new research opportunities and avoiding to be confined on sterile, purely academic issues.
Working on real applications requires the meeting of people from the research part and users with applica tion problems. The recent successes in the field of MBD point out that this requires efforts from both sides. First of all, there must be an effort to create the opportunity for the meeting; after that there must be an effort in understanding each other. A number of conditions facil itated these aspects in the case of MBD. First of all, we stress once again the importance of presenting ourselves to companies as a community with a strong common background. This is not to say that we agree on every thing, but this means that the probability of having com pletely different answers on basic questions and problems when discussing with different researchers is low. This avoid confusion into potential "customers" . · Second, we are a community which already involves people from the application side, working in corporate research centers or in end-user companies. The participation to the annual "Workshop on Principles of Diagnosis" is not only aca demic. This was the result of big efforts; we always tried to involve application people, maybe organizing special events. This interest is now a witness for attracting other people from the application side.
Finally, on the European side an important role has been played by funding from the European Union. A part from many specific projects, an important initia tive is the MONET [Monet, ] network of excellence on model-based systems and qualitative reasoning. MONET involves some 70 nodes, including Universities, research centers and companies and its role is the diffusion and promotion of model-based techniques. The network is very active in this task, especially towards companies, creating thus the background for facilitating the meet ing between research and application and thus the condi tions for new projects. Obviously, for that to happen, it is important that all researchers in the field look around for finding potential applications and that companies in vest on such applications. Past experiences show that there is a chance that the investments do not get lost.
From the research point of view, the analysis in the paper points out that there are many open issues and opportunities for co-operation with many other fields of AI, exactly as it was in the past.
6 Acknowledgements
The authors are indebted to the partners in the VMBD project (BE 95/2128). Special thanks to D. Theseider Dupre, P. Struss, P. Torasso, for their comments.
@inproceedings{ijcai1999p197,
title = {Model-based Diagnosis in the Real World: Lessons Learned and Challenges Remaining},
author = {Console, Luca and Dressler, Oskar},
booktitle = {Proceedings of the Sixteenth International Joint Conference on
Artificial Intelligence, {IJCAI-99}},
publisher = {International Joint Conferences on Artificial Intelligence},
editor = {Thomas Dean},
pages = {1393--1399},
year = {1999},
note = {Invited Speakers},
url = {https://ijcai.org/proceedings/1999/197},
}
TY - CPAPER TI - Model-based Diagnosis in the Real World: Lessons Learned and Challenges Remaining AU - Console, Luca AU - Dressler, Oskar T2 - Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, IJCAI-99 PB - International Joint Conferences on Artificial Intelligence PY - 1999 SP - 1393 EP - 1399 UR - https://ijcai.org/proceedings/1999/197 ER -Copied
Machine-readable versions of this paper, for tools and AI agents: