{"type":"ScholarlyArticle","title":"A Methodology for Evaluating Theory Revision Systems: Results with Audrey II","authors":[{"name":"James Wogulis","slug":"james-wogulis","given":"James","family":"Wogulis","affiliation":"Department of Information & Computer Science University of California Irvine , CA 92717"},{"name":"Michael J. Pazzani","slug":"michael-j-pazzani","given":"Michael J.","family":"Pazzani","affiliation":"Department of Information & Computer Science University of California Irvine , CA 92717"}],"abstract":"Theory revision systems are learning systems that have a goal of making small changes to an original theory to account for new data . A measure for the distance between two theories is proposed. This measure corresponds to the minimum number of edit operations at the literal level required to transform one theory into another. By computing the distance between an original theory and a revised theory, the claim that a theory revision system makes few revisions to a theory may be quantitatively evaluated. We present data using both accuracy and the distance metric on AUDREY II, a first-order theory revision system.","doi":null,"url":"https://ijcai.org/proceedings/1993/156","pdf":"https://ijcai.org/proceedings/1993/0156.pdf","markdown":"https://ijcai.org/proceedings/1993/156.md","bibtex":"https://ijcai.org/proceedings/1993/bibtex/156","volume":{"slug":"1993","title":"Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 1993","url":"https://ijcai.org/proceedings/1993/","year":1993,"location":"Chambéry, France","dates":"","editors":"Ruzena Bajcsy","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Machine Learning","track":"Machine Learning","number":156,"pages":{"first":1128,"last":1133},"keywords":[],"videos":[],"references":[{"position":1,"raw":"Bergadano and Giordana, 1988) F . Bergadano and A. Gior- dana. A knowledge intensive approach to concept induc- tion. In Proceedings of the Fifth International Conference on Machine Learning, pages 305-317, Ann Arbor, Michi- gan, 1988.","title":"A knowledge intensive approach to concept induction","authors":"Giordana Bergadano, F Bergadano, A Giordana","year":"1988","venue":"Proceedings of the Fifth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Morgan Kaufmann .","title":null,"authors":"Morgan Kaufmann","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Cohen, 1992a) W. Cohen. Abductive explanation-based learning: A solution to the multiple inconsistent expla- nation problem. Machine Learning, 8:167-219 , 1992.","title":"Abductive explanation-based learning: A solution to the multiple inconsistent explanation problem","authors":") W Cohen ; A, Cohen","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Cohen, 1992b) W. Cohen. Compiling prior knowledge into an explicit bias. In Proceedings of the Ninth Interna- tional Conference on Machine Learning, pages 100-110, Aberdeen, Scotland, 1992.","title":"Compiling prior knowledge into an explicit bias","authors":") W Cohen ; B, Cohen","year":"1992","venue":"Proceedings of the Ninth International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Morgan Kaufmann . [Davis, 1978)","title":null,"authors":"Morgan Kaufmann","year":"1978","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"R. T. Davis. Model-directed learning of pro- duction rules. In D. Waterman and F. Hayes-Roth, edi- tors, Pattern-directed inference systems. Academic Press, 1978.","title":"Model-directed learning of production rules","authors":"R T Davis","year":"1978","venue":"editors, Pattern-directed inference systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Flann and Dietterich, 1989) N. Flann and T . Dietterich. A study of explanation-based methods for inductive learning. Machine Learning, 4:187-226, 1989.","title":"A study of explanation-based methods for inductive learning","authors":"Dietterich Flann, ) N Flann, T Dietterich","year":"1989","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Hirsh, 1989)","title":null,"authors":"Hirsh","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"H. Hirsh . Combining empirical and analytical learning with version spaces. In Proceedings of the Sixth International Workshop on Machine Learning, pages 29- 33, Ithaca, NY, 1989.","title":"Combining empirical and analytical learning with version spaces","authors":"H Hirsh","year":"1989","venue":"Proceedings of the Sixth International Workshop on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Morgan Kaufmann.","title":null,"authors":"Morgan Kaufmann","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kuhn, 1962] T. Kuhn. The structure of scientific revolu- tions. University of Chicago Press, 1962.","title":"The structure of scientific revolutions","authors":"T Kuhn","year":"1962","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lawler, 1976] E. L. Lawler. Combinatorial optimization: Networks and matroids. Holt, Rinehart and Winston, 1976.","title":"Combinatorial optimization: Networks and matroids","authors":"E L Lawler","year":"1976","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lenat and Guha, 1990] D. Lenat and R: V. Guha. Building large knowledge-based systems: Representation and infer- ence in the CYC project. Addison-Wesley, 1990.","title":"Building large knowledge-based systems: Representation and inference in the CYC project","authors":"D Lenat, R, V Guha","year":"1990","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mooney and Richards, 1992] R. Mooney and B. Richards. Automated debugging of logic programs via theory revi- sion. In Proceedings of the International Workshop on In- ductive Logic Programming, Tokyo, Japan, 1992.","title":"Automated debugging of logic programs via theory revision","authors":"R Mooney, B Richards","year":"1992","venue":"Proceedings of the International Workshop on Inductive Logic Programming","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Muggleton and Buntine, 1988] S. Muggleton and W . Bun- tine. Machine invention of first-order predicates by invert- ing resolution . In Proceedings of the Fifth International Workshop on Machine Learning, pages 339-352, Ann Ar- bor, Ml , 1988.","title":"Machine invention of first-order predicates by inverting resolution","authors":"S Muggleton, W Buntine","year":"1988","venue":"Proceedings of the Fifth International Workshop on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Morgan Kaufmann . [Muggleton and Feng, 1990]","title":"Muggleton and Feng","authors":"Morgan Kaufmann","year":"1990","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"S. Muggleton and C . Feng. Ef- ficient induction of logic programs. In Proceedings of the First Conference on Algorithmic Learning Theory, Tokyo, Japan , 1990. Ohmsha.","title":"Efficient induction of logic programs","authors":"S Muggleton, C Feng","year":"1990","venue":"Proceedings of the First Conference on Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Norvig, 1992] P. Norvig. Paradigms of artificial intelligence programming: Case studies in Common Lisp. Morgan Kaufmann, 1992.","title":"Paradigms of artificial intelligence programming: Case studies in Common Lisp","authors":"P Norvig","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Ourston and Mooney, 1990) D. Ourston and R. Mooney. Changing the rules: A comprehensive approach to theory refinement . In Proceedings of the Eighth National Con- ference on Artificial Intelligence, pages 815-820, Boston, MA, 1990.","title":"Changing the rules: A comprehensive approach to theory refinement","authors":"Ourston, ) D Mooney, R Ourston, Mooney","year":"1990","venue":"Proceedings of the Eighth National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Morgan Kaufmann. [Pazzani and Brunk, 1991]","title":null,"authors":"Morgan Kaufmann, . [ Pazzani, Brunk","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"M. Pazzani and C . Brunk. De- tecting and correcting errors in rule-based expert systems: An integration of empirical and explanation-based learn- ing. Knowledge Acquisition, 3:157-173, 1991.","title":"Detecting and correcting errors in rule-based expert systems: An integration of empirical and explanation-based learning","authors":"M Pazzani, C Brunk","year":"1991","venue":"Knowledge Acquisition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pazzani and Kibler, 1992] M. Pazzani and D. Kibler. The utility of knowledge in inductive learning. Machine Learn- ing, 9:57-94 , 1992.","title":"The utility of knowledge in inductive learning","authors":"M Pazzani, D Kibler","year":"1992","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Quinlan, 1990] J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5:239-266, 1990.","title":"Learning logical definitions from relations","authors":"J R Quinlan","year":"1990","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Richards and Mooney, 1991] B. Richards and R. Mooney. First-order theory revision . In Proceedings of the Eighth International Workshop on Machine Learning, pages 447- 451 , Evanston, IL, 1991.","title":"First-order theory revision","authors":"B Richards, R Mooney","year":"1991","venue":"Proceedings of the Eighth International Workshop on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"Morgan Kaufmann .","title":null,"authors":"Morgan Kaufmann","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Richards, 1992] B. Richards. An operator-based approach to first-order theory revision. PhD thesis, University of Texas, Austin, TX, 1992.","title":"An operator-based approach to first-order theory revision","authors":"B Richards","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Shasha and Zhang, 1990] D. Shasha and K. Zhang. Fast al- gorithms for the unit cost editing distance between trees. Journal of Algorithms, 11:581-621 , 1990.","title":"Fast algorithms for the unit cost editing distance between trees","authors":"D Shasha, K Zhang","year":"1990","venue":"Journal of Algorithms","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Swartout, 1981] W . Swartout. Explaining and justifying in expert consulting programs. In Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pages 203-208, Vancouver, B.C., 1981. Morgan Kaufmann. [Tai, 1979]","title":"Explaining and justifying in expert consulting programs","authors":"W Swartout","year":"1979","venue":"Proceedings of the Seventh International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"K. C . Tai. The tree-to-tree correction problem. J. Assoc. Comput. Mach ., 26:422-433, 1979.","title":"The tree-to-tree correction problem","authors":"K C Tai","year":"1979","venue":"J. Assoc. Comput. Mach","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Towell, 1991] G. Towell. Symbolic knowledge and neural net- works: Insertion, refinement and extraction. PhD thesis, University of Wisconsin, Madison, WI, 1991.","title":"Symbolic knowledge and neural networks: Insertion, refinement and extraction","authors":"G Towell","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wogulis, 1991] J. Wogulis. Revising relational domain the- ories. In Proceedings of the Eighth International Workshop on Machine Learning, pages 462-466 , Evanston, IL, 1991.","title":"Revising relational domain theories","authors":"J Wogulis","year":"1991","venue":"Proceedings of the Eighth International Workshop on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"Morgan Kaufmann.","title":null,"authors":"Morgan Kaufmann","year":null,"venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}