{"type":"ScholarlyArticle","title":"On Automating the Doctrine of Double Effect","authors":[{"name":"Naveen Sundar Govindarajulu","slug":"naveen-sundar-govindarajulu","given":"Naveen Sundar","family":"Govindarajulu","affiliation":"Rensselaer Polytechnic Institute , Troy , NY"},{"name":"Selmer Bringsjord","slug":"selmer-bringsjord","given":"Selmer","family":"Bringsjord","affiliation":"Rensselaer Polytechnic Institute , Troy , NY"}],"abstract":"The doctrine of double effect (DDE) is a long-studied ethical principle that governs when actions that have both positive and negative effects are to be allowed. The goal in this paper is to automate DDE. We briefly present DDE, and use a first-order modal logic, the deontic cognitive event calculus, as our framework to formalize the doctrine. We present formalizations of increasingly stronger versions of the principle, including what is known as the doctrine of triple effect. We then use our framework to simulate successfully scenarios that have been used to test the presence of the principle in human subjects. Our framework can be used in two different modes. One can use it to build DDE-compliant autonomous systems from scratch, or one can use it to verify that a given AI system is DDE-complaint, by applying a DDE layer on an existing system or model. For the latter mode, the underlying AI system can be built using any architecture (planners, deep neural networks, bayesian networks, knowledge-representation systems, or a hybrid); as long as the system exposes a few parameters in its model, such verification is possible. The role of the DDE layer here is akin to a (dynamic or static) software verifier that examines existing software modules. Finally, we end by sketching initial work on how one can apply our DDE layer to the STRIPS-style planning model, and to a modified POMDP model. This is preliminary work to illustrate the feasibility of the second mode, and we hope that our initial sketches can be useful for other researchers in incorporating DDE in their own frameworks.","doi":"10.24963/ijcai.2017/658","url":"https://ijcai.org/proceedings/2017/658","pdf":"https://ijcai.org/proceedings/2017/0658.pdf","markdown":"https://ijcai.org/proceedings/2017/658.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/658","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"AI and autonomy track","track":"AI and autonomy track","number":658,"pages":{"first":4722,"last":4730},"keywords":["Philosophical: Norms","Philosophical: Concepts","Legal: Regulation","Technical: Models"],"videos":[],"references":[{"position":1,"raw":"Schemata K(a,t 1 , Γ), Γ φ, t 1 ≤ t 2 K(a,t 2 , φ)","title":"t 1 , Γ","authors":"K Schemata","year":null,"venue":"Γ φ","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"R K ] B(a,t 1 , Γ), Γ φ, t 1 ≤ t 2 B(a,t 2 , φ)","title":"t 1 , Γ)","authors":"R K ] B","year":null,"venue":"Γ φ","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"B ] C(t, P(a,t, φ) → K(a,t, φ))","title":null,"authors":"P B ] C(t","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"R 1 ]","title":"R","authors":null,"year":null,"venue":"R","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"C(t, K(a,t, φ) → B(a,t, φ))","title":"→ B(a,t, φ)","authors":null,"year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"R 2 ]","title":"R","authors":null,"year":null,"venue":"R","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"C(t, φ) t 1 . . .t K(a 1 ,t 1 , . . . K(a n ,t n , φ) . . .)","title":"φ) t 1","authors":null,"year":null,"venue":"t K","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"C(t, K(a,t 1 , φ 1 → φ 2 )) → K(a,t 2 , φ 1 ) → K(a,t 3 , φ 2 )","title":"t 1 , φ 1 → φ 2 )) → K(a,t 2 , φ 1 ) → K","authors":"K","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"C(t, B(a,t 1 , φ 1 → φ 2 )) → B(a,t 2 , φ 1 ) → B(a,t 3 , φ 2 )","title":"t 1 , φ 1 → φ 2 )) → B(a,t 2 , φ 1 ) → B","authors":"B","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"C(t, C(t 1 , φ 1 → φ 2 )) → C(t 2 , φ 1 ) → C(t 3 , φ 2 )","title":"t 1 , φ 1 → φ 2 )) → C(t 2 , φ 1 ) → C","authors":"C","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"C(, ∀x. φ → φ[x → t])","title":"φ → φ[x → t","authors":"C","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"C(, φ 1 ↔ φ 2 → ¬φ 2 → ¬φ 1 )","title":"φ 1 ↔ φ 2 → ¬φ 2 → ¬φ 1 )","authors":"C","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"C(, [φ","title":null,"authors":"C","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"∧ . . . ∧ φ n → φ] → [φ 1 → . . . → φ n → ψ])","title":"∧ φ n → φ] → [φ 1 →","authors":"∧","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"I(a,t, happens(action(a * , α),t ))","title":"I(a,t, happens","authors":null,"year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"R 14 ]","title":"R","authors":null,"year":null,"venue":"R","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Allsopp, 2011] Michael E. Allsopp. The Doctrine of Double Effect in US Law: Exploring Neil Gorsuchs Analyses. The National Catholic Bioethics Quarterly, 11(1):31-40, 2011.","title":"The Doctrine of Double Effect in US Law: Exploring Neil Gorsuchs Analyses","authors":"E Michael, Allsopp","year":"2011","venue":"The National Catholic Bioethics Quarterly","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Arkoudas and Bringsjord, 2008] Konstantine Arkoudas and Selmer Bringsjord. Toward Formalizing Common-Sense Psy- chology: An Analysis of the False-Belief Task. In T.-B. Ho and Z.-H. Zhou, editors, Proceedings of the Tenth Pacific Rim International Conference on Artificial Intelligence (PRICAI 2008), number 5351 in Lecture Notes in Artificial Intelligence (LNAI), pages 17-29. Springer-Verlag, 2008.","title":"Toward Formalizing Common-Sense Psychology: An Analysis of the False-Belief Task","authors":"Konstantine Arkoudas, Selmer Bringsjord","year":"2008","venue":"Proceedings of the Tenth Pacific Rim International Conference on Artificial Intelligence (PRICAI 2008)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Arnold et al., 2017] Thomas Arnold, Daniel Kasenberg, and Matthias Scheutz. Value Alignment or Misalignment-What Will Keep Systems Accountable?, 2017. Presented at the 3rd Interna- tional Workshop on AI, Ethics and Society at AAAI 2017.","title":"Value Alignment or Misalignment-What Will Keep Systems Accountable?","authors":"Thomas Arnold, Daniel Kasenberg, Matthias Scheutz","year":"2017","venue":"Presented at the 3rd International Workshop on AI, Ethics and Society at AAAI 2017","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Bentzen, 2016] Martin Mose Bentzen. The Principle Of Double Effect Applied to Ethical Dilemmas of Social Robots. Frontiers in Artificial Intelligence and Applications, 2016.","title":"The Principle Of Double Effect Applied to Ethical Dilemmas of Social Robots","authors":"Martin Mose, Bentzen","year":"2016","venue":"Frontiers in Artificial Intelligence and Applications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Berreby et al., 2015] Fiona Berreby, Gauvain Bourgne, and Jean- Gabriel Ganascia. Modelling Moral Reasoning and Ethical Re- sponsibility with Logic Programming. In Logic for Program- ming, Artificial Intelligence, and Reasoning, pages 532-548. Springer, 2015.","title":"Modelling Moral Reasoning and Ethical Responsibility with Logic Programming","authors":"Fiona Berreby, Gauvain Bourgne, Jean-Gabriel Ganascia","year":"2015","venue":"Logic for Programming, Artificial Intelligence, and Reasoning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Boolos et al., 2003] George S. Boolos, John P. Burgess, and Richard C. Jeffrey. Computability and Logic (Fifth Edition). Cambridge University Press, Cambridge, UK, 2003.","title":"Computability and Logic (Fifth Edition)","authors":"George S Boolos, John P Burgess, Richard C Jeffrey","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Bringsjord and Govindarajulu, 2012] Selmer Bringsjord and Naveen Sundar Govindarajulu. Given the Web, What is Intelligence, Really? Metaphilosophy, 43(4):361-532, 2012.","title":"Given the Web, What is Intelligence","authors":"Selmer Bringsjord, Naveen Sundar, Govindarajulu","year":"2012","venue":"Really? Metaphilosophy","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Bringsjord and Taylor, 2012] Selmer Bringsjord and Joshua Tay- lor. The Divine-Command Approach to Robot Ethics. In P. Lin, G. Bekey, and K. Abney, editors, Robot Ethics: The Ethical and Social Implications of Robotics, pages 85-108. MIT Press, Cam- bridge, MA, 2012.","title":"The Divine-Command Approach to Robot Ethics","authors":"Selmer Bringsjord, Joshua Taylor","year":"2012","venue":"Robot Ethics: The Ethical and Social Implications of Robotics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Bringsjord et al., 2014] Selmer Bringsjord, Naveen Sundar Govin- darajulu, Daniel Thero, and Mei Si. Akratic Robots and the Com- putational Logic Thereof. In Proceedings of ETHICS • 2014 (2014 IEEE Symposium on Ethics in Engineering, Science, and Technology), pages 22-29, Chicago, IL, 2014. IEEE Catalog Number: CFP14ETI-POD.","title":"Akratic Robots and the Computational Logic Thereof","authors":"Selmer Bringsjord, Naveen Sundar Govindarajulu, Daniel Thero, Mei Si","year":"2014","venue":"IEEE Symposium on Ethics in Engineering, Science, and Technology)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Bringsjord, 2017] Selmer Bringsjord. A 21st-Century Ethical Hi- erarchy for Robots and Persons: EH . In A World with Robots: International Conference on Robot Ethics: ICRE 2015, vol- ume 84, page 47. Springer, 2017.","title":"A 21st-Century Ethical Hierarchy for Robots and Persons: EH","authors":"Selmer Bringsjord","year":"2017","venue":"A World with Robots: International Conference on Robot Ethics: ICRE 2015","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Cushman et al., 2006] Fiery Cushman, Liane Young, and Marc Hauser. The Role of Conscious Reasoning and Intuition in Moral Judgment Testing Three Principles of Harm. Psychological sci- ence, 17(12):1082-1089, 2006.","title":"The Role of Conscious Reasoning and Intuition in Moral Judgment Testing Three Principles of Harm","authors":"Cushman","year":"2006","venue":"Psychological science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Feldman, 1978] Fred Feldman. Introductory Ethics. Prentice-Hall, Englewood Cliffs, NJ, 1978.","title":"Introductory Ethics","authors":"Feldman","year":"1978","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Fletcher, 1998] George P. Fletcher. Basic Concepts of Criminal Law. Oxford University Press, 1998.","title":"Basic Concepts of Criminal Law","authors":"George P Fletcher","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Francez and Dyckhoff, 2010] Nissim Francez and Roy Dyckhoff. Proof-theoretic Semantics for a Natural Language Fragment. Lin- guistics and Philosophy, 33:447-477, 2010.","title":"Proof-theoretic Semantics for a Natural Language Fragment","authors":"Dyckhoff Francez","year":"2010","venue":"Linguistics and Philosophy","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Gentzen, 1935] Gerhard Gentzen. Investigations into Logical De- duction. In M. E. Szabo, editor, The Collected Papers of Gerhard Gentzen, pages 68-131. North-Holland, Amsterdam, The Nether- lands, 1935. This is an English version of the well-known 1935 German version. Hauser et al., 2007] Marc Hauser, Fiery Cushman, Liane Young, R Kang-Xing Jin, and John Mikhail. A Dissociation Between Moral Judgments and Justifications. Mind & Language, 22(1):1- 21, 2007.","title":"A Dissociation Between Moral Judgments and Justifications","authors":"Gerhard Gentzen, Marc Gentzen, Fiery Hauser, Liane Cushman, R Young, Jin Kang-Xing, John Mikhail","year":"1935","venue":"The Collected Papers of Gerhard Gentzen","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Huxtable, 2004] Richard Huxtable. Get Out Of Jail Free? The Doctrine Of Double Effect In English Law. Palliative Medicine, 18(1):62-68, 2004.","title":"Get Out Of Jail Free? The Doctrine Of Double Effect In English Law","authors":"Richard Huxtable","year":"2004","venue":"Palliative Medicine","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Kamm, 2007] Frances Myrna Kamm. Intricate Ethics: Rights, Re- sponsibilities, And Permissible Harm. Oxford University Press, New York, New York, 2007.","title":"Intricate Ethics: Rights, Responsibilities, And Permissible Harm","authors":"Frances Myrna, Kamm","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Khasidashvili et al., 2009] Zurab Khasidashvili, Mahmoud Ki- nanah, and Andrei Voronkov. Verifying Equivalence of Memo- ries Using a First Order Logic Theorem Prover. In Formal Meth- ods in Computer-Aided Design, 2009. FMCAD 2009, pages 128- 135. IEEE, 2009.","title":"Verifying Equivalence of Memories Using a First Order Logic Theorem Prover","authors":"Zurab Khasidashvili, Mahmoud Kinanah, Andrei Voronkov","year":"2009","venue":"Formal Methods in Computer-Aided Design","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Khatchadourian, 1988] Haig Khatchadourian. Is the Principle of Double Effect Morally Acceptable? International Philosophical Quarterly, 28(1):21-30, 1988.","title":"Is the Principle of Double Effect Morally Acceptable","authors":"Haig Khatchadourian","year":"1988","venue":"International Philosophical Quarterly","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Malle et al., 2015] Bertram F. Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano. Sacrifice One for the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-robot Interac- tion, pages 117-124. ACM, 2015.","title":"Sacrifice One for the Good of Many?: People Apply Different Moral Norms to Human and Robot Agents","authors":"F Bertram, Matthias Malle, Thomas Scheutz, John Arnold, Corey Voiklis, Cusimano","year":"2015","venue":"Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-robot Interaction","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[McIntyre, 2014] Alison McIntyre. Relevance Logic. In Ed- ward Zalta, editor, The Standford Encyclopedia of Philosophy. September, 2014 edition, 2014.","title":"Relevance Logic","authors":"Alison Mcintyre","year":"2014","venue":"The Standford Encyclopedia of Philosophy","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[McNamara, 2010] Paul McNamara. Deontic Logic. In Edward Zalta, editor, The Stanford Encyclopedia of Philosophy. 2010. McNamara's (brief) note on a paradox arising from Kant's Law is given in an offshoot of the main entry. [Mikhail, 2011] John Mikhail. Elements of Moral Cognition: Rawls' Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment. Cambridge University Press, Cambridge, UK, 2011. Kindle edition.","title":"paradox arising from Kant's Law is given in an offshoot of the main entry","authors":"Paul Mcnamara","year":"2010","venue":"Elements of Moral Cognition: Rawls' Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Mueller, 2006] Erik T. Mueller. Commonsense Reasoning: An Event Calculus Based Approach. Morgan Kaufmann, San Fran- cisco, CA, 2006. This is the first edition of the book. The second edition was published in 2014.","title":"Commonsense Reasoning: An Event Calculus Based Approach","authors":"Erik T Mueller","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Pereira and Saptawijaya, 2016a] Luís Moniz Pereira and Ari Saptawijaya. Counterfactuals, Logic Programming and Agent Morality. In Shahid Rahman and Juan Redmond, editors, Logic, Argumentation and Reasoning, pages 85-99. Springer, 2016.","title":"Counterfactuals, Logic Programming and Agent Morality","authors":"Saptawijaya ; Luís Moniz Pereira, Ari Pereira, Saptawijaya","year":"2016","venue":"Logic, Argumentation and Reasoning","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Pereira and Saptawijaya, 2016b] Luís Moniz Pereira and Ari Saptawijaya. Programming Machine Ethics. Springer, Basel, Switzerland, 2016. This book is in Springer's SAPERE series, Vol. 26. Pollock, 1976]","title":"Programming Machine Ethics","authors":"Saptawijaya Pereira","year":"1976","venue":"Luís Moniz Pereira and Ari Saptawijaya","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"John Pollock. Subjunctive Reasoning. D. Reidel, Dordrecht, Holland & Boston, USA, 1976.","title":"Subjunctive Reasoning","authors":"John Pollock","year":"1976","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Rao and Georgeff, 1991] Anand S. Rao and Michael P. Georgeff. Modeling Rational Agents Within a BDI-architecture. In R. Fikes and E. Sandewall, editors, Proceedings of Knowledge Represen- tation and Reasoning (KR&R-91), pages 473-484, San Mateo, CA, 1991.","title":"Modeling Rational Agents Within a BDI-architecture","authors":"Anand S Rao, Michael P Georgeff","year":"1991","venue":"Proceedings of Knowledge Representation and Reasoning (KR&R-91)","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"Morgan Kaufmann. [Stannett and Németi, 2014]","title":"Stannett and Németi","authors":"Morgan Kaufmann","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"Mike Stannett and István Németi. Us- ing Isabelle/HOL to Verify First-order Relativity Theory. Journal of Automated Reasoning, 52(4):361-378, 2014.","title":"Using Isabelle/HOL to Verify First-order Relativity Theory","authors":"Mike Stannett, István Németi","year":"2014","venue":"Journal of Automated Reasoning","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zalta, 1988] Edward N. Zalta. Intensional Logic and the Meta- physics of Intentionality. MIT Press, Cambridge, MA, 1988.","title":"Intensional Logic and the Metaphysics of Intentionality","authors":"Edward N Zalta","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}