{"type":"ScholarlyArticle","title":"Formalisation and Evaluation of Properties for Consequentialist Machine Ethics","authors":[{"name":"Raynaldio Limarga","slug":"raynaldio-limarga","given":"Raynaldio","family":"Limarga","affiliation":"School of Computer Science and Engineering , University of New South Wales , Sydney , Australia"},{"name":"Yang Song","slug":"yang-song","given":"Yang","family":"Song","affiliation":"School of Computer Science and Engineering , University of New South Wales , Sydney , Australia"},{"name":"Abhaya Nayak","slug":"abhaya-nayak","given":"Abhaya","family":"Nayak","affiliation":"Department of Computing , Macquarie University , Australia"},{"name":"David Rajaratnam","slug":"david-rajaratnam","given":"David","family":"Rajaratnam","affiliation":"School of Computer Science and Engineering , University of New South Wales , Sydney , Australia"},{"name":"Maurice Pagnucco","slug":"maurice-pagnucco","given":"Maurice","family":"Pagnucco","affiliation":"School of Computer Science and Engineering , University of New South Wales , Sydney , Australia"}],"abstract":"As artificial intelligence (AI) technologies continue to influence our daily lives, there has been a growing need to ensure that AI enabled decision making systems adhere to principles expected of human decision makers. This need has given rise to the area of Machine Ethics. We formalise several ethical principles from the philosophical literature in the situation calculus framework to verify the ethical permissibility of a plan. Moreover, we propose several important properties, including some of our own that are intuitively appealing, and a number derived from the social choice literature that would appear to be relevant in evaluating the various approaches. Finally we provide an assessment of how our various situation calculus models of Machine Ethics that we examine satisfy the important properties we have identified.","doi":"10.24963/ijcai.2024/49","url":"https://ijcai.org/proceedings/2024/49","pdf":"https://ijcai.org/proceedings/2024/0049.pdf","markdown":"https://ijcai.org/proceedings/2024/49.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/49","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":49,"pages":{"first":440,"last":448},"keywords":["Knowledge Representation and Reasoning: KRR: Other","Knowledge Representation and Reasoning: KRR: Reasoning about actions","Knowledge Representation and Reasoning: KRR: Common-sense reasoning","AI Ethics, Trust, Fairness: ETF: Moral decision making"],"videos":[],"references":[{"position":1,"raw":"[Abel et al., 2016] David Abel, James MacGlashan, and Michael L Littman. Reinforcement learning as a frame- work for ethical decision making. In Workshops at the Thirtieth AAAI conference on Artificial Intelligence, pages 54-61, 2016.","title":"Reinforcement learning as a framework for ethical decision making","authors":"David Abel, James Macglashan, Michael L Littman","year":"2016","venue":"Workshops at the Thirtieth AAAI conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anderson and Anderson, 2011] Michael Anderson and Su- san Leigh Anderson. Machine Ethics. Cambridge Univer- sity Press, 2011.","title":"Michael Anderson and Susan","authors":"Anderson, Anderson ;, Leigh Anderson","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arrow, 1951] Kenneth Joseph Arrow. Social choice and in- dividual values. 1951.","title":"Social choice and individual values","authors":"Kenneth Joseph, Arrow","year":"1951","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Asaro, 2019] Peter M Asaro. AI ethics in predictive polic- ing: From models of threat to an ethics of care. IEEE Technology and Society Magazine, 38(2):40-53, 2019.","title":"AI ethics in predictive policing: From models of threat to an ethics of care","authors":"M Peter, Asaro","year":"2019","venue":"IEEE Technology and Society Magazine","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baum, 2020] Seth D Baum. Social choice ethics in artificial intelligence. AI & Society, 35(1):165-176, 2020.","title":"Social choice ethics in artificial intelligence","authors":"Seth D Baum","year":"2020","venue":"AI & Society","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Beckers, 2021] Sander Beckers. The counterfactual ness definition of causation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pages 6210- 6217, 2021.","title":"The counterfactual ness definition of causation","authors":"Sander Beckers","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bentzen et al., 2018] Martin Mose Bentzen, Felix Lindner, Louise Dennis, and Michael Fisher. Moral permissibil- ity of actions in smart home systems. In Federated Logic Conference 2018, 2018.","title":"Moral permissibility of actions in smart home systems","authors":"Martin Mose Bentzen, Felix Lindner, Louise Dennis, Michael Fisher","year":"2018","venue":"Federated Logic Conference 2018","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Berreby et al., 2017] Fiona Berreby, Gauvain Bourgne, and Jean-Gabriel Ganascia. A declarative modular framework for representing and applying ethical principles. In 16th Conference on AAMAS, Sao Paulo, Brazil, May 2017.","title":"A declarative modular framework for representing and applying ethical principles","authors":"Fiona Berreby, Gauvain Bourgne, Jean-Gabriel Ganascia","year":"2017","venue":"16th Conference on AAMAS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bochman, 2018] Alexander Bochman. Actual causality in a logical setting. In IJCAI, pages 1730-1736, 2018.","title":"Actual causality in a logical setting","authors":"Alexander Bochman","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/239"},{"position":10,"raw":"[Bremner et al., 2019] Paul Bremner, Louise A Dennis, Michael Fisher, and Alan F Winfield. On proactive, trans- parent, and verifiable ethical reasoning for robots. Pro- ceedings of the IEEE, 107(3):541-561, 2019.","title":"On proactive, transparent, and verifiable ethical reasoning for robots","authors":"Paul Bremner, Louise A Dennis, Michael Fisher, Alan F Winfield","year":"2019","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Claßen and Delgrande, 2020] Jens Claßen and James Del- grande. Dyadic obligations over complex actions as de- ontic constraints in the situation calculus. In Proceedings of the International Conference on KR, volume 17, pages 253-263, 2020.","title":"Jens Claßen and James Delgrande. Dyadic obligations over complex actions as deontic constraints in the situation calculus","authors":"Delgrande Claßen","year":"2020","venue":"Proceedings of the International Conference on KR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dennis et al., 2021] Louise A Dennis, Martin Mose Bentzen, Felix Lindner, and Michael Fisher. Verifiable machine ethics in changing contexts. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11470-11478, 2021.","title":"Verifiable machine ethics in changing contexts","authors":"Louise A Dennis, Martin Mose Bentzen, Felix Lindner, Michael Fisher","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Driver, 2011] Julia Driver. Consequentialism. Routledge, 2011.","title":"Consequentialism","authors":"Julia Driver","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Driver, 2013] Julia Driver. Ethics: The Fundamentals. John Wiley & Sons, 2013.","title":"Julia Driver. Ethics: The Fundamentals","authors":"Driver","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ghallab et al., 2004] Malik Ghallab, Dana Nau, and Paolo Traverso. Automated Planning: theory and practice. El- sevier, 2004.","title":"Automated Planning: theory and practice","authors":"Malik Ghallab, Dana Nau, Paolo Traverso","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gotterbarn, 2001] Donald Gotterbarn. Informatics and pro- fessional responsibility. Science and Engineering Ethics, 7(2):221-230, 2001.","title":"Donald Gotterbarn. Informatics and professional responsibility","authors":"Gotterbarn","year":"2001","venue":"Science and Engineering Ethics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Grisez, 1978] Germain Grisez. Against consequentialism. American Journal of Jurisprudence, 23, 1978.","title":"Against consequentialism","authors":"Germain Grisez, Grisez","year":"1978","venue":"American Journal of Jurisprudence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hegde et al., 2020] Aditya Hegde, Vibhav Agarwal, and Shrisha Rao. Ethics, prosperity, and society: Moral evalu- ation using virtue ethics and utilitarianism. In 29th Inter- national Joint Conference on Artificial Intelligence (IJCAI 2020). doi, volume 10, 2020.","title":"Ethics, prosperity, and society: Moral evaluation using virtue ethics and utilitarianism","authors":"Aditya Hegde, Vibhav Agarwal, Shrisha Rao","year":"2020","venue":"29th International Joint Conference on Artificial Intelligence (IJCAI 2020)","doi":null,"url":null,"internal_url":"/proceedings/2020/24"},{"position":19,"raw":"[Hendrycks et al., 2020] Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning AI with shared human values. arXiv preprint arXiv:2008.02275, 2020.","title":"Aligning AI with shared human values","authors":"Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, Jacob Steinhardt","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lindner et al., 2020] Felix Lindner, Robert Mattmüller, and Bernhard Nebel. Evaluation of the moral permissibility of action plans. Artificial Intelligence, 287:103350, 2020.","title":"Evaluation of the moral permissibility of action plans","authors":"Felix Lindner, Robert Mattmüller, Bernhard Nebel","year":"2020","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lindström, 2022] Sten Lindström. A semantic approach to nonmonotonic reasoning. Theoria, 88(3):494-528, 2022.","title":"A semantic approach to nonmonotonic reasoning","authors":"Sten Lindström","year":"2022","venue":"Theoria","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[List, 2022] Christian List. Social Choice Theory. In Ed- ward N. Zalta, editor, The Stanford Encyclopedia of Phi- losophy. Metaphysics Research Lab, Stanford University, Spring 2022 edition, 2022.","title":"Christian List. Social Choice Theory","authors":"List","year":"2022","venue":"The Stanford Encyclopedia of Philosophy","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Loreggia et al., 2018] Andrea Loreggia, Nicholas Mattei, Francesca Rossi, and K Brent Venable. Preferences and ethical principles in decision making. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Soci- ety, pages 222-222, 2018.","title":"Preferences and ethical principles in decision making","authors":"Andrea Loreggia, Nicholas Mattei, Francesca Rossi, Brent Venable","year":"2018","venue":"Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[McCarthy, 1958] John McCarthy. Programs with common- sense. In Proceedings of the Symposium on Mechniza- tion of Thought Processes, volume 1, pages 77-84. Her Majesty's Stationery Office, 1958.","title":"Programs with commonsense","authors":"John Mccarthy","year":"1958","venue":"Proceedings of the Symposium on Mechnization of Thought Processes","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mill, 1863] John Stuart Mill. Utilitarianism. London: Parker, Son, and Bourn, 1863.","title":"Utilitarianism","authors":"John Stuart, Mill","year":"1863","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mill, 1978] John Stuart Mill. On liberty, ed. Elizabeth Ra- paport, Indianapolis: Hackett, 1978.","title":"Elizabeth Rapaport","authors":"John Stuart, Mill","year":"1978","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Moor, 2006] James H Moor. The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4):18-21, 2006.","title":"The nature, importance, and difficulty of machine ethics","authors":"H James, Moor","year":"2006","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Moulin, 1985] Hervé Moulin. Choice functions over a finite set: a summary. Social Choice and Welfare, 2(2):147-160, 1985.","title":"Choice functions over a finite set: a summary","authors":"Hervé Moulin","year":"1985","venue":"Social Choice and Welfare","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Noothigattu et al., 2018] Ritesh Noothigattu, Snehalkumar Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, and Ariel Procaccia. A voting-based system for ethical decision making. In Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018.","title":"A voting-based system for ethical decision making","authors":"Ritesh Noothigattu, Snehalkumar Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, Ariel Procaccia","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Raphael, 1991] David Daiches Raphael. British Moralists, 1650-1800: Hume, volume 2. Hackett Publishing, 1991.","title":"British Moralists","authors":"David Daiches, Raphael","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Reiter, 2001] Raymond Reiter. Knowledge in Action: Logi- cal Foundations for Specifying and Implementing Dynam- ical Systems. MIT press, 2001.","title":"Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems","authors":"Raymond Reiter, Reiter","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Rodriguez-Soto et al., 2021] Manel Rodriguez-Soto, Maite Lopez-Sanchez, and Juan A Rodriguez-Aguilar. Multi- objective reinforcement learning for designing ethical en- vironments. In Proceedings of the 30th IJCAI, pages 545- 551, 2021.","title":"Multiobjective reinforcement learning for designing ethical environments","authors":"Manel Rodriguez-Soto, Maite Lopez-Sanchez, Juan A Rodriguez-Aguilar","year":"2021","venue":"Proceedings of the 30th IJCAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Rossi and Mattei, 2019] Francesca Rossi and Nicholas Mat- tei. Building ethically bounded AI. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9785-9789, 2019.","title":"Building ethically bounded AI","authors":"Francesca Rossi, Nicholas Mattei","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sarmiento et al., 2022] Camilo Sarmiento, Gauvain Bourgne, Daniele Cavalli, Katsumi Inoue, and Jean- Gabriel Ganascia. Action languages based actual causality in ethical decision making contexts. CoRR, abs/2205.02919, 2022.","title":"Action languages based actual causality in ethical decision making contexts","authors":"Camilo Sarmiento, Gauvain Bourgne, Daniele Cavalli, Katsumi Inoue, Jean-Gabriel Ganascia","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Scarre, 2020] Geoffrey Scarre. Utilitarianism. Routledge, 2020.","title":"Utilitarianism","authors":"Geoffrey Scarre","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Smart and Williams, 1973] John Jamieson Carswell Smart and Bernard Williams. Utilitarianism: For and Against. Cambridge University Press, 1973.","title":"Utilitarianism: For and Against","authors":"Williams ; Smart, John Jamieson, Carswell Smart, Bernard Williams","year":"1973","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Speith, 2022] Timo Speith. A review of taxonomies of ex- plainable artificial intelligence (XAI) methods. In 2022 ACM Conference on Fairness, Accountability, and Trans- parency, pages 2239-2250, 2022.","title":"A review of taxonomies of explainable artificial intelligence (XAI) methods","authors":"Timo Speith","year":"2022","venue":"ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Takeshita et al., 2023] Masashi Takeshita, Rzepka Rafal, and Kenji Araki. Towards theory-based moral AI: Moral AI with aggregating models based on normative ethical theory. arXiv preprint arXiv:2306.11432, 2023.","title":"Towards theory-based moral AI: Moral AI with aggregating models based on normative ethical theory","authors":"Masashi Takeshita, Rzepka Rafal, Kenji Araki","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Thornton et al., 2016] Sarah M Thornton, Selina Pan, Stephen M Erlien, and J Christian Gerdes. Incorporat- ing ethical considerations into automated vehicle control. IEEE Transactions on Intelligent Transportation Systems, 18(6):1429-1439, 2016.","title":"Incorporating ethical considerations into automated vehicle control","authors":"Selina Sarah M Thornton, Pan, J Stephen M Erlien, Gerdes Christian","year":"2016","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Tolmeijer et al., 2020] Suzanne Tolmeijer, Markus Kneer, Cristina Sarasua, Markus Christen, and Abraham Bern- stein. Implementations in machine ethics: A survey. ACM Computing Surveys (CSUR), 53(6):1-38, 2020.","title":"Implementations in machine ethics: A survey","authors":"Suzanne Tolmeijer, Markus Kneer, Cristina Sarasua, Markus Christen, Abraham Bernstein","year":"2020","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Winfield et al., 2014] Alan FT Winfield, Christian Blum, and Wenguo Liu. Towards an ethical robot: Internal mod- els, consequences and ethical action selection. In Confer- ence Towards Autonomous Robotic Systems, pages 85-96. Springer, 2014.","title":"Towards an ethical robot: Internal models, consequences and ethical action selection","authors":"Alan Ft Winfield, Christian Blum, Wenguo Liu","year":"2014","venue":"Conference Towards Autonomous Robotic Systems","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhan et al., 2022] Xiao Zhan, Stefan Sarkadi, Natalia Cri- ado, and Jose Such. A model for governing information sharing in smart assistants. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 845-855, 2022.","title":"A model for governing information sharing in smart assistants","authors":"Xiao Zhan, Stefan Sarkadi, Natalia Criado, Jose Such","year":"2022","venue":"Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}