{"type":"ScholarlyArticle","title":"Analyzing Intentional Behavior in Autonomous Agents under Uncertainty","authors":[{"name":"Filip Cano Córdoba","slug":"filip-cano-cordoba","given":"Filip","family":"Cano Córdoba","affiliation":"Graz University of Technology"},{"name":"Samuel Judson","slug":"samuel-judson","given":"Samuel","family":"Judson","affiliation":"Yale University"},{"name":"Timos Antonopoulos","slug":"timos-antonopoulos","given":"Timos","family":"Antonopoulos","affiliation":"Yale University"},{"name":"Katrine Bjørner","slug":"katrine-bj-rner","given":"Katrine","family":"Bjørner","affiliation":"New York University"},{"name":"Nicholas Shoemaker","slug":"nicholas-shoemaker","given":"Nicholas","family":"Shoemaker","affiliation":"Yale University"},{"name":"Scott J. Shapiro","slug":"scott-j-shapiro","given":"Scott J.","family":"Shapiro","affiliation":"Yale University"},{"name":"Ruzica Piskac","slug":"ruzica-piskac","given":"Ruzica","family":"Piskac","affiliation":"Yale University"},{"name":"Bettina Könighofer","slug":"bettina-konighofer","given":"Bettina","family":"Könighofer","affiliation":"Graz University of Technology"}],"abstract":"Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. We model an uncertain environment as a Markov Decision Process (MDP). For a given scenario, we rely on probabilistic model checking to compute the ability of the agent to influence reaching a certain event. We call this the scope of agency. We say that there is evidence of intentional behavior if the scope of agency is high and the decisions of the agent are close to being optimal for reaching the event.  Our method applies counterfactual reasoning to automatically generate relevant scenarios that can be analyzed to increase the confidence of our assessment. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions.","doi":"10.24963/ijcai.2023/42","url":"https://ijcai.org/proceedings/2023/42","pdf":"https://ijcai.org/proceedings/2023/0042.pdf","markdown":"https://ijcai.org/proceedings/2023/42.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/42","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":42,"pages":{"first":372,"last":381},"keywords":["AI Ethics, Trust, Fairness: ETF: Accountability","Planning and Scheduling: PS: Markov decisions processes","AI Ethics, Trust, Fairness: ETF: Moral decision making"],"videos":[],"references":[{"position":1,"raw":"[Agha and Palmskog, 2018] Gul Agha and Karl Palmskog. A survey of statistical model checking. ACM Transactions on Modeling and Computer Simulation, 28(1):1-39, 2018.","title":"Gul Agha and Karl Palmskog. A survey of statistical model checking","authors":"Palmskog Agha","year":"2018","venue":"ACM Transactions on Modeling and Computer Simulation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aleksandrowicz et al., 2017] Gadi Aleksandrowicz, Hana Chockler, Joseph Y. Halpern, and Alexander Ivrii. The computational complexity of structure-based causality. Journal of Artificial Intelligence Research, 58:431-451, 2017.","title":"The computational complexity of structure-based causality","authors":"Gadi Aleksandrowicz, Hana Chockler, Joseph Y Halpern, Alexander Ivrii","year":"2017","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Anscombe, 1957] Gertrude Elizabeth Margaret Anscombe. Intention. Harvard University Press, 1957.","title":"Gertrude Elizabeth Margaret Anscombe. Intention","authors":"Anscombe","year":"1957","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ashok et al., 2020] Pranav Ashok, Mathias Jackermeier, Pushpak Jagtap, Jan Křetínský, Maximilian Weininger, and Majid Zamani. dtControl: decision tree learning algo- rithms for controller representation. In Proceedings of the 23rd International Conference on Hybrid Systems: Com- putation and Control (HSCC'20), pages 17:1-17:7. ACM, 2020.","title":"dtControl: decision tree learning algorithms for controller representation","authors":"Pranav Ashok, Mathias Jackermeier, Pushpak Jagtap, Jan Křetínský, Maximilian Weininger, Majid Zamani","year":"2020","venue":"Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control (HSCC'20)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Baier and Katoen, 2008] Christel Baier and Joost-Pieter Katoen. Principles of Model Checking, chapter 10. Proba- bilistic Systems. MIT Press, 2008.","title":"Principles of Model Checking, chapter 10. Probabilistic Systems","authors":"Christel Baier, Joost-Pieter Katoen","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Baier et al., 2021] Christel Baier, Florian Funke, and Rupak Majumdar. Responsibility Attribution in Parameterized Markovian Models. In Proceedings of the 35th AAAI Con- ference on Artificial Intelligence (AAAI'21), pages 11734- 11743. AAAI Press, 2021.","title":"Responsibility Attribution in Parameterized Markovian Models","authors":"Christel Baier, Florian Funke, Rupak Majumdar","year":"2021","venue":"Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI'21)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Barbier et al., 2019] Mathieu Barbier, Alessandro Renza- glia, Jean Quilbeuf, Lukas Rummelhard, Anshul Paigwar, Christian Laugier, Axel Legay, Javier Ibañez-Guzmán, and Olivier Simonin. Validation of Perception and Decision- Making Systems for Autonomous Driving via Statistical Model Checking. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV'19), pages 252-259. IEEE, 2019.","title":"Validation of Perception and Decision-Making Systems for Autonomous Driving via Statistical Model Checking","authors":"Mathieu Barbier, Alessandro Renzaglia, Jean Quilbeuf, Lukas Rummelhard, Anshul Paigwar, Christian Laugier, Axel Legay, Javier Ibañez-Guzmán, Olivier Simonin","year":"2019","venue":"Proceedings of the IEEE Intelligent Vehicles Symposium (IV'19)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Beckers et al., 2022] Sander Beckers, Hana Chockler, and Joseph Y Halpern. A Causal Analysis of Harm. In Advances in Neural Information Processing Systems (NeurIPS'22), volume 35, pages 2365-2376.","title":"A Causal Analysis of Harm","authors":"Sander Beckers, Hana Chockler, Joseph Y Halpern","year":null,"venue":"Advances in Neural Information Processing Systems (NeurIPS'22)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Braham and van Hees, 2011] Matthew Braham and Martin van Hees. Responsibility voids. The Philosophical Quar- terly, 61(242):6-15, 2011.","title":"Responsibility voids","authors":"Van Braham, Matthew Hees, Martin Braham, Van Hees","year":"2011","venue":"The Philosophical Quarterly","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Braham and van Hees, 2012] Matthew Braham and Martin van Hees. An Anatomy of Moral Responsibility. Mind, 121(483):601-634, 2012.","title":"An Anatomy of Moral Responsibility","authors":"Van Braham, Matthew Hees, Martin Braham, Van Hees","year":"2012","venue":"Mind","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Bratman, 1987] Michael E. Bratman. Intention, Plans, and Practical Reason. Harvard University Press, 1987.","title":"Intention, Plans, and Practical Reason","authors":"E Michael, Bratman","year":"1987","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Bratman, 1999] Michael E. Bratman. Faces of Intention: Selected Essays on Intention and Agency. Cambridge Studies in Philosophy. Cambridge University Press, 1999.","title":"Faces of Intention: Selected Essays on Intention and Agency","authors":"E Michael, Bratman","year":"1999","venue":"Cambridge Studies in Philosophy","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Budde et al., 2021] Carlos E. Budde, Arnd Hartmanns, Michaela Klauck, Jan Křetínskỳ, David Parker, Tim Quat- mann, Andrea Turrini, and Zhen Zhang. On Correctness, Precision, and Performance in Quantitative Verification. In Proceedings of the 9th International Symposium on Lever- aging Applications of Formal Methods (ISoLA'21), pages 216-241. Springer, 2021.","title":"On Correctness, Precision, and Performance in Quantitative Verification","authors":"Carlos E Budde, Arnd Hartmanns, Michaela Klauck, Jan Křetínskỳ, David Parker, Tim Quatmann, Andrea Turrini, Zhen Zhang","year":"2021","venue":"Proceedings of the 9th International Symposium on Leveraging Applications of Formal Methods (ISoLA'21)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Chockler and Halpern, 2004] Hana Chockler and Joseph Y. Halpern. Responsibility and Blame: A Structural-Model Approach. Journal of Artificial Intelligence Research, 22:93-115, 2004.","title":"Responsibility and Blame: A Structural-Model Approach","authors":"Halpern ; ] Chockler, Hana Chockler, Joseph Y Halpern","year":"2004","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":"/proceedings/2003/21"},{"position":15,"raw":"[Clarke et al., 2018] Edmund M. Clarke, Thomas A. Hen- zinger, Helmut Veith, and Roderick Bloem. Handbook of Model Checking. Springer, 2018.","title":"Handbook of Model Checking","authors":"Edmund M Clarke, Thomas A Henzinger, Helmut Veith, Roderick Bloem","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Cohen and Levesque, 1990] Philip R. Cohen and Hector J. Levesque. Intention is choice with commitment. Artificial Intelligence, 42(2):213-261, 1990.","title":"Intention is choice with commitment","authors":"Philip R Cohen, Hector J Levesque","year":"1990","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Davidson, 1963] Donald Davidson. Actions, reasons, and causes. The Journal of Philosophy, 60(23):685-700, 1963.","title":"Actions, reasons, and causes","authors":"Donald Davidson","year":"1963","venue":"The Journal of Philosophy","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Dosovitskiy et al., 2017] Alexey Dosovitskiy, Germán Ros, Felipe Codevilla, Antonio M. López, and Vladlen Koltun. CARLA: An Open Urban Driving Simulator. In Pro- ceedings of the 1st Annual Conference on Robot Learning (CoRL'17), volume 78, pages 1-16. PMLR, 2017.","title":"CARLA: An Open Urban Driving Simulator","authors":"Alexey Dosovitskiy, Germán Ros, Felipe Codevilla, Antonio M López, Vladlen Koltun","year":"2017","venue":"Proceedings of the 1st Annual Conference on Robot Learning (CoRL'17)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Halpern and Kleiman-Weiner, 2018] Joseph Y. Halpern and Max Kleiman-Weiner. Towards formal definitions of blameworthiness, intention, and moral responsibility. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18), pages 1853-1860. AAAI Press, 2018.","title":"Towards formal definitions of blameworthiness, intention, and moral responsibility","authors":"Joseph Y Halpern, Max Kleiman-Weiner","year":"2018","venue":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Halpern and Pearl, 2005a] Joseph Y. Halpern and Judea Pearl. Causes and Explanations: A Structural-Model Ap- proach. Part I: Causes. The British Journal for the Philos- ophy of Science, 56(4):843-887, 2005.","title":"Causes and Explanations: A Structural-Model Approach. Part I: Causes","authors":"Pearl Halpern, Joseph Y Halpern, Judea Pearl","year":"2005","venue":"The British Journal for the Philosophy of Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Halpern and Pearl, 2005b] Joseph Y. Halpern and Judea Pearl. Causes and Explanations: A Structural-Model Ap- proach. Part II: Explanations. The British Journal for the Philosophy of Science, 56(4):889-911, 2005.","title":"Causes and Explanations: A Structural-Model Approach. Part II: Explanations","authors":"Pearl Halpern, Joseph Y Halpern, Judea Pearl","year":"2005","venue":"The British Journal for the Philosophy of Science","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hensel et al., 2022] Christian Hensel, Sebastian Junges, Joost-Pieter Katoen, Tim Quatmann, and Matthias Volk. The probabilistic model checker storm. International Journal on Software Tools for Technology Transfer, 24(4):589-610, 2022.","title":"The probabilistic model checker storm","authors":"Hensel","year":"2022","venue":"International Journal on Software Tools for Technology Transfer","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Jones et al., 2020] David Jones, Chris Snider, Aydin Nassehi, Jason Yon, and Ben Hicks. Characterising the Digital Twin: A systematic literature review. CIRP Jour- nal of Manufacturing Science and Technology, 29:36-52, 2020.","title":"Characterising the Digital Twin: A systematic literature review","authors":"David Jones, Chris Snider, Aydin Nassehi, Jason Yon, Ben Hicks","year":"2020","venue":"CIRP Journal of Manufacturing Science and Technology","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Judson et al., 2023] Samuel Judson, Matthew Elacqua, Filip Cano Córdoba, Timos Antonopoulos, Bettina Könighofer, Scott J. Shapiro, and Ruzica Piskac. 'Put the Car on the Stand': SMT-based Oracles for Investigating Decisions. preprint, arXiv:2305.05731, 2023.","title":"Put the Car on the Stand': SMT-based Oracles for Investigating Decisions","authors":"Judson","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kwiatkowska et al., 2011] Marta Kwiatkowska, Gethin Norman, and David Parker. PRISM 4.0: Verification of Probabilistic Real-time Systems. In Proceedings of the 23rd International Conference on Computer Aided Verification (CAV'11), pages 585-591, 2011.","title":"PRISM 4.0: Verification of Probabilistic Real-time Systems","authors":"Marta Kwiatkowska, Gethin Norman, David Parker","year":"2011","venue":"Proceedings of the 23rd International Conference on Computer Aided Verification (CAV'11)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lewis, 2013] David Lewis. Counterfactuals. John Wiley & Sons, 2013. Originally published in 1973.","title":"Counterfactuals","authors":"David Lewis","year":"1973","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mele, 1992] Alfred R. Mele. Springs of Action: Under- standing Intentional Behavior. Oxford University Press, 1992.","title":"Springs of Action: Understanding Intentional Behavior","authors":"Alfred R Mele","year":"1992","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Menzies and Beebee, 2020] Peter Menzies and Helen Bee- bee. Counterfactual Theories of Causation. In The Stan- ford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, 2020.","title":"Peter Menzies and Helen Beebee","authors":"Beebee Menzies","year":"2020","venue":"The Stanford Encyclopedia of Philosophy","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Miller, 2019] Tim Miller. Explanation in artificial intelli- gence: Insights from the social sciences. Artificial Intelli- gence, 267:1-38, 2019.","title":"Explanation in artificial intelligence: Insights from the social sciences","authors":"Tim Miller, Miller","year":"2019","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ng and Russell, 2000] Andrew Y. Ng and Stuart Russell. Algorithms for Inverse Reinforcement Learning. In Pro- ceedings of the 17th International Conference on Machine Learning (ICML'00), pages 663-670.","title":"Algorithms for Inverse Reinforcement Learning","authors":"Andrew Y Ng, Stuart Russell","year":null,"venue":"Proceedings of the 17th International Conference on Machine Learning (ICML'00)","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"Morgan Kaufmann, 2000. [Pearl, 2009] Judea Pearl. Causality. Cambridge University Press, 2009.","title":"Pearl, 2009] Judea Pearl. Causality","authors":"Morgan Kaufmann","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Pnueli, 1977] Amir Pnueli. The temporal logic of programs. In Proceedings of the 18th Symposium on Foundations of Computer Science (FOCS'77), pages 46-57. IEEE, 1977.","title":"The temporal logic of programs","authors":"Amir Pnueli, Pnueli","year":"1977","venue":"Proceedings of the 18th Symposium on Foundations of Computer Science (FOCS'77)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Pranger et al., 2021] Stefan Pranger, Bettina Könighofer, Lukas Posch, and Roderick Bloem. TEMPEST -Synthe- sis Tool for Reactive Systems and Shields in Probabilistic Environments. In Proceedings of the 19th International Symposium on Automated Technology for Verification and Analysis (ATVA'21), volume 12971 of Lecture Notes in Computer Science, pages 222-228. Springer, 2021.","title":"TEMPEST -Synthesis Tool for Reactive Systems and Shields in Probabilistic Environments","authors":"Stefan Pranger, Bettina Könighofer, Lukas Posch, Roderick Bloem","year":"2021","venue":"Proceedings of the 19th International Symposium on Automated Technology for Verification and Analysis (ATVA'21)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Puiutta and Veith, 2020] Erika Puiutta and Eric Veith. Ex- plainable Reinforcement Learning: A Survey. In Pro- ceedings of the International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD- MAKE'20), pages 77-95. Springer, 2020.","title":"Explainable Reinforcement Learning: A Survey","authors":"Veith Puiutta, Erika Puiutta, Eric Veith","year":"2020","venue":"Proceedings of the International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE'20)","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Quine, 1969] Willard Van Orman Quine. Ontological Rela- tivity and Other Essays. Columbia University Press, 1969.","title":"Ontological Relativity and Other Essays","authors":"Willard Van Orman Quine","year":"1969","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Rao and Georgeff, 1991] Anand S. Rao and Michael P. Georgeff. Modeling Rational Agents within a BDI- Architecture. In Proceedings of the 2nd International Con- ference on Principles of Knowledge Representation and Reasoning (KR'91), pages 473-484.","title":"Modeling Rational Agents within a BDI-Architecture","authors":"Anand S Rao, Michael P Georgeff","year":null,"venue":"Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning (KR'91)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"Morgan Kaufmann, 1991.","title":null,"authors":"Morgan Kaufmann","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Rao and Georgeff, 1995] Anand S. Rao and Michael P. Georgeff. BDI Agents: From Theory to Practice. In Pro- ceedings of the 1st International Conference on Multiagent Systems (ICMAS'95), pages 312-319. MIT Press, 1995.","title":"BDI Agents: From Theory to Practice","authors":"Anand S Rao, Michael P Georgeff","year":"1995","venue":"Proceedings of the 1st International Conference on Multiagent Systems (ICMAS'95)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Richens et al., 2022] Jonathan Richens, Rory Beard, and Daniel H. Thompson. Counterfactual harm. In Advances in Neural Information Processing Systems (NeurIPS'22), volume 35, pages 36350-36365, 2022.","title":"Counterfactual harm","authors":"Jonathan Richens, Rory Beard, Daniel H Thompson","year":"2022","venue":"Advances in Neural Information Processing Systems (NeurIPS'22)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Scanlon, 2010] Thomas Michael Scanlon. Moral Dimen- sions: Permissibility, Meaning, Blame. Harvard University Press, 2010.","title":"Moral Dimensions: Permissibility, Meaning, Blame","authors":"Thomas Michael, Scanlon","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Shapiro, 2014] Scott J. Shapiro. Massively Shared Agency. Rational and Social Agency: The Philosophy of Michael Bratman, pages 257-293, 2014.","title":"Massively Shared Agency","authors":"J Scott, Shapiro","year":"2014","venue":"Rational and Social Agency: The Philosophy of Michael Bratman","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Simari and Parsons, 2011] Gerardo I. Simari and Simon D. Parsons. Markov Decision Processes and the Belief- Desire-Intention Model: Bridging the Gap for Au- tonomous Agents. Springer Science & Business Media, 2011.","title":"Markov Decision Processes and the Belief-Desire-Intention Model: Bridging the Gap for Autonomous Agents","authors":"Gerardo I Simari, Simon D Parsons","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Song et al., 2016] Jinhua Song, Yang Gao, Hao Wang, and Bo An. Measuring the Distance between Finite Markov Decision Processes. In Proceedings of the International Conference on Autonomous Agents & Multiagent Systems (AAMAS'16), pages 468-476. ACM, 2016.","title":"Measuring the Distance between Finite Markov Decision Processes","authors":"Jinhua Song, Yang Gao, Hao Wang, Bo An","year":"2016","venue":"Proceedings of the International Conference on Autonomous Agents & Multiagent Systems (AAMAS'16)","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Sutton and Barto, 2018] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2018.","title":"Reinforcement Learning: An Introduction","authors":"Richard S Sutton, Andrew G Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Thompson, 1980] Dennis F Thompson. The Moral Respon- sibility of Public Officials: The Problem of Many Hands. American Political Science Review, 74(4):905-916, 1980.","title":"The Moral Responsibility of Public Officials: The Problem of Many Hands","authors":"Dennis F Thompson","year":"1980","venue":"American Political Science Review","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Triantafyllou et al., 2021] Stelios Triantafyllou, Adish Singla, and Goran Radanovic. On Blame Attribution for Accountable Multi-Agent Sequential Decision Making. In Advances in Neural Information Processing Systems (NeurIPS '21), volume 34, pages 15774-15786.","title":"On Blame Attribution for Accountable Multi-Agent Sequential Decision Making","authors":"Stelios Triantafyllou, Adish Singla, Goran Radanovic","year":null,"venue":"Advances in Neural Information Processing Systems (NeurIPS '21)","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Van de Poel, 2015] Ibo Van de Poel. The problem of many hands. In Moral responsibility and the problem of many hands, pages 50-92. Routledge, 2015.","title":"Ibo Van de Poel. The problem of many hands","authors":"Poel Van De","year":"2015","venue":"Moral responsibility and the problem of many hands","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Wachter et al., 2017] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual Explanations With- out Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology, 31(2):841-887, 2017.","title":"Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR","authors":"Sandra Wachter, Brent Mittelstadt, Chris Russell","year":"2017","venue":"Harvard Journal of Law & Technology","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Winikoff et al., 2021] Michael Winikoff, Galina Sidorenko, Virginia Dignum, and Frank Dignum. Why bad coffee? explaining BDI agent behaviour with valuings. Artificial Intelligence, 300:103554, 2021.","title":"Why bad coffee? explaining BDI agent behaviour with valuings","authors":"Michael Winikoff, Galina Sidorenko, Virginia Dignum, Frank Dignum","year":"2021","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Yazdanpanah and Dastani, 2016] Vahid Yazdanpanah and Mehdi Dastani. Distant group responsibility in multi- agent systems. In Proceedings of the International Con- ference on Principles of Practice in Multi-Agent Systems (PRIMA'16), pages 261-278. Springer, 2016.","title":"Distant group responsibility in multiagent systems","authors":"Dastani Yazdanpanah, Vahid Yazdanpanah, Mehdi Dastani","year":"2016","venue":"Proceedings of the International Conference on Principles of Practice in Multi-Agent Systems (PRIMA'16)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}