{"type":"ScholarlyArticle","title":"From Automation to Autonomous Systems: A Legal Phenomenology with Problems of Accountability","authors":[{"name":"Ugo Pagallo","slug":"ugo-pagallo","given":"Ugo","family":"Pagallo","affiliation":"University of Turin , Italy , Law School"}],"abstract":"Over the past decades a considerable amount of work has been devoted to the notion of autonomy and the intelligence of robots and of AI systems: depending on the application, several standards on the “levels of automation” have been proposed. Although current AI systems may have the intelligence of a fridge, or of a toaster, some of such autonomous systems have already challenged basic pillars of society and the law, e.g. whether lethal force should ever be permitted to be “fully automated.” The aim of this paper is to show that the normative challenges of AI entail different types of accountability that go hand-in-hand with choices of technological dependence, delegation of cognitive tasks, and trust. The stronger the social cohesion is, the higher the risks that can be socially accepted through the normative assessment of the not fully predictable consequences of tasks and decisions entrusted to AI systems and artificial agents.","doi":"10.24963/ijcai.2017/3","url":"https://ijcai.org/proceedings/2017/3","pdf":"https://ijcai.org/proceedings/2017/0003.pdf","markdown":"https://ijcai.org/proceedings/2017/3.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/3","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Invited Speakers","track":"Invited Speakers","number":3,"pages":{"first":17,"last":23},"keywords":["Agent-based and Multi-agent Systems: Trust and Reputation","Machine Learning: New Problems","Robotics and Vision: Human Robot Interaction"],"videos":[],"references":[{"position":1,"raw":"[Allen and Widdison, 1996] Tom Allen and Robin Widdi- son. Can computers make contracts? Harvard Journal of Law & Technology 9(1): 26-52, 1996.","title":"Tom Allen and Robin Widdison. Can computers make contracts","authors":"Widdison Allen","year":"1996","venue":"Harvard Journal of Law & Technology","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Allen et al., 2000] Colin Allen, Gary Varner and Jason Zinser. Prolegomena to any future artificial moral agent. Journal of Experimental and Theoretical Artificial Intel- ligence, 12: 251-261, 2000.","title":"Prolegomena to any future artificial moral agent","authors":"Colin Allen, Gary Varner, Jason Zinser","year":"2000","venue":"Journal of Experimental and Theoretical Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Barfield, 2005] Woodrow Barfield. Issues of law for soft- ware agents within virtual environments, Presence 14(6): 741-748, 2005.","title":"Woodrow Barfield. Issues of law for software agents within virtual environments","authors":"Barfield","year":"2005","venue":"Presence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burri, 2016] Thomas Burri. The Politics of Robotic Auton- omy. European Journal of Risk Regulation, 7(2): 341- 360, 2016.","title":"The Politics of Robotic Autonomy","authors":"Thomas Burri","year":"2016","venue":"European Journal of Risk Regulation","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cath et al., 2016] Corinne Cath, Sandra Wachter, Brent Mittelstadt, Mariarosaria Taddeo and Luciano Floridi. Arti- ficial Intelligence and the 'Good Society': the US, EU, and UK Approach. Available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=290624 9, December 2016 (last accessed 19 April 2017).","title":"Artificial Intelligence and the 'Good Society': the US, EU, and UK Approach","authors":"Corinne Cath, Sandra Wachter, Brent Mittelstadt, Mariarosaria Taddeo, Luciano Floridi","year":"2016","venue":null,"doi":null,"url":"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2906249","internal_url":null},{"position":6,"raw":"[Coudert, 1995] Allison P. Coudert. Leibniz and the Kabba- lah. Kluwer Academic, Boston-London 1995.","title":"Leibniz and the Kabbalah","authors":"Allison P Coudert","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Davis, 2011] Jim Davis. The (common) Laws of Man over (civilian) Vehicles Unmanned, Journal of Law, Infor- mation and Science, 21(2): 166-179, 2011.","title":"The (common) Laws of Man over (civilian) Vehicles Unmanned","authors":"Jim Davis","year":"2011","venue":"Journal of Law, Information and Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Floridi, 2017] Luciano Floridi. Robots, Jobs, Taxes, and Responsibilities, Philosophy & Technology, 30: 1-4, 2017.","title":"Robots, Jobs, Taxes, and Responsibilities","authors":"Luciano Floridi","year":"2017","venue":"Philosophy & Technology","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Floridi and Sanders, 2004] Luciano Floridi and Jeff Sand- ers. On the Morality of Artificial Agents, Minds and Machines, 14(3): 349-379, 2004.","title":"Luciano Floridi and Jeff Sanders","authors":"Sanders Floridi","year":"2004","venue":"On the Morality of Artificial Agents","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hallevy, 2015] Gabriel Hallevy. Liability for Crimes In- volving Artificial Intelligence Systems. Springer, Dor- drecht, 2015. [Hegel, 1892-6] Lectures on the History of Philosophy. Eng- lish translation by E.S. Haldane, available at https://www.marxists.org/reference/archive/hegel/works/ hp/hpconten.htm (last accessed 19 April 2017).","title":"Liability for Crimes Involving Artificial Intelligence Systems","authors":"Gabriel Hallevy","year":"2015","venue":"Lectures on the History of Philosophy. English translation by E.S. Haldane","doi":null,"url":"https://www.marxists.org/reference/archive/hegel/works/hp/hpconten.htm","internal_url":null},{"position":11,"raw":"[Hildebrandt et al, 2010] Mireille Hildebrandt, Bert-Jaap Koops and David-Olivier Jaquet-Chiffelle. Bridging the accountability gap: Rights for new entities in the infor- mation society? Minnesota Journal of Law, Science & Technology, 11(2): 497-561, 2010.","title":"Bridging the accountability gap: Rights for new entities in the information society?","authors":"Mireille Hildebrandt, Bert-Jaap Koops, David-Olivier Jaquet-Chiffelle","year":"2010","venue":"Minnesota Journal of Law, Science & Technology","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karnow, 1996] Curtis E. A. Karnow. Liability for Distrib- uted Artificial Intelligence, Berkeley Technology and Law Journal, 11: 147-183, 1996.","title":"Liability for Distributed Artificial Intelligence","authors":"E A Curtis, Karnow","year":"1996","venue":"Berkeley Technology and Law Journal","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Minski, 1967] Marvin Minski. Computation: Finite and Infinite Machines. Prentice-Hall, Englewood Cliffs, N.J., 1965.","title":"Computation: Finite and Infinite Machines","authors":"Marvin Minski, Minski","year":"1965","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pagallo, 2005] Ugo Pagallo. Introduzione alla filosofia di- gitale: da Leibniz a Chaitin. Giappichelli, Torino, 2005.","title":"Introduzione alla filosofia digitale: da Leibniz a Chaitin","authors":"Ugo Pagallo","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pagallo, 2013] Ugo Pagallo. The Laws of Robots: Crimes, Contracts, and Torts. Springer, Dordrecht, 2013.","title":"Ugo Pagallo. The Laws of Robots: Crimes, Contracts, and Torts","authors":"Pagallo","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pagallo, 2016a] Ugo Pagallo. Leibniz: una breve biografia intellettuale. Kluwer, Padova, 2016.","title":"Leibniz: una breve biografia intellettuale","authors":"Ugo Pagallo, Pagallo","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pagallo, 2016b] Ugo Pagallo. Even Angels Need the Rules: AI, Roboethics, and the Law. In Gal A Kaminka et al (eds), ECAI 2016. Frontiers in Artificial Intelligence and Applications, pp. 209-215. IOS Press, Amsterdam 2016.","title":"Even Angels Need the Rules: AI, Roboethics, and the Law","authors":"Ugo Pagallo, Pagallo","year":"2016","venue":"ECAI 2016. Frontiers in Artificial Intelligence and Applications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pagallo, 2016c] Ugo Pagallo. Three Lessons Learned for Intelligent Transport Systems that Abide by the Law, JusLetter IT, 24, November 2016. Available at http://jusletter-it.weblaw.ch/issues/2016/24-November- 2016/three-lessons-learne_9251e5d324.html.","title":"Three Lessons Learned for Intelligent Transport Systems that Abide by the Law","authors":"Ugo Pagallo, Pagallo","year":"2016","venue":"JusLetter IT","doi":null,"url":"http://jusletter-it.weblaw.ch/issues/2016/24-November-2016/three-lessons-learne_9251e5d324.html","internal_url":null},{"position":19,"raw":"[Pagallo, 2017a] Ugo Pagallo. The Legal Challenges of Big Data: Putting Secondary Rules First in the Field of EU Data Protection, European Data Protection Law Review, 3(1): 36-46, 2017.","title":"The Legal Challenges of Big Data: Putting Secondary Rules First in the Field of EU Data Protection","authors":"Ugo Pagallo, Pagallo","year":"2017","venue":"European Data Protection Law Review","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pagallo, 2017b] Ugo Pagallo. When Morals Ain't Enough: Robots, Ethics, and the Rules of the Law, Minds and Machines, January 2017.","title":"When Morals Ain't Enough: Robots, Ethics, and the Rules of the Law, Minds and Machines","authors":"Ugo Pagallo, Pagallo","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pagallo and Durante, 2016] Ugo Pagallo and Massimo Du- rante. The Pros and Cons of Legal Automation and its Governance, European Journal of Risk Regulation, 7(2): 323-334, 2016.","title":"The Pros and Cons of Legal Automation and its Governance","authors":"Ugo Pagallo, Massimo Durante","year":"2016","venue":"European Journal of Risk Regulation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Prakken and Sartor, 2015] Henry Prakken and Giovanni Sartor. Law and Logic: A Review from an Argumenta- tion Perspective. Artificial Intelligence 227: 214-245, 2015.","title":"Law and Logic: A Review from an Argumentation Perspective","authors":"Sartor Prakken, Henry Prakken, Giovanni Sartor","year":"2015","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sartor, 2009] Giovanni Sartor. Cognitive automata and the law: Electronic contracting and the intentionality of software agents, Artificial Intelligence and Law 17(4): 253-290, 2009.","title":"Cognitive automata and the law: Electronic contracting and the intentionality of software agents","authors":"Giovanni Sartor, Sartor","year":"2009","venue":"Artificial Intelligence and Law","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Simon, 1965] Herbert Simon. The Shape of Automation for Men and Management. Harper & Row, New York, 1965.","title":"The Shape of Automation for Men and Management","authors":"Herbert Simon","year":"1965","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Solum, 1992] Lawrence B. Solum. Legal personhood for artificial intelligence. North Carolina Law Review, 70: 1231-1287, 1992.","title":"Legal personhood for artificial intelligence","authors":"Lawrence B Solum","year":"1992","venue":"North Carolina Law Review","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weng et al., 2015] Yueh-Hsuan Weng, Yusuke Sugahara, Kenji Hashimoto and Atsuo Takanishi. Intersection of \"Tokku\" Special Zone, Robots, and the Law: A Case Study on Legal Impacts to Humanoid Robots, Interna- tional Journal of Social Robotics, 7(5): 841-857, 2015.","title":"Intersection of \"Tokku\" Special Zone, Robots, and the Law: A Case Study on Legal Impacts to Humanoid Robots","authors":"Yueh-Hsuan Weng, Yusuke Sugahara, Kenji Hashimoto, Atsuo Takanishi","year":"2015","venue":"International Journal of Social Robotics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Weitzenboeck, 2001] Emily Mary Weitzenboeck. Electron- ic Agents and the Formation of Contracts, International Journal of Law and Information Technology, 9(3): 204- 234, 2001.","title":"Electronic Agents and the Formation of Contracts","authors":"Emily Mary, Weitzenboeck","year":"2001","venue":"International Journal of Law and Information Technology","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}