{"type":"ScholarlyArticle","title":"The Design of Human Oversight in Autonomous Weapon Systems","authors":[{"name":"Ilse Verdiesen","slug":"ilse-verdiesen","given":"Ilse","family":"Verdiesen","affiliation":""}],"abstract":"Autonomous Weapon Systems (AWS) can be defined as weapons systems equipped with Artificial Intelligence (AI). They are an emerging technology and are increasingly deployed on the battlefield. In the societal debate on Autonomous Weapon Systems, the concept of Meaningful Human Control (MHC) is often mentioned as requirement, but MHC will not suffice as requirement to minimize unintended consequences of Autonomous Weapon Systems, because the definition of ‘control’ implies that one has the power to influence or direct the course of events or the ability to\r\nmanage a machine. The characteristics autonomy, interactivity and adaptability of AI  in Autonomous Weapon Systems inherently imply that control in strict sense is not possible. Therefore, a different approach is needed to minimize unintended consequences of AWS. Several scholars are describing the concept of Human Oversight in Autonomous Weapon Systems and AI in general. Just recently Taddeo and Floridi (2018) describe that human oversight procedures are necessary to minimize unintended consequences and to compensate unfair impacts of AI. In my PhD project, I will analyse the concepts that are needed to define, model, evaluate and ensure human oversight in Autonomous Weapons and design a technical architecture to implement this.","doi":"10.24963/ijcai.2019/923","url":"https://ijcai.org/proceedings/2019/923","pdf":"https://ijcai.org/proceedings/2019/0923.pdf","markdown":"https://ijcai.org/proceedings/2019/923.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/923","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":923,"pages":{"first":6468,"last":6469},"keywords":["Humans and AI: Ethical Issues in AI"],"videos":[],"references":[{"position":1,"raw":"[Adams, 2001] Thomas. K. Adams. Future warfare and the decline of human decisionmaking. Parameters, 31(4), 57-71, 2001.","title":"Future warfare and the decline of human decisionmaking","authors":"K Thomas, Adams","year":"2001","venue":"Parameters","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[AIV & CAVV, 2016] AIV, & CAVV. Autonomous weapon systems: the need for meaningful human control. (No. 97, No. 26), 2016. Retrieved from http://aiv-advice.nl/8gr Accessed on: 23 June 2019 [","title":"Autonomous weapon systems: the need for meaningful human control","authors":"Cavv ; Aiv, & Cavv","year":"2016","venue":null,"doi":null,"url":"http://aiv-advice.nl/8gr","internal_url":null},{"position":3,"raw":"[Bonnemains et al., 2018] Vincent Bonnemains, Claire Saurel, & Catherine Tessier. Embedded ethics: some technical and ethical challenges. Ethics and Information Technology, 20(1), 41-58, 2018.","title":"Embedded ethics: some technical and ethical challenges","authors":"Vincent Bonnemains, Claire Saurel, Catherine Tessier","year":"2018","venue":"Ethics and Information Technology","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ekelhof, 2015] Merel Ekelhof. Autonome wapens: een verkenning van het concept Meaningful Human Control. Militaire Spectator, 184, 2015.","title":"Autonome wapens: een verkenning van het concept Meaningful Human Control","authors":"Merel Ekelhof","year":"2015","venue":"Militaire Spectator","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Floridi & Sanders, 2004] Luciano Floridi & Jeff Sanders. On the morality of artificial agents. Minds and Machines, 14(3), 349-379, 2004.","title":"Luciano Floridi & Jeff Sanders. On the morality of artificial agents","authors":"Sanders Floridi","year":"2004","venue":"Minds and Machines","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Horowitz & Scharre, 2015] Michael Horowitz & Paul Scharre. Meaningful human control in weapon systems: a primer: Center for a New American Security, 2015.","title":"Meaningful human control in weapon systems: a primer: Center for a New American Security","authors":"Michael Horowitz, Paul Scharre","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[HRW & ICRC, 2012] HRW & IHRC. Losing Humanity: The Case against Killer Robots, 2012. Retrieved from https://www.hrw.org/sites/default/files/reports/arms1112 ForUpload_0_0.pdf Accessed on: 23 June 2019 [ICRC, 2018] ICRC. Ethics and autonomous weapon systems: An ethical basis for human control?, 2018. Retrieved from Geneva: https://www.icrc.org/en/download/file/69961/icrc_ethics _and_autonomous_weapon_systems_report_3_april_201 8.pdf Accessed on: 23 June 2019","title":"ICRC, 2018] ICRC. Ethics and autonomous weapon systems: An ethical basis for human control?","authors":null,"year":"2012","venue":null,"doi":null,"url":"https://www.icrc.org/en/download/file/69961/icrc_ethics_and_autonomous_weapon_systems_report_3_april_2018.pdf","internal_url":null},{"position":8,"raw":"[Roff, 2016] Heather M. Roff. Weapons autonomy is rocketing, 2016. Retrieved from http://foreignpolicy.com/2016/09/28/weapons- autonomy-is-rocketing/ Accessed on: 23 June 2019","title":"Weapons autonomy is rocketing","authors":"M Heather, Roff","year":"2016","venue":null,"doi":null,"url":"http://foreignpolicy.com/2016/09/28/weapons-autonomy-is-rocketing/","internal_url":null},{"position":9,"raw":"[Roff & Moyes, 2016] Heather M. Roff & Richard Moyes. Meaningful human control, artificial intelligence and autonomous weapons. Paper presented at the Briefing Paper Prepared for the Informal Meeting of Experts on Lethal Autonomous Weapons Systems, UN Convention on Certain Conventional Weapons, 2016.","title":"Meaningful human control, artificial intelligence and autonomous weapons","authors":"M Heather, Richard Roff, Moyes","year":"2016","venue":"for the Informal Meeting of Experts on Lethal Autonomous Weapons Systems, UN Convention on Certain Conventional Weapons","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Santoni di Sio Santoni & Van den Hoven, 2018] Filippo Santoni di Sio & Jeroen van den Hoven. Meaningful Human Control over Autonomous Systems: A Philosophical Account. Frontiers in Robotics and AI, 5, 15, 2018.","title":"Meaningful Human Control over Autonomous Systems: A Philosophical Account","authors":"Santoni Di, Sio Santoni, & Van Den Hoven, ; Filippo, Santoni Di Sio & Jeroen Van Den Hoven","year":"2018","venue":"Frontiers in Robotics and AI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Taddeo & Floridi, 2018] Mariarosaria Taddeo & Luciano Floridi. How AI can be a force for good. Science, 361(6404), 751-752, 2018.","title":"Mariarosaria Taddeo & Luciano Floridi. How AI can be a force for good","authors":"Floridi Taddeo","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Vignard, 2014] Kerstin Vignard. The weaponization of increasingly autonomous technologies: considering how meaningful human control might move discussion forward. UNIDIR Resources, 2, 2014.","title":"The weaponization of increasingly autonomous technologies: considering how meaningful human control might move discussion forward","authors":"Kerstin Vignard","year":"2014","venue":"UNIDIR Resources","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Williams, 2015] John Williams. Democracy and regulating autonomous weapons: biting the bullet while missing the point? Global Policy, 6(3), 179-189, 2015.","title":"Democracy and regulating autonomous weapons: biting the bullet while missing the point?","authors":"John Williams","year":"2015","venue":"Global Policy","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}