{"type":"ScholarlyArticle","title":"Ethics and Governance of Artificial Intelligence: A Survey of Machine Learning Researchers (Extended Abstract)","authors":[{"name":"Baobao Zhang","slug":"baobao-zhang","given":"Baobao","family":"Zhang","affiliation":"Syracuse University"},{"name":"Markus Anderljung","slug":"markus-anderljung","given":"Markus","family":"Anderljung","affiliation":""},{"name":"Lauren Kahn","slug":"lauren-kahn","given":"Lauren","family":"Kahn","affiliation":"Council on Foreign Relations"},{"name":"Noemi Dreksler","slug":"noemi-dreksler","given":"Noemi","family":"Dreksler","affiliation":""},{"name":"Michael C. Horowitz","slug":"michael-c-horowitz","given":"Michael C.","family":"Horowitz","affiliation":"University of Pennsylvania"},{"name":"Allan Dafoe","slug":"allan-dafoe","given":"Allan","family":"Dafoe","affiliation":""}],"abstract":"Machine learning (ML) and artificial intelligence (AI) researchers play an important role in the ethics and governance of AI, including through their work, advocacy, and choice of employment. Nevertheless, this influential group's attitudes are not well understood, undermining our ability to discern consensuses or disagreements between AI/ML researchers. To examine these researchers' views, we conducted a survey of those who published in two top AI/ML conferences (N = 524). We compare these results with those from a 2016 survey of AI/ML researchers and a 2018 survey of the US public. We find that AI/ML researchers place high levels of trust in international organizations and scientific organizations to shape the development and use of AI in the public interest; moderate trust in most Western tech companies; and low trust in national militaries, Chinese tech companies, and Facebook. While the respondents were overwhelmingly opposed to AI/ML researchers working on lethal autonomous weapons, they are less opposed to researchers working on other military applications of AI, particularly logistics algorithms. A strong majority of respondents think that AI safety research should be prioritized more and a majority that ML institutions should conduct pre-publication review to assess potential harms. Being closer to the technology itself, AI/ML researchers are well placed to highlight new risks and develop technical solutions, so this novel data has broad relevance. The findings should help to improve how researchers, private sector executives, and policymakers think about regulations, governance frameworks, guiding principles, and national and international governance strategies for AI.","doi":"10.24963/ijcai.2022/811","url":"https://ijcai.org/proceedings/2022/811","pdf":"https://ijcai.org/proceedings/2022/0811.pdf","markdown":"https://ijcai.org/proceedings/2022/811.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/811","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Journal Track","track":"Journal Track","number":811,"pages":{"first":5787,"last":5791},"keywords":["AI Ethics, Trust, Fairness: Societal Impact of AI","AI Ethics, Trust, Fairness: AI and Law, Governance, Regulation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987247","length_sec":89,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985390","length_sec":422,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aiken et al., 2020a] Catherina Aiken, James Dunham, and Remco Zwetsloot. Immigration pathways and plans of AI talent. Technical report, Center for Security and Emerging Technology, 2020.","title":"Immigration pathways and plans of AI talent","authors":"Aiken","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aiken et al., 2020b] Catherina Aiken, Rebecca Kagan, and Michael Page. \"Cool projects\" or \"expanding the effi- ciency of the murderous American war machine?\": AI professionals' views on working with the Department of Defense. Technical report, Center for Security and Emerg- ing Technology, 2020.","title":"Cool projects\" or \"expanding the efficiency of the murderous American war machine?","authors":"Aiken","year":"2020","venue":"AI professionals' views on working with the Department of Defense","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amodei et al., 2016] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. CoRR, abs/1606.06565, 2016.","title":"Concrete problems in AI safety","authors":"Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Anderson et al., 2018] Janna Anderson, Lee Rainie, and Alex Luchsinger. Artificial intelligence and the future of humans. Technical report, Pew Research Center, 12 2018.","title":"Artificial intelligence and the future of humans","authors":"Janna Anderson, Lee Rainie, Alex Luchsinger","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barocas et al., 2019] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairmlbook.org, 2019. http://www.fairmlbook.org.","title":"Fairness and Machine Learning. fairmlbook","authors":"Solon Barocas, Moritz Hardt, Arvind Narayanan","year":"2019","venue":null,"doi":null,"url":"http://www.fairmlbook.org","internal_url":null},{"position":6,"raw":"[Brundage et al., 2018] Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Jacob Steinhardt, Carrick Flynn, Seán Ó h Éigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yam- polskiy, and Dario Amodei. The malicious use of artifi- cial intelligence: Forecasting, prevention, and mitigation. Technical report, Future of Humanity Institute and Uni- versity of Oxford and Centre for the Study of Existential Risk and University of Cambridge and Center for a New American Security and Electronic Frontier Foundation and OpenAI, 2018.","title":"The malicious use of artificial intelligence: Forecasting, prevention, and mitigation","authors":"Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Jacob Steinhardt, Carrick Flynn, Simon Seán Ó H Éigeartaigh, Haydn Beard, Sebastian Belfield, Clare Farquhar, Rebecca Lyle, Owain Crootof, Michael Evans, Joanna Page, Roman Bryson, Dario Yampolskiy, Amodei","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grace et al., 2018] Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. When will AI exceed human performance? Evidence from AI experts. Journal of Artificial Intelligence Research, 62:729-754, 2018.","title":"When will AI exceed human performance? Evidence from AI experts","authors":"Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans","year":"2018","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gruetzemacher et al., 2020] Ross Gruetzemacher, David Paradice, and Kang Bok Lee. Forecasting extreme labor displacement: A survey of AI practitioners. Technological Forecasting and Social Change, 161, 2020. 120323.","title":"Forecasting extreme labor displacement: A survey of AI practitioners","authors":"Ross Gruetzemacher, David Paradice, Kang Bok, Lee","year":"2020","venue":"Technological Forecasting and Social Change","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Horowitz, 2018] Michael C Horowitz. Artificial intel- ligence, international competition, and the balance of power. Texas National Security Review, 1(3), 2018.","title":"Artificial intelligence, international competition, and the balance of power","authors":"C Michael, Horowitz","year":"2018","venue":"Texas National Security Review","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Horowitz, 2019] Michael C. Horowitz. When speed kills: Lethal autonomous weapon systems, deterrence and sta- bility. Journal of Strategic Studies, 42(6):764-788, aug 2019.","title":"When speed kills: Lethal autonomous weapon systems, deterrence and stability","authors":"C Michael, Horowitz","year":"2019","venue":"Journal of Strategic Studies","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kanaan, 2020] Michael Kanaan. T-Minus AI: Humanity's Countdown to Artificial Intelligence and the New Pursuit of Global Power. BenBella Books, Dallas, TX, 2020.","title":"T-Minus AI: Humanity's Countdown to Artificial Intelligence and the New Pursuit of Global Power","authors":"Michael Kanaan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krafft et al., 2020] P. M. Krafft, Meg Young, Michael Katell, Karen Huang, and Ghislain Bugingo. Defining AI in policy versus practice. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES '20, page 72-78, New York, NY, USA, 2020. Association for Com- puting Machinery. [Noble, 2018] Safiya Umoja Noble. Algorithms of Oppres- sion. NYU Press, New York, 2018.","title":"Defining AI in policy versus practice","authors":"P M Krafft, Meg Young, Michael Katell, Karen Huang, Ghislain Bugingo","year":"2018","venue":"Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES '20","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Russell, 2019] Stuart Russell. Human Compatible: Artifi- cial Intelligence and the Problem of Control. Penguin, New York, 2019.","title":"Human Compatible: Artificial Intelligence and the Problem of Control","authors":"Stuart Russell","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sandberg and Bostrom, 2011] Anders Sandberg and Nick Bostrom. Machine intelligence survey. Technical report, Future of Humanity Institute, Oxford University, 12 2011. Technical Report #2011-1.","title":"Anders Sandberg and Nick Bostrom","authors":"Bostrom Sandberg","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Scharre, 2018] Paul Scharre. Army of None: Autonomous Weapons and the Future of War. WW Norton & Company, New York, 2018.","title":"Army of None: Autonomous Weapons and the Future of War","authors":"Paul Scharre","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Solaiman et al., 2019] Irene Solaiman, Jack Clark, and Miles Brundage. GPT-2: 1.5B release. OpenAI Blog. URL: https://perma.cc/PFA8-KTBP, 2019. Accessed: 23 Jul. 2020.","title":"GPT-2: 1.5B release","authors":"Irene Solaiman, Jack Clark, Miles Brundage","year":"2019","venue":null,"doi":null,"url":"https://perma.cc/PFA8-KTBP","internal_url":null},{"position":17,"raw":"[Zhang and Dafoe, 2020] Baobao Zhang and Allan Dafoe. US public opinion on the governance of artificial intelli- gence. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES '20, page 187-193, New York, NY, USA, 2020. Association for Computing Machinery.","title":"US public opinion on the governance of artificial intelligence","authors":"Dafoe Zhang, Baobao Zhang, Allan Dafoe","year":"2020","venue":"Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES '20","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang et al., 2021a] Baobao Zhang, Markus Anderljung, Lauren Kahn, Noemi Dreksler, Michael C. Horowitz, and Allan Dafoe. Ethics and Governance of Artificial Intelli- gence: Evidence from a Survey of Machine Learning Re- searchers. Journal of Artificial Intelligence Research, 71, August 2021.","title":"Ethics and Governance of Artificial Intelligence: Evidence from a Survey of Machine Learning Researchers","authors":"Zhang","year":"2021","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhang et al., 2021b] Daniel Zhang, Saurabh Mishra, Erik Brynjolfsson, John Etchemendy, Deep Ganguli, Barbara Grosz, Terah Lyons, James Manyika, Juan Carlos Niebles, Michael Sellitto, Shoham Yoav, Jack Clark, and Ray- mond Perrault. Artificial Intelligence Index Report 2021. Technical report, AI Index Steering Committee, Human- Centered AI Institute, Stanford University, Stanford, 2021.","title":"Artificial Intelligence Index Report 2021","authors":"Zhang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zwetsloot and Dafoe, 2019] Remco Zwetsloot and Allan Dafoe. Thinking about risks from AI: Accidents, mis- use and structure. Lawfare. URL: https://perma.cc/4J2N- 2KYV, 2019. Accessed: 23 Sep. 2020.","title":"Remco Zwetsloot and Allan Dafoe. Thinking about risks from AI: Accidents, misuse and structure","authors":"Dafoe Zwetsloot","year":"2019","venue":null,"doi":null,"url":"https://perma.cc/4J2N-2KYV","internal_url":null},{"position":21,"raw":"[Zwetsloot et al., 2021] Remco Zwetsloot, Baobao Zhang, Noemi Dreksler, Lauren Kahn, Markus Anderljung, Allan Dafoe, and Michael C. Horowitz. Skilled and mobile: Sur- vey evidence of ai researchers' immigration preferences. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21), May 19-21, 2021, Virtual Event, USA, 2021.","title":"Skilled and mobile: Survey evidence of ai researchers' immigration preferences","authors":"Remco Zwetsloot, Baobao Zhang, Noemi Dreksler, Lauren Kahn, Markus Anderljung, Allan Dafoe, Michael C Horowitz","year":"2021","venue":"Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}