{"type":"ScholarlyArticle","title":"Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (Extended Abstract)*","authors":[{"name":"Songül Tolan","slug":"songul-tolan","given":"Songül","family":"Tolan","affiliation":"European Commission , Joint Research Centre"},{"name":"Annarosa Pesole","slug":"annarosa-pesole","given":"Annarosa","family":"Pesole","affiliation":"European Commission , Joint Research Centre"},{"name":"Fernando Martínez-Plumed","slug":"fernando-martinez-plumed","given":"Fernando","family":"Martínez-Plumed","affiliation":"European Commission , Joint Research Centre; Universitat Politècnica de València"},{"name":"Enrique Fernández-Macías","slug":"enrique-fernandez-macias","given":"Enrique","family":"Fernández-Macías","affiliation":"European Commission , Joint Research Centre"},{"name":"José Hernández-Orallo","slug":"jose-hernandez-orallo","given":"José","family":"Hernández-Orallo","affiliation":"Universitat Politècnica de València; Leverhulme Centre for the Future of Intelligence"},{"name":"Emilia Gómez","slug":"emilia-gomez","given":"Emilia","family":"Gómez","affiliation":"European Commission , Joint Research Centre"}],"abstract":"We present a framework for analysing the impact of AI on occupations. This framework maps 59 generic tasks from different occupational datasets to 14 cognitive abilities and these to  a comprehensive list of 328 AI benchmarks used to evaluate research intensity in AI. The use of cognitive abilities as an intermediate layer allows for an identification of potential AI exposure for tasks for which AI applications have not been explicitly programmed. We provide insights into the abilities through which AI is most likely to affect jobs, and we show how some of the abilities where AI research is currently very intense are linked to tasks with comparatively limited labour input in the labour markets of advanced economies.","doi":"10.24963/ijcai.2022/809","url":"https://ijcai.org/proceedings/2022/809","pdf":"https://ijcai.org/proceedings/2022/0809.pdf","markdown":"https://ijcai.org/proceedings/2022/809.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/809","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Journal Track","track":"Journal Track","number":809,"pages":{"first":5777,"last":5781},"keywords":["Machine Learning: Evaluation","Multidisciplinary Topics and Applications: Economics","AI Ethics, Trust, Fairness: Ethical, Legal and Societal Issues","AI Ethics, Trust, Fairness: Societal Impact of AI"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987240","length_sec":104,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985383","length_sec":750,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Acemoglu and Restrepo, 2018] Daron Acemoglu and Pas- cual Restrepo. The Race between Man and Machine: Im- plications of Technology for Growth, Factor Shares, and Employment. Ame. Economic Review, 108(6):1488-1542, 2018.","title":"The Race between Man and Machine: Implications of Technology for Growth, Factor Shares, and Employment","authors":"Daron Acemoglu, Pascual Restrepo","year":"2018","venue":"Ame. Economic Review","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agrawal et al., 2015] Ajay Agrawal, John Horton, Nicola Lacetera, and Elizabeth Lyons. Digitization and the con- tract labor market: A research agenda. In Economic Analy- sis of the Digital Economy, pages 219-250.","title":"Digitization and the contract labor market: A research agenda","authors":"Ajay Agrawal, John Horton, Nicola Lacetera, Elizabeth Lyons","year":null,"venue":"Economic Analysis of the Digital Economy","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Autor and Dorn, 2013] David Autor and David Dorn. The Growth of Low-Skill Service Jobs and the Polarization of the US Labor Market. Ame. Econ. Rev., 103(5):1553-97, 2013.","title":"The Growth of Low-Skill Service Jobs and the Polarization of the US Labor Market","authors":"Dorn Autor, David Autor, David Dorn","year":"2013","venue":"Ame. Econ. Rev","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Autor, 2015] David Autor. Why Are There Still So Many Jobs? The History and Future of Workplace Automation. Journal of Economic Perspectives, 29(3):3-30, 2015.","title":"Why Are There Still So Many Jobs? The History and Future of Workplace Automation","authors":"David Autor","year":"2015","venue":"Journal of Economic Perspectives","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown et al., 2020] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language Models are Few-Shot Learners. arXiv:2005.14165, 2020.","title":"Language Models are Few-Shot Learners","authors":"Benjamin Tom B Brown, Nick Mann, Melanie Ryder, Jared Subbiah, Prafulla Kaplan, Arvind Dhariwal, Pranav Neelakantan, Girish Shyam, Amanda Sastry, Askell","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brynjolfsson et al., 2018a] Erik Brynjolfsson, Tom Mitchell, and Daniel Rock. What Can Machines Learn, and What Does It Mean for Occupations and the Economy? In AEA Papers and Proceedings, volume 108, pages 43-47, 2018.","title":"What Can Machines Learn, and What Does It Mean for Occupations and the Economy","authors":"Brynjolfsson","year":"2018","venue":"AEA Papers and Proceedings","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brynjolfsson et al., 2018b] Erik Brynjolfsson, Daniel Rock, and Chad Syverson. Artificial Intelligence and the Mod- ern Productivity Paradox: A Clash of Expectations and Statistics. In The Economics of Artificial Intelligence: An Agenda. University of Chicago Press, 2018.","title":"Artificial Intelligence and the Modern Productivity Paradox: A Clash of Expectations and Statistics","authors":"Brynjolfsson","year":"2018","venue":"The Economics of Artificial Intelligence: An Agenda","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Davis and Marcus, 2015] Ernest Davis and Gary Marcus. Commonsense Reasoning and Commonsense Knowledge in AI. Communications of the ACM, 58(9):92-103, 2015.","title":"Commonsense Reasoning and Commonsense Knowledge in AI","authors":"Marcus ; Davis, Ernest Davis, Gary Marcus","year":"2015","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A Large-Scale Hierarchical Image Database. In CVPR, pages 248-255, 2009. [Eurostat, 2014] Eurostat. STRUCTURE OF EARN- INGS SURVEY (SES). https://ec.europa.eu/eurostat/web/ microdata/structure-of-earnings-survey, 2014.","title":"Imagenet: A Large-Scale Hierarchical Image Database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Eurostat. STRUCTURE OF EARN-INGS SURVEY","doi":null,"url":"https://ec.europa.eu/eurostat/web/microdata/structure-of-earnings-survey,2014","internal_url":null},{"position":10,"raw":"[Felten et al., 2018] Edward W Felten, Manav Raj, and Robert Seamans. A Method to Link Advances in Arti- ficial Intelligence to Occupational Abilities. In AEA Vol 108, pages 54-57, 2018.","title":"A Method to Link Advances in Artificial Intelligence to Occupational Abilities","authors":"Manav Edward W Felten, Robert Raj, Seamans","year":"2018","venue":"AEA","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fernández-Macías and Bisello, 2022] Enrique Fernández- Macías and Martina Bisello. A comprehensive taxonomy of tasks for assessing the impact of new technologies on work. Social Indicators Research, 159(2):821-841, 2022.","title":"A comprehensive taxonomy of tasks for assessing the impact of new technologies on work","authors":"Enrique Fernández, -Macías, Martina Bisello","year":"2022","venue":"Social Indicators Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fernández-Macías et al., 2018] Enrique Fernández-Macías, Emilia Gómez, José Hernández-Orallo, Bao Sheng Loe, Bertin Martens, Fernando Martínez-Plumed, and Songül Tolan. A multidisciplinary task-based perspective for eval- uating the impact of AI autonomy and generality on the future of work. CoRR, abs/1807.02416, 2018.","title":"A multidisciplinary task-based perspective for evaluating the impact of AI autonomy and generality on the future of work","authors":"Enrique Fernández-Macías, Emilia Gómez, José Hernández-Orallo, Bao Sheng Loe, Bertin Martens, Fernando Martínez-Plumed, Songül Tolan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ferrucci, 2012] David A Ferrucci. Introduction to \"This is Watson\". IBM Journal of Research and Development, 56(3.4):1-1, 2012.","title":"Introduction to \"This is Watson","authors":"A David, Ferrucci","year":"2012","venue":"IBM Journal of Research and Development","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hernández-Orallo and Vold, 2019] José Hernández-Orallo and Karina Vold. AI Extenders: The Ethical and Soci- etal Implications of Humans Cognitively Extended by AI. In Proc. AAAI/ACM AIES, pages 507-513, 2019.","title":"José Hernández-Orallo and Karina Vold. AI Extenders: The Ethical and Societal Implications of Humans Cognitively Extended by AI","authors":"Hernández-Orallo Vold","year":"2019","venue":"Proc. AAAI/ACM AIES","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hernández-Orallo et al., 2017] José Hernández-Orallo, Marco Baroni, Jordi Bieger, Nader Chmait, David L Dowe, Katja Hofmann, Fernando Martínez-Plumed, Claes Strannegård, and Kristinn R Thórisson. A New AI Evaluation Cosmos: Ready to Play the Game? AI Magazine, 38(3), 2017.","title":"A New AI Evaluation Cosmos: Ready to Play the Game?","authors":"Hernández-Orallo","year":"2017","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hernández-Orallo, 2017] José Hernández-Orallo. Evalu- ation in Artificial Intelligence: From Task-Oriented to Ability-Oriented Measurement. AI Review, 48(3):397- 447, 2017.","title":"José Hernández-Orallo. Evaluation in Artificial Intelligence: From Task-Oriented to Ability-Oriented Measurement","authors":"Hernández-Orallo","year":"2017","venue":"AI Review","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Martinez-Plumed and Hernandez-Orallo, 2018] Fernando Martinez-Plumed and Jose Hernandez-Orallo. Dual indicators to analyze AI benchmarks: Difficulty, discrim- ination, ability, and generality. IEEE Transactions on Games, 12(2):121-131, 2018.","title":"Dual indicators to analyze AI benchmarks: Difficulty, discrimination, ability, and generality","authors":"Fernando Martinez, -Plumed, Jose Hernandez-Orallo","year":"2018","venue":"IEEE Transactions on Games","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Martínez-Plumed et al., 2018] Fernando Martínez-Plumed, Shahar Avin, Miles Brundage, Allan Dafoe, Sean Ó h Éigeartaigh, and José Hernández-Orallo. Account- ing for the Neglected Dimensions of AI Progress. arXiv:1806.00610, 2018.","title":"Accounting for the Neglected Dimensions of AI Progress","authors":"Fernando Martínez-Plumed, Shahar Avin, Miles Brundage, Allan Dafoe, Sean Ó Éigeartaigh, José Hernández-Orallo","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Martínez-Plumed et al., 2020] Fernando Martínez-Plumed, Jose Hernández-Orallo, and Emilia Gómez. Tracking ai: The capability is (not) near. In ECAI 2020, pages 2915- 2916. IOS Press, 2020.","title":"Tracking ai: The capability is (not) near","authors":"Fernando Martínez-Plumed, Jose Hernández-Orallo, Emilia Gómez","year":"2020","venue":"ECAI 2020","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nedelkoska and Quintini, 2018] Ljubica Nedelkoska and Glenda Quintini. Automation, Skills Use and Training. OECD Social, Employment and Migration WP, (202), 2018.","title":"Ljubica Nedelkoska and Glenda Quintini. Automation, Skills Use and Training. OECD Social, Employment and Migration WP","authors":"Quintini Nedelkoska","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shoham et al., 2018] Yoav Shoham, Raymond Perrault, Erik Brynjolfsson, Jack Clark, James Manyika, Juan Car- los Niebles, Terah Lyons, John Etchemendy, and Z Bauer. The AI Index 2018 Annual Report. AI Index Steering Committee, Human-Centered AI Initiative, Stanford Uni- versity., 202018, 2018.","title":"The AI Index 2018 Annual Report. AI Index Steering Committee","authors":"Yoav Shoham, Raymond Perrault, Erik Brynjolfsson, Jack Clark, James Manyika, Juan Carlos Niebles, Terah Lyons, John Etchemendy, Bauer","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tolan et al., 2021] Songül Tolan, Annarosa Pesole, Fer- nando Martínez-Plumed, Enrique Fernández-Macías, José Hernández-Orallo, and Emilia Gómez. Measuring the oc- cupational impact of ai: tasks, cognitive abilities and ai benchmarks. JAIR, 71:191-236, 2021.","title":"Measuring the occupational impact of ai: tasks, cognitive abilities and ai benchmarks","authors":"Tolan","year":"2021","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Webb, 2020] Michael Webb. The Impact of Artificial Intel- ligence on the Labor Market. Department of Economics, Stanford University, Stanford, California, 2020.","title":"The Impact of Artificial Intelligence on the Labor Market","authors":"Michael Webb","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}