{"type":"ScholarlyArticle","title":"Not a Number: Identifying Instance Features for Capability-Oriented Evaluation","authors":[{"name":"Ryan Burnell","slug":"ryan-burnell","given":"Ryan","family":"Burnell","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK"},{"name":"John Burden","slug":"john-burden","given":"John","family":"Burden","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK; Centre for the Study of Existential Risk , University of Cambridge , UK"},{"name":"Danaja Rutar","slug":"danaja-rutar","given":"Danaja","family":"Rutar","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK"},{"name":"Konstantinos Voudouris","slug":"konstantinos-voudouris","given":"Konstantinos","family":"Voudouris","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK"},{"name":"Lucy Cheke","slug":"lucy-cheke","given":"Lucy","family":"Cheke","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK"},{"name":"José Hernández-Orallo","slug":"jose-hernandez-orallo","given":"José","family":"Hernández-Orallo","affiliation":"Leverhulme Centre for the Future of Intelligence , University of Cambridge , UK; Centre for the Study of Existential Risk , University of Cambridge , UK; VRAIN , Universitat Politècnica de València , Spain"}],"abstract":"In AI evaluation, performance is often calculated by averaging across various instances. But to fully understand the capabilities of an AI system, we need to understand the factors that cause its pattern of success and failure. In this paper, we present a new methodology to identify and build informative instance features that can provide explanatory and predictive power to analyse the behaviour of AI systems more robustly. The methodology builds on these relevant features that should relate monotonically with success, and represents patterns of performance in a new type of plots known as ‘agent characteristic grids’. We illustrate this methodology with the Animal-AI competition as a representative example of how we can revisit existing competitions and benchmarks in AI—even when evaluation data is sparse. Agents with the same average performance can show very different patterns of performance at the instance level. With this methodology, these patterns can be visualised, explained and predicted, progressing towards a capability-oriented evaluation rather than relying on a less informative average performance score.","doi":"10.24963/ijcai.2022/392","url":"https://ijcai.org/proceedings/2022/392","pdf":"https://ijcai.org/proceedings/2022/0392.pdf","markdown":"https://ijcai.org/proceedings/2022/392.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/392","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":392,"pages":{"first":2827,"last":2835},"keywords":["AI Ethics, Trust, Fairness: Safety & Robustness","AI Ethics, Trust, Fairness: Explainability and Interpretability","Machine Learning: Evaluation","Machine Learning: Experimental Methodology"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986925","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985068","length_sec":345,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Crosby et al., 2020] Matthew Crosby, Benjamin Beyret, Murray Shanahan, Jose Hernandez-Orallo, Lucy Cheke, and Marta Halina. The animal-AI testbed and competition. Proceedings of Machine Learning Research, (123):164- 176, 2020.","title":"The animal-AI testbed and competition","authors":"Matthew Crosby, Benjamin Beyret, Murray Shanahan, Jose Hernandez-Orallo, Lucy Cheke, Marta Halina","year":"2020","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fawcett, 2006] Tom Fawcett. An introduction to roc analy- sis. Pattern recognition letters, 27(8):861-874, 2006.","title":"Tom Fawcett. An introduction to roc analysis","authors":"Fawcett","year":"2006","venue":"Pattern recognition letters","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ilievski et al., 2021] Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L McGuinness, and Pe- dro Szekely. Dimensions of commonsense knowledge. arXiv preprint arXiv:2101.04640, 2021.","title":"Dimensions of commonsense knowledge","authors":"Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L Mcguinness, Pedro Szekely","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Juliani et al., 2018] Arthur Juliani, Vincent-Pierre Berges, Esh Vckay, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. Unity: A general platform for intelligent agents. CoRR, abs/1809.02627, 2018.","title":"Unity: A general platform for intelligent agents","authors":"Arthur Juliani, Vincent-Pierre Berges, Esh Vckay, Yuan Gao, Hunter Henry, Marwan Mattar, Danny Lange","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Jylhä et al., 2017] Juha Jylhä, Marja Ruotsalainen, Ville Väisänen, Kai Virtanen, Mikko Harju, and Minna Väilä. A machine learning framework for performance prediction of an air surveillance system. In 2017 European Radar Conference (EURAD), pages 187-190, 2017.","title":"A machine learning framework for performance prediction of an air surveillance system","authors":"Juha Jylhä, Marja Ruotsalainen, Ville Väisänen, Kai Virtanen, Mikko Harju, Minna Väilä","year":"2017","venue":"2017 European Radar Conference (EURAD)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kiela et al., 2021] Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ring- shia, et al. Dynabench: Rethinking benchmarking in nlp. arXiv preprint arXiv:2104.14337, 2021.","title":"Dynabench: Rethinking benchmarking in nlp","authors":"Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Koh et al., 2020] Pang Wei Koh, Shiori Sagawa, Hen- rik Marklund, Sang Michael Xie, Marvin Zhang, Ak- shay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. CoRR, abs/2012.07421, 2020.","title":"WILDS: A benchmark of in-the-wild distribution shifts","authors":"Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, Percy Liang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lind and Vinken, 2021] Johan Lind and Vera Vinken. Can associative learning be the general process for intelligent behavior in non-human animals? bioRxiv, 2021.","title":"Can associative learning be the general process for intelligent behavior in non-human animals? bioRxiv","authors":"Lind, Vinken, Johan Lind, Vera Vinken","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Marijan and Gotlieb, 2020] Dusica Marijan and Arnaud Gotlieb. Software testing for machine learning. In AAAI, volume 34, pages 13576-13582, 2020.","title":"Software testing for machine learning","authors":"Gotlieb Marijan, Dusica Marijan, Arnaud Gotlieb","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Martin-Diaz et al., 2018] Ignacio Martin-Diaz, Daniel Morinigo-Sotelo, Oscar Duque-Perez, and Rene J Romero-Troncoso. An experimental comparative eval- uation of machine learning techniques for motor fault diagnosis under various operating conditions. IEEE Transactions on Industry Applications, 54(3):2215-2224, 2018.","title":"An experimental comparative evaluation of machine learning techniques for motor fault diagnosis under various operating conditions","authors":"Ignacio Martin-Diaz, Daniel Morinigo-Sotelo, Oscar Duque-Perez, Rene J Romero-Troncoso","year":"2018","venue":"IEEE Transactions on Industry Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Martínez-Plumed et al., 2019] Fernando Martínez-Plumed, Ricardo BC Prudêncio, Adolfo Martínez-Usó, and José Hernández-Orallo. Item response theory in AI: Analysing machine learning classifiers at the instance level. Artificial Intelligence, 271:18-42, 2019.","title":"Item response theory in AI: Analysing machine learning classifiers at the instance level","authors":"Fernando Martínez-Plumed, Ricardo Bc Prudêncio, Adolfo Martínez-Usó, José Hernández-Orallo","year":"2019","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Neyshabur et al., 2017] Behnam Neyshabur, Srinadh Bho- janapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In NeurIPS, pages 5947- 5956, 2017.","title":"Exploring generalization in deep learning","authors":"Srinadh Behnam Neyshabur, David Bhojanapalli, Nati Mcallester, Srebro","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nie et al., 2019] Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversar- ial nli: A new benchmark for natural language understand- ing. arXiv preprint arXiv:1910.14599, 2019.","title":"Adversarial nli: A new benchmark for natural language understanding","authors":"Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, Douwe Kiela","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Osband and others, 2019] Ian Osband et al. Behaviour suite for reinforcement learning. arXiv preprint arXiv:1908.03568, 2019.","title":"Behaviour suite for reinforcement learning","authors":"Ian Osband","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ribeiro et al., 2020] Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Be- havioral testing of nlp models with checklist. ACL2020, arXiv preprint arXiv:2005.04118, 2020.","title":"Beyond accuracy: Behavioral testing of nlp models with checklist","authors":"Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh","year":"2020","venue":"ACL2020","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Risi and Togelius, 2020] Sebastian Risi and Julian Togelius. Increasing generality in machine learning through proce- dural content generation. Nature Machine Intelligence, page 428-436, 2020.","title":"Increasing generality in machine learning through procedural content generation","authors":"Togelius Risi, Sebastian Risi, Julian Togelius","year":"2020","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rozen et al., 2019] Ohad Rozen, Vered Shwartz, Roee Aha- roni, and Ido Dagan. Diversify your datasets: Analyz- ing generalization via controlled variance in adversarial datasets. arXiv preprint arXiv:1910.09302, 2019.","title":"Diversify your datasets: Analyzing generalization via controlled variance in adversarial datasets","authors":"Ohad Rozen, Vered Shwartz, Roee Aharoni, Ido Dagan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schlangen, 2019] David Schlangen. Language tasks and language games: On methodology in current nat- ural language processing research. arXiv preprint arXiv:1908.10747, 2019.","title":"Language tasks and language games: On methodology in current natural language processing research","authors":"David Schlangen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shoham, 2017] Yoav Shoham. Towards the AI index. AI Magazine, 38(4):71-77, 2017.","title":"Towards the AI index","authors":"Yoav Shoham, Shoham","year":"2017","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sugawara et al., 2020] Saku Sugawara, Pontus Stenetorp, and Akiko Aizawa. Benchmarking machine reading com- prehension: A psychological perspective. arXiv preprint arXiv:2004.01912, 2020.","title":"Benchmarking machine reading comprehension: A psychological perspective","authors":"Sugawara","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sutton, 1996] Richard S Sutton. Generalization in rein- forcement learning: Successful examples using sparse coarse coding. In NeurIPS, pages 1038-1044, 1996.","title":"Generalization in reinforcement learning: Successful examples using sparse coarse coding","authors":"Richard S Sutton","year":"1996","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Thurstone, 1937] LL Thurstone. Ability, motivation, and speed. Psychometrika, 2(4):249-254, 1937.","title":"LL Thurstone. Ability, motivation, and speed","authors":"Thurstone","year":"1937","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zellers et al., 2018] Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. Swag: A large-scale adver- sarial dataset for grounded commonsense inference. arXiv preprint arXiv:1808.05326, 2018.","title":"Swag: A large-scale adversarial dataset for grounded commonsense inference","authors":"Rowan Zellers, Yonatan Bisk, Roy Schwartz, Yejin Choi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zellers et al., 2019] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019.","title":"Hellaswag: Can a machine really finish your sentence","authors":"Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, Yejin Choi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}