{"type":"ScholarlyArticle","title":"Learning Curricula for Humans: An Empirical Study with Puzzles from The Witness","authors":[{"name":"Levi H.S. Lelis","slug":"levi-h-s-lelis","given":"Levi H. S.","family":"Lelis","affiliation":"Alberta Machine Intelligence Institute (Amii ); Department of Computing Science , University of Alberta , Canada; Departamento de Informática , Universidade Federal de Vic ¸osa , Brazil"},{"name":"João G.G.V. Nova","slug":"joao-g-g-v-nova","given":"João G.G.V.","family":"Nova","affiliation":"Departamento de Informática , Universidade Federal de Vic ¸osa , Brazil"},{"name":"Eugene Chen","slug":"eugene-chen","given":"Eugene","family":"Chen","affiliation":"Alberta Machine Intelligence Institute (Amii ); Department of Computing Science , University of Alberta , Canada"},{"name":"Nathan R. Sturtevant","slug":"nathan-r-sturtevant","given":"Nathan R.","family":"Sturtevant","affiliation":"Alberta Machine Intelligence Institute (Amii ); Department of Computing Science , University of Alberta , Canada"},{"name":"Carrie Demmans Epp","slug":"carrie-demmans-epp","given":"Carrie","family":"Demmans Epp","affiliation":"Department of Computing Science , University of Alberta , Canada"},{"name":"Michael Bowling","slug":"michael-bowling","given":"Michael","family":"Bowling","affiliation":"Alberta Machine Intelligence Institute (Amii ); Department of Computing Science , University of Alberta , Canada"}],"abstract":"The combination of tree search and neural networks has achieved super-human performance in challenging domains. We are interested in transferring to humans the knowledge these learning systems generate. We hypothesize the process in which neural-guided tree search algorithms learn how to solve a set of problems can be used to generate curricula for helping human learners. In this paper we show how the Bootstrap learning system can be modified to learn curricula for humans in a puzzle domain. We evaluate our system in two curriculum learning settings. First, given a small set of problem instances, our system orders the instances to ease the learning process of human learners. Second, given a large set of problem instances, our system returns a small ordered subset of the initial set that can be presented to human learners. We evaluate our curricula with a user study where participants learn how to solve a class of puzzles from the game `The Witness.' The user-study results suggest one of the curricula our system generates compares favorably with simple baselines and is competitive with the curriculum from the original `The Witness' game in terms of user retention and effort.","doi":"10.24963/ijcai.2022/538","url":"https://ijcai.org/proceedings/2022/538","pdf":"https://ijcai.org/proceedings/2022/0538.pdf","markdown":"https://ijcai.org/proceedings/2022/538.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/538","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":538,"pages":{"first":3877,"last":3883},"keywords":["Humans and AI: Applications","Search: Heuristic Search","Multidisciplinary Topics and Applications: Computer Games"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987063","length_sec":103,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985206","length_sec":717,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bengio et al., 2009] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Confer- ence on Machine Learning, page 41-48. Association for Computing Machinery, 2009.","title":"Curriculum learning","authors":"Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston","year":"2009","venue":"Proceedings of the 26th Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cakmak and Lopes, 2012] Maya Cakmak and Manuel Lopes. Algorithmic and human teaching of sequential decision tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, page 1536-1542. AAAI Press, 2012.","title":"Maya Cakmak and Manuel Lopes. Algorithmic and human teaching of sequential decision tasks","authors":"Lopes Cakmak","year":"2012","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chi et al., 2011] Min Chi, Kurt VanLehn, Diane Litman, and Pamela Jordan. Empirically evaluating the applica- tion of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Modeling and User-Adapted Interaction, 21(1):137-180, 2011.","title":"Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies","authors":"Chi","year":"2011","venue":"User Modeling and User-Adapted Interaction","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goldman and Kearns, 1995] S.A. Goldman and M.J. Kearns. On the complexity of teaching. Journal of Computer and System Sciences, 50(1):20-31, 1995.","title":"On the complexity of teaching","authors":"S A Goldman, M J Kearns","year":"1995","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Graesser and D'Mello, 2012] Arthur C. Graesser and Sid- ney D'Mello. Chapter Five -Emotions During the Learn- ing of Difficult Material. In Brian H. Ross, editor, Psychol- ogy of Learning and Motivation, volume Volume 57 of The Psychology of Learning and Motivation, pages 183-225. Academic Press, 2012.","title":"Chapter Five -Emotions During the Learning of Difficult Material","authors":"D' Graesser, C Mello ; Arthur, Sidney D' Graesser, Mello","year":"2012","venue":"The Psychology of Learning and Motivation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hart et al., 1968] Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Sci- ence and Cybernetics, 4(2):100-107, 1968.","title":"A formal basis for the heuristic determination of minimum cost paths","authors":"Nils J Peter E Hart, Bertram Nilsson, Raphael","year":"1968","venue":"IEEE transactions on Systems Science and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jabbari Arfaee et al., 2011] Shahab Jabbari Arfaee, San- dra Zilles, and Robert C. Holte. Learning heuristic functions for large state spaces. Artificial Intelligence, 175(16):2075-2098, 2011.","title":"Learning heuristic functions for large state spaces","authors":"Shahab Jabbari Arfaee, Sandra Zilles, Robert C Holte","year":"2011","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Khan et al., 2011] Faisal Khan, Bilge Mutlu, and Jerry Zhu. How do humans teach: On curriculum learning and teach- ing dimension. In Advances in Neural Information Pro- cessing Systems, volume 24, pages 1-9, 2011.","title":"How do humans teach: On curriculum learning and teaching dimension","authors":"Faisal Khan, Bilge Mutlu, Jerry Zhu","year":"2011","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lindsey et al., 2013] Robert V Lindsey, Michael C Mozer, William J Huggins, and Harold Pashler. Optimizing instructional policies. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, edi- tors, Advances in Neural Information Processing Systems, volume 26, 2013.","title":"Optimizing instructional policies","authors":"Michael C Robert V Lindsey, William J Mozer, Harold Huggins, ; C J C Pashler, L Burges, M Bottou, Z Welling, K Q Ghahramani, Weinberger","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Orseau and Lelis, 2021] Laurent Orseau and Levi H. S. Lelis. Policy-guided heuristic search with guarantees. In AAAI Conference on Artificial Intelligence, pages 12382- 12390. AAAI Press, 2021.","title":"Policy-guided heuristic search with guarantees","authors":"Laurent Orseau, H S Levi, Lelis","year":"2021","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Orseau et al., 2018] Laurent Orseau, Levi Lelis, Tor Latti- more, and Theophane Weber. Single-agent policy tree search with guarantees. In Advances in Neural Informa- tion Processing Systems 31, pages 3201-3211.","title":"Single-agent policy tree search with guarantees","authors":"Orseau","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Patil et al., 2014] Kaustubh R Patil, Jerry Zhu, Ł ukasz Kopeć, and Bradley C Love. Optimal teaching for limited- capacity human learners. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, vol- ume 27, 2014.","title":"Optimal teaching for limitedcapacity human learners","authors":"Jerry Kaustubh R Patil, Ł Zhu, Bradley C Ukasz Kopeć, Love","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Rafferty et al., 2011] Anna N. Rafferty, Emma Brunskill, Thomas L. Griffiths, and Patrick Shafto. Faster teaching by pomdp planning. In Gautam Biswas, Susan Bull, Judy Kay, and Antonija Mitrovic, editors, Artificial Intelligence in Education, pages 280-287. Springer Berlin Heidelberg, 2011.","title":"Faster teaching by pomdp planning","authors":"Anna N Rafferty, Emma Brunskill, Thomas L Griffiths, Patrick Shafto","year":"2011","venue":"Artificial Intelligence in Education","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Reinecke and Gajos, 2015] Katharina Reinecke and Krzysztof Z. Gajos. Labinthewild: Conducting large- scale online experiments with uncompensated samples. In Proceedings of the ACM Conference on Computer Supported Cooperative Work & Social Computing, pages 1364-1378. ACM, 2015.","title":"Labinthewild: Conducting largescale online experiments with uncompensated samples","authors":"Katharina Reinecke, Krzysztof Z Gajos","year":"2015","venue":"Proceedings of the ACM Conference on Computer Supported Cooperative Work & Social Computing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self- play. Science, 362(6419):1140-1144, 2018.","title":"A general reinforcement learning algorithm that masters chess, shogi, and go through selfplay","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Taylor, 2009] Matthew E. Taylor. Assisting transfer-enabled machine learning algorithms: Leveraging human knowl- edge for curriculum design. In The AAAI Spring Sympo- sium, pages 141-143. AAAI, 2009.","title":"Assisting transfer-enabled machine learning algorithms: Leveraging human knowledge for curriculum design","authors":"Matthew E Taylor","year":"2009","venue":"The AAAI Spring Symposium","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wald, 1980] Abraham Wald. A Reprint of 'A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Technical report, Center for Naval Analyses Alexandria VA Operations Evaluation Group, July 1980. Section: Technical Reports.","title":"A Reprint of 'A Method of Estimating Plane Vulnerability Based on Damage of Survivors","authors":"Abraham Wald","year":"1980","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhu et al., 2017] Xiaojin Zhu, Ji Liu, and Manuel Lopes. No learner left behind: On the complexity of teaching mul- tiple learners simultaneously. In Proceedings of the Inter- national Joint Conference on Artificial Intelligence, pages 3588-3594, 2017.","title":"No learner left behind: On the complexity of teaching multiple learners simultaneously","authors":"Xiaojin Zhu, Ji Liu, Manuel Lopes","year":"2017","venue":"Proceedings of the International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/502"},{"position":19,"raw":"[Zhu et al., 2018] Xiaojin Zhu, Adish Singla, Sandra Zilles, and Anna N. Rafferty. An overview of machine teach- ing. Technical Report arXiv:1801.05927 [cs.LG], ArXiV, 2018.","title":"An overview of machine teaching","authors":"Xiaojin Zhu, Adish Singla, Sandra Zilles, Anna N Rafferty","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhu, 2015] Xiaojin Zhu. Machine teaching: An inverse problem to machine learning and an approach toward op- timal education. In Proceedings of the AAAI Conference on Artificial Intelligence, 2015.","title":"Machine teaching: An inverse problem to machine learning and an approach toward optimal education","authors":"Xiaojin Zhu","year":"2015","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zilles et al., 2011] Sandra Zilles, Steffen Lange, Robert Holte, and Martin Zinkevich. Models of cooperative teach- ing and learning. Journal of Machine Learning Research, 12(11):349-384, 2011.","title":"Models of cooperative teaching and learning","authors":"Sandra Zilles, Steffen Lange, Robert Holte, Martin Zinkevich","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}