{"type":"ScholarlyArticle","title":"Expected Work Search: Combining Win Rate and Proof Size Estimation","authors":[{"name":"Owen Randall","slug":"owen-randall","given":"Owen","family":"Randall","affiliation":"University of Alberta"},{"name":"Martin Müller","slug":"martin-muller","given":"Martin","family":"Müller","affiliation":"University of Alberta"},{"name":"Ting-Han Wei","slug":"ting-han-wei","given":"Ting-Han","family":"Wei","affiliation":"University of Alberta"},{"name":"Ryan Hayward","slug":"ryan-hayward","given":"Ryan","family":"Hayward","affiliation":"University of Alberta"}],"abstract":"We propose Expected Work Search (EWS), a new game solving algorithm. EWS combines win rate estimation, as used in Monte Carlo Tree Search, with proof size estimation, as used in Proof Number Search. The search efficiency of EWS stems from minimizing a novel notion of Expected Work, which predicts the expected computation required to solve a position. EWS outperforms traditional solving algorithms on the games of Go and Hex. For Go, we present the first solution to the empty 5x5 board with the commonly used positional superko ruleset. For Hex, our algorithm solves the empty 8x8 board in under 4 minutes. Experiments show that EWS succeeds both with and without extensive domain-specific knowledge.","doi":"10.24963/ijcai.2024/774","url":"https://ijcai.org/proceedings/2024/774","pdf":"https://ijcai.org/proceedings/2024/0774.pdf","markdown":"https://ijcai.org/proceedings/2024/774.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/774","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Search","track":"Main Track","number":774,"pages":{"first":7003,"last":7011},"keywords":["Search: S: Heuristic search","Search: S: Applications","Search: S: Game playing"],"videos":[],"references":[{"position":1,"raw":"[Allis et al., 1994] L.Victor Allis, Maarten van der Meulen, and H.Jaap van den Herik. Proof-Number Search. Artifi- cial Intelligence, 66(1):91-124, 1994.","title":"Proof-Number Search","authors":"L Victor Allis, Maarten Van Der Meulen, H Jaap Van Den Herik","year":"1994","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arneson et al., 2010] Broderick Arneson, Ryan Hayward, and Philip Henderson. Solving Hex: Beyond Humans. In Computers and Games, pages 1-10, 2010.","title":"Solving Hex: Beyond Humans","authors":"Arneson","year":"2010","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Browne et al., 2012] Cameron B. Browne, Edward Pow- ley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowl- ing, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games, 4(1):1-43, 2012.","title":"A Survey of Monte Carlo Tree Search Methods","authors":"Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, Simon Colton","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Campbell, 1985] Murray Campbell. The Graph-History In- teraction: On Ignoring Position History. In ACM Annual Conference on the Range of Computing, pages 278-280, 1985.","title":"The Graph-History Interaction: On Ignoring Position History","authors":"Murray Campbell, Campbell","year":"1985","venue":"ACM Annual Conference on the Range of Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Doe et al., 2022] Elliot Doe, Mark H. M. Winands, Jakub Kowalski, Dennis J. N. J. Soemers, Daniel Górski, and Cameron Browne. Proof Number Based Monte-Carlo Tree Search. In IEEE Conference on Games, pages 206-212, 2022.","title":"Proof Number Based Monte-Carlo Tree Search","authors":"Elliot Doe, H M Mark, Jakub Winands, Kowalski, J N J Dennis, Daniel Soemers, Cameron Górski, Browne","year":"2022","venue":"IEEE Conference on Games","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Enzenberger et al., 2011] Markus Enzenberger, Mar- cus Müller, Broderick Arneson, and Richard Segal. Fuego-An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search. IEEE Transactions on Computational Intelligence and AI in Games, 2:259-270, 2011.","title":"Fuego-An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search","authors":"Enzenberger","year":"2011","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ewalds, 2012] Timo V Ewalds. Playing and Solving Ha- vannah. University of Alberta Master's Thesis, 2012. https://github.com/tewalds/morat.","title":"Playing and Solving Havannah","authors":"V Timo, Ewalds","year":"2012","venue":null,"doi":null,"url":"https://github.com/tewalds/morat","internal_url":null},{"position":8,"raw":"[Fawzi et al., 2022] Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mo- hammadamin Barekatain, Alexander Novikov, Francisco J. R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, David Silver, Demis Hassabis, and Pushmeet Kohli. Discovering Faster Matrix Multiplication Algorithms with Reinforce- ment Learning. Nature, pages 47-53, 2022.","title":"Discovering Faster Matrix Multiplication Algorithms with Reinforcement Learning","authors":"Fawzi","year":"2022","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hayward et al., 2004] Ryan Hayward, Yngvi Björnsson, Michael Johanson, Morgan Kan, Nathan Po, and Jack van Rijswijck. Solving 7×7 Hex: Virtual Connections and Game-State Reduction. Advances in Computer Games: Many Games, Many Challenges, pages 261-278, 2004.","title":"Solving 7×7 Hex: Virtual Connections and Game-State Reduction","authors":"Ryan Hayward, Yngvi Björnsson, Michael Johanson, Morgan Kan, Nathan Po, Jack Van Rijswijck","year":"2004","venue":"Advances in Computer Games: Many Games, Many Challenges","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Henderson et al., 2009] Philip Henderson, Broderick Arne- son, and Ryan Hayward. Solving 8x8 Hex. IJCAI Inter- national Joint Conference on Artificial Intelligence, pages 505-510, 2009.","title":"Solving 8x8 Hex","authors":"Philip Henderson, Broderick Arneson, Ryan Hayward","year":"2009","venue":"IJCAI International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kishimoto and Müller, 2004] Akihiro Kishimoto and Mar- tin Müller. A General Solution to the Graph History Inter- action Problem. In Proceedings of the 19th National Con- ference on Artifical Intelligence, pages 644-649, 2004.","title":"Akihiro Kishimoto and Martin Müller. A General Solution to the Graph History Interaction Problem","authors":"Müller Kishimoto","year":"2004","venue":"Proceedings of the 19th National Conference on Artifical Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kishimoto et al., 2012] Akihiro Kishimoto, Mark Winands, Martin Müller, and Jahn Takeshi Saito. Game-Tree Search Using Proof Numbers: The First Twenty Years. ICGA Journal, 35:131-156, 2012.","title":"Game-Tree Search Using Proof Numbers: The First Twenty Years","authors":"Akihiro Kishimoto, Mark Winands, Martin Müller, Jahn Takeshi Saito","year":"2012","venue":"ICGA Journal","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Knuth and Moore, 1975] Donald E. Knuth and Ronald W. Moore. An Analysis of Alpha-Beta Pruning. Artificial Intelligence, 6(4):293-326, 1975.","title":"An Analysis of Alpha-Beta Pruning","authors":"Donald E Knuth, Ronald W Moore","year":"1975","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kocsis and Szepesvári, 2006] Levente Kocsis and Csaba Szepesvári. Bandit Based Monte-Carlo Planning. In Ma- chine Learning: ECML, pages 282-293, 2006.","title":"Bandit Based Monte-Carlo Planning","authors":"Szepesvári Kocsis, Levente Kocsis, Csaba Szepesvári","year":"2006","venue":"Machine Learning: ECML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nagai, 2002] Ayumu Nagai. DF-PN Algorithm for Search- ing AND/OR Trees and its Applications. University of Tokyo Ph.D. Thesis, 2002.","title":"DF-PN Algorithm for Searching AND/OR Trees and its Applications","authors":"Ayumu Nagai","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pawlewicz and Hayward, 2013] Jakub Pawlewicz and Ryan B. Hayward. Scalable Parallel DFPN Search. In Computers and Games, pages 138-150, 2013.","title":"Scalable Parallel DFPN Search","authors":"Jakub Pawlewicz, Ryan B Hayward","year":"2013","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pawlewicz and Lew, 2006] Jakub Pawlewicz and Lukasz Lew. Improving Depth-First PN-Search: 1 + Epsilon Trick. In Proceedings of the 5th International Conference on Computers and Games, pages 160-171, 2006.","title":"Improving Depth-First PN-Search: 1 + Epsilon Trick","authors":"Lew ; Pawlewicz, Jakub Pawlewicz, Lukasz Lew","year":"2006","venue":"Proceedings of the 5th International Conference on Computers and Games","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Randall et al., 2022] Owen Randall, Ting-Han Wei, Ryan Hayward, and Martin Müller. Improving Search in Go Using Bounded Static Safety. In Computers and Games, pages 14-23, 2022.","title":"Improving Search in Go Using Bounded Static Safety","authors":"Owen Randall, Ting-Han Wei, Ryan Hayward, Martin Müller","year":"2022","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Randall et al., 2024] Owen Randall, Martin Müller, Ting Han Wei, and Ryan Hayward. Expected work search: Combining win rate and proof size estimation, 2024. https://arxiv.org/abs/2405.05594.","title":"Expected work search: Combining win rate and proof size estimation","authors":"Owen Randall, Martin Müller, Ting Han Wei, Ryan Hayward","year":"2024","venue":null,"doi":null,"url":"https://arxiv.org/abs/2405.05594","internal_url":null},{"position":20,"raw":"[Rosin, 2010] Christopher Rosin. Multi-armed Bandits with Episode Context. Annals of Mathematics and Artificial Intelligence, 61:203-230, 2010.","title":"Multi-armed Bandits with Episode Context","authors":"Christopher Rosin","year":"2010","venue":"Annals of Mathematics and Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Saffidine and Cazenave, 2012] Abdallah Saffidine and Tris- tan Cazenave. Multiple-outcome proof number search. In ECAI, pages 708-713, 2012.","title":"Abdallah Saffidine and Tristan Cazenave. Multiple-outcome proof number search","authors":"Cazenave Saffidine","year":"2012","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Saffidine and Cazenave, 2013] Abdallah Saffidine and Tris- tan Cazenave. Developments on Product Propagation. In Computers and Games, pages 100-109, 2013.","title":"Abdallah Saffidine and Tristan Cazenave","authors":"Cazenave Saffidine","year":"2013","venue":"Developments on Product Propagation","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Schaeffer and Plaat, 1996] Jonathan Schaeffer and Aske Plaat. New Advances in Alpha-Beta Searching. In Pro- ceedings of the 1996 ACM 24th annual Conference on Computer Science, pages 124-130, 1996.","title":"New Advances in Alpha-Beta Searching","authors":"Plaat Schaeffer, Jonathan Schaeffer, Aske Plaat","year":"1996","venue":"Proceedings of the 1996 ACM 24th annual Conference on Computer Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shih et al., 2021] Chung-Chin Shih, Ti-Rong Wu, Ting Han Wei, and I-Chen Wu. A Novel Approach to Solving Goal- Achieving Problems for Board Games. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, pages 10362-10369, 2021.","title":"A Novel Approach to Solving Goal-Achieving Problems for Board Games","authors":"Chung-Chin Shih, Ti-Rong Wu, Ting Han Wei, I-Chen Wu","year":"2021","venue":"Proceedings of the 36th AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Christopher Maddison, Arthur Guez, Laurent Sifre, George Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the Game of Go with Deep Neural Networks and Tree Search. Na- ture, 529:484-489, 2016.","title":"Mastering the Game of Go with Deep Neural Networks and Tree Search","authors":"David Silver, Aja Huang, Christopher Maddison, Arthur Guez, Laurent Sifre, George Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A General Reinforcement Learning Al- gorithm that Masters Chess, Shogi, and Go through Self- Play. Science, 362(6419):1140-1144, 2018.","title":"A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Slate and Atkin, 1977] David Slate and Larry Atkin. Chess Skill in Man and Machine. In Springer-Verlag, pages 82- 118, 1977.","title":"Chess Skill in Man and Machine","authors":"Atkin Slate, David Slate, Larry Atkin","year":"1977","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sutton and Barto, 1998] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, pages 39-40, 1998.","title":"Reinforcement Learning: An Introduction","authors":"Richard S Sutton, Andrew G Barto","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[van der Werf and Winands, 2009] Erik C. D. van der Werf and Mark H. M. Winands. Solving Go for Rectangular Boards. ICGA Journal, 32(2):77-88, 2009.","title":"Solving Go for Rectangular Boards","authors":"C D Erik, Van Der Werf, H M Mark, Winands","year":"2009","venue":"ICGA Journal","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[van der Werf et al., 2003] Erik van der Werf, H. Herik, and Jos Uiterwijk. Solving Go on Small Boards. ICGA Jour- nal, 26:92-107, 2003.","title":"Solving Go on Small Boards","authors":"Werf Van Der","year":"2003","venue":"ICGA Journal","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Winands et al., 2008] Mark H. M. Winands, Yngvi Björnsson, and Jahn-Takeshi Saito. Monte-Carlo Tree Search Solver. In Computers and Games, pages 25-36, 2008.","title":"Monte-Carlo Tree Search Solver","authors":"H M Mark, Yngvi Winands, Jahn-Takeshi Björnsson, Saito","year":"2008","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wu et al., 2022] Ti-Rong Wu, Chung-Chin Shih, Ting-Han Wei, Meng-Yu Tsai, Wei-Yuan Hsu, and I-Chen Wu. AlphaZero-based Proof Cost Network to Aid Game Solv- ing. In International Conference on Learning Representa- tions, 2022.","title":"AlphaZero-based Proof Cost Network to Aid Game Solving","authors":"Ti-Rong Wu, Chung-Chin Shih, Ting-Han Wei, Meng-Yu Tsai, Wei-Yuan Hsu, I-Chen Wu","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yao et al., 2023] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solv- ing with Large Language Models, 2023.","title":"Tree of Thoughts: Deliberate Problem Solving with Large Language Models","authors":"Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, Karthik Narasimhan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zobrist, 1990] Albert Zobrist. A New Hashing Method with Application for Game Playing. ICGA Journal, 13:69-73, 1990.","title":"A New Hashing Method with Application for Game Playing","authors":"Albert Zobrist","year":"1990","venue":"ICGA Journal","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}