{"type":"ScholarlyArticle","title":"MCTS-Minimax Hybrids with State Evaluations (Extended Abstract)","authors":[{"name":"Hendrik Baier","slug":"hendrik-baier","given":"Hendrik","family":"Baier","affiliation":"Centrum Wiskunde & Informatica , Amsterdam , The Netherlands"},{"name":"Mark H. M. Winands","slug":"mark-h-m-winands","given":"Mark H. M.","family":"Winands","affiliation":"Maastricht University , Maastricht , The Netherlands"}],"abstract":"Monte-Carlo Tree Search (MCTS) has been found to show weaker play than minimax-based search in some tactical game domains. In order to combine the tactical strength of minimax and the strategic strength of MCTS, MCTS-minimax hybrids have been proposed in prior work. This article continues this line of research for the case where heuristic state evaluation functions are available. Three different approaches are considered, employing minimax in the rollout phase of MCTS, as a replacement for the rollout phase, and as a node prior to bias move selection. The latter two approaches are newly proposed. Results show that the use of enhanced minimax for computing node priors results in the strongest MCTS-minimax hybrid in the three test domains of Othello, Breakthrough, and Catch the Lion. This hybrid also outperforms enhanced minimax as a standalone player in Breakthrough, demonstrating that at least in this domain, MCTS and minimax can be combined to an algorithm stronger than its parts.","doi":"10.24963/ijcai.2018/782","url":"https://ijcai.org/proceedings/2018/782","pdf":"https://ijcai.org/proceedings/2018/0782.pdf","markdown":"https://ijcai.org/proceedings/2018/782.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/782","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Journal track","track":"Journal track","number":782,"pages":{"first":5548,"last":5552},"keywords":["Heuristic Search and Game Playing: Heuristic Search","Heuristic Search and Game Playing: Game Playing"],"videos":[],"references":[{"position":1,"raw":"[Baier and Winands, 2015] Hendrik Baier and Mark H. M. Winands. MCTS-Minimax Hybrids. IEEE Transactions on Computational Intelligence and AI in Games, 7(2):167-179, 2015.","title":"MCTS-Minimax Hybrids","authors":"Hendrik Baier, H M Mark, Winands","year":"2015","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Baier and Winands, in press] Hendrik Baier and Mark H. M. Winands. MCTS-Minimax Hybrids with State Evaluations. Journal of Artificial Intelligence Research, in press.","title":"MCTS-Minimax Hybrids with State Evaluations","authors":"Winands Baier, Hendrik Baier, H M Mark, Winands","year":null,"venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Browne et al., 2012] Cameron Browne, Edward J. Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyri- don Samothrakis, and Simon Colton. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Compu- tational Intelligence and AI in Games, 4(1):1-43, 2012.","title":"A Survey of Monte Carlo Tree Search Methods","authors":"Cameron Browne, Edward J Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, Simon Colton","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Coulom, 2007] R. Coulom. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In H. J. van den Herik, P. Ciancarini, and H. H. L. M. Donkers, editors, 5th International Conference on Computers and Games, CG 2006. Revised Papers, volume 4630 of Lecture Notes in Computer Science, pages 72-83, 2007.","title":"Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search","authors":"R Coulom","year":"2007","venue":"5th International Conference on Computers and Games, CG 2006","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Finnsson and Björnsson, 2011] Hilmar Finnsson and Yngvi Björnsson. Game-Tree Properties and MCTS Performance. In IJCAI 2011 Workshop on General Intelligence in Game Playing Agents, GIGA'11, pages 23-30, 2011.","title":"Hilmar Finnsson and Yngvi Björnsson. Game-Tree Properties and MCTS Performance","authors":"Björnsson Finnsson","year":"2011","venue":"IJCAI 2011 Workshop on General Intelligence in Game Playing Agents, GIGA'11","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gelly and Silver, 2007] Sylvain Gelly and David Silver. Combining Online and Offline Knowledge in UCT. In Z. Ghahramani, editor, 24th International Conference on Machine Learning, ICML 2007, volume 227 of ACM In- ternational Conference Proceeding Series, pages 273-280, 2007.","title":"Combining Online and Offline Knowledge in UCT","authors":"Silver Gelly, Sylvain Gelly, David Silver","year":"2007","venue":"24th International Conference on Machine Learning, ICML 2007","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gelly et al., 2006] Sylvain Gelly, Yizao Wang, Rémi Munos, and Olivier Teytaud. Modification of UCT with Patterns in Monte-Carlo Go. Technical report, HAL -CCSd -CNRS, France, 2006.","title":"Modification of UCT with Patterns in Monte-Carlo Go","authors":"Yizao Sylvain Gelly, Rémi Wang, Olivier Munos, Teytaud","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Knuth and Moore, 1975] Donald E. Knuth and Ronald W. Moore. An Analysis of Alpha-Beta Pruning. Artificial Intelligence, 6(4):293-326, 1975.","title":"An Analysis of Alpha-Beta Pruning","authors":"Donald E Knuth, Ronald W Moore","year":"1975","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kocsis and Szepesvári, 2006] Levente Kocsis and Csaba Szepesvári. Bandit Based Monte-Carlo Planning. In J. Fürnkranz, T. Scheffer, and M. Spiliopoulou, editors, 17th European Conference on Machine Learning, ECML 2006, volume 4212 of Lecture Notes in Computer Science, pages 282-293, 2006.","title":"Bandit Based Monte-Carlo Planning","authors":"Szepesvári Kocsis, Levente Kocsis, Csaba Szepesvári","year":"2006","venue":"17th European Conference on Machine Learning, ECML 2006","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lorentz, 2008] Richard J. Lorentz. Amazons Discover Monte-Carlo. In van den Herik et al. [2008], pages 13- 24.","title":"Amazons Discover Monte-Carlo","authors":"Richard J Lorentz","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Nijssen and Winands, 2012] J. A. M. Nijssen and Mark H. M. Winands. Playout Search for Monte-Carlo Tree Search in Multi-player Games. In H. J. van den Herik and Aske Plaat, editors, 13th International Conference on Ad- vances in Computer Games, ACG 2011, volume 7168 of Lecture Notes in Computer Science, pages 72-83, 2012.","title":"Playout Search for Monte-Carlo Tree Search in Multi-player Games","authors":"J A M Nijssen, H M Mark, Winands","year":"2011","venue":"13th International Conference on Advances in Computer Games","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ramanujan et al., 2010a] Raghuram Ramanujan, Ashish Sabharwal, and Bart Selman. On Adversarial Search Spaces and Sampling-Based Planning. In Ronen I. Brafman, Hec- tor Geffner, Jörg Hoffmann, and Henry A. Kautz, editors, 20th International Conference on Automated Planning and Scheduling, ICAPS 2010, pages 242-245, 2010.","title":"On Adversarial Search Spaces and Sampling-Based Planning","authors":"Ramanujan","year":"2010","venue":"20th International Conference on Automated Planning and Scheduling, ICAPS 2010","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ramanujan et al., 2010b] Raghuram Ramanujan, Ashish Sabharwal, and Bart Selman. Understanding Sampling Style Adversarial Search Methods. In Peter Grünwald and Peter Spirtes, editors, 26th Conference on Uncertainty in Artificial Intelligence, UAI 2010, pages 474-483, 2010.","title":"Understanding Sampling Style Adversarial Search Methods","authors":"Ramanujan","year":"2010","venue":"26th Conference on Uncertainty in Artificial Intelligence, UAI 2010","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J. Maddi- son, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Pan- neershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Tim- othy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, Demis Hassabis","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Silver et al., 2017] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy P. Lillicrap, Karen Simonyan, and Demis Hassabis. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. CoRR, abs/1712.01815, 2017.","title":"Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy P Lillicrap, Karen Simonyan, Demis Hassabis","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sturtevant, 2008] Nathan R. Sturtevant. An Analysis of UCT in Multi-Player Games. ICGA Journal, 31(4):195-208, 2008.","title":"An Analysis of UCT in Multi-Player Games","authors":"Nathan R Sturtevant","year":"2008","venue":"ICGA Journal","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[van den Herik et al., 2008] H. J. van den Herik, X. Xu, Z. Ma, and M. H. M. Winands, editors. volume 5131 of Lecture Notes in Computer Science, 2008.","title":"Lecture Notes in Computer Science","authors":"H J Van Den Herik, X Xu, Z Ma, M H M Winands","year":"2008","venue":"Lecture Notes in Computer Science","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Winands and Björnsson, 2011] Mark H. M. Winands and Yngvi Björnsson. Alpha-Beta-based Play-outs in Monte- Carlo Tree Search. In Sung-Bae Cho, Simon M. Lucas, and Philip Hingston, editors, 2011 IEEE Conference on Computational Intelligence and Games, CIG 2011, pages 110-117, 2011.","title":"Alpha-Beta-based Play-outs in Monte-Carlo Tree Search","authors":"H M Mark, Yngvi Winands, Björnsson","year":"2011","venue":"IEEE Conference on Computational Intelligence and Games, CIG 2011","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Winands et al., 2008] Mark H. M. Winands, Yngvi Björns- son, and Jahn-Takeshi Saito. Monte-Carlo Tree Search Solver. In van den Herik et al. [2008], pages 25-36.","title":"Monte-Carlo Tree Search Solver","authors":"H M Mark, Yngvi Winands, Jahn-Takeshi Björnsson, Saito","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Winands et al., 2010] Mark H. M. Winands, Yngvi Björns- son, and Jahn-Takeshi Saito. Monte Carlo Tree Search in Lines of Action. IEEE Transactions on Computational Intelligence and AI in Games, 2(4):239-250, 2010.","title":"Monte Carlo Tree Search in Lines of Action","authors":"H M Mark, Yngvi Winands, Jahn-Takeshi Björnsson, Saito","year":"2010","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}