{"type":"ScholarlyArticle","title":"Reinforcement Nash Equilibrium Solver","authors":[{"name":"Xinrun Wang","slug":"xinrun-wang","given":"Xinrun","family":"Wang","affiliation":"Nanyang Technological University , Singapore"},{"name":"Chang Yang","slug":"chang-yang","given":"Chang","family":"Yang","affiliation":"The Hong Kong Polytechnic University , Hong Kong SAR , China"},{"name":"Shuxin Li","slug":"shuxin-li","given":"Shuxin","family":"Li","affiliation":"Nanyang Technological University , Singapore"},{"name":"Pengdeng Li","slug":"pengdeng-li","given":"Pengdeng","family":"Li","affiliation":"Nanyang Technological University , Singapore"},{"name":"Xiao Huang","slug":"xiao-huang","given":"Xiao","family":"Huang","affiliation":"The Hong Kong Polytechnic University , Hong Kong SAR , China"},{"name":"Hau Chan","slug":"hau-chan","given":"Hau","family":"Chan","affiliation":"University of Nebraska-Lincoln , Lincoln , Nebraska , United States"},{"name":"Bo An","slug":"bo-an","given":"Bo","family":"An","affiliation":"Nanyang Technological University , Singapore; Skywork AI , Singapore"}],"abstract":"Nash Equilibrium (NE) is the canonical solution concept of game theory, which provides an elegant tool to understand the rationalities. Though mixed strategy NE exists in any game with finite players and actions, computing NE in two- or multi-player general-sum games is PPAD-Complete. Various alternative solutions, e.g., Correlated Equilibrium (CE), and learning methods, e.g., fictitious play (FP), are proposed to approximate NE. For convenience, we call these methods as ``inexact solvers'', or ``solvers'' for short. However, the alternative solutions differ from NE and the learning methods generally fail to converge to NE. Therefore, in this work, we propose REinforcement Nash Equilibrium Solver (RENES), which trains a single policy to modify the games with different sizes and applies the solvers on the modified games where the obtained solution is evaluated on the original games. Specifically, our contributions are threefold. i) We represent the games as alpha-rank response graphs and leverage graph neural network (GNN) to handle the games with different sizes as inputs; ii) We use tensor decomposition, e.g., canonical polyadic (CP), to make the dimension of modifying actions fixed for games with different sizes; iii) We train the modifying strategy for games with the widely-used proximal policy optimization (PPO) and apply the solvers to solve the modified games, where the obtained solution is evaluated on original games. Extensive experiments on large-scale normal-form games show that our method can further improve the approximation of NE of different solvers, i.e., alpha-rank, CE, FP and PRD, and can be generalized to unseen games.","doi":"10.24963/ijcai.2024/30","url":"https://ijcai.org/proceedings/2024/30","pdf":"https://ijcai.org/proceedings/2024/0030.pdf","markdown":"https://ijcai.org/proceedings/2024/30.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/30","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":30,"pages":{"first":265,"last":273},"keywords":["Machine Learning: ML: Reinforcement learning","Game Theory and Economic Paradigms: GTEP: Noncooperative games","Agent-based and Multi-agent Systems: MAS: Multi-agent learning","Machine Learning: ML: Game Theory"],"videos":[],"references":[{"position":1,"raw":"[Akiba et al., 2019] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In KDD, pages 2623-2631, 2019.","title":"Optuna: A next-generation hyperparameter optimization framework","authors":"Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama","year":"2019","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aumann, 1974] Robert J Aumann. Subjectivity and corre- lation in randomized strategies. Journal of Mathematical Economics, 1(1):67-96, 1974.","title":"Subjectivity and correlation in randomized strategies","authors":"Robert J Aumann","year":"1974","venue":"Journal of Mathematical Economics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aumann, 1987] Robert J Aumann. Correlated equilibrium as an expression of Bayesian rationality. Econometrica: Journal of the Econometric Society, pages 1-18, 1987.","title":"Correlated equilibrium as an expression of Bayesian rationality","authors":"Robert J Aumann","year":"1987","venue":"Econometrica: Journal of the Econometric Society","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Sandholm, 2015] Noam Brown and Tuomas Sandholm. Simultaneous abstraction and equilibrium find- ing in games. In IJCAI, pages 489-496, 2015.","title":"Simultaneous abstraction and equilibrium finding in games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/75"},{"position":5,"raw":"[Brown and Sandholm, 2017] Noam Brown and Tuomas Sandholm. Safe and nested subgame solving for imperfect- information games. In NeurIPS, pages 689-699, 2017.","title":"Safe and nested subgame solving for imperfectinformation games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brown, 1951] George W Brown. Iterative solution of games by fictitious play. Activity Analysis of Production and Allo- cation, 13(1):374, 1951.","title":"Iterative solution of games by fictitious play","authors":"George W Brown","year":"1951","venue":"Activity Analysis of Production and Allocation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Burch et al., 2014] Neil Burch, Michael Johanson, and Michael Bowling. Solving imperfect information games using decomposition. In AAAI, 2014.","title":"Solving imperfect information games using decomposition","authors":"Neil Burch, Michael Johanson, Michael Bowling","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2009] Xi Chen, Xiaotie Deng, and Shang-Hua Teng. Settling the complexity of computing two-player Nash equilibria. Journal of the ACM, 56(3):1-57, 2009.","title":"Settling the complexity of computing two-player Nash equilibria","authors":"Xi Chen, Xiaotie Deng, Shang-Hua Teng","year":"2009","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Daskalakis et al., 2009] Constantinos Daskalakis, Paul W Goldberg, and Christos H Papadimitriou. The complex- ity of computing a Nash equilibrium. SIAM Journal on Computing, 39(1):195-259, 2009.","title":"The complexity of computing a Nash equilibrium","authors":"Constantinos Daskalakis, Paul W Goldberg, Christos H Papadimitriou","year":"2009","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fudenberg and Tirole, 1991] Drew Fudenberg and Jean Ti- role. Game Theory. MIT press, 1991.","title":"Drew Fudenberg and Jean Tirole","authors":"Tirole Fudenberg","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gemp et al., 2022] Ian Gemp, Rahul Savani, Marc Lanctot, Yoram Bachrach, Thomas Anthony, Richard Everett, An- drea Tacchetti, Tom Eccles, and János Kramár. Sample- based approximation of nash in large many-player games via gradient descent. In AAMAS, pages 507-515, 2022.","title":"Samplebased approximation of nash in large many-player games via gradient descent","authors":"Ian Gemp, Rahul Savani, Marc Lanctot, Yoram Bachrach, Thomas Anthony, Richard Everett, Andrea Tacchetti, Tom Eccles, János Kramár","year":"2022","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Goldberg et al., 2013] Paul W Goldberg, Christos H Pa- padimitriou, and Rahul Savani. The complexity of the ho- motopy method, equilibrium selection, and Lemke-Howson solutions. ACM Transactions on Economics and Computa- tion (TEAC), 1(2):1-25, 2013.","title":"The complexity of the homotopy method, equilibrium selection, and Lemke-Howson solutions","authors":"Christos H Paul W Goldberg, Rahul Papadimitriou, Savani","year":"2013","venue":"ACM Transactions on Economics and Computation (TEAC)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Haarnoja et al., 2018] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In ICML, pages 1861-1870, 2018.","title":"Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor","authors":"Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Harsanyi et al., 1988] John C Harsanyi, Reinhard Selten, et al. A general theory of equilibrium selection in games. MIT Press Books, 1, 1988.","title":"A general theory of equilibrium selection in games","authors":"Reinhard John C Harsanyi, Selten","year":"1988","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hart and Mas-Colell, 2000] Sergiu Hart and Andreu Mas- Colell. A simple adaptive procedure leading to correlated equilibrium. Econometrica, 68(5):1127-1150, 2000.","title":"A simple adaptive procedure leading to correlated equilibrium","authors":"Sergiu Hart, Andreu Mas-Colell","year":"2000","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Heinrich et al., 2015] Johannes Heinrich, Marc Lanctot, and David Silver. Fictitious self-play in extensive-form games. In ICML, pages 805-813, 2015.","title":"Fictitious self-play in extensive-form games","authors":"Johannes Heinrich, Marc Lanctot, David Silver","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Herings and Peeters, 2010] P Herings and Ronald Peeters. Homotopy methods to compute equilibria in game theory. Economic Theory, 42(1):119-156, 2010.","title":"Homotopy methods to compute equilibria in game theory","authors":"Peeters ; P Herings, Ronald Herings, Peeters","year":"2010","venue":"Economic Theory","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kolda and Bader, 2009] Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM Review, 51(3):455-500, 2009.","title":"Tensor decompositions and applications","authors":"G Tamara, Brett W Kolda, Bader","year":"2009","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. In NeurIPS, 2012.","title":"ImageNet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lanctot et al., 2017] Marc Lanctot, Vinicius Zambaldi, Audrūnas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, and Thore Graepel. A unified game- theoretic approach to multiagent reinforcement learning. In NeurIPS, pages 4193-4206, 2017.","title":"A unified gametheoretic approach to multiagent reinforcement learning","authors":"Marc Lanctot, Vinicius Zambaldi, Audrūnas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, Thore Graepel","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lanctot et al., 2019] Marc Lanctot, Edward Lockhart, Jean- Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, et al. OpenSpiel: A frame- work for reinforcement learning in games. arXiv preprint arXiv:1908.09453, 2019.","title":"OpenSpiel: A framework for reinforcement learning in games","authors":"Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lemke and Howson, 1964] Carlton E Lemke and Joseph T Howson, Jr. Equilibrium points of bimatrix games. Jour- nal of the Society for Industrial and Applied Mathematics, 12(2):413-423, 1964.","title":"Equilibrium points of bimatrix games","authors":"E Carlton, Joseph T Lemke, Jr Howson","year":"1964","venue":"Journal of the Society for Industrial and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li and Wellman, 2021] Zun Li and Michael P Wellman. Evo- lution strategies for approximate solution of bayesian games. In AAAI, pages 5531-5540, 2021.","title":"Evolution strategies for approximate solution of bayesian games","authors":"Li, Wellman ; ] Zun Li, Michael P Wellman","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2018] Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Combinatorial optimization with graph convolu- tional networks and guided tree search. In NeurIPS, pages 537-546, 2018.","title":"Combinatorial optimization with graph convolutional networks and guided tree search","authors":"Li","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Marris et al., 2021] Luke Marris, Paul Muller, Marc Lanc- tot, Karl Tuyls, and Thore Graepel. Multi-agent training beyond zero-sum with correlated equilibrium meta-solvers. In ICML, pages 7480-7491, 2021.","title":"Multi-agent training beyond zero-sum with correlated equilibrium meta-solvers","authors":"Marris","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[McKelvey and Palfrey, 1995] Richard D McKelvey and Thomas R Palfrey. Quantal response equilibria for normal form games. Games and economic behavior, 10(1):6-38, 1995.","title":"Quantal response equilibria for normal form games","authors":"D Richard, Thomas R Mckelvey, Palfrey","year":"1995","venue":"Games and economic behavior","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous methods for deep reinforcement learning. In ICML, pages 1928-1937, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Monderer and Shapley, 1996] Dov Monderer and Lloyd S Shapley. Fictitious play property for games with identical interests. Journal of Economic Theory, 68(1):258-265, 1996.","title":"Fictitious play property for games with identical interests","authors":"Shapley Monderer","year":"1996","venue":"Journal of Economic Theory","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Muller et al., 2020] Paul Muller, Shayegan Omidshafiei, Mark Rowland, Karl Tuyls, Julien Perolat, Siqi Liu, Daniel Hennes, Luke Marris, Marc Lanctot, Edward Hughes, et al. A generalized training approach for multiagent learning. In ICLR, 2020.","title":"A generalized training approach for multiagent learning","authors":"Paul Muller, Shayegan Omidshafiei, Mark Rowland, Karl Tuyls, Julien Perolat, Siqi Liu, Daniel Hennes, Luke Marris, Marc Lanctot, Edward Hughes","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Myerson, 1999] Roger B Myerson. Nash equilibrium and the history of economic theory. Journal of Economic Literature, 37(3):1067-1082, 1999.","title":"Nash equilibrium and the history of economic theory","authors":"Roger B Myerson","year":"1999","venue":"Journal of Economic Literature","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Nash Jr, 1950] John F Nash Jr. Equilibrium points in n- person games. PNAS, 36(1):48-49, 1950.","title":"Equilibrium points in nperson games","authors":"John F Nash Jr","year":"1950","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Omidshafiei et al., 2019] Shayegan Omidshafiei, Christos Papadimitriou, Georgios Piliouras, Karl Tuyls, Mark Row- land, Jean-Baptiste Lespiau, Wojciech M Czarnecki, Marc Lanctot, Julien Perolat, and Remi Munos. α-rank: Multi- agent evaluation by evolution. Scientific Reports, 9(1):1-29, 2019.","title":"α-rank: Multiagent evaluation by evolution","authors":"Shayegan Omidshafiei, Christos Papadimitriou, Georgios Piliouras, Karl Tuyls, Mark Rowland, Jean-Baptiste Lespiau, Wojciech M Czarnecki, Marc Lanctot, Julien Perolat, Remi Munos","year":"2019","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Omidshafiei et al., 2020] Shayegan Omidshafiei, Karl Tuyls, Wojciech M Czarnecki, Francisco C Santos, Mark Row- land, Jerome Connor, Daniel Hennes, Paul Muller, Julien Pérolat, Bart De Vylder, et al. Navigating the landscape of multiplayer games. Nature Communications, 11(1):1-17, 2020.","title":"Navigating the landscape of multiplayer games","authors":"Shayegan Omidshafiei, Karl Tuyls, Wojciech M Czarnecki, Francisco C Santos, Mark Rowland, Jerome Connor, Daniel Hennes, Paul Muller, Julien Pérolat, Bart De Vylder","year":"2020","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Schulman et al., 2015] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In ICML, pages 1889-1897, 2015.","title":"Trust region policy optimization","authors":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [Schuster and Sigmund, 1983] Peter Schuster and Karl Sig- mund. Replicator dynamics. Journal of Theoretical Biology, 100(3):533-538, 1983.","title":"Schuster and Sigmund, 1983] Peter Schuster and Karl Sigmund. Replicator dynamics","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"1983","venue":"Journal of Theoretical Biology","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement Learning: An Introduction. MIT press, 2018.","title":"Reinforcement Learning: An Introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Sutton et al., 1999] Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. In NeurIPS, pages 1057-1063, 1999.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David Richard S Sutton, Satinder Mcallester, Yishay Singh, Mansour","year":"1999","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Taylor and Jonker, 1978] Peter D Taylor and Leo B Jonker. Evolutionary stable strategies and game dynamics. Mathe- matical biosciences, 40(1-2):145-156, 1978.","title":"Evolutionary stable strategies and game dynamics","authors":"D Peter, Leo B Taylor, Jonker","year":"1978","venue":"Mathematical biosciences","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2021] Runzhong Wang, Zhigang Hua, Gan Liu, Jiayi Zhang, Junchi Yan, Feng Qi, Shuang Yang, Jun Zhou, and Xiaokang Yang. A bi-level framework for learning to solve combinatorial optimization on graphs. In NeurIPS, 2021.","title":"A bi-level framework for learning to solve combinatorial optimization on graphs","authors":"Runzhong Wang, Zhigang Hua, Gan Liu, Jiayi Zhang, Junchi Yan, Feng Qi, Shuang Yang, Jun Zhou, Xiaokang Yang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Watkins and Dayan, 1992] Christopher JCH Watkins and Pe- ter Dayan. Q-learning. Machine learning, 8(3):279-292, 1992.","title":"Q-learning","authors":"Jch Christopher, Peter Watkins, Dayan","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Yun et al., 2019] Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph trans- former networks. In NeurIPS, pages 11983-11993, 2019.","title":"Graph transformer networks","authors":"Yun","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}