Algorithms or Actions? A Study in Large-Scale Reinforcement Learning

Algorithms or Actions? A Study in Large-Scale Reinforcement Learning

Anderson Rocha Tavares, Sivasubramanian Anbalagan, Leandro Soriano Marcolino, Luiz Chaimowicz

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 2717-2723. https://doi.org/10.24963/ijcai.2018/377

Large state and action spaces are very challenging to reinforcement learning. However, in many domains there is a set of algorithms available, which estimate the best action given a state. Hence, agents can either directly learn a performance-maximizing mapping from states to actions, or from states to algorithms. We investigate several aspects of this dilemma, showing sufficient conditions for learning over algorithms to outperform over actions for a finite number of training iterations. We present synthetic experiments to further study such systems. Finally, we propose a function approximation approach, demonstrating the effectiveness of learning over algorithms in real-time strategy games.
Keywords:
Machine Learning: Reinforcement Learning
Multidisciplinary Topics and Applications: Computer Games
Uncertainty in AI: Markov Decision Processes
Machine Learning Applications: Game Playing