Proceedings

Deep Residual Reinforcement Learning (Extended Abstract)

Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson

Download BibTeX Cite

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. Sister Conferences Best Papers. Pages 4869-4873.

https://doi.org/10.24963/ijcai.2021/668

Keywords

  • Machine Learning: Reinforcement Learning

Abstract

We revisit residual algorithms in both model-free and model-based reinforcement learning settings.

We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in commonly used benchmarks.

Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning.

Compared with the existing TD(k) method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.

Shangtong Zhang, Wendelin Boehmer, and Shimon Whiteson. 2021. Deep Residual Reinforcement Learning (Extended Abstract). In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. IJCAI, 4869–4873. https://doi.org/10.24963/ijcai.2021/668
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards