---
title: "Deep Residual Reinforcement Learning (Extended Abstract)"
authors:
  - "Shangtong Zhang"
  - "Wendelin Boehmer"
  - "Shimon Whiteson"
doi: "10.24963/ijcai.2021/668"
url: "https://ijcai.org/proceedings/2021/668"
pdf: "https://ijcai.org/proceedings/2021/0668.pdf"
venue: "Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"
year: 2021
pages: "4869-4873"
track: "Sister Conferences Best Papers"
keywords:
  - "Machine Learning: Reinforcement Learning"
---

# Deep Residual Reinforcement Learning (Extended Abstract)

Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson

*Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence*, pages 4869-4873. DOI: [10.24963/ijcai.2021/668](https://doi.org/10.24963/ijcai.2021/668)

## Abstract

We revisit residual algorithms in both model-free and model-based reinforcement learning settings. 
We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in commonly used benchmarks. 
Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning. 
Compared with the existing TD(k) method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.