Active Policy Iteration: Efficient Exploration through Active Learning for Value Function Approximation in Reinforcement Learning

Akiyama, Takayuki; Hachiya, Hirotaka; Sugiyama, Masashi

Active Policy Iteration: Efficient Exploration through Active Learning for Value Function Approximation in Reinforcement Learning

Takayuki Akiyama, Hirotaka Hachiya, Masashi Sugiyama

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. .

Abstract

Appropriately designing sampling policies is highly important for obtaining better control policies in reinforcement learning. In this paper, we first show that the least-squares policy iteration (LSPI) framework allows us to employ statistical active learning methods for linear regression. Then we propose a design method of good sampling policies for efficient exploration, which is particularly useful when the sampling cost of immediate rewards is high. We demonstrate the usefulness of the proposed method, named active policy iteration (API), through simulations with a batting robot.

Takayuki Akiyama, Hirotaka Hachiya, and Masashi Sugiyama. 2009. Active Policy Iteration: Efficient Exploration through Active Learning for Value Function Approximation in Reinforcement Learning. In Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09. IJCAI, https://ijcai.org/proceedings/2009/166

@inproceedings{ijcai2009p166,
  title     = {Active Policy Iteration: Efficient Exploration through Active Learning for Value Function Approximation in Reinforcement Learning},
  author    = {Akiyama, Takayuki and Hachiya, Hirotaka and Sugiyama, Masashi},
  booktitle = {Proceedings of the Twenty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-09}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Hiroaki Kitano},
  year      = {2009},
  url       = {https://ijcai.org/proceedings/2009/166},
}

TY  - CPAPER
TI  - Active Policy Iteration: Efficient Exploration through Active Learning for Value Function Approximation in Reinforcement Learning
AU  - Akiyama, Takayuki
AU  - Hachiya, Hirotaka
AU  - Sugiyama, Masashi
T2  - Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2009
UR  - https://ijcai.org/proceedings/2009/166
ER  -