Sample Efficient On-Line Learning of Optimal Dialogue Policies with Kalman Temporal Differences

Pietquin, Olivier; Geist, Matthieu; Chandramohan, Senthilkumar

doi:10.5591/978-1-57735-516-8/IJCAI11-314

Sample Efficient On-Line Learning of Optimal Dialogue Policies with Kalman Temporal Differences

Olivier Pietquin, Matthieu Geist, Senthilkumar Chandramohan

Download BibTeX Cite

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. Pages 1878-1882.

https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-314

Abstract

Designing dialog policies for voice-enabled interfaces is a tailoring job that is most often left to natural language processing experts. This job is generally redone for every new dialog task because cross-domain transfer is not possible. For this reason, machine learning methods for dialog policy optimization have been investigated during the last 15 years. Especially, reinforcement learning (RL) is now part of the state of the art in this domain. Standard RL methods require to test more or less random changes in the policy on users to assess them as improvements or degradations. This is called on policy learning. Nevertheless, it can result in system behaviors that are not acceptable by users. Learning algorithms should ideally infer an optimal strategy by observing interactions generated by a non-optimal but acceptable strategy, that is learning off-policy. In this contribution, a sample-efficient, online and off-policy reinforcement learning algorithm is proposed to learn an optimal policy from few hundreds of dialogues generated with a very simple handcrafted policy.

Olivier Pietquin, Matthieu Geist, and Senthilkumar Chandramohan. 2011. Sample Efficient On-Line Learning of Optimal Dialogue Policies with Kalman Temporal Differences. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11. IJCAI, 1878–1882. https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-314

@inproceedings{ijcai2011p314,
  title     = {Sample Efficient On-Line Learning of Optimal Dialogue Policies with Kalman Temporal Differences},
  author    = {Pietquin, Olivier and Geist, Matthieu and Chandramohan, Senthilkumar},
  booktitle = {Proceedings of the Twenty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-11}},
  publisher = {AAAI Press/International Joint Conferences on Artificial Intelligence},
  editor    = {Toby Walsh},
  pages     = {1878--1882},
  year      = {2011},
  month     = {7},
  doi       = {10.5591/978-1-57735-516-8/IJCAI11-314},
  url       = {https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-314},
}

TY  - CPAPER
TI  - Sample Efficient On-Line Learning of Optimal Dialogue Policies with Kalman Temporal Differences
AU  - Pietquin, Olivier
AU  - Geist, Matthieu
AU  - Chandramohan, Senthilkumar
T2  - Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI-11
PB  - AAAI Press/International Joint Conferences on Artificial Intelligence
PY  - 2011
SP  - 1878
EP  - 1882
DO  - 10.5591/978-1-57735-516-8/IJCAI11-314
UR  - https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-314
ER  -