Commitment Semantics for Sequential Decision Making under Reward Uncertainty

Zhang, Qi; Durfee, Edmund H.; Singh, Satinder; Chen, Anna; Witwicki, Stefan J.

Commitment Semantics for Sequential Decision Making under Reward Uncertainty

Qi Zhang, Edmund H. Durfee, Satinder Singh, Anna Chen, Stefan J. Witwicki

Download BibTeX Cite

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Pages 3315-3322.

Abstract

Cooperating agents can make commitments to help each other, but commitments might have to be probabilistic when actions have stochastic outcomes. We consider the additional complication in cases where an agent might prefer to change its policy as it learns more about its reward function from experience. How should such an agent be allowed to change its policy while still faithfully pursuing its commitment in a principled decision-theoretic manner? We address this question by defining a class of Dec-POMDPs with Bayesian reward uncertainty, and by developing a novel Commitment Constrained Iterative Mean Reward algorithm that implements the semantics of faithful commitment pursuit while still permitting the agent's response to the evolving understanding of its rewards. We bound the performance of our algorithm theoretically, and evaluate empirically how it effectively balances solution quality and computation cost.

Qi Zhang, Edmund H. Durfee, Satinder Singh, Anna Chen, and Stefan J. Witwicki. 2016. Commitment Semantics for Sequential Decision Making under Reward Uncertainty. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16. IJCAI, 3315–3322. https://ijcai.org/proceedings/2016/469

@inproceedings{ijcai2016p469,
  title     = {Commitment Semantics for Sequential Decision Making under Reward Uncertainty},
  author    = {Zhang, Qi and Durfee, Edmund H. and Singh, Satinder and Chen, Anna and Witwicki, Stefan J.},
  booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-16}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Subbarao Kambhampati},
  pages     = {3315--3322},
  year      = {2016},
  month     = {7},
  url       = {https://ijcai.org/proceedings/2016/469},
}

TY  - CPAPER
TI  - Commitment Semantics for Sequential Decision Making under Reward Uncertainty
AU  - Zhang, Qi
AU  - Durfee, Edmund H.
AU  - Singh, Satinder
AU  - Chen, Anna
AU  - Witwicki, Stefan J.
T2  - Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2016
SP  - 3315
EP  - 3322
UR  - https://ijcai.org/proceedings/2016/469
ER  -