Equivalence Relations in Fully and Partially Observable Markov Decision Processes

Castro, Pablo Samuel; Panangaden, Prakash; Precup, Doina

Equivalence Relations in Fully and Partially Observable Markov Decision Processes

Pablo Samuel Castro, Prakash Panangaden, Doina Precup

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. Pages 1653-1657.

Abstract

We explore equivalence relations between states in Markov Decision Processes and Partially Observable Markov Decision Processes. We focus on two different equivalence notions: bisimulation (Givan et al., 2003) and a notion of trace equivalence, under which states are considered equivalent if they generate the same conditional probability distributions over observation sequences (where the conditioning is on action sequences). We show that the relationship between these two equivalence notions changes depending on the amount and nature of the partial observability. We also present an alternate characterization of bisimulation based on trajectory equivalence.Pablo Samuel Castro, Prakash Panangaden, Doina Precup

Pablo Samuel Castro, Prakash Panangaden, and Doina Precup. 2009. Equivalence Relations in Fully and Partially Observable Markov Decision Processes. In Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09. IJCAI, 1653–1657. https://ijcai.org/proceedings/2009/276

@inproceedings{ijcai2009p276,
  title     = {Equivalence Relations in Fully and Partially Observable Markov Decision Processes},
  author    = {Castro, Pablo Samuel and Panangaden, Prakash and Precup, Doina},
  booktitle = {Proceedings of the Twenty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-09}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Hiroaki Kitano},
  pages     = {1653--1657},
  year      = {2009},
  url       = {https://ijcai.org/proceedings/2009/276},
}

TY  - CPAPER
TI  - Equivalence Relations in Fully and Partially Observable Markov Decision Processes
AU  - Castro, Pablo Samuel
AU  - Panangaden, Prakash
AU  - Precup, Doina
T2  - Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, IJCAI-09
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2009
SP  - 1653
EP  - 1657
UR  - https://ijcai.org/proceedings/2009/276
ER  -