---
title: "The Value of Observation for Monitoring Dynamic Systems"
authors:
  - "Eyal Even-Dar"
  - "Sham M. Kakade"
  - "Yishay Mansour"
doi: ""
url: "https://ijcai.org/proceedings/2007/398"
pdf: "https://ijcai.org/proceedings/2007/0398.pdf"
venue: "Proceedings of the Twentieth International Joint Conference on Artificial Intelligence"
year: 2007
pages: "2474-2478"
track: ""
keywords:
---

# The Value of Observation for Monitoring Dynamic Systems

Eyal Even-Dar, Sham M. Kakade, Yishay Mansour

*Proceedings of the Twentieth International Joint Conference on Artificial Intelligence*, pages 2474-2478.

## Abstract

We consider the fundamental problem of monitoring (i.e. tracking) the belief state in a dynamic system, when the model is only approximately correct and when the initial belief state might be unknown. In this general setting where the model is (perhaps only slightly) mis-specified, monitoring (and consequently planning) may be impossible as errors might accumulate over time. We provide a new characterization, the emph{value of observation}, which allows us to bound the error accumulation. The value of observation is a parameter that governs how much information the observation provides. For instance, in Partially Observable MDPs when it is 1 the POMDP is an MDP while for an unobservable Markov Decision Process the parameter is 0. Thus, the new parameter characterizes a spectrum from MDPs to unobservable MDPs depending on the amount of information conveyed in the observations.