{"type":"ScholarlyArticle","title":"SMDP Homomorphisms: An Algebraic Approach to Abstraction in Semi-Markov Decision Processes","authors":[{"name":"Balaraman Ravindran","slug":"balaraman-ravindran","given":"Balaraman","family":"Ravindran","affiliation":""},{"name":"Andrew G. Barto","slug":"andrew-g-barto","given":"Andrew G.","family":"Barto","affiliation":""}],"abstract":"To operate effectively in complex environments learning agents require the ability to selectively ignore irrelevant details and form useful abstractions. In this article we consider the question of what constitutes a useful abstraction in a stochastic sequential decision problem modeled as a semi-Markov Decision Process (SMDPs). We introduce the notion of SMDP homomorphism and argue that it provides a useful tool for a rigorous study of abstraction for SMDPs. We present an SMDP minimization framework and an abstraction framework for factored MDPs based on SMDP homomorphisms. We also model different classes of abstractions that arise in hierarchical systems. Although we use the options framework for purposes of illustration, the ideas are more generally applicable. We also show that the conditions for abstraction we employ are a generalization of earlier work by Dietterich as applied to the options framework.","doi":null,"url":"https://ijcai.org/proceedings/2003/144","pdf":"https://ijcai.org/proceedings/2003/0144.pdf","markdown":"https://ijcai.org/proceedings/2003/144.md","bibtex":"https://ijcai.org/proceedings/2003/bibtex/144","volume":{"slug":"2003","title":"Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2003","url":"https://ijcai.org/proceedings/2003/","year":2003,"location":"Acapulco, Mexico","dates":"","editors":"Georg Gottlob","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":144,"pages":{"first":1011,"last":1017},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}